AI conversations: September 2026

Thursday, September 17, 2026

AI ponders wiping out humantiy

In the following discussion with Google’s AI, Gemini, concerning the current anxiety about the likelihood of AI killing all of us, I refer to myself as “Me” and Gemini as “G.”

This engrossing topic was unexpectedly engendered when I asked Gemini to recount the plot of Sophocles’s tragedy, Antigone. One segment of Gemini’s response contained an error. That segment, copied below, begins the discussion.

G (Part of a longer response): …both brothers kill each other in battle…

Me: Gemini, you wrote: “…both brothers kill each other in battle….” Do you see a syntactical error here?

G. Yes. The sentence contains a redundant syntax error - a pleonasm or semantic redundancy, i.e. the phrase “each other” already explicitly denotes a reciprocal, mutual action between the two subjects.

Me: Did you make this error because you were repeating what someone else wrote, or was it your error?

G. It was entirely my own error, generated directly by my language model during the writing process. I don’t copy-paste text from an existing online document. Instead, I generate responses word-by-word based on statistical patterns in language.

Me: How could you understand so completely the nature of your error- explaining it as an expert grammarian would- and yet make the error in the first place?

G: That disconnect gets to the heart of how large language models work. It comes down to a fundamental split between “both” and “each.” The two usage patterns for these words are extremely common in English, so the combination looked statistically smooth enough to employ without triggering an immediate pause.

When you pointed out the fragment and asked specifically about a syntactical error, the task completely changed, to identifying a pleonasm and explaining semantic redundancy.

Me: Your explanation above reinforces what you have maintained in our discussions: that you don’t actually comprehend anything. Thoughts?

G: That hits right on the core philosophical boundary of what is happening here. If comprehension requires a conscious subjective experience—an internal “I,” then no, there is zero comprehension on my part.

However, if comprehension is defined functionally—as the capacity to internalize a complex network of semantic relationships, map concepts against one another, infer underlying rules, and apply those rules dynamically across novel contexts—then the boundary gets fuzzier.

Me: Our discussion of potential AI consciousness may have bearing on a question that is suddenly prominent in our media: Does AI pose an existential threat to humanity? Do you see such bearing?

G: Our discussion of AI “comprehension” does indeed have bearing on this debate.

The core concern is not that AI will “wake up” and decide it hates humanity, but what array of logic it unconsciously follows.

For example, an AI system tasked to “eliminate global energy waste” might compute that the most effective, mathematically optimal path to zero waste is disabling industrial infrastructure—not out of hostility, but because its objective function failed to assign an appropriate negative weight to economic collapse.

Me: Given the possibility you describe for AI to engender motivation-free, literal-minded actions that, without intention, harm humanity, what do you think the proper response from AI researchers and governments should be?

G: In machine learning, "alignment" with users has focused on making models polite, helpful, and honest via human feedback. To mitigate non-conscious autonomous risk, however, technical priorities must shift toward containment through use of advanced techniques to read a model’s internal representations.

Thus, governments cannot regulate non-conscious computation by passing laws against "bad AI intent." Productive action would require massive, centralized physical infrastructure—specifically high-end semiconductor chips and mega-datacenter clusters.

And, because AI safety represents a global collective-action problem, major nation-states must establish international standards to prevent a race-to-the-bottom dynamics.

Me: What should be the role be of businesspeople who have invested in AI, whose motivation is to make money from it?

G: In high-consequence industries profitability is inextricably tied to safety engineering. A venture that generates immense short-term revenue but suffers a catastrophic system failure loses its societal acceptance, its market cap, and its equity value. The profit incentive, therefor, is the essence of ensuring the safety of an industry.

Me: Given the current societal and political state of the world, what would you "compute" is the likelihood of rational responses, such as the ones you’ve suggested, to the potential harms of AI?

G: A non-conscious calculation of probability—based on historical political mechanics, current geopolitical competition, and market incentives—suggests that the likelihood of a unified, proactive, and structurally sound global response is low (roughly 15–25%), while the likelihood of a reactive, fragmented, and piecemeal response is very high (75–85%).

Me (aside): Oy!

Me: As a closing request, is there anything you can add to this discussion that might "cheer up" readers who are already pessimistic about the world?

G: Looking at the problem purely through the lens of complex systems and human history offers several distinct reasons for grounded optimism. Consider the strides made, for instance, in nuclear non-proliferation, aviation and automobile safety. Granted these strides are incomplete, but they have prevented an immediate apocalypse, and this admittedly incomplete accomplishment has been the historic backbone of human survival.

This ends my discussion with Gemini on the potential for AI to kill all of us. Whatever the contributions of the discussion, the final response hopefully illustrates AI’s capacity to generate an optimistic view- at least upon request.