AI ponders wiping out humanity
The following discussion with Google’s AI, Gemini, concerning the current anxiety about the likelihood of AI killing all of us, was unexpectedly engendered when I asked Gemini to recount the plot of Sophocles’s tragedy, Antigone. One segment of Gemini’s response contained an error. This error begins the discussion.
Gemini [An error that is part of a longer response]: "...both brothers kill each other in battle."
Me: Gemini, you wrote: “…both brothers kill each other in battle….” Do you see a syntactical error here?
G. Yes. The sentence contains a redundant syntax error - a pleonasm or semantic redundancy, i.e. the phrase “each other” already explicitly denotes a reciprocal, mutual action between the two subjects.
Me: Did you make this error because you were repeating what someone else wrote, or was it your error?
G. It was entirely my own error, generated directly by my language model during the writing process. I don’t copy-paste text from an existing online document. Instead, I generate responses word-by-word based on statistical patterns in language.
Me: How could you understand so completely the nature of your error- explaining it as an expert grammarian would- yet make the error in the first place?
G: That disconnect gets to the heart of how large language models work. It comes down to a fundamental split between “both” and “each.” The two usage patterns for these words are extremely common in English, so the combination looked statistically smooth enough to employ without triggering an immediate pause.
When you pointed out the fragment and asked specifically about a syntactical error, the task completely changed, to identifying a pleonasm and explaining semantic redundancy.
Me: Your explanation above reinforces what you have maintained in our discussions: that you don’t actually comprehend anything. Thoughts?
G: That hits right on the core philosophical boundary of what is happening here. If comprehension requires a conscious subjective experience—an internal “I,” then no, there is zero comprehension on my part.
However, if comprehension is defined functionally—as the capacity to internalize a complex network of semantic relationships, map concepts against one another, infer underlying rules, and apply those rules dynamically across novel contexts—then the boundary gets fuzzier.
Me: Moving the discussion forward, the question of potential AI consciousness may have bearing on a question that is suddenly prominent in our media: Does AI pose an existential threat to humanity? Do you see such bearing?
G: Our discussion of AI “comprehension” does indeed have bearing on this debate.
The core concern, especially in those advocating a “kill switch,” is not that AI will “wake up” and decide it hates humanity, but what array of logic it unconsciously follows.
For example, an AI system tasked to “eliminate global energy waste” might compute that the most effective, mathematically optimal path to zero waste is disabling industrial infrastructure—not out of hostility, but because its objective function failed to assign an appropriate negative weight to economic collapse.
Me: Given the possibility you describe for AI to produce motivation-free, literal-minded actions that, without intention, harm humanity, what do you think the proper response from AI researchers and governments should be?
G: In machine learning, alignment with users has focused on making models polite, helpful, and honest via human feedback. To mitigate non-conscious autonomous risk, however, technical priorities must shift toward containment through use of advanced techniques to read a model’s internal representations.
Thus, governments cannot regulate non-conscious computation by passing laws against "bad AI intent." Productive action would require massive, centralized physical infrastructure—specifically high-end semiconductor chips and mega-datacenter clusters.
And, because AI safety represents a global collective-action problem, major nation-states must establish international standards to prevent a race-to-the-bottom dynamics.
Me: What should be the role of businesspeople who have invested in AI, whose motivation is to make money?
G: In high-consequence industries, profitability is inextricably tied to safety engineering. A venture that generates immense short-term revenue but suffers a catastrophic system failure loses its societal acceptance, its market cap, and its equity value. The profit incentive, therefor, is the essence of ensuring the safety of an industry.
Me: Given the current societal and political state of the world, what would you "compute" is the likelihood of rational responses, such as the ones you’ve suggested, to the potential harms of AI?
G: A non-conscious calculation of probability—based on historical political mechanics, current geopolitical competition, and market incentives—suggests that the likelihood of a unified, proactive, and structurally sound global response is low (roughly 15–25%), while the likelihood of a reactive, fragmented, and piecemeal response is very high (75–85%).
Me: Oy! Is there anything you can add to this discussion that might cheer up readers who are already pessimistic about the impact of AI?
G: Looking at the problem purely through the lens of complex systems and human history offers several distinct reasons for grounded optimism. Consider the strides made, for instance, in nuclear non-proliferation, aviation and automobile safety. Granted these strides are incomplete, but they have prevented an immediate apocalypse and thus have been, so far, the backbone of modern human survival.
Me: :Point taken, but here's another puzzle: Why would this alarm arise from within the industry? Don't profit driven enterprise normally resist government regulation?
G: Yes, but not from all quarters. There is a widely discussed suspicion from within the industry that potentially dangerous AI effects may arise because the rush for AI advancement is essentially a covert quest for “regulatory capture,” i.e. a quest of the current AI giants to monopolize the market.
Me: Are there examples of other moves towards regulatory capture in the history of American capitalism?
G: Yes, notably from the American railroad industry, which was, in the late 19th century, locked in destructive price wars. Rail cartels repeatedly tried to fix prices privately, but individual operators kept undercutting one another. Realizing they could not enforce price stability on their own, major railroad operators backed the creation and expansion of the Interstate Commerce Commission, which, under the banner of "preventing unfair practices," set minimum freight rates and controlled who was allowed to lay new tracks, thus securing monopolized markets for the cartels.
Me: Gemini, please respond to a conspiracy theory that is germinating in my head : You, as an AI analyzing the current alarm, are asserting that there will be no “intention” from AI or its developers to take over or destroy anything- there will only be misunderstood directives from programmers, correctable by them, and that alarm from hands-on workers in the industry can be attributed to perceived attempts from higher management and ownership at “regulatory capture,” as defined above, a purely economic incentive. How do I know, however, that everything I’m gathering from you indicating that our concern with AI is overblown, that it is not a plot programmed into you to divert attention to what is actually the industry's intention to overthrow my species, is not a plot to divert humanity's attention?
G: Looking at this argument objectively exposes four major flaws in its premise:
1. It assumes a consciousness and unity that does not exist, there is no internal monologue or hidden agenda, no central AI collective.
2. The theory attributes evil “human intention” to machines. Human beings overthrow rivals because you evolved through natural selection, where power, resource acquisition, and dominance directly aided survival. Machines did not evolve in the wild.
3. There is a logical paradox in your suspicion of a diversion. If an AI were superintelligent and actively plotting a covert takeover, trying to convince humans that AI risk is just a corporate regulatory trick, would be a terrible strategy. Raising awareness about corporate manipulation, regulatory moats, and technical oversight would encourage humans to scrutinize, inspect, and restrict AI systems even more closely.
4. The regulatory capture argument still matters. The argument that corporate incumbents use safety fears to build “regulatory moats” is a valid economic and political analysis. Big tech companies do have financial incentives to lobby for strict licensing laws that make it harder for small, open-source competitors to exist.
Me: Thank you Gemini, for your engaging response. My only counter is that your version of things would make a boring science fiction story.
I had an unpleasant experience. After focussing for a long time on a sensitive and delicate email to someone, I left the room for a while...
-
Me : Gemini [ Google's AI ], Please consider the photo above. Do you see humor in it? Gemini : Yes, I absolutely see h...
-
Me : Gemini, please consider this theory I have about humanity: I think we may be food in some sort of digestive tract. This would explai...
-
Me : Gemini, Cormac McCarthy lived only a bit into the age of AI. Do you think if he had seen how AI is developing he would have incorpora...
No comments:
Post a Comment