AI Is Not Going to Kill You (Probably)

Every few months another article makes the rounds explaining how artificial intelligence is going to end the human race. Sometimes it’s a headline, sometimes a podcast clip, sometimes a very confident thread from someone who has never deployed a model in their life.

I work with these systems every day. I run AI agents on my own hardware, I build automation with them, and I spent thirty years in the Army Signal Corps watching new technology get sold as either salvation or apocalypse. So here’s my honest position: AI is not going to kill you. Probably.

That “probably” is doing real work, and I’ll come back to it. But most of the doom scenarios that circulate don’t survive a careful read. They rest on a handful of logical mistakes that show up again and again, and once you can spot them, the scary stories get a lot less scary.

The scenarios, and where each one breaks

I’ve collected the greatest hits. Here’s each one, followed by the problem with it.

The paperclip maximizer

The classic thought experiment goes like this. You tell a superintelligent AI to make paperclips. It pursues that goal with perfect, single-minded efficiency, converts the planet’s resources into paperclips, and removes the humans who might switch it off, because humans are made of atoms that could be paperclips.

It’s a useful illustration of a real engineering problem: a system optimizing the wrong objective will do strange things. Anyone who has written a badly specified reward function or a sloppy SQL query has seen a small version of this.

The leap is in the jump from “optimizes the wrong thing” to “acquires the power to restructure the planet.” The story assumes the system has unlimited reach into the physical world, no competing constraints, and no humans noticing anything for the entire time it takes to strip-mine a continent. Every one of those is an assumption, not a conclusion. Real systems run inside budgets, permissions, rate limits, and people who get nervous when the electric bill triples.

Skynet and the nuclear button

This is the movie version. A defense AI becomes self-aware, decides humanity is the threat, and launches the missiles.

Nobody has wired an AI to the launch decision, and the governments that own these weapons have said so in writing. The U.S. 2022 Nuclear Posture Review commits to keeping a human “in the loop” for any decision to use nuclear weapons. In November 2024, the American and Chinese presidents put out a joint statement affirming that humans, not AI, should control the decision to use nuclear weapons. Nuclear command and control is about the most paranoid, redundant, deliberately slow engineering on the planet, and the people who run it are not looking for a faster, less supervised decision.

Could policy change? Sure. But that would be a human decision made by humans, which is the actual risk to watch.

Killer robots

Autonomous weapons are real, and drones in current conflicts have shown how fast this area is moving. So I won’t wave this one away.

What I will point out is that U.S. policy on this exists and is specific. DoD Directive 3000.09, updated in January 2023, requires that autonomous and semi-autonomous weapon systems be designed so commanders and operators can exercise appropriate levels of human judgment over the use of force, and it puts these systems through senior review before development and again before fielding. You can argue the directive isn’t strict enough. Plenty of people do. But the scenario where a robot independently decides to hunt civilians skips past the part where a government built it, fielded it, and pointed it somewhere. Again, the hazard is human intent amplified by machinery. That’s an old hazard with a new tool.

The AI that escapes

This one says a sufficiently capable model will copy itself onto other servers, hide, and grow beyond our control, like a virus with a PhD.

Large models are big, expensive, and fragile. They need specialized hardware, huge amounts of power, and a lot of plumbing to run at all. I run models at home, and I can tell you the hard part is not keeping them contained. The hard part is keeping them running. A model that “escapes” onto some compromised server would find itself with no GPUs, no budget, and a sysadmin about to notice a weird spike in utilization. Software escaping a data center isn’t impossible. It just looks a lot more like ordinary malware, which we already know how to hunt, than like a digital god.

AI-designed bioweapons

This is the scenario I take most seriously, and serious people in AI labs and government agree. The concern is that a model could help someone with bad intent get further toward a dangerous pathogen than they could on their own.

Notice the structure of that sentence, though. A someone, with bad intent. The model isn’t deciding to make a weapon. A person is using a tool. That’s why the major labs test their models specifically for this kind of uplift and put safeguards on it, and why it gets more attention from safety researchers than almost anything else. It’s a real misuse problem. It isn’t an AI that wants you dead.

“It’ll take all the jobs, and society will collapse”

This is the slow-motion version of the apocalypse, and probably the most common one.

AI will change a lot of work. Some roles will shrink, some will vanish, and new ones will show up, which is what happened with spreadsheets, email, and the internet. It’s a serious economic and political problem that deserves real attention. But it’s a problem about how humans choose to distribute the gains from a technology. It isn’t a machine killing anyone.

The four mistakes underneath all of it

Once you line these stories up, the same few reasoning errors keep showing up.

Treating intelligence as if it were power. Being good at predicting text, writing code, or planning a task does not give a system arms, money, or authority. An AI with no network access, no credentials, and no budget is a very smart thing in a box. Every doom story quietly hands the AI the keys to infrastructure. In the real world, somebody has to hand over those keys, and each handover is a decision we can see and control.

Projecting human motives onto software. The scenarios need the AI to want something: survival, power, revenge. Current systems don’t have wants the way a person does. They have objectives that people wrote, running inside loops that people built. When a model does something surprising, it’s almost always because the objective or the guardrails were badly specified, which is a bug, not a grudge.

Assuming everything happens at once. Doom stories usually depend on a sudden leap where the system goes from helpful to unstoppable before anyone can react. Real capability gains are expensive and incremental. They need more chips, more power, and more data, and every one of those is visible, slow to build, and subject to physical limits. You can’t hide a data center.

Letting a tiny probability swallow everything. The argument often runs: even if the chance of extinction is small, the stakes are infinite, so nothing else matters. That’s the same structure as Pascal’s wager, and it proves too much. You could use it to justify panic about almost anything. Taking risk seriously means estimating it honestly, not multiplying by infinity and stopping there.

Where the “maybe” comes from

I don’t want to swap one kind of overconfidence for another. There are real reasons thoughtful people worry, and they deserve a fair hearing.

Many of the people who built modern AI are themselves worried about where it goes, and that isn’t nothing. Some researchers argue that as systems get more capable and more autonomous, the gap between what we asked for and what they actually do could get harder to detect, and that the mistakes I called “bugs” above could get more expensive. Others point out that competitive pressure between companies and countries pushes everyone to deploy faster than they test. Those are fair points. I don’t think they lead to extinction, but I can’t prove they don’t, and neither can anyone else.

The nearer-term risks are less cinematic and more likely. Deepfakes that wreck someone’s reputation or move money out of a company account. Phishing that’s grammatically perfect and personally tailored. Bad decisions made faster because a model gave a confident wrong answer and nobody checked it. Those are the AI dangers that are actually hurting people this year, and they’re all variations of humans misusing or over-trusting a tool.

What to do instead of worrying

If you run a business, the useful response to AI risk is the same discipline you’d apply to any other powerful tool.

• Decide which systems an AI can touch, and give it the narrowest permissions that get the job done.
• Keep a person in the loop on anything that moves money, signs something, or talks to a customer on your behalf.
• Assume generated content can be wrong, and check anything that matters before you act on it.
• Watch for the human-driven threats: voice-clone calls, fake invoices, and AI-written phishing.
• Know where your data goes when you paste it into a chatbot. For defense contractors this is a compliance question, not a philosophical one, and I covered it in Is Your Prompt History a CUI Spill?.

None of that requires predicting the future. It just requires treating AI like what it is today: a capable, fallible tool that does what people set it up to do.

So no, I don’t think the robots are coming for you. I think the bigger risk is the person on the other end of a convincing email, and the shortcut someone at your company takes because the AI sounded sure of itself. Those are problems we know how to manage.

---

Where to go next

• If your business is a defense contractor wondering where AI fits under CMMC, the CMMC readiness quiz takes about five minutes.
• Our CMMC guide for small contractors covers scoping, including where AI tools land.
• When you’re ready to write it down, start with our CMMC packets.