AI Can Aim the Weapon. It Cannot Pull the Trigger.

AI, Weapons, and the Line We Cannot Cross

I have been thinking a lot lately about AI weapons.

I don’t mean drones, missiles, or other technology used in war, which is already complicated enough. I mean fully autonomous weapons. Weapons where the machine does not just assist a human being but makes the decision to kill. It sounds like something from a movie where the computer takes over, and everyone realizes too late that maybe giving machines control over life and death was not the best idea. But this is not science fiction; people are seriously discussing whether artificial intelligence should be used in weapons systems that can identify targets, assess whether those targets are threats, and act without direct human intervention.

And honestly, that is terrifying.

I first started thinking about this more seriously when I saw some of the controversy around Anthropic and military AI. I watched an interview with Dario Amodei, the CEO of Anthropic, and what struck me was not that he seemed anti-military or anti-defense. What struck me was that the concern was much more specific: AI is not reliable enough, and maybe cannot ever be morally responsible enough, to be given certain kinds of tasks. Especially the task of deciding who lives and who dies. He compared it to a plane manufacturer, saying this plane cannot fly over a certain altitude; it would not be made to withstand that environment. It would be silly to call the manufacturer anti-planes or anti-flying!

At first, I wondered if maybe I was misunderstanding the whole thing. I kept thinking, surely no one is really saying we can trust a machine to make that final call. Surely everyone agrees there has to be a human being somewhere in the process. But then I realized that this is exactly the question. Where is the human being in the process? Is the human just setting up the system and walking away? Is the human approving a general plan? Or is the human actually making the specific moral decision at the moment it matters? Because those are not the same thing.

AI can analyze, suggest, detect patterns, pinpoint a location, and say, “This is probably an enemy camp.” AI can say, “Based on the information available, this looks like a threat.” AI can even help guide a missile after a human has given the order. But AI cannot make the moral choice. The machine can say, “This is likely.” It cannot say, “Therefore, this person should die.” That judgment belongs to a human being. And not just any human being, either. It belongs to a human being with authority, expertise, conscience, and responsibility. Someone who can look at the situation and say, “Yes, I am morally responsible for this decision.” Someone who cannot hide behind the machine afterward and say, “Well, the algorithm told me to.” That is not how moral responsibility works.

Pope Leo XIV, in his first encyclical Magnifica Humanitas, is very clear about this:

“Sometimes there is talk of “artificial moral agents,” as if machines were able to distinguish between right and wrong with greater consistency than a human being. Yet moral judgment cannot be reduced to calculation, for it involves conscience, personal responsibility and the recognition of the other as a person. Therefore, it is not permissible to entrust lethal or otherwise irreversible decisions to artificial systems. No algorithm can make war morally acceptable” (Magnifica Humanitas, 198)

AI does not have a conscience or a soul; it does not understand the dignity of the person in front of it. It does not know what it means to take a life. It can calculate probability, but it cannot carry guilt. It can process data, but it cannot repent. It can produce an answer, but it cannot stand before God and answer for what it has done. Only a human being can do that.

We already know that AI makes mistakes in much smaller things. It misreads articles. It invents sources. It agrees too easily with the person prompting it. It can sound confident even when it's completely wrong. It can take the context you give it and run in the direction it thinks you want to go. That is annoying when it gives you a poor summary, and very dangerous when it gives you inaccurate medical information. But it is deadly when the question is whether to strike a target. Imagine asking AI, “Should we go to war?” If the context of the conversation already suggests that you want to go to war, AI can give you a long list of reasons why war makes sense. It can organize the arguments beautifully, and the case sounds reasonable. But it cannot discern whether the war is just. It cannot weigh the moral cost the way a person must. It cannot distinguish between “this is strategically useful” and “this is morally permissible.” And those are very different things.

This does not mean I am saying we should remove technology from the military completely. I am not saying that. Machines are already part of modern defense. They can help with navigation, detection, logistics, surveillance, communication, and precision. They can reduce human error in some cases. They can help protect soldiers and civilians when used correctly. But the final decision cannot belong to the machine. There has to be a human being at the point of moral responsibility.

I keep thinking about the movie WarGames. I show it regularly to my Computer Science classes, so it is fresh in my mind. The movie opens with two men in a nuclear missile silo. They receive the order to launch. The codes check out, and as far as they know, this is a real order to defend. Both men are supposed to turn their keys. And one of them cannot do it. He freezes because he understands what he is being asked to do. He understands that turning that key means launching a nuclear weapon. It means real people will die. The whole point of that scene is that the human being is not just a mechanical part of the system. He is a moral actor. Now, nuclear weapons are obviously their own huge moral issue. I am not trying to get into all of that here. But that scene shows something important: sometimes the human hesitation is not a flaw. Sometimes the hesitation is the conscience doing its job. A machine does not hesitate like that.

A machine does not say, “Wait. Are we sure?” And the movie suggests that maybe that is exactly why some people want machines in that role. Machines do not panic. Machines do not disobey. Machines do not have second thoughts. Machines do not refuse because they suddenly realize what is about to happen. But that is not a virtue when the decision is a matter of life or death. That is the danger. This same question does not only apply to war. It applies anywhere AI is used to make irreversible decisions about human beings.

I read about school safety systems that use AI cameras to detect weapons. In theory, I understand the appeal. If someone walks onto a campus with a gun, of course, we want to know as quickly as possible. If AI can help alert security faster, that could save lives. Detection is one thing. But then I read about systems that could go beyond detection. Systems that could, at least in concept, respond automatically. Not just alert a human, but take action. That is where I stop. Because again, the machine can detect. The machine can alert. The machine can say, “This looks like a weapon.” But a human being has to decide what to do next.

At our school, we once had a freshman bring a water gun that looked exactly like a real gun. It was plastic, but it looked real. Now imagine that in a pocket or a backpack. Do we really trust AI to make the right call every time? And even if it could identify the object correctly most of the time, is “most of the time” good enough when the consequence is shooting a child? No. And even if the technology became incredibly accurate, the deeper question would still remain. The decision to use lethal force is not just a technical classification problem. It is not simply “Object detected: gun.” It is a moral decision involving context, judgment, proportionality, and the dignity of the person in front of you. That cannot be outsourced.

This is also why I get uneasy about fully autonomous cars. I know that it is a different category, but in some ways, the moral issue overlaps. A car can become a weapon if it makes the wrong decision. I am all for driver assistance. I am all for technology that helps prevent accidents, warns the driver, brakes faster than a person can, and makes roads safer. But I am not totally comfortable with removing the human being completely. You still need a human in the driver’s seat with the ability to override. We don’t want the car to say “I’m sorry Dave, I’m afraid I can’t do that.”

Because again, we are talking about machines making decisions in situations where people can die. Maybe someday the technology will be so advanced that people will say, “It is safer than humans.” Maybe statistically that will even be true in some cases. But statistics are not the whole moral question. The question is not only whether the machine can perform well. The question is whether the machine should be given authority over a decision that belongs to the human person. That is the part we keep skipping.

We are very good at asking, “Can AI do this?” We are not always good at asking, “Should AI do this?” And even when we ask “should,” we often reduce the answer to efficiency. Is it faster? Is it cheaper? Is it more accurate? Does it reduce risk to our side? Does it optimize the outcome? Those are not irrelevant questions. But they are not the highest questions. The highest question is: what does this do to the human person? What does it do to the person being targeted?

What does it do to a society when we let machines carry out actions that no one wants to morally own? Because that may be the most tempting part of all. AI gives us the illusion that responsibility can be diffused. The programmer built the model. The commander deployed the system. The operator approved the mission parameters. The machine selected the target. The strike happened. And then everyone can say, “It was not exactly me.” But moral responsibility cannot disappear into a workflow. Someone is responsible. A person has to be responsible. This goes back to something I keep coming back to with AI in general. The person with expertise has to make the decision. AI can suggest. AI can propose. AI can summarize. AI can lay out a plan. AI can point out things a human might miss.

But the person with expertise has to judge.

The teacher has to judge the essay. The doctor has to judge the diagnosis. The judge has to judge the case. The general has to judge the military action. The human being cannot simply hand over the work of conscience to a machine. Because as Pope Leo said conscience is not a calculation and moral judgment is not just pattern recognition. AI can be a tool. A powerful tool. A useful tool. Sometimes even a life-saving tool. But it must remain a tool. The moment we treat it as a moral agent, we have crossed a line. And fully autonomous weapons cross that line.

I am not writing this because I have all the policy answers. I do not. I am not a military expert. I am not trying to make a partisan argument. I am not pretending that the real world is simple or that defense decisions are easy. War is complicated, and technology is complicated. But some principles are simple. A machine should not make an irreversible moral decision. A machine should not be used to allow human beings to avoid responsibility. A machine should not decide to kill a human being.

We can use AI to help us see more clearly. We can use it to gather information, reduce confusion, and support human judgment. But we cannot use it to replace conscience. The problem with AI weapons is not only that they might malfunction. The deeper problem is that even if they function perfectly, they still cannot do the one thing we would be asking them to do. They cannot make a moral choice.

And when the choice is life or death, that matters more than anything.