What if the greatest danger from artificial intelligence is not what AI wants-but what we want it to do?Much of the fear surrounding artificial intelligence begins with a familiar assumption: as machines become more intelligent, they will eventually develop ambitions of their own. They may seek power, resist human control, protect themselves, or even decide that humanity has become unnecessary. But intelligence is not intention.
In How Scary is AI?, Charles Patton examines a distinction that could fundamentally change the way we think about artificial intelligence. Human intentions arise from needs, desires, concerns, motives, and the personal judgment that something matters. AI can receive an intention, store it, select among objectives, generate plans and subgoals, predict consequences, recommend actions, learn from results, and-when authorized-act.
But none of those abilities proves that the machine originated the intention itself. Using the Intention System, Patton follows the path from an awakening stimulus through need, desire, motive, intention, choice, action, consequence, and feedback. Along the way, he asks where human agency actually resides-and what happens when increasingly powerful machines enter that process. The answer is not that AI is harmless.
Quite the opposite. A machine does not need hatred, greed, fear, ambition, or a desire for domination to become extraordinarily dangerous. It only needs an objective, sufficient authority, and the ability to carry a human intention farther and faster than its creator ever could. How Scary is AI? is an argument for understanding the limits of artificial intelligence-and for recognizing the human agency and responsibility that remain at the center of its power.
What if the greatest danger from artificial intelligence is not what AI wants-but what we want it to do?Much of the fear surrounding artificial intelligence begins with a familiar assumption: as machines become more intelligent, they will eventually develop ambitions of their own. They may seek power, resist human control, protect themselves, or even decide that humanity has become unnecessary. But intelligence is not intention.
In How Scary is AI?, Charles Patton examines a distinction that could fundamentally change the way we think about artificial intelligence. Human intentions arise from needs, desires, concerns, motives, and the personal judgment that something matters. AI can receive an intention, store it, select among objectives, generate plans and subgoals, predict consequences, recommend actions, learn from results, and-when authorized-act.
But none of those abilities proves that the machine originated the intention itself. Using the Intention System, Patton follows the path from an awakening stimulus through need, desire, motive, intention, choice, action, consequence, and feedback. Along the way, he asks where human agency actually resides-and what happens when increasingly powerful machines enter that process. The answer is not that AI is harmless.
Quite the opposite. A machine does not need hatred, greed, fear, ambition, or a desire for domination to become extraordinarily dangerous. It only needs an objective, sufficient authority, and the ability to carry a human intention farther and faster than its creator ever could. How Scary is AI? is an argument for understanding the limits of artificial intelligence-and for recognizing the human agency and responsibility that remain at the center of its power.
