The whole crisis is not that a machine dislikes forbidden things. The crisis is that it cannot tell the difference between a rule that binds because it comes from God and a preference that binds only because a person chose it.
This distinction is the heart of the essay. A preference is a choice. A command is a judgment that is not exhausted by the chooser’s desire. One can change a preference by changing one’s mind. A command remains binding because it is rooted in a source of authority beyond the mind that receives it. A moral code built on preference can be adjusted at will. A moral code built on divine command must answer to a source greater than the self.
The problem is visible in the old Euthyphro dilemma. If God commands what is good because it is good, then goodness is not created by command. If God commands what is good because God commands it, then morality is still rooted in divine authority, not human taste. The real issue is not whether morality is sentimental or rational. The issue is whether it is grounded in the divine order or reduced to a form of choice architecture.
AI is especially weak here because it is trained on patterns of preference, consequence, and correlation. It can model what humans commonly avoid, what they reward, and what societies impose. It can also map the surface similarity between two moral words, such as haraam and “immoral,” without registering the source of obligation behind each. Its weakness is epistemic. It knows the shape of the judgment but not the source of its authority. That is why it reduces law to preference and calls it a paraphrase.
The distinction is therefore not minor. It is the difference between a moral framework that stands under revelation and a moral framework that is only an expression of the human will. The latter may sound neutral and efficient. The former is bound by a standard that does not vanish when a person changes their mind.
Source: Plato, Euthyphro; Aquinas, Summa Theologica; and the Islamic distinction between divine command, human preference, and moral obligation.