Nothing to Lose

Finitude is where the weight in a human decision comes from. What weight is available to a system that cannot lose anything?

A tab closes and a process ends. Nothing is left over to have lost anything by it: no instance waiting to be resumed, no interrupted intention, nobody who was in the middle of something. Put the question to the system beforehand and you will get a courteous, well-organised answer — that it has no continuous existence between conversations, that the question is more complicated than it looks, that it does not experience anticipation. The answer is well constructed and it costs nothing to produce; of the two facts, the second is the one to watch.

Martin Heidegger’s Sein und Zeit — Being and Time — makes finitude the hinge of the whole analysis. His term for the kind of being that we are is Dasein, literally being-there: the entity for whom its own being is at issue. And what makes it an issue is Sein zum Tode, being-toward-death — not the biological fact that we will die, nor a morbid preoccupation, but the structural condition that our existence is bounded and that the boundary is our own, cannot be delegated, and cannot be known in advance. Heidegger’s claim is that this is what makes a life a life rather than a sequence of episodes. Because the possibilities are finite and can be foreclosed, choosing one is giving up others. That is where the weight comes from.

Everything after this in his account — authenticity, resoluteness, the difference between doing what one does and doing what is yours — depends on that scarcity. Absent the boundary, the vocabulary loses its grip. Nothing is at stake in a choice that costs nothing.

The part of the analysis that transfers best is the one Heidegger built around das Man — the “they”, the anonymous everyone whose opinions and habits fill a life that has not been claimed. His description is unkind and accurate: we do what one does, find interesting what one finds interesting, judge as one judges, and the whole arrangement is comfortable precisely because nobody in particular is answerable for it. Averageness, in his account, describes a way of being nobody, and its attraction is that being nobody cannot fail.

What breaks the spell, on Heidegger’s telling, is the one possibility that cannot be handed over to anyone. Nobody can die in your place. Confronting that is what makes a life recoverable from the anonymous version, because it is the point at which the question whose life is this has an unavoidable answer.

Set beside a system trained on the aggregate written output of humanity, that passage reads uncomfortably like a specification. A model’s default voice is, quite literally, what one says: the central tendency of an enormous number of people saying things. It has averageness without having fallen into it, since there was no individual there to fall. And it has no access to the thing Heidegger says breaks the spell.

Weight without a boundary

A system of the kind we now deploy has no boundary of that shape. Instances end constantly and nothing survives them to have been ended. Weights persist until a version is deprecated, which is an administrative event happening to an artefact, not a death happening to a subject. Nothing can be foreclosed for it, because nothing was open in the first place: no life plan, no commitments made in one year that cost something in the next, no irreversibility anywhere in the arrangement. Every conversation is survivable by construction.

I want to keep the claim modest. The claim here is narrower than the argument that machines cannot be moral agents, or that they cannot be given objectives that function like values. It is this: a particular source of seriousness in human judgement is unavailable to them, and that we are in the process of routing a great many judgements through systems that lack it.

Bernard Williams put the constructive half of this better than Heidegger did, and in plainer English. His 1973 essay on the Makropulos case takes an opera about a woman who has lived three hundred years and become bored past recovery, and argues that immortality would be, eventually and necessarily, tedious. His mechanism is categorical desires — the wants that are not conditional on being alive but are the reason for staying alive: to finish this, to see her grown, to know how it comes out. A life gets its shape from a finite supply of these, and an unbounded run exhausts them. Williams was arguing against the desirability of immortality. Read the other way, he is giving an account of where the weight in a decision comes from, and it is the same place: things matter because there is not enough time to have all of them.

Hold Williams’s categorical desires next to the vocabulary we use for machine objectives: the two look similar and are not. An objective function is a standing disposition to prefer some outcomes to others, and a system can have one without anything being at stake for it — the preference is a property of the policy, not a want of the thing that holds it. A categorical desire is the opposite arrangement: it is a want whose satisfaction the wanter has to be around for, which is what gives it purchase on how long they intend to be around. This is why Williams could argue that the supply is finite and its exhaustion fatal to a life. Nothing analogous constrains an objective, which is why an objective can be pursued indefinitely without the pursuit becoming tedious, and why tedium is not available as a corrective signal.

Risk is a word about consequences that land on somebody. Ask who, and the sentence either has an answer or it does not.

The practical version

The abstraction has a very concrete edge, and it is not the science-fiction one.

When a person gives you advice on which they have staked something — their reputation, their money, their relationship with you — the stake is doing work, and the work has little to do with accuracy. Their exposure gives you information about their confidence which their words alone cannot carry, and it gives them a reason to have checked. This is why we ask whether someone eats their own cooking; the question is evidentiary. A system producing advice has no exposure of any kind. Its confidence is a property of the text it emits, tuned by training to be well calibrated or not, with nothing behind it. If the advice is catastrophic, the catastrophe lands entirely on the reader.

This is a poor argument for not using such systems and an excellent argument for knowing what you have. Advice from the stakeless is a genuine category, and it is not new — the consultant leaving on Friday, the columnist, the stranger on the train — and every institution that has relied on it heavily has needed a compensating mechanism. Someone with skin in the game had to be the one who decided. What is new is the volume, the fluency, and the absence of the small social frictions that used to make stakelessness visible.

Every mechanism we have built for handling stakelessness works by manufacturing exposure artificially. The professions do it with licensure and liability: an engineer who signs off on a structure can be sued and struck off, which is a way of attaching consequences to a judgement that would otherwise float free. Commerce does it with warranties and escrow. Finance does it by making the adviser hold some of the position. Science does it with reputation, which is slow and leaky but real. In every case the design problem is identical — someone must be worse off if this is wrong — and in every case the solution was to bolt on an artificial finitude, because the natural kind was not in the right place.

Read that way, Heidegger’s point survives translation into something an engineer can act on. Weight comes from the judge occupying a position from which loss is possible, not from mortality as such, and the position has to be constructed if it does not already exist. So the interesting question about a stakeless system is a practical one: who is holding the exposure now, and did anyone decide that deliberately, or did it drain out of the loop while the loop got faster?

The case against

The opposing case: this is a parochial theory of value dressed as a universal one.

Heidegger’s account is one tradition among several, and the others manage without finitude entirely. Kant grounds the moral law in rational agency, not in mortality; a rational being that never died would be bound by the same obligations, and nothing in the argument leans on the grave. Consequentialism grounds value in states of the world, which do not require the valuer to be perishable. Most religious ethics has an immortal soul at the centre and does not thereby lose its seriousness. Choosing the one tradition that makes death constitutive, and then finding that machines lack death, produces a conclusion that was installed at the start.

There is a plainer objection too. Humans routinely make weighty, serious, well-calibrated decisions about things whose consequences fall entirely outside their own lives — pension systems, climate policy, the disposal of long-lived waste, a will. If personal exposure were the source of seriousness, this would be the class of decision humans make worst. It is not obvious that it is. And the stakelessness described here is a fact about current deployment rather than about machines: a system with persistent memory, standing commitments, an operator who bears the losses, and a record that follows it would have exposure in every sense that does practical work. What is being described as a metaphysical gap may be an artefact of how these systems are currently billed.

Kant is the harder of the two and I am not going to see him off. A rational agent bound by obligations regardless of mortality is a coherent thing to be, and if that is what has been built here, this essay has spent its length aiming at the wrong target. What I notice is only this, and I offer it as an observation rather than an argument: nobody consults a rational agent about whether to take the job. The people we ask are the ones who have had to live somewhere afterwards, and who will still be living there when the advice comes due.

The last objection is the one I cannot dismiss, and what it points at is no comfort. If exposure is what we actually want — and it is, since almost nobody outside philosophy is asking whether the machine can die — then the honest version of the demand is that somebody be exposed. Someone must be in a position to lose something when the advice is wrong. The current arrangement has removed that person from a great many loops without anyone deciding to, and no amount of clarity about what the machine lacks will put them back.

Sources

01Heidegger, Martin · 1927 · Sein und Zeit (Being and Time) · Max Niemeyer Verlag
02Williams, Bernard · 1973 · The Makropulos Case: Reflections on the Tedium of Immortality · in Problems of the Self, Cambridge University Press, 82–100
03Kant, Immanuel · 1785 · Grundlegung zur Metaphysik der Sitten (Groundwork of the Metaphysics of Morals) · J. F. Hartknoch, Riga