Beware underestimating the probability of very bad outcomes: Historical examples against future optimism

It may be tempting to view history through a progressive lens that sees humanity as climbing toward ever greater moral progress and wisdom. As the famous quote popularized by Martin Luther King Jr. goes: “The arc of the moral universe is long, but it bends toward justice.”

Yet while we may hope that this is true, and do our best to increase the probability that it will be, we should also keep in mind that there are reasons to doubt this optimistic narrative. For some, the recent rise of right-wing populism is a salient reason to be less confident about humanity’s supposed path toward ever more compassionate and universal values. But it seems that we find even stronger reasons to be skeptical if we look further back in history. My aim in this post is to present a few historical examples that in my view speak against confident optimism regarding humanity’s future.


Contents

  1. Germany in year 1900
  2. Shantideva around year 700
  3. Lewis Gompertz and J. Howard Moore in the 19th century

Germany in year 1900

In 1900, Germany was far from being a paragon of moral advancement. They were a colonial power, antisemitism was widespread, and bigoted anti-Polish Germanisation policies were in effect. Yet Germany anno 1900 was nevertheless far from being like Germany anno 1939-1945, in which it was the main aggressor in the deadliest war in history and the perpetrator of the largest genocide in history.

In other words, Germany had undergone an extreme case of moral regress along various dimensions by 1942 (the year the so-called Final Solution was formulated and approved by the Nazi leadership) compared to 1900. And this development was not easy to predict in advance. Indeed, for historian of antisemitism Shulamit Volkov, a key question regarding the Holocaust is: “Why was it so hard to see the approaching disaster?”

If one had told the average German citizen in 1900 about the atrocities that their country would perpetrate four decades later, would they have believed it? What probability would they have assigned to the possibility that their country would commit atrocities on such a massive scale? I suspect it would be very low. They might not have seen more reason to expect such moral regress than we do today when we think of our future.

A lesson that we can draw from Germany’s past moral deterioration is, to paraphrase Volkov’s question, that approaching disasters can be hard to see in advance. And this lesson suggests that we should not be too confident as to whether we ourselves might currently be headed toward disasters that are difficult to see in advance.

Shantideva around year 700

Shantideva was a Buddhist monk who lived in ca. 685-763. He is best known as the author of A Guide to the Bodhisattva’s Way of Life, which is a remarkable text for its time. The core message is one of profound compassion for all sentient beings, and Shantideva not only describes such universally compassionate ideals, but he also presents stirring encouragements and cogent reasoning in favor of acting on those ideals.

That such a universally compassionate text existed at such an early time is a deeply encouraging fact in one sense. Yet in another sense, it is deeply discouraging. That is, when we think about all the suffering, wars, and atrocities that humanity has caused since Shantideva expounded these ideals — centuries upon centuries of brutal violence and torment imposed upon human and non-human beings — it seems that a certain pessimistic viewpoint gains support.

In particular, it seems that we should be pessimistic about notions along the lines of “compassionate ideals presented in a compelling way will eventually create a benevolent world”. After all, even today, 1300 years later, where we generally pride ourselves of being far more civilized and morally developed than our ancestors, we are still painfully far from observing the most basic of compassionate ideals in relation to other sentient beings.

Of course, one might think that the problem is merely that people have yet to be exposed to compassionate ideals such as those of Shantideva — or those of Mahavira or Mozi, both of whom lived more than a thousand years before Shantideva. But even if we grant that this is the main problem, it still seems that historical cases like these give us some reason to doubt whether most people ever will be exposed to such compassionate ideals, or whether most people would accept such ideals upon becoming exposed to them, let alone be willing to act on them. The fact that these memes have not caught on to a greater degree than they have, despite existing in such developed forms a long time ago, is some evidence that they are not nearly as virulent as many of us would have hoped.

Speaking for myself at least, I can say that I used to think that people just needed to be exposed to certain compassionate ideals and compassion-based arguments, and then they would change their minds and behaviors due to the sheer compelling nature of these ideals and arguments. But my experience over the years, e.g. with animal advocacy, have made me far more pessimistic about the force of such arguments. And the limited influence of highly sophisticated expositions of these ideals and arguments made many centuries ago is further evidence for that pessimism (relative to my previous expectations).

Of course, this is not to say that we can necessarily do better than to promote compassion-based ideals and arguments. It is merely to say that the best we can do might be a lot less significant — or be less likely to succeed — than what many of us had initially expected.

Lewis Gompertz and J. Howard Moore in the 19th century

Lewis Gompertz (ca. 1784-1861) and J. Howard Moore (1862-1916) both have a lot in common with Shantideva, as they likewise wrote about compassionate ethics relating to all sentient beings. (And all three of them touched on wild-animal suffering.) Yet Gompertz and Moore, along with other figures in the 19th century, wrote more explicitly about animal rights and moral vegetarianism than did Shantideva. Two observations seem noteworthy with regard to these writings.

One is that Gompertz and Moore both wrote about these topics before the rise of factory farming. That is, even though authors such as Gompertz and Moore made strong arguments against exploiting and killing other animals in the 19th century, humanity still went on to exploit and kill beings on a far greater scale than ever before in the 20th century, indeed on a scale that is still increasing today.

This may be a lesson for those who are working to reduce risks of astronomical suffering at present: even if you make convincing arguments against a moral atrocity that humanity is committing or otherwise heading toward, and even if you make these arguments at an early stage where the atrocity has yet to (fully) develop, this might still not be enough to prevent it from happening on a continuously expanding scale.

The second and closely related observation is that Gompertz and Moore both seem to have focused exclusively on animal exploitation as it existed in their own times. They did not appear to focus on preventing the problem from getting worse, even though one could argue, in hindsight, that such a strategy might have been more helpful overall.

Indeed, even though Moore’s outlook was quite pessimistic, he still seems to have been rather optimistic about the future. For instance, in the preface to his book The Universal Kinship (1906), he wrote: “The time will come when the sentiments of these pages will not be hailed by two or three, and ridiculed or ignored by the rest; they will represent Public Opinion and Law.”

Gompertz appeared similarly optimistic about the future, as he in his Moral Inquiries (1824, p. 48) wrote: “… though I cannot conceive how any person can shut his eyes to the general state of misery throughout the universe, I still think that it is for a wise purpose; that the evils of life, which could not properly be otherwise, will in the course of time be rectified …” None of them seem to have predicted that animal exploitation would be getting far worse in many ways (e.g. the horrible conditions of factory farms) or that it would increase vastly in scale.

This second observation might likewise carry lessons for animal activists and suffering reducers today. If these leading figures of 19th-century animal activism tacitly underestimated the risk that things might get far worse in the future, and as a result paid insufficient attention to such risks, could it be the case that most activists today are similarly underestimating and underprioritizing future risks of things getting even worse still? This question is at least worth pondering.

On a general and concluding note, it seems important to be aware of our tendencies to entertain wishful thinking and to be under the spell of the illusion of control. Just because a group of people have embraced some broadly compassionate values, and in turn identified ongoing atrocities and future risks based on those values, it does not mean that those people will be able to steer humanity’s future such that we avoid these atrocities and risks. The sad reality is that universally compassionate values are far from being in charge.

Reasons to include insects in animal advocacy

I have seen some people claim that animal activists should primarily be concerned with certain groups of numerous vertebrates, such as chickens and fish, whereas we should not be concerned much, if at all, with insects and other small invertebrates. (See e.g. here.) I think there are indeed good arguments in favor of emphasizing chickens and fish in animal advocacy, yet I think those same arguments tend to support a strong emphasis on helping insects as well. My aim in this post is to argue that we have compelling reasons to include insects and other small vertebrates in animal advocacy.


Contents

  1. A simplistic sequence argument: Smaller beings in increasingly large numbers
    1. The sequence
    2. Why stop at chickens or fish?
  2. Invertebrate vs. vertebrate nervous systems
    1. Phylogenetic distance
    2. Behavioral and neurological evidence
    3. Nematodes and extended sequences
  3. Objection based on appalling treatment
  4. Potential biases
    1. Inconvenience bias
    2. Smallness bias
    3. Disgust and fear reflexes
    4. Momentum/status quo bias
  5. Other reasons to focus more on small invertebrates
    1. Neglectedness
    2. Opening people’s eyes to the extent of suffering and harmful decisions
    3. Risks of spreading invertebrates to space: Beings at uniquely high risk of suffering due to human space expansion
    4. Qualifications and counter-considerations
  6. My own view on strategy in brief
  7. Final clarification: Numbers-based arguments need not assume that large amounts of mild suffering can be worse than extreme suffering
  8. Acknowledgments

A simplistic sequence argument: Smaller beings in increasingly large numbers

As a preliminary motivation for the discussion, it may be helpful to consider the sequence below.

I should first of all clarify what I am not claiming in light of the following sequence. I am not making any claims about the moral relevance of the neuron counts of individual beings or groups of beings (that is a complicated issue that defies simple answers). Nor am I claiming that we should focus mostly on helping beings such as land arthropods and nematodes. The claim I want to advance is a much weaker one, namely that, in light of the sequence below, it is hardly obvious that we should focus mostly on helping chickens or fish.

The sequence

At any given time, there are roughly:

  • 780 million farmed pigs, with an estimated average neuron count of 2.2 billion. Total neuron count: ~1.7 * 10^18.
  • 33 billion farmed chickens, with an estimated average neuron count of 200 million. Total neuron count: ~6.6 * 10^18.
  • 10^15 fish (the vast majority of whom are wild fish), with an estimated average neuron count of 1 million neurons (this number lies between the estimated neuron count of a larval zebrafish and an adult zebrafish; note that there is great uncertainty in all these estimates). Total neuron count: ~10^21. It is estimated that humanity kills more than a trillion fish a year, and if we assume that they likewise have an average neuron count of around 1 million neurons, the total neuron count of these beings is ~10^18.
  • 10^19 land arthropods, with an estimated average neuron count of 15,000 neurons (some insects have brains with more than a million neurons, but most arthropods appear to have considerably fewer neurons). Total neuron count: ~1.5*10^23. If humanity kills roughly the same proportion of land arthropods as the proportion of fish that we kill (e.g. through insecticides and insect farming), then the total neuron count of the land arthropods we kill is ~10^20.
  • 10^21 nematodes, with an estimated average neuron count of 300 neurons. Total neuron count: ~3 * 10^23.

Why stop at chickens or fish?

The main argument that supports a strong emphasis on chickens or fish is presumably their large numbers (as well as their poor treatment, which I discuss below). Yet the numbers-based argument that supports a strong emphasis on chickens and fish could potentially also support a strong emphasis on small invertebrates such as insects. It is thus not clear why we should place a strict boundary right below chickens or fish beyond which this numbers-based argument no longer applies. After all, each step of this sequence entails a similar pattern in terms of crude numbers: we have individual beings who on average have 1-3 orders of magnitude fewer neurons yet who are 1-5 orders of magnitude more numerous than the beings in the previous step.

Invertebrate vs. vertebrate nervous systems

A defense that one could give in favor of placing a relatively strict boundary below fish is that we here go from vertebrates to invertebrates, and we can be significantly less sure that invertebrates suffer compared to vertebrates.

Perhaps this defense has some force. But how much? Our confidence that the beings in this sequence have the capacity to suffer should arguably decrease at least somewhat in each successive step, yet should the decrease in confidence from fish to insects really be that much bigger than in the previous steps?

Phylogenetic distance

Based on the knowledge that we ourselves can suffer, one might think that a group of beings’ phylogenetic distance from us (i.e. how distantly related they are to us) can provide a tentative prior as to whether those beings can suffer, and regarding how big a jump in confidence we should make for different kinds of beings. Yet phylogenetic distance per se arguably does not support a substantially greater decrease in confidence in the step from fish to insects compared to the previous steps in the sequence above. 

The last common ancestor of humans and insects appears to have lived around 575 million years ago, whereas the last common ancestor of humans and fish lived around 400-485 million years ago (depending on the species of fish; around 420-460 million years for the most numerous fish). By comparison, the last common ancestor of humans and chickens lived around 300 million years ago, while the last common ancestor of humans and pigs lived around 100-125 million years ago.

Thus, when we look at different beings’ phylogenetic distance from humans in these temporal terms, it does not seem that the step between fish and insects (in the sequence above) is much larger than the step between fish and chickens or between chickens and pigs. In each case, the increase in the “distance” appears to be something like 100-200 million years.

Behavioral and neurological evidence

Of course, “phylogenetic distance from humans” does not represent strong evidence as to whether a group of beings has the capacity to suffer. After all, humans are more closely related to starfish (~100 neurons) than to octopuses (~500 million neurons), and we have much stronger reasons to think that the latter can suffer, based on behavioral and neurological evidence (cf. the Cambridge Declaration on Consciousness).

Does such behavioral and neurological evidence support a uniquely sharp drop in confidence regarding insect sentience compared to fish sentience? Arguably not, as there is mounting evidence of pain in (small) invertebrates, both in terms of behavioral and neuroscientific evidence. Additionally, there are various commonalities in the respective structures and developments of arthropod and vertebrate brains.

In light of this evidence, it seems that a sharp drop in confidence regarding pain in insects (versus pain in fish) requires a justification.

Nematodes and extended sequences

I believe that a stronger decrease in confidence is warranted when comparing arthropods and nematodes, for a variety of reasons: the nematode nervous system consists primarily of a so-called nerve ring, which is quite distinct from the brains of arthropods, and unlike the neurons of arthropods (and other animals), nematode neurons do not have action potentials or orthologs of sodium-channels (e.g. Nav1 and Nav2), which appear to play critical roles to pain signaling in other animals.

However, the evidence of pain in nematodes should not be understated either. The probability of pain in nematodes still seems non-negligible, and it arguably justifies substantial concern for (the risk of) nematode pain, even if it does not overall warrant a similarly strong concern and priority as does the suffering of chickens, fish, and arthropods.

This discussion also hints at why the sequence argument above need not imply that we should primarily focus on risks of suffering in bacteria or atoms, as one may reasonably hold that the probability of such suffering decreases by a greater rate than the number of the purported sufferers increases in such extended sequences.

Objection based on appalling treatment

Another reason one could give in favor of focusing on chickens and fish is that they are treated in particularly appalling ways, e.g. they are often crammed in extremely small spaces and killed in horrific ways. I agree that humanity’s abhorrent treatment of chickens and fish is a strong additional reason to prioritize helping them. Yet it seems that this same argument also favors a focus on insects.

After all, humanity poisons vast numbers of insects with insecticides that may cause intensely painful deaths, and in various insect farming practices — which are sadly growing — insects are commonly boiled, fried, or roasted alive. These practices seem no less cruel and appalling than the ways in which we treat and kill chickens and fish.

Potential biases

There are many reasons to expect that we are biased against giving adequate moral consideration to small invertebrates such as insects (in addition to our general speciesist bias). The four plausible biases listed below are by no means exhaustive.

Inconvenience bias

It is highly inconvenient if insects can feel pain, as it would imply that 1) we should be concerned about far more beings, which greatly complicates our ethical and strategic considerations (compared to if we just focused on vertebrates); 2) the extent of pain and suffering in the world is far greater than we would otherwise have thought, which may be a painful conclusion to accept; and 3) we should take far greater care not to harm insects in our everyday lives. All these inconveniences likely motivate us to conclude that insects are not sentient or that they are not that important in the bigger picture.

Smallness bias

Insects tend to be rather small, even compared to fish, which might make us reluctant to grant them moral consideration. In other words, our intuitions plausibly display a general sizeist bias. As a case in point, ants have more than twice as many neurons as lobsters, and there does not seem to be any clear reason to think that ants are less able to feel pain than are lobsters. Yet ants are obviously much smaller than lobsters, which may explain why people seem to show considerably more concern for lobsters than for ants, and why the number of people who believe that lobsters can feel pain (more than 80 percent in a UK survey) is significantly larger than the number of people who believe that ants can feel pain (around 56 percent). Of course, this pattern may also be partially explained by the inconvenience bias, since the acceptance of pain in lobsters seems less inconvenient than does the acceptance of pain in ants; but size likely still plays a significant role. (See also Vinding, 2015, “A Short Note on Insects”.)

Disgust and fear reflexes

It seems that many people have strong disgust reactions to (at least many) small invertebrates, such as cockroaches, maggots, and spiders. Some people may also feel fear toward these animals, or at least feel that they are nuisance. Gut reactions of this kind may well influence our moral evaluations of small invertebrates in general, even though they ideally should not.

Momentum/status quo bias

The animal movement has not historically focused on invertebrates, and hence there is little momentum in favor of focusing on their plight. That is, our status quo bias seems to favor a focus on helping the vertebrates whom the animal movement have traditionally focused on. To be sure, status quo bias also works against concern for fish and chickens to some degree (which is worth controlling for as well), yet chickens and fish have still received considerably more focus from the animal movement, and hence status quo bias likely negates concern for insects to an even stronger extent.

These biases should give us pause when we are tempted to reflexively dismiss the suffering of small invertebrates.

Other reasons to focus more on small invertebrates

In addition to the large number of arthropods and the evidence for arthropod pain, what other reasons might support a greater focus on small invertebrates?

Neglectedness

An obvious reason is the neglect of these beings. As hinted in the previous section, a focus on helping small invertebrates has little historical momentum, and it is still extremely neglected in the broader animal movement today. This seems to me a fairly strong reason to focus more on invertebrates on the margin, or at the very least to firmly include invertebrates in one’s advocacy.

Opening people’s eyes to the extent of suffering and harmful decisions

Another, perhaps less obvious reason is that concern for smaller beings such as insects might help reduce risks of astronomical suffering. This claim should immediately raise some concerns about suspicious convergence, and as I have argued elsewhere, there is indeed a real risk that expanding the moral circle could increase rather than reduce future suffering. Partly for this reason, it might be better to promote a deeper concern for suffering than to promote wider moral circles (see also Vinding, 2020, ch. 12).

Yet that being said, I also think there is a sense in which wider moral circles can help promote a deeper concern for suffering, and not least give people a more realistic picture of the extent of suffering in the world. Simply put, a moral outlook that includes other vertebrates besides humans will see far more severe suffering and struggle in the world, and a perspective that also includes invertebrates will see even more suffering still. Indeed, not only does such an outlook open one’s eyes to more existing suffering, but it may also open one’s eyes (more fully) to humanity’s capacity to ignore suffering and to make decisions that actively increase it, even today.

Risks of spreading invertebrates to space: Beings at uniquely high risk of suffering due to human space expansion

Another way in which greater concern for invertebrate suffering might reduce risks of astronomical suffering is that small invertebrates seem to be among the animals who are most likely to be sent into space on a large scale in the future (e.g. because they may survive better in extreme environments). Indeed some invertebrates — including fruit flies, crickets, and wasps — have already been sent into space, and some tardigrades were even sent to the moon (though the spacecraft crashed and probably none survived). Hence, the risk of spreading animals to space plausibly gives us additional reason to include insects in animal advocacy.

Qualifications and counter-considerations

To be clear, the considerations reviewed above merely push toward increasing the emphasis that we place on small beings such as insects — they are not necessarily decisive reasons to give primary focus to those beings. In particular, these arguments do not make a case for focusing on helping insects over, say, new kinds of beings who might be created in the future in even larger numbers.

It is also worth noting that there may be countervailing reasons not to emphasize insects more. One is that it could risk turning people away from the plight of non-human animals and the horror of suffering, which many people might find difficult to relate to if insect suffering constitutes the main focus at a practical level. This may be a reason to favor a greater focus on the suffering of larger and (for most people) more relatable animals.

I think the considerations on both sides need to be taken into account, including considerations about future beings who may become even more numerous and more neglected than insects. The upshot, to my mind, is that while focusing primarily on helping insects is probably not the best way to reduce suffering (for most of us), it still seems likely that 1) promoting greater concern for insects, as well as 2) promoting concrete policies that help insects, both constitute a significant part of the optimal portfolio of aims to push for.

My own view on strategy in brief

While questions about which beings seem most worth helping (on the margin) can be highly relevant for many of our decisions, there are also many strategic decisions that do not depend critically on how we answer these questions.

Indeed, my own view on strategies for reducing animal suffering is that we generally do best by pursuing robust and broad strategies that help many beings simultaneously, without focusing too narrowly on any single group of beings. (Though as hinted above, I think there are many situations where it makes sense to focus on interventions that help specific groups of beings.)

This is one of the reasons why I tend to favor an antispeciesist approach to animal advocacy, with a particular emphasis on the importance of suffering. Such an approach is still compatible with highlighting the scale and neglectedness of the suffering of chickens, fish, and insects, as well as the scale and neglectedness of wild-animal suffering. That is, a general approach thoroughly “scope-informed” about the realities on the ground.

And such a comprehensive approach seems further supported when we consider risks of astronomical suffering (despite the potential drawbacks alluded to earlier). In particular, when trying to help other animals today, it is worth asking how our efforts might be able to help future beings as well, since failing to do so could be a lost opportunity to spare large numbers of beings from suffering. (For elaboration, see “How the animal movement could do even more good” and Vinding, 2022, sec. 10.8-10.9.)

Final clarification: Numbers-based arguments need not assume that large amounts of mild suffering can be worse than extreme suffering

An objection against numbers-based arguments for focusing more on insects is that small pains, or a high probability of small pains, cannot be aggregated to be worse than extreme suffering.

I agree with the view that small pains do not add up to be worse than extreme suffering, yet I think it is mistaken to think that this view undermines any numbers-based argument for emphasizing insects more in animal advocacy. The reason, in short, is that we should also assign some non-negligible probability to the possibility that insects experience extreme suffering (e.g. in light of the evidence for pain in insects cited above). And this probability, combined with the very large number of insects, implies that there are many instances of extreme suffering occurring among insects in expectation. After all, the vast number of insects should lead us to believe that there are many beings who have experiences at the (expected) tail-end of the very worst experiences that insects can have.

As a concluding thought experiment that may challenge comfortable notions regarding the impossibility of intense pain among insects, consider that you were given the choice between A) living as a chicken inside a tiny battery cage for a full day, or B) being continually born and reborn as an insect who has the experience of being burned or crushed alive, for a total of a million days (for concreteness, you may imagine that you will be reborn as a butterfly like the one pictured at the top of this post).

If we were really given this choice, I doubt that we would consider it an easy choice in favor of B. I doubt that we would dismiss the seriousness of the worst insect suffering.

Acknowledgments

For their helpful comments, I am grateful to Tobias Baumann, Simon Knutsson, and Winston Oswald-Drummond.

Far From Omelas

The following is a slightly edited excerpt from my book Effective Altruism: How Can We Best Help Others? (2018/2022).


I should like to re-emphasize a tragic fact that is all too easily forgotten by our wishful and optimistic minds, that fact being that the world we inhabit is hopelessly far from Omelas. For our world is unfortunately nothing like a near-paradisiacal city predicated on the misery of a single child. Rather, in our world, there are millions of starving children, and millions of children who die from such starvation or otherwise readily preventable causes, every single year. And none of this misery serves to support a paradise or anything close to it.

We do not live in a world where a starving child confined to a basement is anywhere near the worst forms of suffering that exist. Sadly, our world contains an incomprehensibly larger number of horrors of incomprehensibly greater severity, forms of suffering that make the sufferer wish dearly for a fate as “lucky” as that of the unfortunate child in Omelas. This is, of course, true even if we only consider the human realm, yet it is even more true if we also, as we must, consider the realm of non-human individuals.

Humanity subjects billions of land-living beings to conditions similar to those of the child in Omelas, and we inflict extreme suffering upon a significant fraction of them, by castrating them without anesthetics, boiling them alive, suffocating them, grinding them alive, etc. And our sins toward aquatic animals are greater still, as we kill them in far greater numbers, trillions on some estimates; and most tragically, these deaths probably involve extreme suffering more often than not, as we slowly drag these beings out of the deep, suffocate them, and cut off their heads without stunning or mercy. And yet even this horror story of unfathomable scale still falls hopelessly short of capturing the true extent of suffering in the world, as the suffering created by humanity only represents a fraction of the totality of suffering on the planet. The vast majority of this suffering is found in the wild, where non-human animals suffer and die from starvation, parasitism, and disease, not to mention being eaten alive, which is a source of extreme suffering for countless beings on the planet every single second.

Sadly, our condition is very far from Omelas, implying that if one would choose to walk away from Omelas, it seems impossible to defend supporting the spread of our condition, or anything remotely like it, beyond Earth. The extent of suffering in the world is immense and overshadowing, and our future priorities should reflect this reality.

Why I don’t prioritize consciousness research

For altruists trying to reduce suffering, there is much to be said in favor of gaining a better understanding of consciousness. Not only may it lead to therapies that can mitigate suffering in the near term, but it may also help us in our large-scale prioritization efforts. For instance, clarifying which beings can feel pain is important for determining which causes and interventions we should be working on to best reduce suffering.

These points notwithstanding, my own view is that advancing consciousness research is not among the best uses of marginal resources for those seeking to reduce suffering. My aim in this post is to briefly explain why I hold this view.


Contents

  1. Reason I: Scientific progress seems less contingent than other important endeavors
  2. Reason II: Consciousness research seems less neglected than other important endeavors
    1. Objection: The best consciousness research is also neglected
  3. Reason III: Prioritizing the fundamental bottleneck — the willingness problem
  4. Reason IV: A better understanding of consciousness might enable deliberate harm
    1. Objection: Consciousness research is the best way to address these problems
    2. Objection: We should be optimistic about solving these problems
  5. Acknowledgments

Reason I: Scientific progress seems less contingent than other important endeavors

Scientific discoveries generally seem quite convergent, so much so that the same discovery is often made independently at roughly the same time (cf. examples of “multiple discovery”). This is not so surprising: if we are trying to uncover an underlying truth — as per the standard story of science — we should expect our truth-seeking efforts to eventually converge upon the best explanation, provided that our hypotheses can be tested.

This is not to say that there is no contingency whatsoever in science, which there surely is — after all, the same discovery can be formalized in quite different ways (famous examples include the competing calculus notations of Newton and Leibniz, as well as distinct yet roughly equivalent formalisms of quantum mechanics). But the level of contingency in science still seems considerably lower than the level of contingency found in other domains, such as when it comes to which values people hold or what political frameworks they embrace.

To be clear, it is not that values and political frameworks are purely contingent either, as there is no doubt some level of convergence in these respects as well. Yet the convergence still seems significantly lower (and the contingency higher). For example, compare two of the most important events in the early 20th century in these respective domains: the formulation of the general theory of relativity (1915) and the communist revolution in Russia (roughly 1917-1922). While the formulation of the theory of general relativity did involve some contingency, particularly in terms of who and when, it seems extremely likely that the same theory would eventually have been formulated anyway (after all, many of Einstein’s other discoveries were made independently, roughly at the same time).

In comparison, the outcome of the Russian Revolution appears to have been far more contingent, and it seems that greater foreign intervention (as well as other factors) could easily have altered the outcome of the Russian Civil War, and thereby changed the course of history quite substantially.

This greater contingency of values and political systems compared to that of scientific progress suggests that we can generally make a greater counterfactual difference by focusing on the former, other things being equal.

Reason II: Consciousness research seems less neglected than other important endeavors

Besides contingency, it seems that there is a strong neglectedness case in favor of prioritizing the promotion of better values and political frameworks over the advancement of consciousness research.

After all, there are already many academic research centers that focus on consciousness research. By contrast, there is not a single academic research center that focuses primarily on the impartial reduction of suffering (e.g. at the level of values and political frameworks). To be sure, there is a lot of academic work that is relevant to the reduction of suffering, yet only a tiny fraction of this work adopts a comprehensive perspective that includes the suffering of all sentient beings across all time; and virtually none of it seeks to clarify optimal priorities relative to that perspective. Such impartial work seems extremely rare.

This difference in neglectedness likewise suggests that it is more effective to promote values and political frameworks that aim to reduce the suffering of all sentient beings — as well as to advance our macrostrategic insights into effective suffering reduction — than to push for a better scientific understanding of consciousness.

Objection: The best consciousness research is also neglected

One might object that certain promising approaches to consciousness research (that we could support) are also extremely neglected, even if the larger field of consciousness research is not. Yet granting that this is true, I still think work on values and political frameworks (of the kind alluded to above) will be more neglected overall, considering the greater convergence of science compared to values and politics.

That is, the point above regarding scientific convergence suggests that uniquely promising approaches to understanding consciousness are likely to be discovered eventually. Or at least it suggests that these approaches will be significantly less neglected than will efforts to promote values and political systems centered on effective suffering reduction for all sentient beings.

Reason III: Prioritizing the fundamental bottleneck — the willingness problem

Perhaps the greatest bottleneck to effective suffering reduction is humanity’s lack of willingness to this end. While most people may embrace ideals that give significant weight to the reduction of suffering in theory, the reality is that most of us tend to give relatively little priority to the reduction of suffering in terms of our revealed preferences and our willingness to pay for the avoidance of suffering (e.g. in our consumption choices).

In particular, there are various reasons to think that our (un)willingness to reduce suffering is a bigger bottleneck than is our (lack of) understanding of consciousness. For example, if we look at what are arguably the two biggest sources of suffering in the world today — factory farming and wild-animal suffering — it seems that the main bottleneck to human progress on both of these problems is a lack of willingness to reduce suffering, whereas a greater knowledge of consciousness does not appear to be a key bottleneck. After all, most people in the US already report that they believe many insects to be sentient, and a majority likewise agree that farmed animals have roughly the same ability to experience pain as humans. Beliefs about animal sentience per se thus do not appear to be a main bottleneck, as opposed to speciesist attitudes and institutions that disregard non-human suffering.

In general, it seems to me that the willingness problem is best tackled by direct attempts to address it, such as by promoting greater concern for suffering, by reducing the gap between our noble ideals and our often less than noble behavior, and by advancing institutions that reflect impartial concern for suffering to a greater extent. While a better understanding of consciousness may be helpful with respect to the willingness problem, it still seems unlikely to me that consciousness research is among the very best ways to address it. 

Reason IV: A better understanding of consciousness might enable deliberate harm

A final reason to prioritize other pursuits over consciousness research is that a better understanding of consciousness comes with significant risks. That is, while a better understanding of consciousness would allow benevolent agents to reduce suffering, it may likewise allow malevolent agents to increase suffering.

This risk is yet another reason why it seems safer and more beneficial to focus directly on the willingness problem and the related problem of keeping malevolent agents out of power — problems that we have by no means found solutions to, and which we are not guaranteed to find solutions to in the future. Indeed, given how serious these problems are, and how little control we have in regard to risks of malevolent individuals in power — especially in autocratic states — it is worth being cautious about developing tools and insights that can potentially increase humanity’s ability to cause harm.

Objection: Consciousness research is the best way to address these problems

One might argue that consciousness research is ultimately the best way to address both the willingness problem and the risk of malevolent agents in power, or that it is the best way to solve at least one of those problems. Yet this seems doubtful to me, and like somewhat of a suspicious convergence. Given the vast range of possible interventions we could pursue to address these problems, we should be a priori skeptical of any intervention that we may propose as the best one, particularly when the path to impact is highly indirect.

Objection: We should be optimistic about solving these problems

Another argument in favor of consciousness research might be that we have reason to be optimistic about solving both the willingness problem and the malevolence problem, since the nature of selection pressure is about to change. Thanks to modern technological tools, benevolent agents will soon be able to design the world with greater foresight. We will deliberately choose genes and institutions to ensure that benevolence becomes realized to an ever greater extent, and in effect practically solve both the willingness problem and the malevolence problem.

But this argument seems to overlook two things. First, there is no guarantee that most humans will make actively benevolent choices, even if their choices will not be outright malevolent either. Most people may continue to optimize for things other than impartial benevolence, such as personal status and prestige, and they may continue to show relatively little concern for non-human beings.

Second, and perhaps more worryingly, modern technologies that enable intelligent foresight and deliberation for benevolent agents could be just as empowering for malevolent agents. The arms race between cooperators and exploiters is an ancient one, and I think we have strong reasons to doubt that this arms race will disappear in the next few decades or centuries. On the contrary, I believe we have good grounds to expect this arms race to get intensified, which to my mind is all the more reason to focus directly on reducing the risks posed by malevolent agents, and to promote norms and institutions that favor cooperation. And again, I am skeptical that consciousness research is among the best ways to achieve these aims, even if it might be beneficial overall.

Acknowledgments

For their comments, I thank Tobias Baumann, Winston Oswald-Drummond, and Jacob Shwartz-Lucas.

The dismal dismissal of suffering-focused views

Ethical views that give a foremost priority to the reduction of suffering are often dismissed out of hand. More than that, it is quite common to see such views discussed in highly uncharitable ways, and to even see them described with pejorative terms.

My aim in this post is to call attention to this phenomenon, as I believe it can distort public discourse and individual thinking about the issue. That is, if certain influential people consistently dismiss certain views without proper argumentation, and in some cases even use disparaging terms to describe such views, then this is likely to bias people’s evaluations of these views. After all, most people will likely feel some social pressure not to endorse views that their intellectual peers call “crazy” or “monstrously toxic”. (See also what Simon Knutsson writes about social mechanisms that may suppress talk about, and endorsements of, suffering-focused views.)

Many of the examples I present below are not necessarily that significant on their own, but I think the general pattern that I describe is quite problematic. Some of the examples involve derogatory descriptions, while others involve strawman arguments and uncharitable rejections of suffering-focused views that fail to engage with the most basic arguments in favor of such views.

My overall recommendation is simply to meet suffering-focused views with charitable arguments rather than with strawman argumentation or insults — i.e. to live up to the standards that are commonly accepted in other realms of intellectual discourse.


Contents

  1. “Crazy” and “transparently silly” views
  2. Lazari-Radek and Singer’s cursory rejection
  3. “Arguably too nihilistic and divorced from humane values to be worth taking seriously”
  4. “Anti-natalism is neurotic self-hatred”
  5. More examples
  6. Conclusion

“Crazy” and “transparently silly” views

In his essay “Why I’m Not a Negative Utilitarian” (2013), Toby Ord writes that “you would have to be crazy” to choose a world with beings who experience unproblematic states over a world with beings who experience pure happiness (negative utilitarianism would be indifferent between the two, and according to some versions of negative utilitarianism, unproblematic mental states and pure happiness are the same thing).

Ord also writes that the view that happiness does not contribute to a person’s wellbeing independently of its effects on reducing problematic states is a “crazy view”, without engaging with any of the arguments that have been made in favor of the class of views that he is thereby dismissing — i.e. views according to which wellbeing consists in the absence of problematic states or frustrated desires (see e.g. Schopenhauer, 18191851; Fehige, 1998; O’Keefe, 2009, ch. 12).

These may not seem like particularly problematic claims, yet I believe that Ord would consider it poor form if similar claims were made about his preferred view — for example, if someone claimed that “you would have to be crazy to choose to create arbitrarily large amounts of extreme suffering in order to create a ‘sufficient’ amount of pleasure” (cf. the Very Repugnant Conclusion; Creating Hell to Please the Blissful; and Intense Bliss with Hellish Cessation). 

Similarly, Rob Bensinger writes that negative utilitarianism is “transparently false/silly”. Bensinger provides a brief justification for his claim that I myself and others find unconvincing, and it is in any case not a justification that warrants calling negative utilitarianism “transparently false/silly”.

Lazari-Radek and Singer’s cursory rejection

In their book The Point of View of the Universe, Lazari-Radek and Singer seek to defend the classical utilitarian view of Henry Sigdwick. It would be natural, in this context, to provide an elaborate discussion of the moral symmetry between happiness and suffering that is entailed by classical utilitarianism — after all, such a symmetry has been rejected by various philosophers in a variety of ways, and it is arguably one of the most controversial features of classical utilitarianism (cf. Mayerfeld, 1996, p. 335).

Yet Lazari-Radek and Singer barely broach the issue at all. The only thing that comes close is a single page worth of commentary on the views of David Benatar, which unfortunately amounts to a misrepresentation of Benatar’s views. Lazari-Radek and Singer claim that Benatar argues that “to have a desire for something is to be in a negative state” (p. 362). To my knowledge, this is not a claim that Benatar defends, and the claim is at any rate not critical to the main procreative asymmetry that he argues for (Benatar, 2006, ch. 2).

Lazari-Radek and Singer briefly rebut the claim about desires that they (I suspect wrongly) attribute to Benatar, by which they fail to address Benatar’s core views in any meaningful way. They then proceed to write the following, which as far as I can tell is the closest they get to a defense of a moral symmetry between happiness and suffering in their entire book: “for people who are able to satisfy the basic necessities of life and who are not suffering from depression or chronic pain, life can reasonably be judged positively” (pp. 362-362).

This is, of course, not much of a defense of a moral symmetry. First of all, no arguments are provided in defense of the claim that such lives “can reasonably be judged positively” (a claim that one can reasonably dispute). Second, even if we grant that certain lives “can be judged positively” (in terms of the intrinsic value of their contents), it still does not follow that such lives that are “judged positively” can also morally outweigh the most horrific lives (cf. what I have elsewhere called “The Big Neglected Question”). This is an all-important question for the classical utilitarian to address, and yet Lazari-Radek and Singer proceed as though their claim that “life can reasonably be judged positively” also applies to the world as a whole, even when we factor in all of its most horrific lives. Put briefly, Lazari-Radek and Singer’s cursory rejection of asymmetric and suffering-focused views is highly unsatisfactory.

(In a vein similar to the dismissive remarks covered in the previous section, Lazari-Radek and Singer also later write that “any sane person will agree” that a scenario in which 100 percent of humanity dies is worse than a scenario in which 99 percent of humanity dies, cf. p. 375. Regardless of the plausibility of that claim — which one might agree with even from a purely suffering-focused perspective — it is bad form to imply that people are not sane if they disagree with it, not least since the latter scenario could well involve far more suffering overall. Likewise, in a response to a question on Reddit, Singer dismisses negative utilitarianism as “hopeless” without providing any reasons as to why.)

“Arguably too nihilistic and divorced from humane values to be worth taking seriously”

The website utilitarianism.net is co-authored by William MacAskill, Richard Yetter Chappell, and Darius Meissner. The aim of the website is to provide “a textbook introduction to utilitarianism at the undergraduate level”, and it is endorsed by Peter Singer (among others), who blurbs it as “the place to go for clear, full and fair accounts of what utilitarianism is, the arguments for it, the main objections to it, special issues like population ethics, and what living as a utilitarian involves.”

Yet the discussion found on the website is sorely lacking when it comes to fundamental questions and objections concerning the relative importance of suffering versus happiness. In particular, like Lazari-Radek and Singer’s Point of View of the Universe, the website contains no discussion of the moral symmetry between suffering and happiness that is entailed by classical utilitarianism, despite it being among the most disputed features of that view (see e.g. Popper, 1945; Mayerfeld, 19961999; Wolf, 199619972004; O’Keefe, 2009; Knutsson, 2016; Mathison, 2018; Vinding, 2020).

Similarly, the discussion of population ethics found on the website is extremely one-sided and uncharitable in its discussion of suffering-focused and asymmetric views in population ethics, especially for a text that is supposed to serve as an introductory textbook.

For instance, they write the following in a critique of the Asymmetry in population ethics (the Asymmetry is roughly the idea that it is bad to bring miserable lives into the world but not good to bring happy lives into the world):

But this brings us to a deeper problem with the procreative asymmetry, which is that it has trouble accounting for the idea that we should be positively glad that the world (with all its worthwhile lives) exists

There is much to take issue with in this sentence. First, it presents the idea that “we should be positively glad that the world exists” as though it is an obvious and supremely plausible idea; yet it is by no means obvious, and it has been questioned by many philosophers. A truly “full and fair” introductory textbook would have included references to such counter-perspectives. Indeed, the authors of utilitarianism.net call it a “perverse conclusion” that an empty world would be better than a populated one, without mentioning any of the sources that have defended that “perverse conclusion”, and without engaging with the arguments that have been made in its favor (e.g. Schopenhauer, 18191851; Benatar, 19972006; Fehige, 1998; Breyer, 2015; Gloor, 2017; St. Jules, 2019; Frick, 2020; Ajantaival, 2021/2022). Again, this falls short of what one would expect from a “full and fair” introductory textbook.

Second, the quote above may be critiqued for bringing in confounding intuitions, such as intuitions about the value of the world as a whole, which is in many ways a different issue from the question of whether it can be good to add new beings to the world for the sake of these beings themselves.

Third, the notion of “worthwhile lives” is not necessarily inconsistent with a procreative asymmetry, since lives may be deemed worthwhile in the sense that their continuation is good even if their creation is not (cf. Benatar, 19972006; Fehige, 1998; St. Jules, 2019; Frick, 2020). Additionally, one can think that a life is worthwhile — both in terms of its continuation and creation — because it has beneficial effects for others, even if it can never be better for the created individual themself that they come into existence.

The authors go on to write:

when thinking about what makes some possible universe good, the most obvious answer is that it contains a predominance of awesome, flourishing lives. How could that not be better than a barren rock? Any view that denies this verdict is arguably too nihilistic and divorced from humane values to be worth taking seriously.

This quote effectively dismisses all of the views cited above — the views of Schopenhauer, Fehige, Benatar, and Frick, as well as the Nirodha View in the Pali Buddhist tradition — in one fell swoop by claiming that they are “arguably too nihilistic and divorced from humane values to be worth taking seriously”. That is, to put it briefly, a lazy treatment that again falls short of the minimal standards of a fair introductory textbook.

After all, classical utilitarians would probably also object if a textbook introduction were to effectively dismiss classical utilitarianism (and similar views) with the one-line claim that “views that allow the creation of lives full of extreme suffering in order to create pleasure for others are arguably too divorced from humane values to be worth taking seriously.” Yet the dismissal is just as unhelpful and uncharitable when made in the other direction. 

Finally, the authors also omit any mention of the Very Repugnant Conclusion, although one of the co-authors, William MacAskill, has stated that he considers it the strongest objection against his favored version of utilitarianism. It is arguably bad form to omit any discussion — or even a mention — of what one considers the strongest objection against one’s favored view, especially if one is trying to write a fair and balanced introductory textbook that features that view prominently.

“Anti-natalism is neurotic self-hatred”

Psychologist Geoffrey Miller has given several talks about effective altruism, including one at EA Global, and he has also taught a full university course on the psychology of effective altruism. At the time of writing, Miller has more than 120,000 followers on Twitter, which makes him one of the most widely followed people associated with effective altruism, with more followers than Peter Singer.

Having such a large audience arguably raises one’s responsibility to communicate in an intellectually honest and charitable manner. Yet Miller has repeatedly misrepresented the views of David Benatar and written highly uncharitable statements about antinatalism and negative utilitarianism, without seriously engaging with the arguments made in favor of these views.

For example, Miller has written on Twitter that “anti-natalism is neurotic self-hatred”, and he has on several occasions falsely implied that David Benatar is a negative utilitarian, such as when he writes that “[Benatar’s] negative utilitarianism assumes that only suffering counts, & pleasure can never offset it”; or when he writes that “Benatar’s view boils down to the claim that all the joy, beauty, & love in the world can’t offset even a drop of suffering in any organism anywhere. It’s a monstrously toxic & nihilistic philosophy.”

Yet the views that Miller attributes to Benatar are not views that Benatar in fact defends, and anyone familiar with Benatar’s position knows that he does not think that “only suffering counts” (cf. his rejection of the Epicurean view of death, Benatar, 2006, ch. 7).

Miller also betrays a failure to understand Benatar’s view when he writes:

The asymmetry thesis is empirically false for humans. Almost all people report net positive subjective well-being in hundreds of studies around the world. Benatar is basically patronizing everyone, saying ‘All you guys are wrong; you’re actually miserable’.

First, Benatar discusses various reasons as to why self-assessments of one’s quality of life may be unreliable (Benatar, 2006, pp. 64-69; see also Vinding, 2018). This is not fundamentally different from, say, evolutionary psychologists who argue that people’s self-reported motives may be wrong. Second, and more importantly, the main asymmetry that Benatar defends is not an empirical one, but rather an evaluative asymmetry between the presence and absence of goods versus the presence and absence of bads (Benatar, 2006, ch. 2). This evaluative asymmetry is not addressed by Miller’s claim above.

One might object that Miller’s statements have all been made on Twitter, and that tweets should generally be held to a lower standard than other forms of writing. Yet even if we grant that tweets should be held to a lower standard, we should still be clear that Miller blatantly misrepresents Benatar’s views, which is bad form on any platform and by any standard.

Moreover, one could argue that tweets should in some sense be held to a higher standard, since tweets are likely to be seen by more people compared to many other forms of writing (such as the average journal article), and perhaps also by readers who are less inclined to verify scholarly claims made by a university professor compared to readers of other media.

More examples

Additional examples of uncharitable dismissals of suffering-focused views include statements from:

  • Writer and EA Global speaker Riva-Melissa Tez, who wrote that “anti-natalism and negative utilitarianism is true ‘hate speech’”.
  • YouTuber Robert Miles (>100k subscribers), who wrote: “Looks like it’s time for another round of ‘Principled Negative Utilitarianism or Undiagnosed Major Depressive Disorder?’” (See also here.)
  • Daniel Faggella, who wrote: “If I didn’t know so many negative utilitarians who I liked as people, I’d call it a position of literal cowardice – even vice.” (The original post was even stronger in its tone: “If I didn’t know and respect so many negative utilitarians, I would openly call it a vice, and a position of childish, seething cowardice.”)
    • I find the remark about cowardice to be quite strange, as it seems to me that it takes a lot of courage to face up to the horror of suffering, and to set out to alleviate suffering with determination. And socially, too, it can take a lot of courage to embrace strongly suffering-focused views in a social environment that often ridicules such views, and which often insinuates that there is something wrong with the adherents of these views.
  • R. N. Smart, who wrote that negative utilitarianism allows “certain absurd and even wicked moral judgments”, without providing any arguments as to whether competing moral views imply less “absurd or wicked” moral judgments, and without mentioning that classical utilitarianism — which Smart seems to express greater approval toward — has similar and arguably worse theoretical implications (cf. Knutsson, 2021; Ajantaival, 2022).

The following anecdotal example illustrates how uncharitable remarks can influence people’s motivations and make people feel unwelcome in certain communities: An acquaintance of mine who took part in an EA intro fellowship heard a fellow participant dismiss antinatalism quite uncharitably, saying something along the lines of “antinatalism is like high school atheism, but edgier”. My acquaintance thought that antinatalism was a plausible view, and the remark left them feeling unwelcome and discouraged from engaging further with effective altruism.

Conclusion

To be clear, my point is by no means that people should refrain from criticizing suffering-focused views, even in strong terms. My recommendation is simply that critics should strive to be even-handed, and to not misrepresent or unfairly malign views with which they disagree.

If we are trying to think straight about ethics, we should be keen not to let uncharitable claims and social pressures distort our thinking, especially since these factors tend to influence our views in hidden ways. After all, few people consciously think — let alone say — that social pressure exerts a strong influence on their views. Yet it is likely a potent factor all the same.

Research vs. non-research work to improve the world: In defense of more research and reflection

When trying to improve the world, we can either pursue direct interventions, such as directly helping beings in need and doing activism on their behalf, or we can pursue research on how we can best improve the world, as well as on what improving the world even means in the first place.

Of course, the distinction between direct work and research is not a sharp one. We can, after all, learn a lot about the “how” question by pursuing direct interventions, testing out what works and what does not. Conversely, research publications can effectively function as activism, and may thereby help bring about certain outcomes quite directly, even when such publications do not deliberately try to do either.

But despite these complications, we can still meaningfully distinguish more or less research-oriented efforts to improve the world. My aim here is to defend more research-oriented efforts, and to highlight certain factors that may lead us to underinvest in research and reflection. (Note that I here use the term “research” to cover more than just original research, as it also covers efforts to learn about existing research.)


Contents

  1. Some examples
    1. I. Cause Prioritization
    2. II. Effective Interventions
    3. III. Core Values
  2. The steelman case for “doing”
    1. We can learn a lot by acting
    2. Direct action can motivate people to keep working to improve the world
    3. There are obvious problems in the world that are clearly worth addressing
    4. Certain biases plausibly prevent us from pursuing direct action
  3. The case for (more) research
    1. We can learn a lot by acting — but we are arguably most limited by research insights
      1. Objections: What about “long reflection” and the division of labor?
    2. Direct action can motivate people — but so can (the importance of) research
    3. There are obvious problems in the world that are clearly worth addressing — but research is needed to best prioritize and address them
    4. Certain biases plausibly prevent us from pursuing direct action — but there are also biases pushing us toward too much or premature action
  4. The Big Neglected Question
  5. Conclusion
  6. Acknowledgments

Some examples

Perhaps the best way to give a sense of what I am talking about is by providing a few examples.

I. Cause Prioritization

Say our aim is to reduce suffering. Which concrete aims should we then pursue? Maybe our first inclination is to work to reduce human poverty. But when confronted with the horrors of factory farming, and the much larger number of non-human animals compared to humans, we may conclude that factory farming seems the more pressing issue. However, having turned our gaze to non-human animals, we may soon realize that the scale of factory farming is small compared to the scale of wild-animal suffering, which might in turn be small compared to the potentially astronomical scale of future moral catastrophes.

With so many possible causes one could pursue, it is likely suboptimal to settle on the first one that comes to mind, or to settle on any one of them without having made a significant effort considering where one can make the greatest difference.

II. Effective Interventions

Next, say we have settled on a specific cause, such as ending factory farming. Given this aim, there is a vast range of direct interventions one could pursue, including various forms of activism, lobbying to influence legislation, or working to develop novel foods that can outcompete animal products. Yet it is likely suboptimal to pursue any of these particular interventions without first trying to figure out which of them have the best expected impact. After all, different interventions may differ greatly in terms of their cost-effectiveness, which suggests that it is reasonable to make significant investments into figuring out which interventions are best, rather than to rush into action mode (although the drive to do the latter is understandable and intuitive, given the urgency of the problem).

III. Core Values

Most fundamentally, there is the question of what matters and what is most worth prioritizing at the level of core values. Our values ultimately determine our priorities, which renders clarification of our values a uniquely important and foundational step in any systematic endeavor to improve the world.

For example, is our aim to maximize a net sum of “happiness minus suffering”, or is our aim chiefly to minimize extreme suffering? While there is significant common ground between these respective aims, there are also significant divergences between them, which can matter greatly for our priorities. The first view implies that it would be a net benefit to create a future that contains vast amounts of extreme suffering as long as that future contains a lot of happiness, while the other view would recommend the path of least extreme suffering.

In the absence of serious reflection on our values, there is a high risk that our efforts to improve the world will not only be suboptimal, but even positively harmful relative to the aims that we would endorse most strongly upon reflection. Yet efforts to clarify values are nonetheless extremely neglected — and often completely absent — in endeavors to improve the world.

The steelman case for “doing”

Before making a case for a greater focus on research, it is worth outlining some of the strongest reasons in favor of direct action (e.g. directly helping other beings and doing activism on their behalf).

We can learn a lot by acting

  • The pursuit of direct interventions is a great way to learn important lessons that may be difficult to learn by doing pure research or reflection.
  • In particular, direct action may give us practical insights that are often more in touch with reality than are the purely theoretical notions that we might come up with in intellectual isolation. And practical insights and skills often cannot be compensated for by purely intellectual insights.
  • Direct action often has clearer feedback loops, and may therefore provide a good opportunity to both develop and display useful skills.

Direct action can motivate people to keep working to improve the world

  • Research and reflection can be difficult, and it is often hard to tell whether one has made significant progress. In contrast, direct action may offer a clearer indication that one is really doing something to improve the world, and it can be easier to see when one is making progress (e.g. whether people altered their behavior in response to a given intervention, or whether a certain piece of legislation changed or not).

There are obvious problems in the world that are clearly worth addressing

  • For example, we do not need to do more research to know that factory farming is bad, and it seems reasonable to think that evidence-based interventions that significantly reduce the number of beings who suffer on factory farms will be net beneficial.
  • Likewise, it is probably beneficial to build a healthy movement of people who aim to help others in effective ways, and who reflect on and discuss what “helping others” ideally entails.

Certain biases plausibly prevent us from pursuing direct action

  • It seems likely that we have a passivity bias of sorts. After all, it is often convenient to stay in one’s intellectual armchair rather than to get one’s hands dirty with direct work that may fall outside of one’s comfort zone, such as doing street advocacy or running a political campaign.
  • There might also be an omission bias at work, whereby we judge an omission to do direct work that prevents harm less harshly than an equivalent commission of harm.

The case for (more) research

I endorse all the arguments outlined above in favor of “doing”. In particular, I think they are good arguments in favor of maintaining a strong element of direct action in our efforts to improve the world. Yet they are less compelling when it comes to establishing the stronger claim that we should focus more on direct action (on the current margin), or that direct action should represent the majority of our altruistic efforts at this point in time. I do not think any of those claims follow from the arguments above.

In general, it seems to me that altruistic endeavors tend to focus far too strongly on direct action while focusing far too little on research. This is hardly a controversial claim, at least not among aspiring effective altruists, who often point out that research on cause prioritization and on the cost-effectiveness of different interventions is important and neglected. Yet it seems to me that even effective altruists tend to underinvest in research, and to jump the gun when it comes to cause selection and direct action, and especially when it comes to the values they choose to steer by.

A helpful starting point might be to sketch out some responses to the arguments outlined in the previous section, to note why those arguments need not undermine a case for more research.

We can learn a lot by acting — but we are arguably most limited by research insights

The fact that we can learn a lot by acting, and that practical insights and skills often cannot be substituted by pure conceptual knowledge, does not rule out that our potential for beneficial impact might generally be most bottlenecked by conceptual insights.

In particular, clarifying our core values and exploring the best causes and interventions arguably represent the most foundational steps in our endeavors to improve the world, suggesting that they should — at least at the earliest stages of our altruistic endeavors — be given primary importance relative to direct action (even as direct action and the development of practical skills also deserve significant priority, perhaps even more than 20 percent of the collective resources we spend at this point in time).

The case for prioritizing direct action would be more compelling if we had a lot of research that delivered clear recommendations for direct action. But I think there is generally a glaring shortage of such research. Moreover, research on cause prioritization often reveals plausible ways in which direct altruistic actions that seem good at first sight may actually be harmful. Such potential downsides of seemingly good actions constitute a strong and neglected reason to prioritize research more — not to get perpetually stuck in research, but to at least map out the main considerations for and against various actions.

To be more specific, it seems to me that the expected value of our actions can change a lot depending on how deep our network of crucial considerations goes, so much so that adding an extra layer of crucial considerations can flip the expected value of our actions. Inconvenient as it may be, this means that our views on what constitutes the best direct actions have a high risk of being unreliable as long as we have not explored crucial considerations in depth. (Such a risk always exists, of course, yet it seems that it can at least be markedly reduced, and that our estimates can become significanctly better informed even with relatively modest research efforts.)

At the level of an individual altruist’s career, it seems warranted to spend at least one year reading about and reflecting on fundamental values, one year learning about the most important cause areas, and one year learning about optimal interventions within those cause areas (ideally in that order, although one may fruitfully explore them in parallel to some extent; and such a full year’s worth of full-time exploration could, of course, be conducted over several years). In an altruistic career spanning 40 years, this would still amount to less than ten percent of one’s work time focused on such basic exploration, and less than three percent focused on exploring values in particular.

A similar argument can be made at a collective level: if we are aiming to have a beneficial influence on the long-term future — say, the next million years — it seems warranted to spend at least a few years focused primarily on what a beneficial influence would entail (i.e. clarifying our views on normative ethics), as well as researching how we can best influence the long-term future before we proceed to spend most of our resources on direct action. And it may be even better to try to encourage more people to pursue such research, ideally creating an entire research project in which a large number of people collaborate to address these questions.

Thus, even if it is ideal to mostly focus on direct action over the entire span of humanity’s future, it seems plausible that we should focus most strongly on advancing research at this point, where relatively little research has been done, and where the explore-exploit tradeoff is likely to favor exploration quite strongly.

Objections: What about “long reflection” and the division of labor?

An objection to this line of reasoning is that heavy investment into reflection is premature, and that our main priority at this point should instead be to secure a condition of “long reflection” — a long period of time in which humanity focuses on reflection rather than action.

Yet this argument is problematic for a number of reasons. First, there are strong reasons to doubt that a condition of long reflection is feasible or even desirable, given that it would seem to require strong limits to voluntary actions that diverge from the ideal of reflection.

To think that we can choose to create a condition of long reflection may be an instance of the illusion of control. Human civilization is likely to develop according to its immediate interests, and seems unlikely to ever be steered via a common process of reflection. And even if we were to secure a condition of long reflection, there is no guarantee that humanity would ultimately be able to reach a sufficient level of agreement regarding the right path forward — after all, it is conceivable that a long reflection could go awfully wrong, and that bad values could win out due to poor execution or malevolent agents hijacking the process.

The limited feasibility of a long reflection suggests that there is no substitute for reflecting now. Failing to clarify and act on our values from this point onward carries a serious risk of pursuing a suboptimal path that we may not be able to reverse later. The resources we spend pursuing a long reflection (which is unlikely to ever occur) are resources not spent on addressing issues that might be more important and more time-sensitive, such as steering away from worst-case outcomes.

Another objection might be that there is a division of labor case favoring that only some people focus on research, while others, perhaps even most, should focus comparatively little on research. Yet while it seems trivially true that some people should focus more on research than others, this is not necessarily much of a reason against devoting more of our collective attention toward research (on the current margin), nor a reason against each altruist making a significant effort to read up on existing research.

After all, even if only a limited number of altruists should focus primarily on research, it still seems necessary that those who aim to put cutting-edge research into practice also spend time reading that research, which requires a considerable time investment. Indeed, even when one chooses to mostly defer to the judgments of other people, one will still need to make an effort to evaluate which people are most worth deferring to on different issues, followed by an effort to adequately understand what those people’s views and findings entail.

This point also applies to research on values in particular. That is, even if one prioritizes direct action over research on fundamental values, it still seems necessary to spend a significant amount of time reading up on other people’s work on fundamental values if one is to be able to make at least a somewhat qualified judgment regarding which values one will attempt to steer by.

The division of altruistic labor is thus consistent with the recommendation that every dedicated altruist should spend at least a full year reading about and reflecting on fundamental values (just as the division of “ordinary” labor is consistent with everyone spending a certain amount of time on basic education). And one can further argue that the division of altruistic labor, and specialized work on fundamental values in particular, is only fully utilized if most people spend a decent amount of time reading up on and making use of the insights provided by others.

Direct action can motivate people — but so can (the importance of) research

While research work is often challenging and difficult to be motivated to pursue, it is probably a mistake to view our motivation to do research as something that is fixed. There are likely many ways to increase our motivation to pursue research, not least by strongly internalizing the (highly counterintuitive) importance of research.

Moreover, the motivating force provided by direct action might be largely maintained as long as one includes a strong component of direct action in one’s altruistic work (by devoting, say, 25 percent of one’s resources toward direct action).

In any case, reduced individual motivation to pursue research seems unlikely to be a strong reason against devoting a greater priority to research at the level of collective resources and priorities (even if it might play a significant role in many individual cases). This is partly because the average motivation to pursue these respective endeavors seems unlikely to differ greatly — after all, many people will be more motivated to pursue research over direct action — and partly because urgent necessities are worth prioritizing and paying for even if they happen to be less than highly motivating.

By analogy, the cleaning of public toilets is also worth prioritizing and paying for, even if it may not be the most motivating pursuit for those who do it, and the same point arguably applies even more strongly in the case of the most important tasks necessary for achieving altruistic aims such as reducing extreme suffering. Moreover, the fact that altruistic research may be unusually taxing on our motivation (e.g. due to a feeling of “analysis paralysis”) is actually a reason to think that such taxing research is generally neglected and hence worth pursuing on the margin.

Finally, to the extent one finds direct action more motivating than research, this might constitute a bias in one’s prioritization efforts, even if it represents a relevant data point about one’s personal fit and comparative advantage. And the same point applies in the opposite direction: to the extent that one finds research more motivating, this might make one more biased against the importance of direct action. While personal motivation is an important factor to consider, it is still worth being mindful of the tendency to overprioritize that which we consider fun and inspiring at the expense of that which is most important in impartial terms.

There are obvious problems in the world that are clearly worth addressing — but research is needed to best prioritize and address them

Knowing that there are serious problems in the world, as well as interventions that reduce those problems, does not in itself inform us about which problems are most pressing or which interventions are most effective at addressing them. Both of these aspects — roughly, cause prioritization and estimating the effectiveness of interventions — seem best advanced by research.

A similar point applies to our core values: we cannot meaningfully pursue cause prioritization and evaluations of interventions without first having a reasonably clear view of what matters, and what would constitute a better or worse world. And clarifying our values is arguably also best done through further research rather than through direct action (even as the latter may be helpful as well).

Certain biases plausibly prevent us from pursuing direct action — but there are also biases pushing us toward too much or premature action

The putative “passivity bias” outlined above has a counterpart in the “action bias”, also known as “bias for action” — a tendency toward action even when action makes no difference or is positively harmful. A potential reason behind the action bias relates to signaling: actively doing something provides a clear signal that we are at least making an effort, and hence that we care (even if the effect might ultimately be harmful). By comparison, doing nothing might be interpreted as a sign that we do not care.

There might also be individual psychological benefits explaining the action bias, such as the satisfaction of feeling that one is “really doing something”, as well as a greater feeling of being in control. In contrast, pursuing research on difficult questions can feel unsatisfying, since progress may be relatively slow, and one may not intuitively feel like one is “really doing something”, even if learning additional research insights is in fact the best thing one can do.

Political philosopher Michael Huemer similarly argues that there is a harmful tendency toward too much action in politics. Since most people are uninformed about politics, Huemer argues that most people ought to be passive in politics, as there is otherwise a high risk that they will make things worse through ignorant choices.

Whatever one thinks of the merits of Huemer’s argument in the political context, I think one should not be too quick to dismiss a similar argument when it comes to improving the long-term future — especially considering that action bias seems to be greater when we face increased uncertainty. At the very least, it seems worth endorsing a modified version of the argument that says that we should not be eager to act before we have considered our options carefully.

Furthermore, the fact that we evolved in a condition that was highly action-oriented rather than reflection-oriented, and in which action generally had far more value for our genetic fitness than did systematic research (indeed, the latter was hardly even possible), likewise suggests that we may be inclined to underemphasize research relative to how important it is for optimal impact from an impartial perspective.

This also seems true when it comes to our altruistic drives and behaviors in particular, where we have strong inclinations toward pursuing publicly visible actions that make us appear good and helpful (Hanson, 2015; Simler & Hanson, 2018, ch. 12). In contrast, we seem to have much less of an inclination toward reflecting on our values. Indeed, it seems plausible that we generally have an inclination against questioning our instinctive aims and drives — including our drive to signal altruistic intentions with highly visible actions — as well as an inclination against questioning the values held by our peers. After all, such questioning would likely have been evolutionarily costly in the past, and may still feel socially costly today.

Moreover, it is very unnatural for us to be as agnostic and open-minded as we should ideally be in the face of the massive uncertainty associated with endeavors that seek to have the best impact for all sentient beings (see also Vinding, 2020, 9.1-9.2). This suggests that we may tend to be overconfident about — and too quick to conclude — that some particular direct action happens to be the optimal path for helping others.

Lastly, while some kind of omission bias plausibly causes us to discount the value of making an active effort to help others, it is not clear whether this bias counts more strongly against direct action than against research efforts aimed at helping others, since omission bias likely works against both types of action (relative to doing nothing). In fact, the omission bias might count more strongly against research, since a failure to do important research may feel like less of a harmful inaction than does a failure to pursue direct actions, whose connection to addressing urgent needs is usually much clearer.

The Big Neglected Question

There is one question that I consider particularly neglected among aspiring altruists — as though it occupies a uniquely impenetrable blindspot. I am tempted to call it “The Big Neglected Question”.

The question, in short, is whether anything can ethically outweigh or compensate for extreme suffering. Our answer to this question has profound implications for our priorities. And yet astonishingly few people seem to seriously ponder it, even among dedicated altruists. In my view, reflecting on this question is among the first, most critical steps in any systematic endeavor to improve the world. (I suspect that a key reason this question tends to be shunned is that it seems too dark, and because people may intuitively feel that it fundamentally questions all positive and meaning-giving aspects of life — although it arguably does not, as even a negative answer to the question above is compatible with personal fulfillment and positive roles and lives.)

More generally, as hinted earlier, it seems to me that reflection on fundamental values is extremely neglected among altruists. Ozzie Gooen argues that many large-scale altruistic projects are pursued without any serious exploration as to whether the projects in question are even a good way to achieve the ultimate (stated) aims of these projects, despite this seeming like a critical first question to ponder.

I would make a similar argument, only one level further down: just as it is worth exploring whether a given project is among the best ways to achieve a given aim before one pursues that project, so it is worth exploring which aims are most worth striving for in the first place. This, it seems to me, is even more neglected than is exploring whether our pet projects represent the best way to achieve our (provisional) aims. There is often a disproportionate amount of focus on impact, and comparatively little focus on what is the most plausible aim of the impact.

Conclusion

In closing, I should again stress that my argument is not that we should only do research and never act — that would clearly be a failure mode, and one that we must also be keen to steer clear of. But my point is that there are good reasons to think that it would be helpful to devote more attention to research in our efforts to improve the world, both on moral and empirical issues — especially at this early point in time.


Acknowledgments

For helpful comments, I thank Teo Ajantaival, Tobias Baumann, and Winston Oswald-Drummond.

“Team victory” as a key hidden motive

Simler and Hanson’s The Elephant in the Brain has been hugely influential on me. The core claim of the book is that our beliefs and behaviors often serve hidden motives, and that these motives are commonly less pretty than the more noble motives that we usually proclaim.

A key point that is mentioned in the book is the significance of coalitions and coalitional conflicts in human life and human evolution. Specifically, the authors note how, in small-scale coalition politics, “coalitions compete for control, and individuals seek to ally themselves with powerful coalitions”.

Yet it seems that there is more to be said about the significance of coalitional conflicts for our hidden motives than what is covered in The Elephant in the Brain (as the authors would surely agree). Indeed, the book is explicitly an open invitation for others to identify or suggest additional hidden motives, and I will here take up that invitation and suggest a general hidden motive that plausibly plays a large role in much human behavior, namely coalitional success, or “team victory”.

Different categories of hidden motives

It seems to me that we can meaningfully distinguish at least four categories of hidden motives (even as these categories are overlapping, and the motives not always hidden):

  • To signal impressiveness (e.g. by showing that you are impressively knowledgeable, athletic, or hard-working)
  • To signal loyalty (e.g. by wearing a sports jersey or a religious symbol)
  • To gain “team victory” (e.g. helping to ensure that your team gains more power than the rival team)
  • To gain “individual success” (e.g. actually getting the calories or sex needed for survival and reproduction)

Of course, from an evolutionary perspective, these motives must ultimately all translate into success in terms of “individual success”. Yet seeking individual success very directly is often a bad way to achieve such success for humans — hence these other motives and strategies. (Though it is worth noting that in certain circumstances, it would be fitness-enhancing to pursue “individual success” even at the expense of these other adaptive drives, meaning that there are cases where humans can gain “individual success” by doing things that are positively unimpressive, disloyal, or detrimental to team victory. These are sometimes called scandals.)

Hidden motives: Not all about signaling

An important point implied by the categories above is that hidden motives are not all about signaling, and that our signaling motives sometimes take a backseat to other hidden motives that are even stronger.

For example, signaling loyalty and ensuring team victory seem to be fairly convergent aims for the most part, yet there are probably still many cases where the “team victory” motive is stronger than the loyalty-signaling motive, such as when our actions have a significant influence on the probability of team victory (e.g. slightly reducing our perceived team loyalty — from 10 to 9, say — in exchange for a huge gain in our team’s success would likely have been adaptive in many cases).

Indeed, even when our actions do not have a high probability of influencing outcomes, such as in large-scale politics involving millions of other actors, it is likely that our evolved instincts — which were adapted for small-scale coalition politics — will in many cases still care as much or more about collective team victory as they care about individual loyalty-signaling to that team.

Reasons to think that “team victory” is a strong motive

What reasons do we have for thinking that “team victory” is a strong motive underlying much of human behavior?

The importance of coalitional success

First, there is the fact that individual human success often depended crucially on coalitional success, at the level of intra- as well as inter-group competition (both of which could be lethal). And merely signaling loyalty to one’s own coalition(s) — while important — would often not be sufficient to secure coalitional victory. A serious drive and effort toward actually winning was likely paramount.

As hinted above, actions that optimize for loyalty-signaling and actions that optimize for group victory are probably correlated to a significant extent, but not perfectly so, and individuals whose motives and instincts were optimized purely for loyalty-signaling would probably be less effective at achieving coalitional success than would individuals whose motives and drives were optimized more for that aim (i.e. individuals whose motives were to some degree optimized both for intra-group loyalty-signaling and for securing inter-group success and power).


Given the importance of inter-group success in our ancestral environment, it seems reasonable to think that our motives also reflect drives for such success to a significant extent. Hence we should not restrict our focus to intra-group success alone when analyzing human motives. (Note that this picture applies both to group struggles within and between tribes; after all, human social life consists of multiple nested coalitions.)

Some evolutionary theorists, including John Tooby and Leda Cosmides, have provided more elaborate arguments for the claim that we humans have strong coalitional instincts, similarly based on the importance of coalitional acumen in our ancestral environment. Related is Jonathan Haidt’s argument that “groupishness” is a deep feature of human nature.

Empirical data and informative examples

In empirical terms, there are studies that show that we often prefer policies that disadvantage our outgroup (e.g. people in a foreign country), even when we have the option to choose win-win policies that benefit our own group as well. Such findings lend some support to a significant “team victory” motive in human decisions and behavior. (Of relevance, too, are “minimal group paradigm”; “realistic conflict theory”; Sapolsky, 2017, ch. 11; Clark et al, 2019.)

Examples where the “team victory” motive seems to positively eclipse the loyalty-signaling motive include cases in which people secretly cheat in order to secure “team victory” — behaviors that the cheaters sometimes know will get them hated among their ingroup if they get exposed. Some of the instances of cheating mentioned here appear to fit this pattern.

Board games in which different teams compete against each other may be another example. It seems that people are often more concerned about winning than about signaling loyalty to, and having a good standing among, their team mates; so much so that they sometimes even deride their entire team while being in a relentless rush to win. And this phenomenon also happens at times in sports. (Board games and sports are arguably both supernormal stimuli that trigger the players’ drive for “team victory” in more overt and systematic ways than do our everyday — mostly hidden — coalitional competitions.)

The fact that the derisive and practically anti-loyal behaviors described above seem to occur with some frequency in competitive domains — despite team cohesion generally being important for team victory, and despite loyalty-signaling in particular seeming reasonably correlated with team victory — suggests that the “team victory” motive is probably also lurking in more ordinary circumstances (where it is expressed in more group-aligned ways). Indeed, one can argue that the drive for “team victory” must be quite strong for it to override and to some extent counteract our otherwise powerful pro-cohesion and loyalty-signaling motives in this way, even if only occasionally.

Introspection

Similarly, consider your own direct experience when you play team sports or board games on a team. Do you plausibly most want to signal loyalty to your team or do you most want to win? While we should not base our views only or even mostly on introspective observations of this kind, they can still provide at least some additional evidence, especially if our felt drive for team victory is particularly strong. (There is, of course, individual variation in terms of how strongly people are motivated by “team victory” — for instance, some people do not seem to care at all whether they win in team board games. Yet the same is true of other hidden motives: some people do not seem particularly keen to signal loyalty or impressiveness in the usual ways, but that hardly undermines the claim that these are significant motives for most people.)

An explanation of “Sudden Patriotic Sports Obsession”?

Finally, one may argue that the “team victory” motive is supported by some of the surprising predictions that follow from the conjecture that we have strong desires and drives for “team victory”. For example, a prediction that arguably follows from this hypothesis is that people should generally feel a desire to see their national team win in major sports events that are highly publicized (e.g. the FIFA World Cup). And importantly, this should be true even of many people who do not usually follow that sport, and even if they do not identify strongly with their nationality. (Only “many people” because of factors such as individual psychological variation and a lack of exposure to the relevant media channels.)

As far as I can tell, this hypothesis is strongly vindicated. During widely popularized sports events, people who are usually neither sports fans nor patriots indeed tend to become mysteriously preoccupied with the fate of their national team (and I must admit that this is also true of myself: I somehow care about it, despite trying not to, and despite not watching any games).

Yet this phenomenon makes perfect sense if we have strong drives for “team victory” that can readily be triggered by a perception of direct competition between “our group” and “other groups”. (This is not to say that an instinctive drive for “team victory” is the only factor that explains this phenomenon of “Sudden Patriotic Sports Obsession”, but it does seem a plausible explanatory factor.)

Is the motive of “team victory” really hidden?

One may agree that the “team victory” motive is common and strong, yet dispute that it is at all hidden. It is, after all, unmistakably clear in the expressed desires and behaviors of athletes and dedicated sports fans, as well as in many other explicitly competitive arenas of human life.

However, in supposedly nobler and more cooperative spheres, such as in academia or in activist circles, the “team victory” motive does indeed appear quite hidden. Here, attempts to undermine the status of opposing groups, and to increase the status and influence of one’s own group, seem to often be packaged as “intellectual criticism” and “strategic disagreements”. In other words, the text of the conversation may be a technical discussion about some obscure claim, while the subtext — the underlying driver of the dispute — may be a fight for coalitional victory and dominance.

Of course, these are not the kinds of motives that sophisticated and prosocial folks are supposed to have, and hence such folks are forced to find more indirect and sophisticated ways to act them out.

Hanson makes a similar point about our lust for power — something that we would usually gain through “team victory” in our ancestral environment:

We humans evolved to lust after power and those who wield power, but to pretend our pursuit of power is accidental; we mainly just care about beauty, stories, exciting contests, and intellectual progress. Or so we say.

Indeed, like the other hidden motives identified by Simler and Hanson, our motive to achieve “team victory” and power is probably mostly unconscious in situations where we are not supposed to act on this motive, since being unconscious about such a norm-transgressing motive might make us better able to deny the accusation that we are acting on it.

Even in politics, which is obviously competitive, we almost always frame our motives purely in terms of impartial motives to “help the world” and the like (cf. Simler and Hanson, 2018, ch. 16). We rarely frame them in terms of wanting our team to win, even though there is much evidence that this is in fact a strong motive underlying our political behavior.

Tentative critique

A point of criticism I would raise regarding The Elephant in the Brain is that it seems to focus almost exclusively on hidden signaling motives, and that it thereby underemphasizes other hidden motives, such as “team victory”. Yet to consistently give overriding weight to signaling explanations relative to other, often more disturbing and unflattering explanatory motives would seem to require a justification. After all, signaling explanations — e.g. explanations that invoke loyalty-signaling motives over “team victory” motives — are not more plausible by default.

The following are some examples from the book where I think the “team victory” motive is likely to play a significant role (to be clear, I am not claiming that “team victory” necessarily plays a greater role than the hidden motives identified by Simler and Hanson; my claim is merely that the “team victory” motive plausibly also plays at least some significant role in these areas):

Conversation

Simler and Hanson emphasize impressiveness-signaling as the key hidden motive of our conversations, including when it comes to academic conversations in particular. This seems right to me. But it appears that “team victory” is also an important hidden motive in our conversations, and that it sometimes even overrides the impressiveness motive. In particular, many academic conversations and disputes are plausibly more driven by a crude desire for “team victory” than by a motive to signal impressiveness — especially when these disputes are chiefly impressive in terms of how primitively tribal they are.

Art

On art, the authors again highlight the individual motive to impress as the key hidden motive, and I again think they are right. But even here, I suspect that “team victory” can play a surprisingly significant role, beyond just the (also significant) motive of wanting to personally affiliate with impressive artists. For example, beautiful cities, such as Florence and Budapest, are themselves pieces of art that can provide a strong sense of pride and “team victory” to the local inhabitants — including their leaders — which might help explain the creation of all this art (even if “team victory” may not be the main motive). And note that this is arguably an even more cynical motive than is bare impressiveness; “we’re creating all this art to make a good impression on you” seems considerably more prosocial than “we’re creating all this art to beat your team”.

Likewise, people sometimes seem to view their best artists in much the same way that they view their best athletes: as individuals who can symbolically match and beat those on the other team. (The same appears true of the way people sometimes view their best scientists, intellectuals, fashion models, etc. Our most famous and prestigious people can serve as tokens of team status and “team victory”.)


“Our church is bigger than yours”

“But our parliament can beat your parliament”

Charity

Simler and Hanson argue that the main hidden motives behind charity are to signal our wealth and empathy. Again, I think they are right. But it seems plausible that charitable behavior can also be motivated to some extent by a desire for “team victory”, such as when people donate toward the promotion of their own religion, political faction, or activist ingroup.

Religion

The hidden motives the authors ascribe to religious behavior is community bonding and loyalty-signaling, which seems right. But “team victory” is probably also an influential motive (cf. Tuschman, 2013, ch. 7). An extreme example might be religious wars, in which one religion would essentially try to beat another, plausibly motivated in part by a drive for “team victory”. A less extreme example might be apologists and missionaries who seek to defend their faith and convert others — for many such people, the “team victory” motive plausibly plays some role, even if they also have other motives (e.g. being impressive to the ingroup, seeking to get into heaven, or genuinely trying to help other people).

Politics

The authors identify loyalty-signaling as a key hidden motive underlying our political behavior. This seems right. But as noted earlier, it is plausible that we are also strongly motivated by “team victory”. After all, even when following an election in private, partisan voters still seem to fervently root for the victory of “their team”, not too unlike people who eagerly want their team to win in board games or in sports. And again, just as many sports fans would be willing to quietly take off their sports jersey (i.e. their personal signal of loyalty) if they thought it significantly increased their team’s chances of winning, it seems that many political actors would likewise be willing to quietly forego loyalty signaling to a significant extent provided it could help their political team bring home the desired win.

Potential biases that follow from this?

Lastly, it is worth briefly pondering how this drive for “team victory” might bias our outlooks and priorities. The most plausible bias I see is a tendency to overstate the extent to which “our team winning” is the key to creating better outcomes from an impartial perspective.

That is, our coalitional intuitions might at some level hold that “if our coalition wins, that is a total success; if their coalition wins, that is a total disaster”. After all, in terms of reproductive success, this was probably often true in the context of intense coalitional conflicts in our ancestral environment. But it seems considerably less true from an impartial perspective, especially in the context of modern political competition between similar parties, or among different factions of activists who have broadly similar aims.

In other words, our intuitions are plausibly much too afraid of (reasonably similar) “outgroups” in the modern political and altruistic landscape, and we may well overestimate how much better “our group” would do compared to “their group” when it comes to creating beneficial outcomes for everyone.


Acknowledgments

I am grateful to Tobias Baumann and Robin Hanson for helpful feedback.

Reasoned Politics

How can we do politics better?

In Reasoned Politics, Magnus Vinding lays out a path toward politics based on ethical reasoning and empirical evidence. He argues that a better approach to politics is both conceivable and realistic. Modern discoveries in political psychology hint at new, improved norms for political discourse and cooperation, while also pointing to concrete ways in which such improvements can gradually be realized.

Having outlined a general framework for reasoned politics, Vinding proceeds to apply this framework to real-world policy issues. Based on an ethical foundation that takes the suffering of all sentient beings into account, he explores various lines of evidence to infer which policies seem most helpful for alleviating severe suffering.

Amazon (Kindle, paperback, and hardcover)
Apple BooksBarnes & NobleKobo, ScribdVivlioThalia
Smashwords
Paperback PDF

Why I have written a book about politics



“We missed it, now we have it. The Magnum Opus for a Reasoned Politics for all, humans and animals alike. I heartily recommend it to anyone who is interested in a rational approach to politics.”
Sabine Brels, international animal lawyer, author of Le droit du bien-être animal dans le monde

“In a time of heated political debate, Magnus Vinding provides a strong case for pursuing reason in politics, while cautioning us about the dangers of giving up on it. Vinding practices what he preaches — the book engages with relevant research from different areas to make its case in a reasoned way. It combines a wide-ranging view with topical applications. Even if not agreeing on every topic, the reader will come out enlightened.”
Tiago Ribeiro Dos Santos, author of Why Not Parliamentarism?

“A compelling case for a new kind of politics. Politics shouldn’t be conducted in the interests of any one ethnic group or species, but instead to promote the interests of all sentient beings. The text combines a masterly command of the academic literature with a minimum of scholarly clutter. Vinding’s plea for an alliance of reason and compassion deserves the widest possible audience. Highly recommended.”
David Pearce, author of The Hedonistic Imperative and Can Biotechnology Abolish Suffering?

“Magnus Vinding’s extensively researched and lucidly written work is a welcome antidote to the bold claims and strong opinions that permeate politics and activism. He carefully proposes aims and approaches that may inch us towards a world with less intense suffering of all sentient beings, based on empirical findings from sociology, psychology and other fields. A must-read for any changemaker concerned about how to reduce suffering over the long term.”
Jonathan Leighton, founder of the Organisation for the Prevention of Intense Suffering, author of The Battle for Compassion: Ethics in an Apathetic Universe

“This book is unlike any other I know. Reasoned Politics shows us how we can adopt a form of politics thoughtfully informed by the right kind of values. To do this, we need to clarify our moral priorities and to identify the individual and collective political choices that best honor them. This task requires disciplined reflection, awareness of cognitive biases, patient empirical research, and inclusive deliberation. As Vinding argues, the reduction of suffering, human and non-human, must be central to any plausible political ideal. He then considers the political structures and norms that will advance the reduction of suffering and other paramount values. This leads him to an illuminating discussion of how to understand the concepts of liberty, equality, justice, and democracy. Unlike most political theorists, Vinding never lets his readers forget the urgency of ending our species’ indefensibly cruel treatment of non-human animals. This book is filled with insight, wisdom, and critical information. Vinding models the virtues that he recommends in political discourse: he is observant, clear-minded, humane, sensible, honest, and unafraid. Political theorists should take a break from what they are doing and read Reasoned Politics.”
Jamie Mayerfeld, professor of political science at the University of Washington, author of Suffering and Moral Responsibility and The Promise of Human Rights

Priorities for reducing suffering: Reasons not to prioritize the Abolitionist Project

I discussed David Pearce’s Abolitionist Project in Chapter 13 of my book on Suffering-Focused Ethics. The chapter is somewhat brief and dense, and its main points could admittedly have been elaborated further and explained more clearly. This post explores and elaborates on some of these points.


A good place to start might be to highlight some of the key points of agreement between David Pearce and myself.

  • First and most important, we both agree that minimizing suffering should be our overriding moral aim.
  • Second, we both agree that we have reason to be skeptical about the possibility of digital sentience — and at the very least to not treat it as a foregone conclusion — which I note from the outset to flag that views on digital sentience are unlikely to account for the key differences in our respective views on how to best reduce suffering.
  • Third, we agree that humanity should ideally use biotechnology to abolish suffering throughout the living world, provided this is indeed the best way to minimize suffering.

The following is a summary of some of the main points I made about the Abolitionist Project in my book. There are four main points I would emphasize, none of which are particularly original (at least two of them are made in Brian Tomasik’s Why I Don’t Focus on the Hedonistic Imperative).

I.

Some studies suggest that people who have suffered tend to become more empathetic. This obviously does not imply that the Abolitionist Project is infeasible, but it does give us reason to doubt that abolishing the capacity to suffer in humans should be among our main priorities at this point.

To clarify, this is not a point about what we should do in the ideal, but more a point about where we should currently invest our limited resources, on the margin, to best reduce suffering. If we were to focus on interventions at the level of gene editing, other traits (than our capacity to suffer) seem more promising to focus on, such as increasing dispositions toward compassion and wisdom. And yet interventions focused on gene editing may themselves not be among the most promising things to focus on in the first place, which leads to the next point.

II.

For even if we grant that the Abolitionist Project should be our chief aim, at least in the medium term, it still seems that the main bottleneck to its completion is found not at the technical level, but rather at the level of humanity’s values and willingness to do what would be required. I believe this is also a point David and I mostly agree on, as he has likewise hinted, in various places, that the main obstacle to the Abolitionist Project will not be technical, but sociopolitical. This would give us reason to mostly prioritize the sociopolitical level on the margin — especially humanity’s values and willingness to reduce suffering. And the following consideration provides an additional reason in favor of the same conclusion.

III.

The third and most important point relates to the distribution of future (expected) suffering, and how we can best prevent worst-case outcomes. Perhaps the most intuitive way to explain this point is with an analogy to tax revenues: if one were trying to maximize tax revenues, one should focus disproportionately on collecting taxes from the richest people rather than the poorest, simply because that is where most of the money is.

The visual representation of the income distribution in the US in 2019 found below should help make this claim more intuitive.

The point is that something similar plausibly applies to future suffering: in terms of the distribution of future (expected) suffering, it seems reasonable to give disproportionate focus to the prevention of worst-case outcomes, as they contain more suffering (in expectation).

Futures in which the Abolitionist Project is completed, and in which our advocacy for the Abolitionist Project helps bring on its completion, say, a century sooner, are almost by definition not the kinds of future scenarios that contain the most suffering. That is, they are not worst-case futures in which things go very wrong and suffering gets multiplied in an out-of-control fashion.

Put more generally, it seems to me that advocating for the Abolitionist Project is not the best way to address worst-case outcomes, even if we assume that such advocacy has a positive effect in this regard. A more promising focus, it seems to me, is again to increase humanity’s overall willingness and capacity to reduce suffering (the strategy that also seems most promising for advancing the Abolitionist Project itself). And this capacity should ideally be oriented toward the avoidance of very bad outcomes — outcomes that to me seem most likely to stem from bad sociopolitical dynamics.

IV.

Relatedly, a final critical point is that there may be some downsides to framing our goal in terms of abolishing suffering, rather than in terms of minimizing suffering in expectation. One reason is that the former framing may invoke our proportion bias, or what is known in the literature as proportion dominance: our tendency to intuitively care more about helping 10 out of 10 individuals rather than helping 10 out of 100, even though the impact is in fact the same.

Minimizing suffering in expectation would entail abolishing suffering if that were indeed the way to minimize suffering in expectation, but the point is that it might not be. For instance, it could be that the way to reduce the most suffering in expectation is to instead focus on reducing the probability and mitigating the expected badness of worst-case outcomes. And framing our aim in terms of abolishing suffering, rather than the more general and neutral terms of minimizing suffering in expectation, can hide this possibility somewhat. (I say a bit more about this in Section 13.3 in my book.)

Moreover, talking about the complete abolition of suffering can leave the broader aim of reducing suffering particularly vulnerable to objections — e.g. the objection that completely abolishing suffering seems risky in a number of ways. In contrast, the aim of reducing intense suffering is much less likely to invite such objections, and is more obviously urgent and worthy of priority. This is another strategic reason to doubt that the abolitionist framing is optimal.

Lastly, it would be quite a coincidence if the actions that maximize the probability of the complete abolition of suffering were also exactly those actions that minimize extreme suffering in expectation; even as these goals are related, they are by no means the same. And hence to the extent that our main goal is to minimize extreme suffering, we should probably frame our objective in these terms rather than in abolitionist terms.

Reasons in favor of prioritizing the Abolitionist Project

To be clear, there are also things to be said in favor of an abolitionist framing. For instance, many people will probably find a focus on the mere alleviation and reduction of suffering to be too negative and insufficiently motivating, leading them to disengage and drop out. Such people may find it much more motivating if the aim of reducing suffering is coupled with an inspiring vision about the complete abolition of suffering and increasingly better states of superhappiness.

As a case in point, I think my own focus on suffering was in large part inspired by the Abolitionist Project and the The Hedonistic Imperative, which gradually, albeit very slowly, eased my optimistic mind into prioritizing suffering. Without this light and inspiring transitional bridge, I may have remained as opposed to suffering-focused views as I was eight years ago, before I encountered David’s work.

Brian Tomasik writes something similar about the influence of these ideas: “David Pearce’s The Hedonistic Imperative was very influential on my life. That book was one of the key factors that led to my focus on suffering as the most important altruistic priority.”

Likewise, informing people about technologies that can effectively reduce or even abolish certain forms of suffering, such as novel gene therapies, may give people hope that we can do something to reduce suffering, and thus help motivate action to this end.

But I think the two reasons cited above count more as reasons to include an abolitionist perspective in our “communication portfolio”, as opposed to making it our main focus. Especially in light of the four considerations mentioned above that count against the abolitionist framing and focus.

A critical question

The following question may capture the main difference between David’s view and my own.

In previous conversations, David and I have clarified that we both accept that the avoidance of worst-case outcomes is, plausibly, the main priority for reducing suffering in expectation.

This premise, together with our shared moral outlook, seems to recommend a focus on minimizing the risk and ameliorating the badness of worst-case outcomes. Specifically, it follows that we should pursue the best causes and interventions for preventing such worst-case outcomes.

The critical question is thus: What reasons do we have to think that prioritizing and promoting the Abolitionist Project is the single best way, or even among the best ways, to address worst-case outcomes?

As noted above, I think there are good reasons to doubt that such a focus is among the most promising strategies to this end (say, among the top 10 causes to pursue), even if we grant that it has positive effects overall, including on worst-case outcomes in particular.

Specifically, worst-case scenarios will probably tend to be ones in which compassionate agents are not in charge, and in which “we” have very limited control over what happens. In other words, while the illusion of control is strong in general, it is plausible that our intuitive sense of how much control “we” have over the future is especially unreliable as far as worst-case outcomes are concerned.

The worst-case outcomes we should worry about are probably mostly ones in which sensible agents do not have their hands on the steering wheel, and hence our main objective should plausibly be to prevent such “low-control” outcomes, and to mitigate their badness in case they happen. Talking about futures in which advanced civilization phases out the biology of suffering is already to direct our attention toward relatively good outcomes. These scenarios are hardly among the, say, 5th percentile of worst outcomes — i.e. the outcomes that arguably deserve the greatest priority. And the actions that are best for ameliorating the badness of these worst-case outcomes are, most likely, rather different from the actions that are best for improving the, say, 50th percentile of best-case outcomes.

Possible responses

Analogy to smallpox

A way to respond may be to invoke the example of smallpox: eradicating smallpox was plausibly the best way to minimize the “risk of astronomical smallpox”, as opposed to focusing on other, indirect measures.

I think this is an interesting line of argument, but I think the case of smallpox is disanalogous in various ways. First, smallpox is in a sense a much simpler and circumscribed phenomenon than is suffering. In part for this reason, the eradication of smallpox was much easier than the abolition of suffering would be. As an infectious disease, smallpox, unlike suffering, has not evolved to serve any functional role in animals. It could thus not only be eradicated more easily, but also without unintended negative effects on, say, the function of the human mind.

Second, if we were primarily concerned about not spreading smallpox to space, and minimizing “smallpox-risks” in general, I think it is indeed plausible that the short-term eradication of smallpox would not be the ideal thing to prioritize with marginal resources. (Again, it’s important to here distinguish what humanity at large should ideally do versus what the, say, 1,000 most dedicated suffering reducers should do with most of their resources, on the margin, in our imperfect world.)

One reason such a short-term focus may be suboptimal is that the short-term eradication of smallpox is already — or would already be, if it still existed — prioritized by mainstream organizations and governments around the world, and hence additional marginal resources would likely have a rather limited counterfactual impact to this end. Work to minimize the risk of spreading life forms vulnerable to smallpox is far more neglected, and hence does seem a fairly reasonable priority from a “smallpox-risk minimizing” perspective. Granted, this is not intuitive, but the negative potential of trillions of stars combined with an expected value framework, along with marginal thinking, will often suggest rather unintuitive conclusions.

(Of course, minimizing “smallpox risk” is also intuitively crazy for another reason that is worth flagging, namely that, in the real world, there are countless other sources of suffering worth prioritizing. Hence, focusing purely on minimizing this particular risk, at the opportunity cost of neglecting all other risks, including far greater risks, is indeed transparently unreasonable. Yet striving to minimize suffering risks in general is not unreasonable in this way, given the broad scope of s-risk reduction.)

Third, and most significant I believe, there is the sad point that the suffering of virtually all sentient beings — and hence suffering as a general phenomenon — is extremely neglected. Humanity showed a relatively high willingness to eradicate smallpox, whereas in the case of the suffering of non-human beings, people are often willing to pay for “products” that entail the active infliction of intense suffering. Smallpox is thus disanalogous in that the willingness situation was fundamentally different than it is in the case of suffering — especially as far as the suffering of all sentient beings is concerned.

This relates to Point II above: the main bottleneck, not just to suffering reduction in general but also to the Abolitionist Project in particular, is likely humanity’s willingness to reduce suffering. And hence any analogy in which the willingness problem is essentially solved would seem disanalogous to the original problem in what is arguably the most crucial respect.

Sources of unwillingness

Another response may be to argue that humanity’s unwillingness to reduce suffering derives mostly from the sense that the problem of suffering is intractable, and hence the best way to increase our willingness to alleviate and prevent suffering is to set out technical blueprints for its prevention. In David’s words, “we can have a serious ethical debate about the future of sentience only once we appreciate what is — and what isn’t — technically feasible.”

I think there is something to be said in favor of this argument, as noted above in the section on reasons to favor the Abolitionist Project. Yet unfortunately, my sense is that humanity’s unwillingness to reduce suffering does not primarily stem from a sense that the problem is too vast and intractable. Sadly, it seems to me that most people give relatively little thought to the urgency of (others’) suffering, especially when it comes to the suffering of non-human beings. As David notes, factory farming can be said to be “the greatest source of severe and readily avoidable suffering in the world today”. This is but a subset of the vast problem of suffering, and solving it is clearly tractable and avoidable at a collective level. Yet most people still actively contribute to it rather than work against it, despite its solution being technically straightforward.

What is the best way to motivate humanity to prevent suffering?

This is an empirical question. But I would be surprised if setting out abolitionist blueprints turned out to be the single best strategy, especially for motivating efforts to mitigate worst-case outcomes (which this framing can risk neglecting, as argued in Point IV above). Other candidates that seem more promising to me include informing people about horrific examples of suffering, as well as presenting reasoned arguments in favor of prioritizing suffering. Again, this is not to say that abolitionist blueprints cannot be beneficial and have their place. They are just unlikely to be the best or main thing to invest in to this end, in my view.

To clarify, I am not arguing for any efforts to conserve suffering. The issue here is rather about what we should prioritize with our limited resources. The following analogy may help clarify my view: When animal advocates argue in favor of prioritizing the suffering of farm animals or wild animals rather than, say, the suffering of companion animals, they are not thereby urging us to conserve let alone increase the suffering of companion animals. The argument is rather that our limited resources seem to reduce more suffering if we spend them on these other things, even as we grant that it is a very good thing to reduce the suffering of companion animals.

In terms of how we rank the cost-effectiveness of different causes and interventions (cf. this distribution), I would still consider abolitionist advocacy to be quite positive all things considered, and probably significantly better than the vast majority of activities we could do. Not least because it highlights the urgency of suffering in a way that may be uniquely encouraging to people, which is also a good reason to include abolitionist ideas in our core portfolio of ideas. But I would not quite rank it at the tail-end of the cost-effectiveness distribution, for some of the reasons outlined above.

Effective altruism and common sense

Thomas Sowell once called Milton Friedman “one of those rare thinkers who had both genius and common sense”.

I am not here interested in Sowell’s claim about Friedman, but rather in his insight into the tension between abstract smarts and common sense, and particularly how it applies to the effective altruism (EA) community. For it seems to me that there sometimes is an unbalanced ratio of clever abstractions to common sense in EA discussions.

To be clear, my point is not that abstract ideas are unimportant, or even that everyday common sense should generally be favored over abstract ideas. After all, many of the core ideas of effective altruism are highly abstract in nature, such as impartiality and the importance of numbers, and I believe we are right to stand by these ideas. But my point is that common sense is underutilized as a sanity check that can prevent our abstractions from floating into the clouds. More generally, I seem to observe a tendency to make certain assumptions, and to do a lot of clever analysis and deductions based on those assumptions, but without spending anywhere near as much energy exploring the plausibility of these assumptions themselves. Somewhat akin to treating mathematical modeling as an exercise in pure mathematics.

Below are three examples that I think follow this pattern.

Boltzmann brains

A highly abstract idea that is admittedly intriguing to ponder is that of a Boltzmann brain: a hypothetical conscious brain that arises as the product of random quantum fluctuations. Boltzmann brains are a trivial corollary given certain assumptions: let some basic combinatorial assumptions hold for a set amount of time, and we can conclude that a lot of Boltzmann brains must exist in this span of time (at least as a mater of statistical certainty, similar to how we can derive and be certain of the second law of thermodynamics).

But this does not mean that Boltzmann brains are in fact possible, as the underlying assumptions may well be false. Beyond the obvious possibility that the lifetime of the universe could be too short, it is also conceivable that the combinatorial assumptions that allow a functioning 310 K human brain to emerge in ~ 0 K empty space do not in fact obtain, e.g. because it falsely assumes a combinatorial independence concerning the fluctuations that happen in each neighboring “bit” of the universe (or for some other reason). If any such key assumption is false, it could be that the emergence of a 310 K human brain in ~ 0 K space is not in fact allowed by the laws of physics, even in principle, meaning that even an infinite amount of time would never spontaneously produce a 310 K human Boltzmann brain.

Note that I am not claiming that Boltzmann brains cannot emerge in ~ 0 K space. My claim is simply that there is a big step from abstract assumptions to actual reality, and there is considerable uncertainty about whether the starting assumptions in question can indeed survive that step.

Quantum immortality

Another example is the notion of quantum immortality — not in the sense of merely surviving an attempted quantum suicide for improbably long, but in the sense of literal immortality because a tiny fraction of Everett branches continue to support a conscious survivor indefinitely.

This is a case where I think skeptical common sense and a search for erroneous assumptions is essential. Specifically, even granting a picture in which, say, a victim of a serious accident survives for a markedly longer time in one branch than in another, there are still strong reasons to doubt that there will be any branches in which the victim will survive for long. Specifically, we have good reason to believe that the measure of branches in which the victim survives will converge rapidly toward zero.

An objection might be that the measure indeed will converge toward zero, but that it never actually reaches zero, and hence there will in fact always be a tiny fraction of branches in which the victim survives. I believe this rests on a false assumption. For our understanding of physics suggests that there is only — and could only be — a finite number of distinct branches, meaning that even if the measure of branches in which the victim survives is approximated well by a continuous function that never exactly reaches zero, the critical threshold that corresponds to a zero measure of actual branches with a surviving victim will in fact be reached, and probably rather quickly.

Of course, one may argue that we should still assign some probability to quantum immortality being possible, and that this possibility is still highly relevant in expectation. But I think there are many risks that are much less Pascallian and far more worthy of our attention.

Intelligence explosion

Unlike the two previous examples, this last example has become quite an influential idea in EA: the notion of a fast and local “intelligence explosion“.

I will not here restate my lengthy critiques of the plausibility of this notion (or the critiques advanced by others). And to be clear, I do not think the effective altruism community is at all wrong to have a strong focus on AI. But the mistake I think I do see is that there are many abstractly grounded assumptions pertaining to a hypothetical intelligence explosion that have received an insufficient amount of scrutiny from common sense and empirical data (Garfinkel, 2018 argues along similar lines).

I think part of the problem stems from the fact that Nick Bostrom’s book Superintelligence framed the future of AI in a certain way. Here, for instance, is how Bostrom frames the issue in the conclusion of his book (p. 319):

Before the prospect of an intelligence explosion, we humans are like small children playing with a bomb. … We have little idea when the detonation will occur, though if we hold the device to our ear we can hear a faint ticking sound. … Some little idiot is bound to press the ignite button just to see what happens.

I realize Bostrom is employing a metaphor here, and I realize that he assigns a substantial credence to many different future scenarios. But the way his book is framed is nonetheless mostly in terms of such a metaphorical bomb that could ignite an intelligence explosion (i.e. FOOM). And it seems that this kind of scenario in effect became the standard scenario many people assumed and worked on, with comparatively little effort going into the more fundamental question of how plausible this future scenario is in the first place. An abstract argument about (a rather vague notion of) “intelligence” recursively improving itself was given much weight, and much clever analysis focusing on this FOOM picture and its canonical problems followed.

Again, my claim here is not that this picture is wrong or implausible, but rather that the more fundamental questions about the nature and future of “intelligence” should be kept more alive, and that our approach to these questions should be more informed by empirical data, lest we misprioritize our resources.


In sum, our fondness for abstractions is plausibly a bias we need to control for. We can do this by applying common-sense heuristics to a greater extent, by spending more time considering how our abstract models might be wrong, and by making a greater effort to hold our assumptions up against empirical reality.

Blog at WordPress.com.

Up ↑