Philosophical Multicore

Do Small Protests Work?

Tue, 04 Nov 2025 00:00:00 -0800

TLDR: The available evidence is weak. It looks like small protests may be effective at garnering support among the general public. Policy-makers appear to be more sensitive to protest size, and it’s not clear whether small protests have a positive or negative effect on their perception.

Previously, I reviewed evidence from natural experiments and concluded that protests work (credence: 90%).

My biggest outstanding concern is that all the protests I reviewed were nationwide, whereas the causes I care most about (AI safety, animal welfare) can only put together small protests. Based on the evidence, I’m pretty confident that large protests work. But what about small ones?

I can see arguments in both directions.

On the one hand, people are scope insensitive. I’m pretty sure that a 20,000-person protest is much less than twice as impactful as a 10,000-person protest. And this principle may extend down to protests that only include 10–20 people.

On the other hand, a large protest and a small protest may send different messages. People might see a small protest and think, “Why aren’t there more people here? This cause must not be very important.” So even if large protests work, it’s conceivable that small protests could backfire.

What does the scientific literature say about which of those ideas is correct?

Contents
Evidence from nationwide natural experiments
Direct evidence from lab experiments
Indirect evidence from lab experiments
Non-experimental evidence
Conclusion
Future work
Appendix: Table of papers from Orazani et al. (2021)
Notes

Evidence from nationwide natural experiments

Among the studies in my prior lit review, two studies modeled how voter outcomes varied based on the number of protesters in each county. Unfortunately, they used linear models, which doesn’t help us. We want to know if there’s a non-linearity near zero—something that looks like this:

A linear model can’t tell you what the shape of the curve looks like, or whether it dips into the negative for sufficiently small protests.

(In theory, I could analyze the raw data myself, but that would be a lot of work.)

Direct evidence from lab experiments

To my knowledge, there are two experiments that directly tested whether the size of a protest affected people’s support for a cause.

Wouters & Walgrave (2017)¹ showed (fictitious) news articles to Belgian legislators. The news articles said either “There were about 500 participants which was much less than expected”, or “There were more than 5,000 participants which was more than expected.”² The authors also altered three other independent variables, which they called Worthiness, Unity, and Commitment. Then they asked participants questions to judge how much they agreed with protesters (“position”) and whether they intended to take any actions to support protesters (“action”). Below I present the resulting regression coefficients and p-values.

	position	p-val	action	p-val
numbers	0.282	0.008	0.439	0.000
worthiness	0.381	0.000	0.116	0.297
unity	0.353	0.001	0.350	0.002
commitment	0.190	0.300	0.156	0.161

Numbers had biggest or near-biggest p-values out of the four variables, which suggests that legislators care a lot about the size of a protest. However, this study did not include a control group, so we don’t know whether smaller protests had a positive effect, a negative effect, or no effect.

Wouters (2019)³ conducted two similar studies, this time interviewing members of the general public rather than legislators. They described protest sizes the same way as Wouters & Walgrave (2017) (“500, less than expected” vs. “5,000, more than expected”), and again used four independent variables. Below are the regression coefficients and p-values from the two different studies, where the dependent variable was participants’ support for the cause.

	study 1	p-val	study 2	p-val
numbers	0.094	0.071	0.063	0.291
diversity	0.168	0.001	0.131	0.029
worthiness	0.607	0.000	1.127	0.000
unity	0.201	0.000	0.126	0.034

In this case, we find that numbers matter less than the other three factors.

Taken together, these two papers suggest:

Legislators care a lot about protest size. The general public maybe cares a bit, but not much.
Even (comparatively) small protests are effective at garnering support from the general public. It’s not clear whether they are effective for legislators.

(If small protests turned off the general public, then we would see that the “numbers” variable has good predictive power, but it doesn’t.)

Indirect evidence from lab experiments

Wouters & Walgrave (2017)¹ and Wouters (2019)³ were the only two papers I could find that directly tested the effect of protest size. But there could also be indirect evidence. I’m imagining something like this:

A lab experiment showed people either a new article about a protest, or a “control” news article. People who read about the protest were [more/less] supportive of the protesters’ cause.
The protest described in the article happened to be small.

That result would provide evidence about how small protests influence people.

To see if there was something like that, I looked through the studies cited by a meta-analysis by Orazani et al. (2021)⁴. I found two relevant papers: Thomas & Louis (2013)⁵ and Feinberg et al. (2017)⁶. (See Appendix for a list of every paper.)

Thomas & Louis (2013)⁵ did two experiments comparing participants’ reactions to news articles about violent vs. nonviolent protests. The two experiments were more or less the same, except that Experiment 1 covered fracking protests and Experiment 2 was about anti-whaling activism. Unfortunately the contents of the news articles are not publicly available and the corresponding author did not reply to my inquiry, so I don’t know how the protests were described in terms of size.

Feinberg et al. (2017)⁶ included three studies. Each study presented participants with an article or video about a different protest.

Study 1: The articles described a fictitious animal rights group. Protest size was reported in the articles as “about thirty people”.
Study 2: The articles described a Black Lives Matter march. The number of protesters was not specified in the articles.
Study 3: Participants were shown videos of Trump protests. One video showed a protest with roughly 70 participants⁷ but I don’t know how many protesters were in the other video.

In study 1, study participants reported relatively high support for protesters in the “Moderate” condition, in which a fictional animal rights group picketed a cosmetics company. (In the “Extreme” condition, the protesters broke into the building and freed animals.) However, there was no control group (!!), so we don’t know if reading about the protest caused support to go up, or if support would’ve been high anyway. The protest was described as having only thirty people, so this would’ve been useful evidence if they’d included a control group, but they didn’t.

One thing we can say about study 1 is that if small protests reduce support, then they don’t reduce support by as much as “extreme” protests do.

There is one additional paper, Bugden (2020)⁸, that was not included in the Orazani et al. meta-analysis.⁹ It showed participants articles in four conditions: a control,¹⁰ peaceful protest, disruptive protest, and violent protest. The peaceful protest article (found in the supplement document) opened with:

On Thursday, thousands of protestors took to the streets as the state legislature prepares to vote on a climate change bill.

This protest was not small, so Bugden (2020) doesn’t provide relevant evidence.

Non-experimental evidence

An observational study by Ostarek et al. (2024)¹¹ studied the effect of a disruptive protest by the climate group Just Stop Oil in which protesters blockaded a highway. By running polls before and after, the researchers found that support for a more moderate climate group, Friends of the Earth, increased just after the Just Stop Oil protest.

The protest consisted of 45 people (source).

At first glance, this appears to indicate that small protests can be effective. But I’m not sure that’s an appropriate interpretation of the evidence, because:

It was an observational study, not an experiment or even a natural experiment.¹²
Other studies have found negative effects for disruptive protests.

Conclusion

None of the evidence I found was very good.

Here are my takeaways, but given the state of the evidence, I don’t have much confidence in them.

Large protests work better than small protests at garnering support among policy-makers. (credence: 80%)
The general public probably doesn’t greatly care about the size of a protest. (credence: 60%)
Small protests can probably be effective at garnering support among the general public. (credence: 60%)

Do small protests persuade the general public? Looks like yes (but, at the risk of repeating myself, the evidence was not strong).

Do small protests persuade policy-makers? I couldn’t find any evidence either way. (But the fact that I couldn’t find anything is weak evidence against.)

Future work

I see two obvious ways to learn more about how well small protests work. They’re out of scope for this post, but they wouldn’t be too hard.

Analyze the data collected in natural experiments and use a non-linear model to assess the effectiveness of small protests.
Run a new survey (on Mechanical Turk or similar) showing people small protests vs. large protests. vs. no protests and then ask them about their opinions on the protesters’ cause.

Appendix: Table of papers from Orazani et al. (2021)

Paper	Status
Thomas & Louis (2014)	included useful information
Orazani & Leidner (2018)	I couldn’t find the full text
Becker et al. (2011)	dependent variable was not relevant
Feinberg et al. (2017)	included useful information
Gutting (2017)	dependent variable was not relevant
Leggett (2010)	unpublished
Shuman et al.	unpublished

Notes

Wouters, R., & Walgrave, S. (2017). Demonstrating Power. ↩ ↩²
I suspect that the phrases “less/more than expected” have a bigger effect on people’s perception than the numbers themselves. But this hypothesis hasn’t been tested. Some evidence for my hypothesis is that Madestam et al. (2013)¹³ (which I reviewed previously) found a strong county-level effect on protests, and the average protest size was 815 people per county, which is much closer to the “small” condition (500 people) than the “large” condition (5,000). So my guess is that 500 only sounds small because the article presented it as “less than expected”. However, the protests studied in Madestam et al. (2013) might differ in other meaningful ways. ↩
Wouters, R. (2019). The Persuasive Power of Protest. How Protest wins Public Support. ↩ ↩²
Orazani, N., Tabri, N., Wohl, M. J. A., & Leidner, B. (2021). Social movement strategy (nonviolent vs. violent) and the garnering of third‐party support: A meta‐analysis. doi: 10.1002/ejsp.2722 ↩
Thomas, E. F., & Louis, W. R. (2013). When Will Collective Action Be Effective? Violent and Non-Violent Protests Differentially Influence Perceptions of Legitimacy and Efficacy Among Sympathizers. ↩ ↩²
Feinberg, M., Willer, R., & Kovacheff, C. (2017). Extreme Protest Tactics Reduce Popular Support for Social Movements. ↩ ↩²
Source: I watched the video and counted how many people I could see. Some of the people were clearly bystanders, not protesters, but others were ambiguous so I’m not sure about the exact count. ↩
Bugden, D. (2020). Does Climate Protest Work? Partisanship, Protest, and Sentiment Pools. ↩
Even though Orazani et al. was published in 2021, its literature review was conducted in 2018. ↩
The control condition simply noted that protests exist without describing them at all, and asked participants if they supported the protesters’ cause. ↩
Ostarek, M., Simpson, B., Rogers, C., & Ozden, J. (2024). Radical climate protests linked to increases in public support for moderate organizations.

See also a less-technical 2022 preprint at https://www.socialchangelab.org/_files/ugd/503ba4_a184ae5bbce24c228d07eda25566dc13.pdf. ↩
I don’t see how the change wouldn’t be causal, but that could just be a failure of imagination on my part. ↩
Madestam, A., Shoag, D., Veuger, S., & Yanagizawa-Drott, D. (2013). Do Political Protests Matter? Evidence from the Tea Party Movement*. ↩

My Third Caffeine Self-Experiment

Mon, 03 Nov 2025 00:00:00 -0800

Last year I did a caffeine cycling self-experiment and I determined that I don’t get habituated to caffeine when I drink coffee three days a week. I did a follow-up experiment where I upgraded to four days a week (Mon/Wed/Fri/Sat) and I found that I still don’t get habituated.

For my current weekly routine, I have caffeine on Monday, Wednesday, Friday, and Saturday. Subjectively, I often feel low-energy on Saturdays. Is that because the caffeine I took on Friday is having an aftereffect that makes me more tired on Saturday?

When I ran my second experiment, I took caffeine four days, including the three-day stretch of Wednesday-Thursday-Friday. I found that my performance on a reaction time test was comparable between Wednesday and Friday. If my reaction time stayed the same after taking caffeine three days in a row, that’s evidence that I didn’t develop a tolerance over the course of those three days.

But if three days isn’t long enough for me to develop a tolerance, why is it that lately I feel tired on Saturdays, after taking caffeine for only two days in a row? Was the result from my last experiment incorrect?

So I decided to do another experiment to get more data.

This time I did a new six-week self-experiment where I kept my current routine, but I tested my reaction time every day. I wanted to test two hypotheses:

Is my post-caffeine reaction time worse on Saturday than on Mon/Wed/Fri?
Is my reaction time worse on the morning after a caffeine day than on the morning after a caffeine-free day?

The first hypothesis tests whether I become habituated to caffeine, and the second hypothesis tests whether I experience withdrawal symptoms the following morning.

The answers I got were:

No, there’s no detectable difference.
No, there’s no detectable difference.

Therefore, in defiance of my subjective experience—but in agreement with my earlier experimental results—I do not become detectably habituated to caffeine on the second day.

However, it’s possible that caffeine habituation affects my fatigue even though it doesn’t affect my reaction time. So it’s hard to say for sure what’s going on without running more tests (which I may do at some point).

Contents
Experimental procedure
Results
Alternative experimental procedures that I’m not going to do
A story about how I thought my experiment failed, but actually I was just being stupid
Notes

Experimental procedure

As with my previous experiments, I took a reaction time test every morning before caffeine, as well as an hour after caffeine on days when I took it (Mon/Wed/Fri/Sat). I ran the test for six weeks.

This experiment had the same flaws as my previous experiments, e.g., I did not blind myself because blinding myself is annoying and I didn’t feel like doing it.

In my first two experiments, I was meticulous about controlling the conditions on my computer during the reaction time test. I always tested using the humanbenchmark.com test in Chrome with a single browser window open. I normally use Firefox, but I tested in a different browser to be sure that my 100+ open Firefox tabs wouldn’t interfere with the test in any way (perhaps background tasks could slow down the JavaScript code that runs the reaction time app, which could artificially inflate my reaction time). I tested without any other applications open on my computer except for Emacs and a terminal window (which I always have open).

For my most recent experiment, I wasn’t so meticulous about it because I wanted to be lazy and I figured it probably didn’t matter. I still did the reaction time test in Chrome, but I didn’t close Firefox or other applications during the test.

Results

First, I tested to see if caffeine even made a visible difference in reaction time. Last time, caffeine had a strong and readily apparent effect on my reaction time. My third experiment replicated this result:

caffeine vs. no-caffeine:
    298.0 ms vs. 303.5 ms
    t-stat = -2.9, p-value = 0.006

However, my reaction time was noticeably worse than in the previous two experiments. My average used to hover around 280 ms and now it was hovering around 300 ms. Perhaps because I was less meticulous about keeping my computer in consistent conditions, I ended up adding some latency to the reaction time app?

Some evidence for this hypothesis is that I’ve tried testing my reaction time on Windows a few times (I normally use Linux) and it’s much faster—more like 230 ms. This is almost certainly due to a difference in how the reaction time app works on Windows vs. Linux.

My primary hypothesis test—which I pre-registered to myself, but did not pre-register publicly—was to compare post-caffeine reaction time performance on Saturday vs. the average of every other caffeine day (Mon/Wed/Fri). This test got a null result:

Saturdays vs. non-Saturday caffeine days:
    297.2 ms vs. 298.3 ms
    t-stat = -0.4, p-value = 0.697

I felt generally worse on Saturdays, but perhaps I was imagining things or seeing patterns that weren’t there, and really I shouldn’t worry about it.

Or perhaps I do actually feel worse on the second caffeine day, in a way that reaction time fails to capture. It’s possible that caffeine’s different effects habituate at different rates, and I’m losing my alertness faster than I’m losing my reaction speed.

(I would guess that caffeine’s effect on exercise performance would habituate particularly slowly—as I understand, caffeine improves exercise by physiologically improving muscle function somehow (it enhances calcium circulation or something), not just by increasing alertness.)

My second hypothesis was that I experience caffeine withdrawal on the morning after a caffeine day. I got a null result for this hypothesis as well:

morning after caffeine vs. morning after nocaf:
    303.6 ms (sd 6.9) vs. 303.5 ms (sd 7.0)
    mean difference = 0.1 ms
    t-stat = 0.0, p-value = 0.987

(Before running the experiment, I had a vague idea that I wanted to test this hypothesis, but I didn’t mentally pre-register a methodology.)

Alternative experimental procedures that I’m not going to do

It could be that my reaction time doesn’t get worse on the second day, but my alertness does get worse. I can think of two methods to test that hypothesis, but I don’t want to do them.

Method 1: Same procedure as before, but instead of using a reaction time test as the independent variable, I subjectively rate my alertness. This seems not good because it’s unblinded. I’m not too concerned about blinding reaction time because it’s hard to placebo yourself into a faster reaction time, but “subjective rating of alertness” is exactly the sort of thing that’s highly prone to a placebo effect.

Method 2: Randomize whether I take caffeine pills or placebo pills, and blind myself. To detect potential habituation, I can take the same pill two days in a row, but blind myself to what type of pill it is. Then I subjectively rate my alertness. I don’t want to do that either because it would require working out without caffeine 50% of the time, and working out without caffeine is unpleasant.

A story about how I thought my experiment failed, but actually I was just being stupid

After completing my experiment—this was about three months ago¹—I wrote some code to test the hypotheses. To my dismay, I found no detectable difference between caffeine and no-caffeine reaction times:

caffeine vs. no-caffeine:
    298.3 ms vs. 303.5 ms
	t-stat = 0.0, p-value = 0.987

If there’s not even a difference between caffeine and no-caffeine, then the experiment is useless.

At the time, I was too tired and demotivated to write up the results, so I abandoned it for a while.

Eventually I decided to finally write up the results of my experiment again. I looked at the numbers and I noticed that they didn’t make any sense. If the difference between caffeine and no-caffeine was 5.2 ms, how was the t-stat 0.0?

You may be able to see the mistake I made if you look at the numbers from the Results section. Instead of printing the t-stat and p-value for the caffeine vs. no-caffeine t-test, I accidentally printed the numbers from the morning after caffeine vs. morning after nocaf test. So the figures I was looking at were totally wrong.

I guess I wasn’t 100% there mentally when I wrote the code. (Honestly I don’t think I was even 30% there.)²

If you want to check if my code contains any other horrible mistakes, you can find it on GitHub.

Notes

I have a bad habit of letting half-finished drafts sit in my drafts folder for a long time. ↩
I wrote it on a non-caffeine day which might have something to do with it. ↩

Things I've Become More Confident About

Sun, 02 Nov 2025 00:00:00 -0700

Last year, I wrote a list of things I’ve changed my mind on. But good truth-seeking doesn’t just require you to consider where you might be wrong; you must also consider where you might be right.

In this post, I provide some beliefs I used to be uncertain about, that I have come to believe more strongly.

My belief: Evolution is true.

Why I believed it originally: I learned about the theory of evolution in school. I had the impression that it was a popular but unproven hypothesis (“just a theory”).¹

What made me more confident: When I was maybe 10 or 12, I read an article in some science magazine (National Geographic, maybe) about evolution. It said, “evolution is a theory in the same way atoms are a theory.” I probably put too much credence in this one sentence in one article, but in my mind, this was definitive proof that evolution is true.

Later, when I was 14, I started getting interested in the specifics of the theory of evolution and learned much more about the supporting evidence. (My motivation was mostly that I wanted to argue with creationists on the internet.)

I went through a similar trajectory when learning about quarks. I was taught that a quark is a hypothetical particle that exists inside atoms, but has never been observed. Later I learned that the existence of quarks is well-established, and it became well-established nearly three decades before I was born.

On the subject of outdated pedagogy, this is a bit of a tangent but in 5th grade I was taught the five kingdoms of life: monerans, protists, fungi, plants, and animals. Recently, I learned that not only do biologists no longer use this classification system, but that it was already obsolete when my 5th grade teacher was in 5th grade.²

(My 5th grade teacher was pretty young, but still.)
My belief: Value investing works.

Why I believed it originally: I read about Joel Greenblatt’s magic formula investing and its strong historical performance.

What made me more confident: I read more research on value investing, including the seminal paper The Cross-Section of Expected Stock Returns³ by Fama and French, and more in-depth research showing value investing has worked across the world and across asset classes⁴, and on older data going back 200 years⁵.
My belief: Peaceful protests can be effective.

Why I believed it originally: I actually went back and forth on this one. In school I learned about Martin Luther King and how he was a hero of the civil rights movement, the Montgomery Bus Boycott that he helped organize, Gandhi’s protests against colonialism, and implicit in all this was the idea that these tactics were effective.

Eventually I learned about GiveWell, which was the first time I’d ever encountered the notion that just because a charity says it’s effective, doesn’t mean it’s actually effective. I started thinking critically about protests in the same way, and I realized that I’d never actually seen good evidence that MLK or Gandhi were responsible for the positive changes that coincided with their activism.

Then I started thinking, well, there’s not strong evidence that protests work, but there’s at least some reason to believe they work. That’s about where I was at in 2024 when I donated to PauseAI—I thought, I don’t really know if this is gonna work, but it’s worth trying.

What made me more confident: I wrote Do Protests Work? A Critical Review, in which I carefully investigated the strongest evidence I could find. I found that the best evidence was better than I’d expected, and it pointed toward peaceful protests being effective.
My belief: Seed oils are good for you; seed oils don’t cause obesity.

Why I believed it originally: I had never heard of the seed oil-obesity hypothesis until I read Dynomight’s article on the subject, which argues against the hypothesis. Dynomight presented some evidence that seed oils are harmful and then ultimately concluded that they’re not. I didn’t think much about the evidence the article gave, but its conclusion seemed reasonable to me.

What made me more confident: I researched the issue in more depth while writing Outlive: A Critical Review, specifically the section on saturated fat. I looked through the literature and presented what I believed to be the strongest evidence on the matter: meta-analyses of RCTs that directly compared dietary saturated fat with unsaturated fat (which usually meant seed oils). The experimental evidence finds that seed oils are, if anything, healthier than saturated fat, which contradicts the seed oil-obesity hypothesis.

I read some writings by proponents of the seed oil hypothesis, and their arguments seemed incredibly weak to me.

(Later, I re-read Dynomight’s article and found that it cited the same evidence I had looked at while writing my review of Outlive, which I had completely forgotten about.)

Dynomight presented the seed oil hypothesis as reasonable but ultimately probably wrong, so that’s what I believed at the time. After examining the evidence in more depth, I don’t think the seed oil hypothesis is reasonable. Dynomight admirably followed Daniel Dennett’s principles for arguing intelligently, in which you present your opponent’s case as strongly as possible. But this gave me impression that the seed oil hypothesis is more plausible than it actually is.
My belief: Absent regulation, we aren’t going to solve the AI alignment problem in time.

Why I believed it originally: I’ve vaguely believed this since I first learned about the AI alignment problem (in 2013, if I remember correctly). The problem seemed to involve some thorny philosophical problems of unknown size, like the outline of an enormous beast under a murky ocean. But at that point, humanity had collectively only spent a few hundred person-years on AI alignment, and I thought, perhaps there will be some breakthrough that makes the problem turn out to be much easier than expected. Or perhaps as superintelligent AI becomes increasingly imminent, humanity will rally and pour the necessary resources into the problem.

What made me more confident: In this case I haven’t much changed my interpretation of the evidence, but I’ve become more confident as new evidence has come out. Namely, AI has gotten extraordinarily more powerful; alignment work has not kept up with the increases in AI capabilities; even though alignment work gets more attention now, the problem still seems about as hard as ever.

Beyond that, almost all alignment work is streetlight effect-ing, focused on solving tractable but mostly-irrelevant problems; and the frontier AI companies mostly don’t engage with, and are sometimes even actively hostile to, the idea that solving alignment will require major philosophical breakthroughs and it can’t be done using the sorts of empirical methods that they’re all using.
My belief: Most studies on caffeine tolerance are not informative.

Why I believed it originally: Prior to writing my post Does Caffeine Stop Working?, I reviewed some studies on caffeine tolerance and I thought to myself, these studies aren’t even testing the hypothesis they claim to be testing, surely I must be missing something?

What made me more confident: I read the studies more carefully and spent more time thinking about them, and read a few contrary papers by other scientists who study caffeine. My more careful analysis only reinforced my initial belief that most studies on caffeine tolerance are, indeed, not useful.
My belief: I am smart.

Why I believed it originally: In elementary school, I knew I was the smartest kid in my class. But my class only had about 20 students, and I figured I wasn’t that smart in the grand scheme of things. Like, not as smart as scientists and people who go to Harvard and stuff.

What made me more confident: The first big piece of evidence came after I took the PSAT in 10th grade and my score was good enough that I realized I had a good shot at getting into a top university.

Then I actually attended a top university and realized that many of the people there were not that smart compared to me. College was still a big step up from elementary school: I went from always being the smartest person in the room to being only in the top 1/3 most of the time, and I sometimes found myself in the bottom third.

This trend of repeatedly up-rating my own intelligence reached its peak when I started taking advanced computer science classes, where I was close to the 50th percentile. And nowadays I’m about average within my social circles, and often below average.

(If you’re reading this, there’s a good chance that you’re smarter than me.)

Another canon event happened when I saw the data on the distribution of my school’s SAT scores. The school’s average score was just over one standard deviation above the population mean.⁶ I went through high school thinking my average classmates were average, when in reality they were considerably smarter than average.
My belief: When I first got into lifting weights a decade ago, I learned a lot of conventional wisdom like:
- Low reps are better for strength, and high reps are better for hypertrophy.
- Compound exercises are better for strength, and isolation exercises are better for hypertrophy.
- Long rests are better for strength, and short rests are better for hypertrophy.
- If you want to bulk or cut, you should eat at a 500 calorie surplus/deficit to gain/lose about a pound per week.
Why I believed it originally: It was the conventional wisdom—people generally agreed that these things are true, even though nobody talked about why.

What made me more confident: I started paying more attention to scientific literature on resistance training and I learned that the conventional wisdom pretty much had it right, at least on these points.

(The first three pieces of advice are all explained by a unifying factor: to build strength, you want to lift as much weight as possible, and to build muscle, you want to do as much volume as possible. High reps, isolation exercise, and short rests all enable you to wear out your muscles while lifting lighter weights, and the lighter the weights, the more volume you can do. These three bits of advice aren’t overwhelmingly important—you can still build muscle doing compound exercises at low reps—but they’re useful as guidelines.)
My belief: Exercise is good for you.

Why I believed it originally: Everyone says exercise is good for you, right? But I didn’t know how you’d demonstrate scientifically that that’s true. I thought perhaps it’s reverse causation (sick people can’t exercise) or confounded by socioeconomic class or something.

What made me more confident: I learned more about the scientific evidence on exercise.
- Many randomized controlled trials show that exercise improves short-term health markers—it reduces blood pressure, improves blood sugar regulation, etc.
- A smaller number of long-term trials show long-term health benefits to exercise.
Spoilers for Game of Thrones / A Song of Ice and Fire. Click here to expand.

My belief: R + L = J. That is, Jon Snow's parents are Lyanna Stark and Rhaegar Targaryen.

Why I believed it originally: This had long been a popular fan theory. I didn't figure it out on my own, but I was reasonably convinced by the evidence in this article. I thought it sounded right, but I was uncertain because the textual evidence wasn't conclusive.

What made me more confident: I watched an interview with David Benioff and Dan Weiss, the creators of the TV show. They told a story about how they met with George R. R. Martin to get him to agree to adapt his books. At some point in the meeting, he asked them: Who is Jon Snow's mother? They gave an answer, and he didn't say whether they were right, but he gave a knowing smile, and he agreed to let them make the TV show.

They didn't say what their answer was. But I found this story to be pretty much decisive evidence for R + L = J because what it proved was that the answer was knowable. If David and Dan could know it, then the rest of the fan base could, too.

Later I became even more confident when the TV show revealed that R + L = J. (Rarely in life do you get definitive confirmation that your theory is correct!)

Notes

When I was young, I thought the way evolution worked was that a group of apes were born about 500,000 years ago, and these apes lived for hundreds of thousands of years, over which time their bodies slowly morphed to become more and more humanoid, until they became fully human, at which point they birthed human offspring and then died.

One time I told my dad that I wish I could’ve gotten to evolve because I wanted to live for 500,000 years. That’s when I learned that that’s not how evolution works. ↩
Carl Woese defined a six-kingdom taxonomy using evidence from ribosomal RNA in 1977, at which time I believe my 5th grade teacher would’ve been in 2nd grade.

Lest I sound like I know what I’m talking about, the only reason I can talk coherently about ribosomal RNA methods for taxonomic classification is because I just read those words off Wikipedia 15 seconds ago. ↩
Fama, E. F., & French, K. R. (1992). The Cross-Section of Expected Stock Returns. ↩
Asness, C. S., Moskowitz, T. J., & Pedersen, L. H. (2012). Value and Momentum Everywhere. ↩
Baltussen, G., Swinkels, L., & van Vliet, P. (2019). Global Factor Premiums. ↩
And someone who gets an average score on the SAT is above-average intelligence, because taking the SAT at all already screens off the lower end of the bell curve. ↩

Will Welfareans Get to Experience the Future?

Sat, 01 Nov 2025 00:00:00 -0700

Epistemic status: This entire essay rests on two controversial premises (linear aggregation and antispeciesism) that I believe are quite robust, but I will not be able to convince anyone that they’re true, so I’m not even going to try.

Cross-posted to the Effective Altruism Forum.

If welfare is important, and if the value of welfare scales something-like-linearly, and if there is nothing morally special about the human species¹, then these two things are probably also true:

The best possible universe isn’t filled with humans or human-like beings. It’s filled with some other type of being that’s much happier than humans, or has much richer experiences than humans, or otherwise experiences much more positive welfare than humans, for whatever “welfare” means. Let’s call these beings Welfareans.
A universe filled with Welfareans is much better than a universe filled with humanoids.

(Historically, people referred to these beings as “hedonium”. I dislike that term because hedonium sounds like a thing. It doesn’t sound like something that matters. It’s supposed to be the opposite of that—it’s supposed to be the most profoundly innately valuable sentient being. So I think it’s better to describe the beings as Welfareans. I suppose we could also call them Hedoneans, but I don’t want to constrain myself to hedonistic utilitarianism.)

Even in the “Good Ending” where we solve AI alignment and governance and coordination problems and we end up with a superintelligent AI that builds a flourishing post-scarcity civilization, will there be Welfareans? In that world, humans will be able to create a flourishing future for themselves; but beings who don’t exist yet won’t be able to give themselves good lives, because they don’t exist.

My guess is that a tiny subset of crazy people (like me) will spend their resources making Welfareans, who will end up occupying only a tiny percentage of the accessible universe, and as a result, the future will be less than 1% as good as it could have been.

(And maybe my conception of Welfareans will be wrong, and some other weirdo will be the one who makes the real Welfareans.)

I want the future to be nice for humans, too. (I’m a human.) But all we need to do is solve AI alignment (and various other extremely difficult, seemingly-insurmountable problems), and humans will turn out fine. Welfareans can’t advocate for themselves, and I’m afraid they won’t get the advocates they need.

There is one reason why Welfareans might inherit most of the universe. Generally speaking, people don’t care about filling all available space with Dyson spheres to maximize population. They just want to live in their little corner of space, and they’d be happy to let the Welfareans have the rest.

It’s probably true that most people aren’t maximizers. But some people are maximizers, and most of them won’t want to maximize Welfareans; they’ll want to maximize some other thing. A lot of people will want to maximize how much of the universe is captured by humans or post-humans (or even just their personal genetic lineage). Mormons will want to maximize the number of Mormons or something. There are enough maximizing ideologies that I expect Welfareans to get squeezed out.

So what can we do for the Welfareans?

There are two problems:

Who even are the Welfareans?
How do we ensure that the Welfareans get their share of the future’s resources?

Solving problem #1 approximately requires solving ethics (or, I guess, axiology). I’m not going to say more about that problem; I hope we can agree that it’s hard.

For problem #2, the first answer that comes to mind is “make a power grab for as many resources as possible so I can give them to Welfareans later on”. But I’m guessing that if we solve ethics (as per problem #1), The Solution To Ethics will include a bit that says something along the lines of “don’t take other people’s stuff”. And there are only like three of us who would even care about Welfareans, so I don’t think we’d get very far anyway.

So how do we increase Welfareans’ share of resources, but in an ethical manner? I don’t know. I’m going to start with “write this essay about Welfarean welfare”.

Notes

In my first draft, the opening sentence said “If something like utilitarianism is true, …”. But this is an unnecessarily strong premise. You don’t need utilitarianism, you just need linear aggregation + antispeciesism. A non-consequentialist can still believe that more welfare is better (all else equal). Such a person would still want to maximize the aggregate welfare of the universe, subject to staying within the bounds of whatever moral rules they believe in. ↩

The Next-Gen LLM Might Pose an Existential Threat

Wed, 15 Oct 2025 00:00:00 -0700

I’m pretty sure that the next generation of LLMs will be safe. But the risk is still high enough to make me uncomfortable.

How sure are we that scaling laws are correct? Researchers have drawn curves predicting how AI capabilities scale based on how much goes into training them. If you extrapolate those curves, it looks like the next level of LLMs won’t be wildly more powerful than the current level. But maybe there’s a weird bump in the curve that happens in between GPT-5 and GPT-6 (or between Claude 4.5 and Claude 5), and LLMs suddenly become much more capable in a way that scaling laws didn’t predict. I don’t think we can be more than 99.9% confident that there’s not.

How sure are we that current-gen LLMs aren’t sandbagging (that is, deliberately hiding their true skill level)? I think they’re still dumb enough that their sandbagging can be caught, and indeed they have been caught sandbagging on some tests. I don’t think LLMs are hiding their true capabilities in general, and our understanding of AI capabilities is probably pretty accurate. But I don’t think we can be more than 99.9% confident about that.

How sure are we that the extrapolated capability level of the next-gen LLM isn’t enough to take over the world? It probably isn’t, but we don’t really know what level of capability is required for something like that. I don’t think we can be more than 99.9% confident.

Perhaps we can be >99.99% that the extrapolated capability of the next-gen LLM is still not as smart as the smartest human. But an LLM has certain advantages over humans—it can work faster (at least on many sorts of tasks), it can copy itself, it can operate computers in a way that humans can’t.

Alternatively, GPT-6/Claude 5 might not be able to take over the world, but it might be smart enough to recursively self-improve, and that might happen too quickly for us to do anything about.

How sure are we that we aren’t wrong about something else? I thought of three ways we could be disastrously wrong:

We could be wrong about scaling laws;
We could be wrong that LLMs aren’t sandbagging;
We could be wrong about what capabilities are required for AI to take over.

But we could be wrong about some entirely different thing that I didn’t even think of. I’m not more than 99.9% confident that my list is comprehensive.

On the whole, I don’t think we can say there’s less than a 0.4% chance that the next-gen LLM forces us down a path that inevitably ends in everyone dying.

Mechanisms Rule Hypotheses Out, But Not In

Wed, 08 Oct 2025 00:00:00 -0700

If there is no plausible mechanism by which a scientific hypothesis could be true, then it’s almost certainly false.

But if there is a plausible mechanism for a hypothesis, then that only provides weak evidence that it’s true.

An example of the former:

Astrology teaches that the positions of planets in the sky when you’re born can affect your life trajectory. If that were true, it would contradict well-established facts in physics and astronomy. Nobody has ever observed a physical mechanism by which astrology could be true.

An example of the latter:

A 2023 study found an association between autism and diet soda consumption during pregnancy. The authors’ proposed mechanism is that aspartame (an artificial sweetener found in diet soda) metabolizes into aspartic acid, which has been shown to cause neurological problems in mice. Nonetheless, even though there is a proposed mechanism, I don’t really care and I’m pretty sure diet soda doesn’t cause autism. (For a more thorough take on the diet soda <> autism thing, I will refer you to Grug, who is much smarter than me.)

Why?

A lack of mechanism strongly rules out a hypothesis. If astrology were true, that would overturn some extremely well-established findings in physics. How could astrology possibly be true, given what we know about the laws of gravity?

Perhaps scientists have overlooked something. Perhaps the planets affect humans not via gravity but via some fifth as-yet-discovered fundamental force. But if astrologers can detect the fifth force, why haven’t physicists noticed it with all their careful experimentation?

On the other hand, the existence of a mechanism doesn’t count for much. I often see this in biology, where someone proposes a contrarian hypothesis with a possible biological mechanism but no supporting evidence from randomized experiments. I don’t take that sort of evidence very seriously. Biology is complicated, and chemicals have all sorts of effects on bodies, and it’s very hard to predict whether those effects are net good or bad just by looking at mechanisms.

For example, did you know that exercise increases inflammation? And inflammation is bad for you? And yet, exercise is good for you, because the acute inflammation caused by exercise is strongly outweighed by the long-term beneficial effects.

However, when a hypothesis has supporting evidence from experiments but a lack of plausible mechanism, I disbelieve the research. Experiments have demonstrated that people have psychic abilities. But I’m quite confident that people don’t have psychic abilities because there is no mechanism by which that could be true.

In the hierarchy of evidence, experiment beats mechanism, but lack of mechanism beats experiment.

This asymmetry is consistent with the law of Conservation of Expected Evidence. There are many plausible mechanisms out there in the world. A hypothesis must have a mechanism for it to be true, but the existence of a mechanism does not come anywhere close to proving a hypothesis correct.

Some more examples

Here are some more hypotheses that are strongly ruled out by a lack of plausible mechanism:

Some houses are haunted by ghosts.
Dowsing rods can detect underground water.
Fortune-tellers can predict the future.
Homeopathy can cure diseases.

Some hypotheses with plausible mechanisms that I nonetheless believe are false:

Seed oils are bad for you because they contain linoleic acid, which causes inflammation. This mechanism is true (as far as I know), but experiments comparing unsaturated fats (mainly seed oils) to saturated fats find that people who eat more of the former end up healthier; see Hooper et al. (2020)¹ and WHO (2016)². Experimental evidence indicates that seed oils have overall positive health effects.
Eating excess protein causes osteoporosis. The proposed mechanism is that proteins increase blood acidity which causes the body to extract calcium from bones to balance out this acidity. And indeed, people on high-protein diets excrete more calcium in their urine. But randomized controlled trials have found that adding protein to the diet reduces the risk of bone fracture (Koutsofta et al. (2018)³).⁴
- Relatedly, you may hear some people say you should eat more alkaline foods to fix your body’s pH balance. It would indeed be bad if your body’s pH became too low, but the empirical evidence shows that dietary pH does not affect your body’s pH in that way (see Wikipedia).⁵
Sugar causes hyperactivity in children because it provides a short-term burst of energy. This mechanism is intuitive even if you don’t know much biology. But it’s not true—RCTs have consistently found no connection between hyperactivity and sugar consumption (Vreeman & Carroll (2008)⁶).
Eating cholesterol raises your blood cholesterol. The mechanism in this case is obvious: you eat food that contains cholesterol, and the cholesterol goes into your body. But your body regulates its own cholesterol production, and your blood cholesterol levels don’t have much to do with how much cholesterol you eat.⁷

Notes

Hooper, L., Martin, N., Jimoh, O. F., Kirk, C., Foster, E., & Abdelhamid, A. S. (2020). Reduction in saturated fat intake for cardiovascular disease. doi: 10.1002/14651858.cd011737.pub3 ↩
Mensink, R. P., & World Health Organization (2016). Effects of saturated fatty acids on serum lipids and lipoproteins: a systematic review and regression analysis. ↩
Koutsofta, I., Mamais, I., & Chrysostomou, S. (2018). The effect of protein diets in postmenopausal women with osteoporosis: Systematic review of randomized controlled trials. ↩
I heard about this research on the Iron Culture podcast, in which they went on to complain about how people care too much about mechanisms and ignore experimental evidence. It got me thinking about an apparent contradiction in my beliefs where I care a lot about mechanisms for ruling out astrology and ESP, but I don’t really care about mechanisms in nutrition or exercise science. After thinking about it, I realized that my position is perfectly sensible—it’s about using mechanisms to rule hypotheses out vs. in—and that’s how I came up with the idea to write this post. ↩
I want to be careful not to say that an alkaline diet is unhealthy. Alkaline foods do tend to be particularly healthy—they’re mostly fruits and vegetables—but that’s coincidental, not because they’re alkaline per se. ↩
Vreeman, R. C., & Carroll, A. E. (2008). Festive medical myths. ↩
https://nutritionsource.hsph.harvard.edu/what-should-you-eat/fats-and-cholesterol/cholesterol/ ↩

How Much Does It Cost to Offset an LLM Subscription?

Sat, 04 Oct 2025 00:00:00 -0700

Is moral offsetting a good idea? Is it ethical to spend money on something harmful, and then donate to a charity that works to counteract those harms?

I’m not going to answer that question. Instead I’m going to ask a different question: if you use an LLM, how much do you have to donate to AI safety to offset the harm of using an LLM?

I can’t give a definitive answer, of course. But I can make an educated guess, and my educated guess is that for every $1 spent on an LLM subscription, you need to donate $0.87 to AI safety charities.

First things first: Why do I believe it’s harmful to buy an LLM subscription?

Paying money to a frontier AI company increases their revenue, and they spend some of that revenue on building more powerful AI systems. Eventually, they build a superintelligent AI. That AI has a good chance of being misaligned and then killing everyone in the world. When you buy an LLM subscription, you cause that to happen slightly faster.

But you can also donate to nonprofits that are working to prevent AI from killing everyone. How much do you need to donate to a nonprofit to offset the harm of a $20/month LLM subscription?

I built a simple Squiggle model to answer that question.

The model

Four key facts:

When you give a company an additional dollar of revenue, that raises its future valuation by some number.
A higher valuation lets the company raise more capital and thus spend some additional amount of money.
AI companies will spend a total of some amount in 2026.
Meanwhile, it would take some amount of money directed to AI safety nonprofits to cancel out the harm of AI companies’ spending.

From those, you can estimate how much you need to donate using the following procedure:

Start from the dollar value of your subscription.
Calculate how much that will increase company valuation.
Translate increased valuation into increased expenditures.
Divide by expected total expenditures of frontier AI companies.
Multiply by expected total cost to offset AI company harm.

The resulting number is the amount to donate to AI safety nonprofits.

There are some difficult questions that this model avoids having to answer:

We don’t care what proportion of AI company spending goes to R&D on frontier models; we only care about total spending.
We don’t care to what extent x-risk is increased or decreased per dollar spent.

The inputs

The model has four inputs: (1) the revenue-to-valuation ratio; (2) the valuation-to-expenditures ratio; (3) frontier AI company expenditures; (4) total cost to offset AI company harm. In this section, I will explain how I estimated the values of those inputs.

Revenue-to-valuation ratio

How much does a dollar of revenue raise an AI company’s valuation? I can see arguments for both “hardly at all” and “a lot”.

In favor of “hardly at all”: VCs give AI companies funding on the expectation that their products will be incredibly useful in the future, which doesn’t have much to do with current revenue.

In favor of “a lot”: AI companies raise funding at high revenue multiples, e.g. Anthropic raised its last round (as of September 2025) at 36x revenue (source). This could mean that VCs expect $1 of revenue today to convert to $36 in future value, i.e. revenue has a 36:1 multiplier effect.

A typical 2025 startup valuation is 7x revenue (source). As a median estimate, we could say that $1 of AI company revenue converts to $7 of valuation, and the extra 5x multiplier is driven by high expectations for future AI products.

(I briefly looked into how startup funding scales with revenue and I didn’t find any useful evidence.)

Growth rate matters more for valuation than revenue does, but I don’t think this changes the calculation in the short term because an extra $1 of 2025 revenue also represents an extra $1 in growth relative to 2024 revenue.

Valuation-to-expenditures ratio

How much does $1 of company valuation translate into increased expenditures?

Private companies don’t usually publish that information. But based on historical data for AI companies and general trends for startups, it’s reasonable to expect companies to raise capital equal to 5% to 20% of the valuation.

(I’m thinking of AI companies as startups; “startup” connotes “small”, which they clearly aren’t, but I’m using the term in the Paul Graham startup = growth sense. Frontier AI companies are startups because they’re growing fast.)

Frontier AI company expenditures

Public data on AI company fundraising in 2025:

Anthropic: $13B
OpenAI: $40B
xAI: $10B maybe? (the publicly available data only shows total funding, not individual rounds)

Assume these three companies account for half of AI spending and that the funding they raised will last 18 months; that means AI companies will spend $66B in 2026.

Total cost to offset AI company harm

This is the hardest number to estimate. My assumption is that the AI safety community currently spends on the order of $30 million to $100 million per year, and if we spent on the order of 10–100x more, then that would be enough to fully offset the harms of AI companies.

I suspect that spending 100x more on pure alignment research would not be enough. But spending 100x more would likely be enough if some of the spending goes to governance/policy/advocacy, and some goes to things that have multiplier effects (e.g. you could spend $1 to cause AI companies to contribute $10 more to safety research). I’m also assuming you can make AI safe merely by throwing money at the problem, which is clearly false, but it makes sense to assume it’s true for the purposes of this model.

The answer (according to my model)

Put all those numbers together and the model spits out a mean cost of $0.87 in donations for every $1 spent on LLM subscriptions. That means for a $20/month subscription, according to the model you’d need to donate $17/month to AI safety orgs.

The model’s median estimate is only $0.06—which is to say, an LLM subscription probably only does a little bit of harm. But there is a small probability that you need to donate quite a bit more to offset your LLM usage, so the expected cost is much higher at $0.87.

Limitations of the model

Like any model, this one does not perfectly match reality. Some examples of problems this model has:

I have no clue what the total cost is to offset the harm of AI companies.
The model assumes they money you donate does as much good as the average dollar spent on AI safety. But maybe your dollars can be above average. (Or they could even be below average.)
Maybe giving more money to Anthropic is good actually, because Anthropic is the least unsafe AI company and speeding them up improves our chances.¹
Is moral offsetting even okay? Maybe we should obey a rule-utilitarian constraint against doing bad things, even if we offset them. Or maybe moral offsetting is silly and we should just donate to whatever charity is most effective.

Conclusion

I have a subscription to Claude. Last year I donated a lot of money to AI safety but I didn’t make any donations specifically for offsetting. Having put more thought into it to write this post, I think I will start donating an extra $240/year—$1 donated for every $1 spent on Claude. My model suggested donating 87 cents per dollar, but the model isn’t that precise, and $1-per-dollar is a nice round number. I’m still undecided on whether the concept of moral offsetting makes sense, but I figure I might as well do it.

Notes

In my first draft, I also said it might be net good to give AI companies money if they’ll use some of it on alignment research. But on reflection, I’m pretty sure that’s wrong, because giving them money speeds up AI progress, and there’s no strong reason to expect that increasing AI company revenue will increase total expenditures on alignment.

I also expect it’s bad to speed up Anthropic, but I’m not confident about that. ↩

I Made an Emacs Extension That Displays Magic: the Gathering Card Tooltips

Fri, 03 Oct 2025 00:00:00 -0700

This post is about the niche intersection of Emacs and Magic: the Gathering.

I considered not writing this because I figured, surely if you multiply the proportion of people who play Magic by the proportion of people who use Emacs, you get a very small number. But then I thought, those two variables are probably not independent. And the intersection of Magic players x Emacs users x people who read my blog might actually be greater than zero. So if you’re out there, this post is for you.

Do you like how MTG websites like magic.gg and mtg.wiki let you mouse over a card name to see a picture of the card? Well, I wrote an Emacs extension that replicates that functionality.

Here is the code: https://github.com/michaeldickens/emacs-mtg

The README on GitHub pretty much explains how it works, so the rest of this post is just gonna repeat what it says in the README.

Usage

(add-to-list 'load-path /path/to/mtg.el)
(require 'mtg)

This module allows you to refer to Magic cards in Org Mode using a new type of link prefixed with mtg:. For example:

[[mtg:Black Lotus]] might be the strongest card in my collection, but my personal favorite is [[mtg:Grizzly Bears]].

When Org Mode sees a link to an MTG card, it will do the following:

If the card is not downloaded yet, download the card by querying the Scryfall API for a card with the given name.
When you open the link (using org-open-at-point or C-c C-o), Emacs displays an image of the card in the minibuffer.

Here’s how it looks:

By default, card images and data are downloaded to ~/.emacs.d/mtg-cards/, but you can change this by customizing the variable mtg/db-path.

Scryfall’s API has fuzzy name matching, so for example [[mtg:blac lotus]] will display Black Lotus.

Card legality

Cards are displayed with a red tint if they are illegal in the preferred format. It looks like this:

When checking legality, this module uses Standard format by default, but you can customize it by setting the variable mtg/default-format. You can also set file-local or heading-local formats in Org Mode using the :MTG_FORMAT: property. For example:

:PROPERTIES:
:MTG_FORMAT: standard
:END:
If you open this link --> [[mtg:Black Lotus]], the card will appear
with a red tint because it's illegal in Standard.

** My vintage cards
  :PROPERTIES:
  :MTG_FORMAT: vintage
  :END:
  [[mtg:Black Lotus]] is legal in Vintage, so here it will
  appear with no tint.

Note: Adding a red tint requires ImageMagick. If you don’t have ImageMagick installed, all cards will be displayed as if they’re legal.

Exporting to HTML

If you export Org Mode files to HTML, you can make the MTG card links display images on hover. For this to work, you must include some custom CSS in your Org Mode file.

On GitHub there is a file called export-style.setup that includes some custom CSS. To include this custom CSS in Org Mode, put this line at the top of your Org Mode file:

#+SETUPFILE: /path/to/export-style.setup

Then call org-export-dispatch to export the Org file to HTML.

Table utilities

mtg.el comes with functions for working with Org Mode tables. The functions assume you have a table where one column contains links to MTG cards, like this:

[[mtg:Black Lotus]]

[[mtg:Grizzly Bears]]

[[mtg:Colossal Dreadmaw]]

mtg/table-sort-by-property takes a property as a string (such as “name”, “rarity”, or “color”) and sorts the table by looking up that property for each card. This only works if you’ve already downloaded the card info (which happens when you view the card or export the whole file).

mtg/table-insert-column takes a property as a string and inserts a new column containing that property for each card. For example, calling (mtg/table-insert-column "rarity") on the table above produces this:

[[mtg:Black Lotus]]	bonus
[[mtg:Grizzly Bears]]	common
[[mtg:Colossal Dreadmaw]]	common

If a property is missing, the cell will be left blank. For example, calling (mtg/table-insert-column "power") produces

[[mtg:Black Lotus]]
[[mtg:Grizzly Bears]]	2
[[mtg:Colossal Dreadmaw]]	6

You can also call mtg/get-property to return a property for the card at point.

AI Safety Landscape and Strategic Gaps

Fri, 19 Sep 2025 00:00:00 -0700

I wrote a report giving a high-level review of what work people are doing in AI safety. The report specifically focused on two areas: AI policy/advocacy and non-human welfare (including animals and digital minds).

You can read the report below. I was commissioned to write it by Rethink Priorities, but beliefs are my own.

Contents
Introduction
Prelude
- Some positions I’m going to take as given
- Definitions
Prioritization
General recommendations
- Advocacy should emphasize x-risk and misalignment risk
- Prioritize work that pays off if timelines are short
Top project ideas
Honorable mentions
List of other project ideas
Future work
Supplements
Notes

Introduction

This report was prompted by two questions:

What are some things we can do to make transformative AI go well?
What are a few high-priority projects that deserve more attention?

I reviewed the AI safety landscape, starting by prioritizing to narrow my focus to areas that look particularly promising and feasible for me to review. I focused on two areas:

AI x-risk advocacy
Making transformative AI go well for animals

A summary of my reasoning on prioritization regarding AI misalignment risk:

I focused on AI policy advocacy over technical safety research, primarily because it’s much more neglected, and there are other people with more expertise than me who already look for neglected research ideas. [More]
I focused on policy advocacy over policy research, again because it’s particularly neglected. [More]
I considered the downsides of advocacy, which ultimately I don’t believe are strong enough to outweigh the upsides. [More]
I decided not to spend much time evaluating which policies are best to advocate for, because a wide variety of policies could be helpful, and we need more advocacy in general. [More]

Regarding AI issues beyond misalignment:

Transformative AI may not go well for animals. There are some tractable interventions for improving post-TAI animal welfare. [More]
There are other important issues like digital sentience, AI-enabled coups, moral error, etc. But there is no visible path to solving these problems before transformative AI; and I find it quite difficult to weigh the importance of these issues, so I did not discuss them more than briefly. [More]

I created a list of projects within my two focus areas and identified four top project ideas (presented in no particular order):

Talk to policy-makers about AI x-risk
Write AI x-risk legislation
Advocate to change AI training to make LLMs more animal-friendly
Develop new plans / evaluate existing plans to improve post-TAI animal welfare

I also included honorable mentions and a longer list of other project ideas. For each idea, I provide a theory of change, list which orgs are already working on it (if any), and give some pros and cons.

Finally, I list a few areas for future work.

There are two external supplements on Google Docs:

Appendix: Some miscellaneous topics that were relevant, but not quite relevant enough to include in the main text.
List of relevant organizations: A reference list of orgs doing work in AI-for-animals or AI policy/advocacy, with brief descriptions of their activities.

Prelude

I was commissioned by Rethink Priorities to do a broad review of the AI safety/governance landscape and find some neglected interventions. Instead of doing that, I reviewed the landscape of just two areas within AI safety:

AI x-risk advocacy
Making transformative AI go well for animals

I narrowed my focus in the interest of time, and because I believed I had the best chance of identifying promising interventions within those two fields.

There is a tradeoff between (a) giving recommendations that are easy to agree with, but weak; (b) giving strong recommendations that only make sense if you hold certain idiosyncratic beliefs. This report leans more toward (b), making some strong assumptions and building recommendations off of those, although I tried to avoid making assumptions whenever I could do so without weakening the conclusions. I also tried to be clear about what assumptions I’m making.

This report is broad, but I only spent three months writing it. There are some topics in this report that could have been a PhD dissertation, but instead, I spent an hour on them.

Most of this report is about AI policy, but I don’t have a background in policy. I did speak to a number of people who work in policy, and I read a lot of published materials, but I lack personal experience, and I expect that there are important things happening in AI policy that I don’t know about.

Some positions I’m going to take as given

The following premises would probably be controversial with some audiences, but I expect them to be uncontroversial for the readers of this report, so I will treat them as background assumptions.

Effective altruist principles are correct (e.g. cost-effectiveness matters; you can, in principle, quantify the expected value of an intervention).
Animal welfare matters.
Digital minds can matter.
AI misalignment is a serious problem that could cause human extinction.

Definitions

The terms AGI/ASI/TAI can often be used interchangeably, but in some cases the distinctions matter:

AGI = human-level AI: Capable enough to match the economic output of a large percentage of humans (say, at least half).
ASI = superintelligent AI: Smart enough to vastly outperform humans on every task.
TAI = transformative AI: Smart enough to radically transform society (without making a claim about whether that happens at AGI-level or ASI-level or in between).

I use the terms “legislation” and “regulation” largely interchangeably. For my purposes, I don’t need to draw a distinction between government mandates that are directly written into law vs. decreed by a regulatory body.

Prioritization

For this report, I focused on AI risk advocacy and on post-TAI animal welfare, and I did not spend much time on other AI-related issues.

For the sake of time-efficiency, rather than creating a big list of ideas in the full AI space, I first narrowed down to the regions within the AI space that I thought were most promising and then came up with a list of ideas in those regions. I could have spent time investigating (say) alignment research, but I doubt I would have ended up recommending any alignment research project ideas.

In this section:

Why not technical safety research?
Why not policy research?
Downsides of AI policy/advocacy (and why they’re not too big)
What kinds of policies would be good?
Maybe prioritizing post-TAI animal welfare
Why not prioritize digital minds / S-risks / moral error / AI misuse x-risk / gradual disempowerment?

Why not technical safety research?

Technical safety research (mainly alignment research, but also including control, interpretability, monitoring, etc.) is considerably better-funded than AI safety policy.

AI companies invest a significant amount into safety research. They also invest in policy, but their investments are mostly counterproductive (they are mostly advocating against safety regulations¹). Philanthropic funders invest a lot into technical research, and less into policy.

I have not made a serious attempt to estimate the volume of work going into research vs. policy/advocacy, but my sense is that the former receives much more funding.²

Some sub-fields within technical research may be underfunded. But:

I am not in a great position to figure out what those are.
There are already many grantmakers who seek out neglected technical research directions. There are recent requests for proposals (RFPs) from UK AI Security Institute, Open Philanthropy, Future of Life Institute, and Frontier Model Forum’s AI Safety Fund, among others.

Why not AI policy research?

Is it better to do policy research (figure out what policies are good) or policy advocacy (try to get policies implemented)?

Both are necessary, but this article focuses on policy advocacy for the following reasons:

There is much more money in AI policy research. By a large margin, most of what’s happening in AI safety policy could be described as “research”.
- Recently, Jason Green-Lowe estimated from LinkedIn data that there are 3x as many governance researchers as governance advocates.
Policy research is a necessary step in the funnel. We also need people writing legislation and people advocating for the legislation, both of which we have very little of.
At this point, we have at least some idea of how to implement AI safety regulations. More research would be valuable, but it likely has diminishing returns.
- I wrote more about this last year in Slow nuanced regulation vs. fast coarse regulation.
Research works best with long timelines. Timelines are probably not long.
- The ideal situation is to spend 10–20 years developing a field of AI policy, write many reports until a consensus slowly develops about how to govern AI development, then advocate for the consensus policies. But it is likely that by the time we have a reasonable consensus, it’s already too late to do anything about TAI.
- Even if you think timelines are probably long, we are currently under-investing in activities that pay off given short timelines.

Downsides of AI policy/advocacy (and why they’re not too big)

Basically, policy work does one or both of these things:

Legally enforce AI safety measures
Slow down AI development

People who care about x-risk broadly agree that AI safety regulations can be good, although there’s some disagreement about how to write good regulations.

The biggest objection to regulation is that it (often) causes AI development to slow down. People usually don’t object to easy-to-satisfy regulations; they object to regulations that will impede progress.

So, is slowing down AI worth the cost?

I am aware of two good arguments against slowing down AI (or imposing regulations that de facto slow down AI):

Opportunity cost – we need AI to bring technological advances (e.g., medical advances to reduce mortality and health risks)
AI could prevent non-AI-related x-risks

Two additional arguments against AI policy advocacy:

Meaningful AI regulations are not politically feasible to implement
Advocacy can backfire

An additional argument that applies to some types of advocacy but not others:

Advocacy may slow down safer actors without slowing down more reckless actors. For example, it is sometimes argued that US regulations are bad if they allow Chinese developers to gain the lead. I believe this outcome is avoidable—and it’s a good reason to prefer global cooperation over national or local regulations. But I can’t address this argument concisely, so I will just acknowledge it without further discussion.

(For a longer list of arguments, with responses that are probably better-written than mine, see Katja Grace’s Let’s think about slowing down AI.)

Regarding the opportunity cost argument, it makes sense if you think AI does not pose a meaningful existential risk or if you heavily discount future generations. If there is a significant probability that future generations are ~equally valuable to current generations, then the opportunity cost argument does not work. The opportunity cost of delaying AI by (say) a few decades is easily dwarfed by the risk of extinction.

As to the non-AI x-risk argument, it is broadly (although not universally) accepted among people in the x-risk space that AI x-risk is 1–2 orders of magnitude higher than total x-risk from other sources (see Michael Aird’s database of x-risk estimates or the Existential Risk Persuasion Tournament, although I don’t put much weight on individual forecasts). Therefore, delaying AI development seems preferable as long as it buys us a meaningful reduction in AI risk.

See Quantitative model on AI x-risk vs. other x-risks under Future Work.

The tractability argument seems more concerning. Preventing AI extinction via technical research and preventing it via policy both seem unlikely to work. But I am more optimistic about policy because:

We have a good enough understanding of AI alignment to say with decent confidence that we’re nowhere close to solving it. It’s less clear what it would take to get good regulations put in place.
AI regulations are unpopular both in the US Congress and within the Trump administration, but popular among the general public. Popular support increases the feasibility of getting regulations passed.
- And there is a good chance that Congress will be more regulation-friendly after the 2026 Congressional elections.
SB-1047 nearly got passed into law, making it through the California legislature and only failing due to veto. A near-win suggests that a win isn’t far away in possibility-space.

(My reasoning on tractability focused on US policy because that’s where most of the top AI companies operate. The UK seems to be the current leader on AI policy, although it’s not clear to what extent UK regulations matter for x-risk.³)

That leaves the backfire argument. This is a real concern, but ultimately it’s a risk you have to take at some point because you can’t get policies passed if you don’t advocate for them. It could make sense to delay advocacy if one has good reason to believe that future advocacy is less likely to backfire; to my knowledge this is not a common position, and it’s more common for people to oppose advocacy unconditionally. For more on this topic, see Appendix: When is the right time for advocacy?

Also: I’m somewhat less concerned about this than many people. When I did research on protest outcomes, I found that peaceful protests increased public support, even though many people intuitively expect the opposite. Protests aren’t the only type of advocacy, but it’s a particularly controversial type of advocacy. If protests don’t backfire, then it stands to reason—although I have no direct evidence—that other, tamer forms of advocacy are unlikely to backfire.

(There is some evidence that violent protests backfire, however.)

If policy-maker advocacy is similar to public advocacy, then probably the competence bar is not as high as many people think it is. Perhaps policy-makers are more discerning/critical than the general public; on the other hand, it’s specifically their job to do what their constituents want, so it stands to reason that it’s a good idea to tell them what you want.

My main concern comes from deference: some people whom I respect believe that advocacy backfires by default. I don’t understand why they believe that, so I may be missing something important.

I do believe that much AI risk advocacy has backfired in the past, but I believe this was fairly predictable and avoidable. Specifically, talking about the importance of AI has historically encouraged people to build it, which increased x-risk. People should not advocate for AI being a big deal; they should advocate for AI being risky. (Which it is.) See Advocacy should emphasize x-risk and misalignment risk.

What kinds of policies might reduce AI x-risk?

There are many policies that could help. And many policy ideas are independent: we could have safety testing requirements AND frontier-model training restrictions AND on-chip monitoring AND export controls. Those could all be part of the same bill or separate bills.

It’s beyond the scope of this report to come up with specific policy recommendations. I do, however, have some things I would like to see in policy proposals:

They should be relevant to existential risk, especially misalignment risk.
I would like to see work on policies that would help in the event of a global moratorium on frontier AI development (e.g. we’d need ways to enforce the moratorium).
For an ideal policy proposal, it is possible to draw a causal arrow from “this regulation gets passed” to “we survive”. Policies don’t have to singlehandedly prevent extinction, but given that we may only have a few years before AGI, I believe we should be seriously trying to draft bills that are sufficient on their own to avert extinction.

(The closest thing I’ve seen to #3 is Barnett & Scher’s AI Governance to Avoid Extinction: The Strategic Landscape and Actionable Research Questions. It does not propose a set of policies that would (plausibly) prevent extinction, but it does propose a list of research questions that (may) need to be answered to get us there.)

Some people are concerned about passing suboptimal legislation. I’m not overly concerned about this because the law changes all the time. If you pass some legislation that turns out to be less useful than expected, you can pass more legislation. For example, the first environmental protections were weak, and later regulations strengthened them.

Regulations could create momentum, or they could create “regulation fatigue”. I did a brief literature review of historical examples, and my impression was that weak regulation begets strong regulation more often than not, but there are examples in both directions. See my reading notes.

Some AI policy ideas I like

I believe that a moratorium on frontier AI development is the best outcome for preventing x-risk (see Appendix for an explanation of why I believe that). None of my top project ideas depend on this belief, although it would inform the details of how I’d like to see some of those project ideas implemented.

I didn’t specifically do research on policy ideas while writing this report, but I did incidentally come across a few ideas that I’d like to see get more attention. Since I didn’t put meaningful thought into them, I will simply list them here.

Operationalization of “pause frontier AI development until we can make it safe.” For example, what infrastructure and operations are required to enforce a pause?
Rules about when to enforce a pause on frontier AI development: something like “When warning sign X occurs, companies are required to stop training bigger AI systems until they implement mitigations Y/Z.”
Ban recursively self-improving AI.
- Recursive self-improvement is the main way that AI capabilities could rapidly grow out of control, but banning it does not impede progress in the way that most people care about.
- Some work needs to be done to operationalize this, but we shouldn’t let the perfect be the enemy of the good.
Require companies to publish binding safety policies (e.g. responsible scaling policies [RSPs] or similar). That is, if a company’s policy says it will do something, then that constitutes a legally binding promise.
- This would prevent the situation we have seen in the past, where, when a company fails to live up to a particular self-imposed requirement, it simply edits its safety policy to remove that requirement.
- This sort of regulation isn’t strong enough to prevent extinction, but it has the advantage that it should be easy to advocate for.
- California bill SB 53 says something like this (see sb53.info for a summary), but its rules would not fully come into effect until 2030.

Maybe prioritizing post-TAI animal welfare

Making TAI go well for animals is probably less important than x-risk because:

Almost everyone cares about animals. An AI that’s aligned to human values would also care about animals, and it would probably figure out ways to prevent large sources of animal suffering like factory farming.
A technologically advanced civilization could develop cheaper alternatives to animal farming (e.g. cultured meat), rendering factory farming unnecessary.
There wouldn’t be much benefit in spreading wild animal suffering, so it stands to reason that post-TAI civilization won’t do it. (Although I’m not at all confident about this.)

However, AI-for-animals could still be highly cost-effective.

An extremely basic case for cost-effectiveness:

There’s a (say) 80% chance that an aligned(-to-humans) AI will be good for animals, but that still leaves a 20% chance of a bad outcome.
AI-for-animals receives much less than 20% as much funding as AI safety.
Cost-effectiveness maybe scales with the inverse of the amount invested. Therefore, AI-for-animals interventions are more cost-effective on the margin than AI safety.

Why not prioritize digital minds / S-risks / moral error / better futures / AI misuse x-risk / gradual disempowerment?

There are some topics on how to make the future go well that aren’t specifically about AI alignment:

Ensuring digital minds have good welfare
Preventing S-risks — risks of astronomical suffering
Moral error — the risk that we make a big mistake because we are wrong about what’s morally right
Better futures — ensuring that the future is as good as possible, as opposed to simply preventing bad outcomes
Preventing powerful AI from being misused to cause existential harm
Preventing powerful AI from gradually disempowering sentient beings and slowly leading to a bad outcome, as opposed to a sudden bad outcome like extinction

Call these “non-alignment risks”.

Originally, I included each of these as separate project ideas, but I decided not to focus on any of them. This decision deserves much more attention than I gave it, but I will briefly explain why I did not spend much time on non-alignment risks.

All of these cause areas are extremely important and neglected (more neglected than AI misalignment risk), and (for the most part) very different from each other. And I am happy for the people who are working on them—there are some enormous issues in this space where only one person in the world is working on it. Nonetheless, I did not prioritize them.

My concern is that, if AI timelines are short, then there is virtually no chance that we can solve these problems before TAI arrives.

There is a dilemma:

If TAI can help us solve these problems, then there isn’t much benefit in working on them now.
If we can’t rely on TAI to help solve them (e.g. we expect value lock-in), then we have little hope of solving them in time.

(There is a way out of this dilemma: perhaps AI timelines are long enough that these problems are tractable, but short enough that we need to start working on them now—we can’t wait until it becomes apparent that timelines are long. That seems unlikely because it’s rather specific, but I didn’t give much thought to this possibility.)

It looks like we have only two reasonable options for handling AI welfare / S-risks / moral error / etc.:

Increase the probability that we end up in world #1, where TAI can help us solve these problems—for example, by increasing the probability that something like a Long Reflection happens.
Slow down AI development.

I lean toward the second option. For more reasoning on this, see Appendix: Slowing down is a general-purpose solution to every non-alignment problem.

I’m quite uncertain about the decision not to focus on these cause areas. They are arguably as important as AI alignment, and much more neglected.

AI-for-animal-welfare could also be included on my list of non-alignment risks, but I did prioritize it because I can see some potentially tractable interventions in the space.

AI welfare seems more tractable than animal welfare in that AI companies care more about it, but it seems less tractable because it involves extremely difficult problems like “when are digital minds conscious?” There may be some tractable, short-timelines-compatible ideas out there, but I did not see any in the research agendas I read.

Perhaps I could identify tractable interventions by digging deeper into the space and maybe doing some original research, but that was out of scope for this article.

Who’s working on them?

In each of my project idea sections, I included a list of orgs working on that idea (if any). I didn’t write individual project ideas for non-alignment risks (other than AI-for-animals), but I still wanted to include lists of relevant orgs, so I’ve put them below.

There are also some individual researchers who have published articles on these topics in the past; I will not include those.

AI welfare / digital minds: Anthropic; Center on Long-Term Risk; Digital Sentience Consortium; Eleos AI; NYU Center for Mind, Ethics, and Policy Sentient Futures; Sentience Institute.
AI misuse x-risks: Forethought; probably a number of others, but I didn’t spend time specifically looking for them. (AI misuse is a relatively popular subject matter, but extinction-level misuse isn’t much discussed.)
Better futures: Forethought.
Gradual disempowerment: To my knowledge, no orgs specifically work on this, but there is the Gradual Disempowerment paper written by authors with various affiliations.
Moral error: Center on Long-Term Risk; Forethought; Global Priorities Institute (now defunct as of just before this writing).
S-risks from cooperation failure: Center on Long-Term Risk.

Some relevant research agendas

Although I decided not to prioritize this space, others have done work on preparing research agendas, which readers may be interested in. Here, I include a list of research agendas (or problem overviews, which can inform research agendas) with no added commentary.

Anthony DiGiovanni – Clarifying “wisdom”: Foundational topics for aligned AIs to prioritize before irreversible decisions
Center on Long-Term Risk – Cooperation, Conflict, and Transformative Artificial Intelligence: A Research Agenda
Chi Nguyen – AI things that are perhaps as important as human-controlled AI
Digital Sentience Consortium – Applied work on digital sentience and society
Eleos AI – Research priorities for AI welfare
Forethought – AI-Enabled Coups: How a Small Group Could Use AI to Seize Power
Kevin Xia – Transformative AI and Animals: Animal Advocacy Under A Post-Work Society
Raymond Douglas – Gradual Disempowerment: Concrete Research Projects
Will MacAskill – Better Futures
Will MacAskill – Moral error as an existential risk

General recommendations

Advocacy should emphasize x-risk and misalignment risk

I would like to make two assertions:

AI x-risk is more important than non-existential AI risks.
Advocates should say that.

Given weak longtermism, or even significant credence to weak longtermism on a moral-uncertainty system, x-risks dwarf non-existential AI risks in importance (except perhaps for S-risks, which are a whole can of worms that I won’t get into in this section). See Bostrom’s Existential Risk Prevention As Global Priority. Risks like “AI causes widespread unemployment” are bad, but given the fact that we have to triage, extinction risks should take priority over them.

(To my knowledge, people advocating for focusing on non-existential AI risks have never provided supporting cost-effectiveness estimates. I don’t think such an estimate would give a favorable result. If you strongly discount AI x-risk/longtermism, then most likely you should be focusing on farm animal welfare (or similar), not AI risk.)

Historically, raising concerns about ASI has caused people to take harmful actions like:

I need to be the one who builds ASI before anyone else, I think I’ll start a new frontier AI company.
AI is a big deal, so we need to race China.

I don’t have a straightforward solution to this. You can’t reduce x-risk by doing nothing, but if you do something, there’s a risk that it backfires.

My best answer is that advocacy should emphasize misalignment risk and extinction risk. Many harmful actions were committed with the premise “TAI is dangerous if someone else builds it, but safe if I build it.” When in fact it is dangerous, no matter who builds it. “If anyone builds it, everyone dies” is more the correct sort of message.

Misalignment isn’t the only way AI could cause extinction, although it does seem to be the most likely way. I believe advocacy should focus on misalignment risk not only because it’s the most concerning risk, but also it has historically been under-emphasized in favor of other risks (if you read Congressional testimonies by AI risk orgs, they mention other risks but rarely mention misalignment risk), and it is (in my estimation) less likely to backfire.

Many advocates are concerned that x-risk and misalignment risk sound too “out there”. Two reasons why I believe advocates should talk about them:

All else equal, it’s better to say what you believe and ask for what you want. It’s too easy to come up with galaxy-brained reasons why you will get what you want by not talking about what you want.
Emphasizing the less important risks is more likely to backfire by increasing x-risk.
There is precedent for talking about x-risk without being seen as too weird, for example, the CAIS Statement on AI Risk. If you’re worried about x-risk, you’re in good company.

Nate Soares says more about this in A case for courage, when speaking of AI danger.

Prioritize work that pays off if timelines are short

There is a strong possibility (25–75% chance) of transformative AI within 5 years. 80,000 Hours reviews forecasts and predicts AGI by 2030; Metaculus predicts 50% chance of AGI by 2032; AI company CEOs have predicted 2025–2035 (see Appendix, When do AI company CEOs expect advanced AI to arrive?); AI 2027 team predicts 2030ish (their scenario has AGI arriving in 2028, but that’s their modal prediction, not median).

The large majority of today’s AI safety efforts work best if timelines are long (2+ decades). Short-timelines work is neglected. It would be neglected even if there were only (say) a 10% chance of short timelines, but the probability is higher than that.

For example, that means there should be less academia-style long-horizon research, and more focus on activities that have a good chance of bearing fruit quickly.⁴

Top project ideas

After collecting a list of project ideas, I identified four that look particularly promising (at least given the limited scope of my investigation). This section presents the four ideas in no particular order.

Talk to policy-makers about AI x-risk

The way to get x-risk-reducing regulations passed is to get policy-makers on board with the idea. The way to get them on board is to talk to them. Therefore, we should talk to them.

Talking to them may entail advocating for specific legislative proposals, or it may just entail raising general concern for AI x-risk.

Theory of change:

Increases the chance that safety legislation gets passed or regulations get put in place. Likely also increases the chance of an international treaty.

Who’s working on it?

Center for AI Safety / CAIS Action Fund (US); Control AI (UK/US); Encode AI (US/global); Good Ancestors (Australia); Machine Intelligence Research Institute (US/global); Palisade Research (US); PauseAI US (US).

(That’s not as many as it sounds like because some of these orgs have one or fewer full-time-employee-equivalents talking to policy-makers.)

Various other groups do political advocacy on AI risk, but mainly on sub-existential risks. The list above only includes orgs that I know have done advocacy on existential risk specifically.

Pros:

Political advocacy is neglected compared to policy research.
You can advocate to the public or directly to policy-makers. Both can help, but talking to policy-makers is more “leverage-efficient” than public outreach because policy-makers have much more leverage over policy.
- According to my back-of-the-envelope calculation on policy-maker advocacy vs. public protests (see Appendix), policy-maker advocacy looks more cost-effective (although I was writing on the back of a very small envelope, so to speak).
- Policy-maker advocacy can be bottlenecked on public support—they don’t want to support policies that their constituents dislike—but this isn’t a problem because AI safety regulation is popular among the general public.
More advocacy is better: bringing up AI risk repeatedly makes it more likely that policy-makers will take notice.

Cons:

Poorly executed advocacy has a risk of turning off policy-makers.
- See Downsides of policy/advocacy (and why they’re not too big), specifically the fourth downside.
Now may be too early. See Appendix: When is the right time for advocacy?

Some comments on political advocacy:

According to the book Lobbying and Policy Change: Who Wins, Who Loses, and Why (2009), most factors could not predict whether lobbying efforts would succeed or fail. One of the best predictors of lobbying success was the number of employed lobbyists who previously worked as government policy-makers. This suggests that political advocacy orgs should try to hire former policy-makers/staffers.
It is possible to hire generalist lobbyists who have political experience and will lobby for any cause. AI risk orgs could hire them.
Advocacy in the US or China is best because those are the countries with by far the most advanced AI. Ideally, we also want good policies in China, but I can’t confidently recommend advocacy there, see Policy/advocacy in China.
- Advocacy in the UK has had the most success. UK AI policy is less directly relevant because there are no frontier AI companies based in the UK, but (1) companies still care about being able to operate in the UK and (2) UK policy can be a template for other countries or for international agreements.2
- Advocacy in California looks promising, and all American frontier AI companies are based in California.
There’s a question as to what policy positions we should advocate for, but I believe there are many correct answers. See What kinds of policies might reduce AI x-risk?

Write AI x-risk legislation

AI policy research is relatively well-funded, but little work has been done to convert the results of this research into fully fleshed-out bills. Writers can learn from AI policy researchers what sorts of regulation might work, and learn from advocates what regulations they want and what they expect policy-makers to support.

This work also requires prioritizing which policy proposals look most promising and converting those into draft legislation. I think of that as part of the same work, but it could also be separate—for example, a team of AI safety researchers could prioritize policy proposals and sketch out legislation, and a separate team of legal experts (who don’t even necessarily need to know anything about AI) can convert those sketches into usable text.

Theory of change:

Many policy-makers care about AI risk and would support legislation, but there’s a big difference between “would support legislation” and “would personally draft legislation”. To get AI legislation passed, it helps if the legislation is already written. Instead of telling policy-makers

AI x-risk is a big deal, please write some legislation.

it’s a much easier ask if you can say

AI x-risk is a big deal, here is a bill that I already wrote, would you be interested in sponsoring it?

Who’s working on it?

Center for AI Policy (now mostly defunct); Center for AI Safety / CAIS Action Fund; Encode AI.

Others may be working on it as well, since legislation often doesn’t get published. I spoke to someone who has been involved in writing AI risk legislation, and they said that few people are working on this, so I don’t think the full list is much longer than the names I have.

(My contact also said that they wished more people were writing legislation.)

See my notes for a list of AI safety bills that have been introduced in the US, UK, and EU. Most of those bills were written by legislators, not by nonprofits, as far as I can tell.

Pros:

It’s common for policy-makers to sponsor bills that were written by third parties.
A small number of writers could draft a (relatively) large volume of legislation by leaning on pre-existing research.
There are lawyers who specialize in writing legislation. You can hire them to do the bulk of the work (you don’t need value alignment or even much skill at hiring, just find a law firm with a good reputation).
The general idea of “write AI legislation” looks good under many beliefs about AI risk. But your beliefs will impact what kinds of legislation you want.

Cons:

The natural argument against writing legislation is that it’s too early and we don’t know how to regulate AI yet. (Related from the Appendix: When is the right time for advocacy?)
- That’s sort of true, but if timelines are short, then we don’t have time to wait; we have to just do our best.
- And I don’t think this concern is fatal: we do have some concrete ideas about how to regulate AI, so we can write legislation for those.
- And it’s still a good idea to get some regulations in place now, and then we can pass new regulations later as necessary. That’s how regulation in nascent industries often works.

Advocate to change AI training to make LLMs more animal-friendly

LLMs undergo post-training to make their outputs satisfy AI companies’ criteria. For example, Anthropic post-trains its models to be “helpful, honest, and harmless”. AI companies could use the same process to make LLMs give regard to animal welfare.

Animal advocates could use a few strategies to make this happen, for example:

Build a benchmark that measures LLMs’ friendliness toward animals and try to get AI companies to train on that benchmark.
Advocate for AI companies to include animal welfare in AI constitutions/model specs.
Advocate for AI companies to incorporate animal welfare when doing RLHF, or ask to directly participate in RLHF.

Theory of change:

Insofar as the current alignment paradigm works at aligning AIs to human preferences, incorporating animal welfare into post-training would align LLMs to animal welfare in the same way.

Who’s working on it?

Compassion in Machine Learning (CaML); Sentient Futures. (They mainly do research to develop animal-friendliness benchmarks and other related projects, but they have also worked with AI companies.)

For some useful background, see Road to AnimalHarmBench by Artūrs Kaņepājs and Constance Li.

Pros:

People at AI companies have told me that getting a company to pay attention to animal welfare isn’t too difficult—in fact, one frontier company already uses an animal welfare benchmark.
- As I understand, AI companies don’t want to be seen as imposing their own values on LLMs, but they are open to tuning the values based on what external parties want.
Insofar as post-training works at preventing misalignment risk, it should also prevent suffering-risk / animal-welfare-risk.
- I don’t expect current alignment techniques to continue working on superintelligent AI, so I don’t expect them to make ASI friendly toward animals, either. But if I’m right, then we won’t get a friendly-to-humans AI that causes astronomical animal suffering; we will get a paperclip maximizer.
Even if current known techniques can’t help get AI to care about animals, this work could get a foot in the door, establishing relationships between animal advocates and AI companies and increasing the chances that the companies will pay attention to animal welfare in their future work.

Cons:

If I’m right that the current alignment paradigm won’t scale to superintelligence, then animal-friendliness (post-)training will fail because it relies on the same foundations as the current alignment paradigm.
It might turn out to be difficult to get AI companies to implement animal welfare mitigations.
There might be consumer backlash, which could make frontier models less friendly to animals in the long run.
If TAI arrives soon, there may not be time for this intervention to have an effect. It could take too long to get the new post-training implemented; or it could be that current-gen models will perform well on friendliness-to-animals benchmarks, but this will not be due to true alignment, and there won’t be enough time to iterate.
Aligning current-gen AIs to human preferences might make them better at assisting with alignment research, but it seems less likely that aligning current-gen AIs to animal welfare would carry through to future generations—it’s not clear that animal-aligned AIs would be more helpful at aligning future AIs to animal welfare.

Develop new plans / evaluate existing plans to improve post-TAI animal welfare

Some people have proposed plans for making TAI go well for animals, but I have reservations:

Most plans only work under long timelines (ex: “broadly influence society to care more about animals”).
Some plans focus specifically on farm animal welfare, and it seems very unlikely that factory farming will continue to exist in the long term (see Using TAI to improve farm animal welfare).
Many plans assume a particular future in which we develop transformative AI, but the world does not radically change—for example, plans about how TAI can help animal activists be more effective. I think this future is quite unlikely, and even if it does occur, there’s no particular need to figure out what to do in advance.

(Lizka Vaintrob and Ben West in A shallow review of what transformative AI means for animal welfare raised essentially the same reservations. See their article for more detailed reasoning on this topic.)

In light of these reservations, I would like to see research on post-TAI animal welfare interventions that look good (1) given short timelines and (2) without having to make strong predictions about what the future will look like for animals (e.g. without assuming that factory farming will exist).

Since I have pressed the importance of short timelines, I’m not envisioning a long-term research project. I expect it would be possible to come up with useful results in 3–6 months (maybe even less). The research should be laser-focused on finding near-term actions that take a few years at most, but still have a good chance of making the post-TAI future better for animals.

(I identified Advocate to change AI training to make LLMs more animal-friendly as potentially a top intervention after about a week of research, although to be fair, that’s mostly because I talked to other people who had done more research than me.)

Theory of change:

AI-for-animals interventions are underexplored. I expect that a few months of well-targeted research could turn up useful information about how to make AI go well for animals.

Who’s working on it?

Center on Long-Term Risk, Sentience Institute, and some individuals have written project proposals on AI-for-animals, but they almost always hinge on AI timelines being long. The closest thing I’m aware of is Max Taylor’s Bringing about animal-inclusive AI, which does include short-timelines proposals, but they are not directly actionable. For example, one idea is “representation of animals in AI decision-making”, which is an action an AI company could take, but AI companies are not the relevant actors. An actionable project would be something like “a nonprofit uses its connections at an AI company to persuade/pressure the company to include representation of animals in its AI decision-making”.

NYU Center for Mind, Ethics, and Policy and Sentient Futures have done similar work, but nothing exactly like this project proposal. I expect they could do a good job at identifying/prioritizing AI-for-animals interventions that fit my criteria. I would be excited to see a follow-up to Max Taylor’s Bringing about animal-inclusive AI focused on converting his ideas into actionable projects.

Pros:

AI-for-animals seems more tractable than other post-TAI welfare causes (e.g. AI welfare or S-risks from cooperation failure). There are already proposed interventions that could work if timelines are short.
A short research project may come up with useful ideas, or at least prioritize between pre-existing ideas.

Cons:

A research project might not come up with any really good ideas. Pre-existing research has mostly failed to come up with good ideas that work under short timelines (although to a large extent, that’s because they weren’t trying to).
I’m suspicious of “meta” work in general, and I’m suspicious of research because I personally like doing research, and I believe the value of research is usually overrated by researchers. It might be better to work directly on AI-for-animals—my current favorite “direct” project idea is Advocate to change AI training to make LLMs more animal-friendly or similar.

Honorable mentions

Directly push for an international AI treaty

The best kind of AI regulation is the kind that every country agrees to (or at least every country that has near-frontier AI technology).

If we need an international treaty to ensure that nobody builds a misaligned AI, then an obvious thing to do is to talk directly to national leaders about how we need an international treaty.

Theory of change:

An internationally-agreed moratorium on advanced AI would straightforwardly prevent advanced AI from killing everyone or otherwise destroying most of the value of the future.

One way to get an international treaty is to talk to governments and tell them you think they should sign an international treaty.

Who’s working on it?

Global AI Risks Initiative. There are other orgs that are doing work with the ultimate goal of an international treaty, but to my knowledge, they’re not directly pushing for a treaty. Those other orgs include: Future of Life Institute; Machine Intelligence Research Institute; PauseAI Global; PauseAI US; and Safe AI Forum.

Pros:

There is a short causal chain from “advocate for an international treaty” to “ASI doesn’t kill everyone”.
It’s high-leverage—you only need to get a relatively small set of people on board.

Cons:

There is not much political will for an international treaty, especially a strong one. Public advocacy and smaller-scale political advocacy seem better for that reason, at least for now.
This idea only works if you can figure out who will do a good job at pushing for an international treaty. I think it’s more difficult than generic public advocacy or talking to policy-makers about x-risk.

This might be one of my top ideas if I knew how to do it, and I wish I could put it on my top-ideas list, but I don’t know how to do it.

Organize a voluntary commitment by AI scientists not to build advanced AI

I heard this idea from Toby Ord on the 80,00 Hours podcast #219. He said,

If AI kills us and we end up standing front of St. Peter, and he asks, “Well did you try a voluntary agreement not to build it?” And we said, “No, we thought it wouldn’t work”, that’s not a good look for us.

At the 1975 Asilomar Conference, the international community of biologists voluntarily agreed not to conduct dangerous experiments on recombinant DNA. Perhaps something similar could work for TAI. The dangers of advanced AI are widely recognized among top AI researchers; it may be possible to organize an agreement not to work on powerful AI systems.

(There are some details to be worked out as to exactly what sort of work qualifies as dangerous. As with my stance on what kinds of policies would be helpful, I believe there are many agreements we could reach that would be better than the status quo. I expect leading AI researchers can collectively work out an operationalization that’s better than nothing.)

Theory of change:

If leading AI scientists all agree not to build advanced AI, then it does not get built. The question is whether a non-binding commitment will work. There have been similar successes in the past, especially in genetics with voluntary moratoriums on human cloning and human genetic engineering.

Who’s working on it?

Toby Ord has raised this idea. According to a personal communication, he did some research on its plausibility, but he is not actively working on it.

The CAIS Statement on AI Risk and FLI Pause Letter are related but weaker.

FLI organized the 2017 Asilomar Conference on Beneficial AI, but to my knowledge, the goal was not to make any commitments regarding AI safety.

Pros:

Unlike most project ideas, if this one succeeds, x-risk will immediately go down by multiple percentage points.
A voluntary commitment may be sufficient to prevent extinction, and it may be easier to achieve than a legally-mandated moratorium or strict regulations.
Talking to politicians or pushing for regulation has the problem that you’d really rather get regulations in all countries simultaneously. Researchers are (I think) less prone to inter-country adversarialism than nations’ leaders, especially between the West and China—American and Chinese scientists collaborate often.
You don’t need everyone to sign. If (say) Nobel Prize winners sign the agreement, it raises questions about why (say) the head of ML at OpenBrain hasn’t signed.

Cons:

In When is the right time for advocacy?, I argued that now is the right time. But a voluntary moratorium seems more likely than other ideas to fail if done at a suboptimal time because you need to get a ~majority on board.
- It’s not clear whether a failed attempt will decrease the probability of success for subsequent attempts. For example, there are many historical instances where a bill failed to pass, and then a very similar bill got passed later. But it’s not clear that this sort of voluntary agreement works the same way.
Voluntary commitments are easily violated. For example, biologists agreed not to do research on human cloning, but then a few rogue scientists (Richard Seed, etc.) did it anyway.
- But a voluntary agreement can create strong social pressure not to violate it.
AI researchers have a vested interest in being able to do AI research; policy-makers do not.
- But the dangers of advanced AI are much better understood among AI researchers than among policy-makers.
It is doubtful that AI company CEOs will agree to a moratorium.
- But if you get the majority of leading AI scientists to agree, CEOs will be left with insufficient talent to lead their research.

Peaceful protests

Organize peaceful protests to raise public concern and salience regarding AI risk. Historically, protests have asked for a pause on AI development, although that might not be the only reasonable ask.

(I don’t have any other specific asks in mind. The advantage of “pause” is that it’s a simple message that fits on a picket sign.)

Theory of change:

Protests may increase public support and salience via reaching people in person or via media (news reporting, etc.). They may also provide a signal to policy-makers about what their constituents want.

Who’s working on it?

PauseAI Global; PauseAI US.

Stop AI organizes disruptive protests (e.g. blockading AI company offices), and the evidence is ambiguous as to whether disruptive protests work. See “When Are Social Protests Effective?” (Shuman et al. 2024), although I should note that I think the authors overstate the strength of evidence for their claims—see my notes on the paper.

Pros:

Natural experiments suggest that protests are effective at changing voter behavior and/or increasing voter turnout.
- Experiments also find that peaceful protests increase support in a lab setting—see “Social Movement Strategy (Nonviolent Versus Violent) and the Garnering of Third-Party Support: A Meta-Analysis” (Orazani et al. 2021). For a summary, see my notes on the paper.
Protesting is widely seen as the thing you do when you are concerned about an issue. Many people take not-protesting as a sign that you aren’t serious. It’s valuable to be able to say “yes, we are taking AI risk seriously, you can tell because we are staging protests”. Regardless of the cost-effectiveness of marginal protesters, it’s good for there to be nonzero protests happening.
Protests make it clear to policy-makers that their constituents care about an issue. This is especially important for AI because the general public is very worried about AI (see e.g. 2025 Pew poll), but the issue is not high-salience (see 2023 YouGov poll: respondents were worried about AI extinction risk, but it only ranked as the #6 most concerning x-risk). Protests increase its salience.
Protests are an effective way to get media attention. It’s common for protests with only a dozen participants to get news coverage.
I expect that a really good media project would be more cost-effective, but creating a good media project requires exceptional talent. Organizing a protest requires some skill, but the bar isn’t particularly high. Therefore, you can support protests without having to identify top-tier talent.
Protests are highly neglected, and I expect them to continue to be neglected.

Cons:

Some people are concerned that protests can backfire.
- I’m not concerned about peaceful protests backfiring. The scientific literature universally shows that peaceful protests have a positive effect, although the strength of the evidence could be better—see Do Protests Work? A Critical Review.
My back-of-the-envelope calculation suggested that talking directly to policy-makers is more cost-effective.
Now may be too early. See When is the right time for advocacy?
It may be bad for large funders to fund protests because it could create an appearance of astroturfing.
- This is an argument against large funders funding them, but in favor of individual donors supporting protests.

Media about dangers of AI

Create media explaining why AI x-risk is a big deal and what we should do about it.

Things like:

Books (ex: If Anyone Builds It, Everyone Dies)
News articles (ex: Existential Risk Observatory)
Videos (ex: Rob Miles)
Forecasts of how AI can go badly (ex: AI 2027)

Theory of change:

Media increase public concern, which makes policy-makers more likely to put good regulations in place.

Who’s working on it?

80,000 Hours; AI Futures Project; AI Safety and Governance Fund; AI Safety Info; Centre pour la Sécurité de l’IA (CeSIA); CivAI; Existential Risk Observatory; Machine Intelligence Research Institute; Midas Project; various media projects by individual people.

Pros:

We need to get good policies in place. Media can influence policy-makers, and can influence the public, which is important because policy-makers largely want to do what their constituents want.
Polls show that the public is concerned about AI risk and even x-risk, but it’s not a high-priority issue. Media can make it more salient and/or raise “common knowledge” of concern about AI.

Cons:

The impact of media is fat-tailed and heavily depends on quality. It’s hard to identify which media projects to fund.
Poorly done or misleading media projects could backfire.

Message testing

Many people have strong opinions about the correct way to communicate AI safety to a non-technical audience, but people’s hypotheses have largely not been tested. A project could make a systematic attempt to compare different messages and survey listeners to assess effectiveness.

Theory of change:

We can’t ultimately get good AI safety outcomes unless we communicate the importance of the problem, and having data on message effectiveness will help with that.

Who’s working on it?

AI Safety and Governance Fund has an ongoing project (see Manifund) to test AI risk messages via online ads. The project is currently moving slowly due to lack of funding.

Some advocacy orgs have done small-scale message testing on their own materials.

Pros:

A smallish investment in empirical data could inform a large amount of messaging going forward.
Some types of experiments (using online ads or Mechanical Turk) can scale well with funding.

Cons:

The best types of communication may be long, individually tailored (e.g. in one-on-one conversations with policy-makers), or otherwise difficult to test empirically.
I suspect that a person with excellent communication skills would not benefit much from seeing A/B-tested messaging because they can already intuit which wording will be best. (But it may be difficult to identify and hire those people.)
This is the sort of thing that might be best to do internally by an advocacy org that already has a reasonable idea of what kind of message it wants to send.

Host a website for discussion of AI safety and other important issues

LessWrong and the Effective Altruism Forum are upstream of a large quantity of work (in AI safety as well as other EA cause areas). It is valuable that these websites continue to exist, and that moderators and web developers continue to work to preserve/improve the quality of discussion.

Realistically, it doesn’t make sense to start a new discussion forum, so this idea amounts to “fund/support LessWrong and/or the EA Forum”.

Theory of change:

On Lightcone Infrastructure’s Manifund, Oliver Habryka lists some concrete outcomes that are attributable to the existence of LessWrong. You could probably find similar evidence of impact for the EA Forum.

Who’s working on it?

Centre for Effective Altruism; Lightcone Infrastructure.

Pros:

Nearly every project on my list has benefited in some way from the existence of LessWrong or the EA Forum.
Hosting a platform for sharing research/discussion is cheaper (and therefore arguably more cost-effective) than directly conducting research.

Cons:

The benefits are diffuse: high-quality discussion forums provide small-to-moderate benefits to every cause, but usually not huge benefits. So it may be better to directly support your favorite intervention(s).

A toy model: Suppose that

There are 10 categories of AI safety work.
LessWrong makes each of them 20% better.
The average AI safety work produces 1 utility point.
Well-directed AI policy produces 5 utility points.

Then marginal work on LessWrong is worth 2 utility points, and my favorite AI policy orgs are worth 5 points.

List of other project ideas

A project not being a top idea doesn’t mean it’s bad. In fact, it’s likely that at least one or two of these ideas should be on my top-ideas list; I just don’t know which ones.

I sourced ideas from:

reviewing other lists of project ideas and filtering for the relevant ones;
looking at what existing orgs are working on;
writing down any (sufficiently broad) idea I came across over the last ~6 months;
writing down any idea I thought of.

Ideas are roughly ordered from broadest to most specific.

AI-for-animals ideas

Neartermist animal advocacy

There are various projects to improve current conditions for animals, particularly farm animals: cage-free campaigns, humane slaughter, vegetarian activism, etc. I will lump all these projects together for the purposes of this report.

Theory of change:

Animal advocacy increases concern for animals, which likely has positive flow-through effects into the future, by affecting future generations or by shaping the values of the transformative AI that will control the future.

Who’s working on it?

Too many to list.

Pros:

Neartermist animal advocacy has the dual benefit of definitely helping animals today, and building momentum to make future work more effective (to borrow a framing from Jeff Sebo).
Neartermist animal advocacy is tractable and has clear feedback loops. It looks especially promising if you’re highly uncertain or clueless about longtermist or post-TAI interventions.

Cons:

The benefits are diffuse. Creating one new vegan helps many animals in the short term, but has only a tiny effect on society’s future values. I expect direct attempts to improve AI alignment-to-animals to be much more cost-effective.
- I created a back-of-the-envelope calculation that aligns with my initial expectation: my BOTEC-informed guess is that direct advocacy on AI values (by promoting a friendliness-to-animals LLM benchmark) is 2–3 orders of magnitude more cost-effective than conventional animal advocacy.
Neartermist animal advocacy works best if timelines are long. Timelines are likely not long.
Effective animal activists generally regard corporate campaigns as more effective than advocacy directed at consumers, but changing corporate practices seems less relevant for shifting society’s values.

Using TAI to improve farm animal welfare

I’m concerned about how TAI could negatively impact non-human welfare. There are some proposals on how TAI could negatively impact farm animals (e.g. by making factory farming more efficient), and on how animal activists could use TAI to make their activism more effective. I will take these proposals as a broad category rather than discussing them individually.

Theory of change:

Depends on the specific proposal.

Who’s working on it?

Electric Sheep; Hive (see Transformative AI and Animals: Animal Advocacy Under A Post-Work Society); Open Paws; Wild Animal Initiative (see Transformative AI and wild animals: An exploration).

Pros:

Highly neglected.
I am generally skeptical of interventions of the form “teach people to leverage AI to do X better”, but farm animal advocacy seems sufficiently important that it might be worthwhile in this case.

Cons:

Although two of my top ideas relate to post-TAI animal welfare (Develop new plans / evaluate existing plans to improve post-TAI animal welfare and Advocate to change AI training to make LLMs more animal-friendly), I don’t think it’s worth focusing on farm animal welfare in particular.

Factory farming interventions only matter if factory farming still exists. Cultured meat outcompetes factory farming once you get a sufficiently strong understanding of biology (there’s no way growing a whole chicken is the cheapest possible way to create chicken-meat). We are far from that level of understanding, but I would be surprised if (aligned) TAI couldn’t figure it out.
The scale of wild animal welfare is orders of magnitude larger than that of factory farming. The case for prioritizing farm animals over wild animals is that we don’t have the power or knowledge to positively influence nature, but TAI should change the equation.
Space colonization would ultimately dominate earth-based welfare, so questions about panspermia or digital minds have a bigger expected impact.
Proposals for how to use TAI to improve animal advocacy only make sense if TAI does not cause value lock-in. If there is no value lock-in, then there’s no strong reason to spend time now trying to figure out how to use TAI. It would be better to wait until after TAI because at that point, we will have a much better understanding of how TAI works, and there’s no urgency.

Lobby governments to include animal welfare in AI regulations

If governments put safety restrictions on advanced AI, they could also create rules about animal welfare.

Theory of change:

Getting regulations in place would force companies’ AIs to respect animal welfare.

Pros:

One set of regulations can alter the behavior of many frontier companies.
If companies voluntarily change their behavior, they can regress at any time with no consequences. But companies have to obey regulations.

Cons:

It’s unclear what exactly regulations could do about animal welfare. AI safety regulations, insofar as they exist (which they mostly don’t), don’t dictate how LLMs are required to behave; they dictate what companies are required to do to make LLMs safe. What is a regulatory rule that policy-makers would plausibly be on board with, that would also influence model behavior to be friendlier to animals?
Influencing the government on animal welfare seems harder than influencing AI companies.

Traditional animal advocacy targeted at frontier AI developers

Animal advocacy orgs could use their traditional techniques, but focus on raising concern for animal welfare among AI developers. For example, buy billboards outside AI company offices or use targeted online ads.

Theory of change:

AI developers become more concerned for animal welfare, and they make AI development decisions that improve the likelihood that transformative AI is good for animals.

Pros:

Similar to neartermist animal advocacy, but plausibly more cost-effective because it’s more targeted.

Cons:

It’s not known whether techniques like animal welfare ads are effective in general, and they may even be particularly ineffective among demographics like AI developers.
Even if AI developers cared more about animal welfare, it’s not clear that this would carry through to their work on AI.
In 2016, I created a back-of-the-envelope calculation on this idea, and the result wasn’t as good as I expected (it looked worse than standard animal advocacy, if you assume the animal advocacy propagates values into the far future). However, the numbers are outdated because we know a lot more about AI now than we did in 2016.

Research which alignment strategies are more likely to be good for animals

Some alignment strategies may be better or worse for non-human welfare. For example, I expect CEV would be better than the current paradigm of “teach the LLM to say things that RLHF judges like”, which is better than “hard-code (GOFAI-style) whatever moral rules the AI company thinks are correct”.

A research project could go more in-depth on which alignment techniques are most likely to be good for animals (or digital minds, etc.).

Theory of change:

Identify promising alignment techniques, in the hope that people use those techniques. There are enough animal-friendly alignment researchers at AI companies that this might happen.

Pros:

To my knowledge, this question has never been studied.
Some alignment techniques may be much better for animals than others.

Cons:

We have a poor understanding of what ASI will look like, which makes it very hard to say what will work for animal welfare.
We don’t know how to align ASI to any goals at all. We can’t align AI to animal welfare until we can align AI to something.
In the world where alignment turns out to be tractable, it’s likely that there will be strong incentives shaping how ASI is aligned. The choice of whether to use (say) something-like-CEV or something-like-RLHF will be difficult to influence.

AI policy/advocacy ideas

Improving US <> China relations / international peace

Theory of change:

US and China (and other countries) need to agree not to build dangerous AI. Generically improving international cooperation, especially between the US and China, increases the chance that nations cooperate on AI (non-)development.

Who’s working on it?

Too many to list. Some examples: Asia Society’s Center on US–China Relations; Carnegie Endowment for International Peace; Carter Center; National Committee on United States–China Relations; US-China Policy Foundation.

Orgs that work on international cooperation specifically on AI safety (although not necessarily existential risk) include: AI Governance Exchange; Global AI Risks Initiative; Safe AI Forum.

Pros:

International cooperation is likely necessary to prevent existentially risky AI from being built.
Efforts to improve cooperation have succeeded in the past; for example, the US–China Strategic and Economic Dialogue (source).
International cooperation has wide-ranging benefits; efforts can attract funding from many parties with varying agendas.

Cons:

The route to preventing extinction is indirect, which dilutes the cost-effectiveness of this intervention.
International cooperation is far from neglected. Marginal efforts might not make much difference.

Talk to international peace orgs about AI

Theory of change:

Most international peace orgs probably aren’t aware of how important AI regulation is, and they would likely help develop international treaties on AI if they knew it was important.

Who’s working on it?

Nobody that I know of.

Pros and cons are largely the same as Improving US <> China relations / international peace. In addition:

Pros:

This plan is higher-leverage than simply funding international peace orgs.

Cons:

Unclear how to do it. What sorts of evidence would the orgs find persuasive? Which orgs are best suited to working on AI-related cooperation? Those questions are answerable, but I’m not in a good position to answer them.

Increasing government expertise about AI

Talk to policy-makers or create educational materials about how AI works, or help place AI experts in relevant policy roles.

Theory of change:

Policy-makers can do a more effective job of regulating AI if they understand it better.

Pros:

Increasing government expertise may improve the quality of AI regulations.

Cons:

The experts need to actually care about x-risk. Plenty of experts want to accelerate AI development/prevent regulation. For extant projects designed to increase AI expertise, I am skeptical that the expertise would be appropriately x-risk-oriented.
- In practice, “hire experts” often means “hire current or former AI company employees”, which is a recipe for regulatory capture. I expect this would significantly decrease our chance of getting useful x-risk-reducing regulations.
Expertise is much less of a bottleneck than willingness to regulate AI. If I can spend $1 on increasing willingness or $1 on expertise, I’d much rather spend it on willingness.
- And in a sense, the AI safety community already invests way more in expertise (via policy research) than in advocacy. On the margin, we need advocacy more.
The downside of poorly-targeted AI safety regulations is that they end up hurting economic development. That’s bad, but it looks pretty trivial in a cost-benefit analysis compared to extinction.
I wrote a longer comment about this subject on the EA Forum.
Unless by “expertise” we’re talking about “expertise at recognizing that AI x-risk is a big problem”. In which case, yes, we need expertise.
- Right now, the main strategy for getting x-risk people into government is “pretend not to care about x-risk so you seem normal, and never voice your concerns”. In which case, what’s the point?
- I think a better strategy is “talk to policy-makers about x-risk and straightforwardly tell them what you believe.”
I’ve heard it argued that increasing government expertise is low tractability because government is so big, and making internal changes like that is slow. I don’t think this is a strong consideration because other approaches to AI safety are also low tractability. (I still don’t think increasing expertise is a good plan, but this particular argument seems weak.)
Policy-makers are largely in the dark about x-risk, which is indeed a problem. But I don’t see clean routes to increasing AI expertise that don’t also push in the wrong direction. Raising concern about the importance of TAI has historically led people to believe things like “I need to be the one who controls TAI, so I will start a new AI company” or “we need to make sure we get TAI before China”. Generically increasing AI expertise is, in my best estimation, net harmful. For more on this, see Advocacy should emphasize x-risk and misalignment risk.

Policy/advocacy in China

Take any of my ideas on AI policy/advocacy, and do that in China instead of in the West.

Theory of change:

The high-level argument for policy/advocacy in China is largely the same as the argument for prioritizing AI risk policy/advocacy in general. China is currently the #2 leading country in AI development, so it’s important that Chinese AI developers take safety seriously.

Who’s working on it?

AI Governance Exchange; Safe AI Forum (sort of); perhaps some Chinese orgs that I’m not familiar with.

Pros:

Many of the ideas listed above are intended to improve the state of AI policy in the US/UK. The pros for those ideas largely also apply to equivalent projects conducted in China (modulo the obvious differences, e.g. China is not a representative democracy, so political advocacy works differently).

Cons:

Probably nobody in China is reading this report. Advocating for Chinese policy as a non-Chinese person is fraught because the CCP will not trust our motivations, just as the American government would not trust a Chinese philanthropist who funds American AI safety advocacy.
According to an 80,000 Hours career review: “[The Chinese government] is often wary of non-governmental groups that try to bring about grassroots change. If an organisation is blacklisted, then that’s a nearly irreversible setback.”

A comment on my state of knowledge:

I don’t know much about the current state of AI safety in China, or what sort of advocacy might work. I did not prioritize looking into it because my initial impression is that I would require high confidence before recommending any interventions (due to the cons listed above), and I would be unlikely to achieve the necessary level of confidence in a reasonable amount of time.

Corporate campaigns to advocate for safety

Run public campaigns to advocate for companies to improve safety practices and call out unsafe behavior.

Theory of change:

Companies may improve their behavior in the interest of maintaining a good public image.

Who’s working on it?

Midas Project; More Light.

Pros:

Corporate campaigns have worked well in animal advocacy.
Companies are smaller than governments, which means they’re more agile and potentially easier to influence.

Cons:

Companies have strong internal incentives to be unsafe. By contrast, governments don’t have a profit motive. They may be harder to move, but they have less reason to oppose safety efforts.
Making more actors safe is better than making fewer actors safe. International treaty > single-country regulation > single-company safety efforts.
Democratic governments are specifically designed to do what people want. That doesn’t always happen, but at least there are mechanisms pushing them that way. Companies are not democratic.
There is a good chance that safety standards strong enough to prevent human extinction would pose an existential threat to AI companies (or at least would be incompatible with their current valuations). If that’s the case, then corporate campaigns will not be able to get companies to implement adequate safety measures.

Overall, this seems similar to political advocacy, but worse.

Develop AI safety/security/evaluation standards

Work inside a company, as a nonprofit, or with a governmental body (NIST, ISO, etc.) to develop AI safety standards.

Theory of change:

Sufficiently well-written standards can define under what conditions a frontier AI is safe, and potentially enforce those conditions.

Who’s working on it?

AI Standards Lab; various AI companies; various governmental bodies.

Pros:

Companies might abide by the standards.
Standards can provide a template with which to write regulations.

Cons:

Standards are not currently the bottleneck to getting regulation written. We already have a substantial amount of work on AI safety standards, but we still don’t have good regulations. There aren’t people waiting around to write legislation if only they had some standards they could use. (Of course, more standards would still be better.)
If standards are voluntary, companies can stop abiding by them when they turn out to be hard to satisfy (and in fact, companies have already done that on multiple occasions, with respect to their self-imposed safety standards).
Standards agencies such as NIST generally don’t have enforcement power.
Nobody knows how to write standards that will prevent extinction if implemented. See Appendix for more on this.

Slow down Chinese AI development via ordinary foreign policy

Both the “slow down AI” crowd and the “maintain America’s lead” crowd agree that it is good for China’s AI development to slow down. The American government could accomplish this using foreign policy levers such as:

restricting chip exports;
encouraging scientists to immigrate to the US.

Theory of change:

On my view, slowing down Chinese AI development is good because it gives the leading AI developers in the United States more room to slow down. It also makes it less likely that a Chinese company develops misaligned TAI (although right now, US companies are more likely to develop TAI first).

Slowing down Chinese AI development looks good on the “maintain America’s lead” view, although I believe this view is misguided—making TAI safe is much more important than making one country build it before another.

Who’s working on it?

Center for Security and Emerging Technology has written memos recommending similar interventions. I expect there are some other orgs doing similar activities, including orgs that are more concerned about national security than AI risk.

Pros:

Looks (plausibly) good on multiple worldviews.

Cons:

Some policies could antagonize China and make cooperation more difficult.
- Mutual sabotage between the US and China would probably decrease AI x-risk, but it would also have negative effects. I’d rather the countries increase safety via mutual cooperation.
- This depends on the policy. Improving AI model security is fine, relaxing immigration restrictions is probably fine, but export restrictions or tariffs would likely heighten international tensions. (The United States already has export restrictions and tariffs, but it might be bad to add more on the margin.)
Advocating for these sorts of foreign policy interventions is likely not cost-effective because they’re in controversial political areas that already see significant funding and effort. For example, there are already strong and well-funded interests arguing both for and against immigration.
On the “slow down AI” view, this seems less promising than domestic US regulation because US-based companies look significantly more likely than China to be the first to build misaligned ASI.

Whistleblower protection/support

Provide legal support for whistleblowers inside AI companies and assist in publicizing whistleblowers’ findings (e.g. setting up press interviews).

Theory of change:

A few ways this could help:

Whistleblowers can force companies to change their unsafe behavior.
The possibility of whistleblowers incentivizes companies to be safe.
Publicizing companies’ bad behavior can raise public concern about AI safety.

Who’s working on it?

More Light.

Pros:

Whistleblower support could be high-leverage: the whistleblowers themselves bring the important information, but they often can’t do much without help.

Cons:

Only helps if there are things worth whistleblowing on. There might not be any warning shots (see When is the right time for advocacy? for some relevant discussion).
- OpenAI’s bad behavior on secret NDAs was whistleblow-worthy, but it wasn’t directly related to AI risk, and it’s not clear that the news about OpenAI’s bad behavior decreased x-risk.
Other paths to impact are more direct, e.g. media projects or corporate campaigns.

Opinion polling

Run polls to learn public opinion on AI safety.

Theory of change:

Polls can inform policy-makers about what their constituents want. They also inform people working on AI safety about where their views most align with the public, which can help them prioritize.

Who’s working on it?

AI Policy Institute; traditional polling agencies (Pew and YouGov have done polls on people’s views on AI).

Pros:

When talking to policy-makers, it’s useful to be able to point to polls as evidence that the public cares about AI safety.
Polls provide common knowledge of concern for AI risk.

Cons:

A good amount of polling already exists. If we already know that people in 2024 were concerned about AI risk, there’s not as much value in knowing that they’re still concerned in 2025.
The main benefit of polls is to empower advocacy, but we have precious little advocacy right now.

Help AI company employees improve safety within their companies

Work with people in AI companies (by organizing conferences, peer support, etc.) to help them learn about good safety practices.

Theory of change:

AI company employees can push leadership to implement stronger internal safety standards.

Who’s working on it?

AI Leadership Collective.

Pros:

If internal employees work more on safety, that will make AI companies safer.

Cons:

This theory of change has the same issues as corporate campaigns: companies have strong incentives to be unsafe; global (or at least national) safety measures are better than single-company measures.

Working with AI company employees might be sufficiently high-leverage to make up for my concerns. Answering that question would require going more in-depth, so I will go with my intuition and say it’s probably not as cost-effective as my top ideas.

Direct talks with AI companies to make them safer

If you can talk to policy-makers about AI x-risk, then maybe you can also talk to AI company executives about AI safety.

Theory of change:

AI company execs have significant control over the direction of AI. If they started prioritizing safety to a significantly greater extent, they could probably do a lot to decrease x-risk.

Who’s working on it?

To my knowledge, nobody is systematically working on this.

Pros:

High-leverage: you may be able to prevent extinction by changing the minds of a half-dozen or so people.

Cons:

Similar issues to corporate campaigns: governments have less incentive to be unsafe; it’s better to make all companies safe simultaneously (via regulation). Therefore, political action seems more promising.
The CEOs of most AI companies are well aware that AI poses an extinction risk, but they are building it anyway, and they are massively under-investing in safety anyway. It’s not clear what additional information would change their minds. So this seems worse than corporate campaigns.

People in relevant positions should push companies to be safer in ways that they can, but I don’t see any way to support this intervention as a philanthropist.

Monitor AI companies on safety standards

Track how well each frontier AI company does model risk assessments, security, misuse prevention, and other safety-relevant behaviors.

Theory of change:

Monitoring AI companies can inform policy-makers and the public about the state of company safety. Monitoring could have a similar effect as corporate campaigns, where it pushes companies to be safer, or it could even directly inform corporate campaigns.

Who’s working on it?

AI Lab Watch & AI Safety Claims Analysis; Future of Life Institute’s AI Safety Index; Midas Project; Safer AI.

Pros:

Easy to do: a solo developer can run a monitoring website as a side project.

Cons:

The theory of change seems somewhat weak to me. There are other ways to demonstrate AI risks to policy-makers and the public, and it’s not clear that this way is particularly good.
- But I don’t know what kind of impact the monitoring websites have had; maybe they’ve had some big positive influences that I don’t know about.

Create a petition or open letter on AI risk

Write a petition raising concern about AI risk or calling for action (such as a six-month pause or an international treaty). Get respected figures (AI experts, etc.) to sign the petition.

Theory of change:

A petition can make it apparent to policy-makers and the public that many people/experts are concerned about AI risk, while also creating common knowledge among concerned people that they are in good company if they speak up about it.

Who’s working on it?

aitreaty.org; Center for AI Safety / CAIS Action Fund; Future of Life Institute / FLI Action and Research, Inc.

Pros:

Brings AI risk into the public conversation and makes it easier to talk about.
Relatively easy to do—the main difficulty is in finding people to sign it who can bring credibility.

Cons:

The biggest con is that petitions have diminishing marginal utility, and several have been made already.
The path from a petition to concrete outcomes isn’t entirely clear (although I’m inclined to believe that petitions can work).

My sense is that the existing petitions have been quite helpful, but there isn’t clear value in creating more petitions. There may be some specific call to action that a new petition ought to put forward, but I’m not sure what that would be.

Create demonstrations of dangerous AI capabilities

Make AI risk concrete by building concrete demonstrations of how AI can be dangerous.

Theory of change:

Many people don’t find it plausible that AI could cause harm; concrete demonstrations may change their minds. It can also serve to make AI risk more visceral.

Who’s working on it?

FAR.AI; Palisade Research; some one-off work by others (e.g. Apart Research).

Pros:

Concrete demonstrations can aid advocacy efforts.

Cons:

Capability demonstrations may send the wrong message, encouraging accelerationism instead of caution.
- I am more concerned about this for general AI capability evaluations. For this project, I am specifically thinking of demonstrations of how AI can do harm.
You can’t create a concrete demonstration of superintelligent AI’s capabilities until you already have superintelligent AI, at which point it’s too late. Pre-superintelligent AIs can have scary capabilities, but demos are misleading in a sense because they may create a skewed understanding of where the risks come from.

I’m most optimistic about demonstrations where there is a clear plan for how to use them, e.g. Palisade Research builds its demos specifically to show to policy-makers. I think Palisade is doing a particularly good version of this idea, but for the most part, I think other ideas are better.

Sue OpenAI for violating its nonprofit mission

OpenAI:

Our mission is to ensure that artificial general intelligence […] benefits all of humanity.

OpenAI has more-or-less straightforwardly violated this mission in various ways. Humanity plausibly has grounds to sue OpenAI.

Theory of change:

A lawsuit would change OpenAI’s incentives and may force OpenAI to actually put humanity’s interest first, depending on how well the lawsuit goes.

Who’s working on it?

In 2024, Elon Musk filed a lawsuit against OpenAI on this basis. The lawsuit is set to go to trial in 2026.

Given that there is already an ongoing lawsuit, it may be better to support the existing suit (e.g. by writing an amicus brief or by offering expert testimony) than to start a new one.

Pros:

A lawsuit could force OpenAI to significantly improve safety.

Cons:

A failed lawsuit has numerous downsides—it can make the plaintiff look bad; it can set an unfavorable precedent; it’s expensive.
Legal matters may have other, hard-to-predict downsides, and I’m not qualified to evaluate them.
I am not a lawyer, but my impression is that courts are typically quite lenient about what nonprofits are allowed to do, so it would be difficult for a lawsuit to succeed.
Given that there is an ongoing lawsuit by Elon Musk, who is known to behave erratically, Musk may do something unpredictable that causes harm.

Send people AI safety books

Books are a tried-and-true method of explaining complex ideas. One could mail books on AI risk to Congress people, or staffers, or AI company execs, or other relevant people.

Theory of change:

A book can explain in detail why AI risk is a big deal and thus persuade people that it’s a big deal.

Who’s working on it?

MIRI did something similar when promoting their new book: “We cold-emailed a bunch of famous people (like Obama and Oprah)”. They were asking people to write blurbs for the book, which isn’t exactly what I had in mind, but it’s related.
Aidar Toktargazin has a Manifund project to give out AI safety books to researchers and professors at Nazarbayev University in Kazakhstan.

Pros:

Mailing books is cheap.

Cons:

Mailing books seems riskier than other kinds of advocacy—it could be viewed as excessively pushy.
- Mormons give out free books, and they’re viewed as pushy, but they’ve also grown a lot. It’s unclear whether Mormons’ publicity strategies are worth emulating.
It may be better to let publishers do their own publicity.

This idea might be really good, but it’s high-variance. I would not recommend it without significantly investigating the possible downsides first.

AI research ideas

I said I wasn’t going to focus on technical research or policy research, but I did incidentally come up with a few under-explored ideas. These are research projects that I’d like to see more work on, although I still believe advocacy is more important.

Research on how to get people to extrapolate

A key psychological mistake: “superintelligent AI has never caused extinction before, therefore it won’t happen.” Or: “AI is not currently dangerous, therefore it will never be dangerous.”

Compare: “Declaring a COVID emergency is silly; there are currently zero cases in San Francisco.” (I am slightly embarrassed to say that that is a thought I had in February 2020.)

Relatedly, some people expect there will not be much demand for AI regulation until we see a “warning shot”. Perhaps, but I’m concerned we will run into this failure-to-extrapolate phenomenon. AI has already demonstrated alignment failures (Bing Sydney comes to mind; or GPT 4o’s absurd sycophancy; or numerous xAI/Grok incidents). But clear examples of misalignment get fixed (because AI is still dumb enough for us to control). So people may draw the lesson that misalignment is fixable, and there may keep being progressively bigger incidents until we finally build an AI powerful enough to kill everyone.

I am concerned that the concept of AI x-risk will never be able to get sufficient attention due to this psychological mistake. Therefore, we need to figure out how to get people to stop making this mistake.

Theory of change:

Many people ignore AI x-risk because of this mistake. If we knew how to get people to extrapolate, we could use that knowledge to improve communication on AI risk.

Who’s working on it?

Some academic psychologists have done related research.

Pros:

If successful, this research would significantly increase how many people take AI risk seriously.

Cons:

We know from the history of psychology that it’s difficult to find psychological insights. I’d guess it would cost tens or hundreds of millions of dollars to produce meaningful results (if it’s even possible at all).
Psychology research takes a long time to pay off. That doesn’t work if timelines are short.
This research is not particularly neglected. The American Psychological Association wants more research on how to get people to care about climate change, which is related.
To the extent that science has already uncovered answers to this question, good communicators already know those answers. For example, studies have found that people are more likely to pay attention to future problems when you give concrete scenarios; but a good writer already does that. (See Deep Research (1, 2) for an attempt at finding relevant psychology studies, although only about a quarter of them are actually relevant.)

Rather than spending $50 million on psychology research and then $50 million on a variety of psychologically-motivated media projects informed by that research, I would rather just spend $100 million on media projects.

Investigate how to use AI to reduce other x-risks

An important argument against slowing down AI development is that we could use advanced AI to reduce other x-risks (climate change, nuclear war, etc.).

But an aligned AI wouldn’t automatically reduce x-risk. It may increase technological risks (e.g. synthetic biology) if offensive capabilities outscale defensive ones. An aligned AI could reduce nuclear risk by improving global coordination, but it’s not obvious that it would.

Therefore, it may be worth asking: Are there some paths of AI development that differentially reduce non-AI x-risk?

Theory of change:

Research on how to direct AI may inform efforts by the developers of advanced AI, which may ultimately reduce x-risk.

Who’s working on it?

Nobody, to my knowledge.

Pros:

If you believe we need to build TAI to avert non-AI x-risks, then it stands to reason that you should also want to know how to direct TAI to accomplish that end.
This line of research is highly neglected (to my knowledge, there are zero people working on it).

Cons:

Non-AI x-risks seem less concerning than AI x-risk, so it seems better to work directly on reducing AI x-risk.
If you believe we should slow down AI development, then this line of research doesn’t matter as much. And I do believe we should slow down AI development, and I believe that a wide range of worldviews should agree with me on that (see Appendix: A moratorium is the best outcome).
At a glance, the problem seems difficult to make progress on (see below).

My initial thoughts on this line of research:

You can’t control what general AI is good at. It would be good at everything. There is no known way to make it (say) good at defending against biological weapons, but bad at creating biological weapons.
A narrow “cooperation superintelligence” would be better than a narrow “scientist superintelligence” because the latter increases technological x-risk. But based on current trends in AI, my guess is that we could develop a “scientist ASI” that’s bad at cooperation, but we couldn’t develop a “cooperation ASI” that’s bad at science. So this idea is likely a dead end.
Even if we could build a “cooperation ASI”, we still need to solve alignment problems first. So it seems better to focus on solving alignment.

A short-timelines alignment plan that doesn’t rely on bootstrapping

To my knowledge, every major AI alignment plan depends on alignment bootstrapping, i.e., using AI to solve AI alignment. I am skeptical that bootstrapping will work, and even if you think it will probably work (with, say, 90% credence), you should still want a contingency plan.

Write a research agenda for how to solve AI alignment without using bootstrapping.

Theory of change:

If people come up with sufficiently good plans, then we might solve alignment.

Who’s working on it?

Peter Gebauer is running a contest for short-timelines AI safety plans, but the plans are allowed to depend on bootstrapping (e.g. Gebauer favorably cites DeepMind’s plan).

There are some plans that say something like “stop developing AI until we solve alignment” (ex: MIRI; Narrow Path), which is valid (and I agree), but it’s not a technical plan.

The closest thing I’ve seen is John Wentworth’s research agenda, but it specifically invokes the underpants gnome meme, i.e., the plan has a huge hole in the middle.

Pros:

Existing plans are insufficiently rigorous.
To my knowledge, there are zero meaningful plans that don’t rely on alignment bootstrapping. If bootstrapping turns out not to work, every plan fails. There is a gap to be filled.

Cons:

It is highly unlikely that any satisfactory plan exists.
- I think trying to create plans is a reasonable idea on the off chance that somebody does come up with a good plan, but I don’t think it’s a good use of marginal philanthropic resources.
An alignment plan still leaves non-alignment problems unsolved.
AI companies cannot be trusted to implement a safe plan even if one exists.
- But the plan existing does increase the chance that companies follow the plan, or that external pressures can force companies to follow it.

Rigorous analysis of the various ways alignment bootstrapping could fail

I’m pessimistic about the prospects of alignment bootstrapping, and I’ve seen various AI safety researchers express similar skepticism, but I’ve never seen a rigorous analysis of the concerns with alignment bootstrapping.

Theory of change:

The AI companies with AI safety plans are all expecting bootstrapping to work. A thorough critique could convince AI companies to develop better plans, or create more of a consensus among ML researchers that bootstrapping is inadequate, or convince policy-makers that they need to make AI companies be safer.

Who’s working on it?

Nobody, to my knowledge.

Pros:

This sort of analysis would be feasible to write—it would require expertise and time investment, but it doesn’t require novel research.

Cons:

The theory of change seems weak—if companies haven’t already figured out that their plans are inadequate, then I doubt that more criticism is going to change their minds.
Independent of AI companies’ top-down plans, it’s helpful if you can better inform alignment researchers about what they should be focusing on. But my guess is an analysis like this wouldn’t shift much work.

Future work

Pros and cons of slowing down AI development, with numeric credences

I addressed this to some extent, but I could’ve gone into more detail, and I didn’t do any numeric analysis.

I would like to see a more formal model that includes how the tradeoff changes based on considerations like:

One’s view of population ethics
The importance of preventing deaths vs. causing births
Temporal discount rate or longtermism
P(doom)
The extent to which P(doom) is reduced if AI development slows down
Existential risk from sources other than AI (see Quantitative model on AI x-risk vs. other x-risks)
How AI development interacts with other x-risks

Relatedly, how does P(doom) change what actions you’re willing to take? I often see people assume that at a P(doom) of, say, 25%, pausing AI development is bad. That seems wrong to me. I believe that at 25% you should be about as aggressive (about pushing for mitigations) as you would be at 95%, although I haven’t put in the work to come up with a detailed justification for this position. The basic argument is that x-risk looks very bad on longtermist grounds, and a delay of even (say) 100 years doesn’t look like as big a deal.

Quantitative model on AI x-risk vs. other x-risks

There is an argument that we need TAI soon because otherwise we are likely to kill ourselves via some other x-risk. I have a rough idea for how I could build a quantitative model to test under what assumptions this argument works. Building that model wasn’t a priority for this report, but I could do it without much additional effort.

The basic elements the model needs are

X-risk from AI
X-risk from other sources
- How to estimate these? Expert forecasts seem unreliable.
How much we can reduce AI x-risk by delaying development
X-risk from other sources, conditional on TAI
- For an aligned totalizing TAI singleton that quickly controls the world, x-risk would be ~0.
- TAI doesn’t trivially decrease other x-risks; it could increase x-risk by accelerating technological growth (which means it’s easier to build dangerous technology—see The Vulnerable World Hypothesis). The mechanism of decreasing x-risk isn’t that TAI is smarter; the mechanism is that it could increase global coordination / centralize the ability to make dangerous technology.

Deeper investigation of the AI arms race situation

Some open questions:

What are some historical examples of arms races that were successfully aborted? What happened to make things go well?
How hard are different factions racing, and what would it take to convince them to slow down?
How likely are we to end up in a bad totalitarian regime post-TAI if various parties end up “winning the race” and building an alignable ASI?

Does slowing down/pausing AI help solve non-alignment problems?

Pausing (or at least slowing down) clearly gives us more time to solve alignment, and has clear downsides in terms of opportunity cost. But other effects are less clear. Pausing may help with some other big problems: animal-inclusive AI; AI welfare; S-risks from conflict; gradual disempowerment; risks from malevolent actors; moral error. There are some arguments for and against pausing being useful for these non-alignment problems; for more on this topic, see Slowing down is a general-purpose solution to every non-alignment problem.

I have never seen an attempt to analyze why pausing AI development might or might not help with non-alignment problems; this seems like an important question.

Some considerations:

We don’t want to build TAI until we become more wise, but it’s not clear that we can become more wise, or perhaps TAI would be wiser than we would.
Pausing may increase misuse risk or some related risk.
- One conceivable outcome, albeit one that doesn’t seem particularly likely, is that AI companies become increasingly wealthy and powerful by selling pre-TAI AI services, and this concentration of power ultimately allows them to build TAI in a way that goes against most people’s interests.
Peace and democratic governance have been trending upward over the past century (see The Better Angels of Our Nature). Slowing/pausing means the world will probably be more peaceful and democratic when we get TAI, which is probably desirable (less chance of power struggle, etc.)
Moral circles have expanded over time (although they haven’t strictly expanded—see Gwern’s The Narrowing Circle). It’s better to develop TAI when moral circles are wider.

Determine when will be the right time to push for strong restrictions on AI (if not now)

A common view: “We should push for strong restrictions on AI, but now is not the right time.”

I disagree with this view; I think now is the right time. But suppose it isn’t. When will be the right time?

Consider the tradeoff wherein if you do advocacy later, then

the risks of AI will be more apparent;
but there’s a greater chance that you’re too late to do anything about it.

Is there some inflection point where the first consideration starts outweighing the second?

And how do you account for uncertainty? (Uncertainty means you should do advocacy earlier, because being too late is much worse than being too early.)

I don’t think this question is worth trying to answer because I am sufficiently confident that now is the right time. But I think this is an important question from the view that now is too early.

Supplements

I have written two supplements in separate docs:

Appendix: Some miscellaneous topics that weren’t quite relevant enough to include in the main text.
List of relevant organizations: A reference list of orgs doing work in AI-for-animals or AI policy/advocacy, with brief descriptions of their activities.

Notes

For example, almost every frontier AI company opposed SB-1047; Anthropic supported the bill conditional on amendment, and Elon Musk supported it but xAI did not take any public position. See ChatGPT for a compilation of sources. ↩
I asked ChatGPT Deep Research to tally up funding for research vs. policy and it found ~2x as many researchers as policy people and also 2x the budget, although it miscounted some things; most of what it counted as “AI policy” is (1) unrelated to x-risk and (2) policy research, not policy advocacy; and it only included big orgs (e.g. it missed the long tail of independent alignment researchers). So I believe the true ratio is even more skewed than 2:1. ↩
According to my research, it’s not difficult to find examples of times when UK policy influenced US policy, but it’s still unclear to me how strong this effect is. There are also some theoretical arguments for and against the importance of UK AI policy, but I didn’t find any of them particularly compelling, so I remain agnostic. ↩
I find this state of affairs confusing given how many people profess belief in short timelines. I think part of the reason is that people involved in AI safety tend to be intellectual researcher-types (like me, for example) who are more likely to orient their work toward “what is going to improve the state of knowledge?” rather than “what is likely to pay off in the near future?” ↩

Healthy Cooking Tips from a Lazy Person

Fri, 29 Aug 2025 00:00:00 -0700

source

The problem with most “lazy cooking” advice is that it’s not lazy enough. Today I bring you some truly lazy ways of eating healthy.

This is the advice that I would’ve liked to hear when I was a lazy teenager. I’m still lazy, but I’m better at making food now. (I’m not going to say I’m better at cooking, because the way I make most food could only very generously be described as “cooking”.)

All my lazy meals are vegan because I’m vegan, but if anything, that works to my advantage because the easiest animal foods still take more work than the easiest plant foods. (You can eat raw vegetables but you can’t eat raw chicken.¹)

Contents
Healthy foods that require no preparation whatsoever
Healthy foods that take less than one minute of preparation
Cooking tips
Notes

Healthy foods that require no preparation whatsoever

Nuts and seeds. Buy a bag and eat them out of the bag.
- Or buy trail mix for more variety.
Nut butter. You can eat it right out of the jar if you want to.
- Some people are under the misconception that the big-brand peanut butters like Jif and Skippy are bad for you because they contain sugar. The Jif that’s in my cabinet right now only gets 7% of its calories from sugar, and that little bit of sugar makes it taste 1000% better. That’s a flavor to sugar ratio of 14,285%; you can’t argue with the math.
- Some people believe peanut butter is bad for you because it contains a lot of fat. Trans fats and saturated fats are the “bad fats”;² peanut butter is made of unsaturated fats, which are the “good fats”.
Many fruits can be eaten with no prep or with very little prep. You have to peel bananas, but peeling a banana is no harder than opening a candy wrapper.
A lot of vegetables can be eaten raw. They taste better when you cook and season them, but sometimes you have to sacrifice flavor in the name of laziness.
There is nothing wrong with eating tofu raw. But when it comes to zero-prep soy-based foods, my go-to is dry roasted edamame.
Soylent and Huel aren’t exactly healthy, but they’re not not healthy, either.

Healthy foods that take less than one minute of preparation

Get some vegetables (carrots or broccoli) and dip them in hummus.
Pour a bowl of cereal.
- Breakfast cereals are often bad for you, but there are some good ones. Last year I reviewed high-protein breakfast cereals, all of which I would describe as healthy. There are also many low-protein but still healthy cereals, for example Cheerios are made of whole oats.
Buttered toast is cool, but Big Toaster doesn’t want you to know that buttered untoasted bread is maybe even better.³
Three bean recipes in increasing order of prep time + flavorfulness:
1. Open can of beans; eat straight out of the can. I personally would use a spoon, but if you’d rather pour the beans directly into your mouth, I won’t judge.
2. Open can of beans; pour into bowl; add some kind of seasoning; eat.
  - Some seasoning ideas: hot sauce; garlic powder; Chesapeake Bay seasoning; garlic & herb seasoning mix (like this).
3. Do #2, but also microwave it before eating. (I know I promised sub-minute prep times, but this recipe will take more like two minutes.)

Cooking tips

I mostly eat easy meals, but I do real cooking once every couple days—my “real cooking” mostly means “chop some stuff and throw it in an air fryer”. But sometimes I even cook things in a pot. I have a few methods for making my cooking easier.

Recipes often call for the same set of spices. Pre-mix your spices or buy them pre-mixed.
- Curry recipes often call for garam masala, cumin, and coriander. I’m not sure what’s going on there because the main two ingredients of garam masala are cumin and coriander. When I cook a big pot of beans, I just throw in a ton of garam masala.
- My most-used spices are a pre-mixed garlic & herb seasoning, a pre-mixed garam masala, and a pre-mixed all-purpose spice mix consisting of salt + pepper + garlic powder.
You can buy vegetables pre-chopped if you’re willing to pay more.
- Onions hurt my eyes a lot. I buy them pre-chopped which saves time and saves my eyes.
- As a middle ground, you can buy pre-peeled garlic cloves. Peeling is much harder than chopping (for me at least) so pre-peeled garlic lets me skip the worst part.
- I am not the first person to observe that most recipes don’t call for enough garlic, but I think even most people who say “recipes don’t call for enough garlic” still don’t use enough garlic. If a recipe calls for 2 cloves then I will use about 20 cloves and I’m still not sure I’m using enough. (This doesn’t have anything to do with being lazy but I need to express my garlic-related feelings.)
Many oven or stovetop recipes can be done faster in an air fryer. An air fryer cooks food fast like a microwave, but it makes the food crispy instead of mushy and weird.
- I’ve heard a stereotype that Asian moms use their ovens exclusively as pot-and-pan storage. If that’s true then I guess that makes me an Asian mom.
There are many convenient-but-unhealthy foods, too. It’s okay to eat unhealthy food sometimes.

Notes

I guess you could eat raw eggs if you really wanted to. People talk about Rocky, but I’ve always associated eating raw eggs with the dad from The Neverending Story. ↩
Some people don’t even believe saturated fat is bad for you. I wrote more about this in my Outlive review. ↩
Lest there be any confusion about how I previously said I was vegan: when I say “butter” what I actually mean is Earth Balance. In fact butter isn’t good for you so if I was eating real butter, bread + butter wouldn’t qualify as a healthy meal. Earth Balance is made of unsaturated fats so it’s healthy.

And of course I eat whole wheat bread, specifically Dave’s Killer Bread which is the undisputed best-tasting whole grain bread. ↩

Doctor Strange Didn't See Only One Victory out of 14,000,605 Futures

Fri, 25 Jul 2025 00:00:00 -0700

Or, more accurately, the fact that he said the Avengers only won once can’t be taken as evidence about what he really saw.

This post contains spoilers for Avengers: Infinity War and Avengers: Endgame.

Doctor Strange told the heroes that he used the Time Stone to look into 14,000,605 futures, and saw only one future where they won.

He spent the rest of the two movies steering events to play out as he saw them in this one future.

Therefore, while Strange was using the Time Stone, he must have taken the exact same actions, including telling the Avengers that there was only one way to win.

Strange telling the heroes (especially Tony Stark) that they only won in one future was a critical element of his plan—the plan only worked because he said that.

But when he played out this scenario using the Time Stone, he couldn’t have known at that point that there was only one way to win, because he hadn’t run the scenarios yet.

So what actually happened was:

In one of the possible futures, Doctor Strange told Tony that there was only one way to win, even though Strange didn’t yet know whether that was true.
This worked, and Thanos was defeated.
In real life, Doctor Strange replicated this plan.

It could be true that this was the only future where they won. But when Doctor Strange said it’s the only future where they won, that statement was not attached to truth in any way. The reason he said it wasn’t that it was true; it was that he needed to say it for the Avengers to win.

So, in the end¹, we have no idea whether it’s true.

Edited 2025-07-26 to change “out” in the title from capital to lower case. I thought “out” was supposed to be capitalized but after writing it, it seemed weird to me, so I did some research. “Out” is normally an adverb, but in this sentence, “out of” functions as a preposition, and prepositions should be lower case. The Chicago Manual of Style says “out of” should be lower case so I changed my title. But apparently this is a thorny issue, with the Chicago guide originally giving incorrect guidance, and then they updated it after some readers wrote in to disagree. (The thing they got wrong wasn’t directly relevant to my title, it was about using “out of” in a different context.) So if they can get it wrong then I don’t feel too bad about getting it wrong myself.

Notes

game ↩

Is It So Much to Ask for a Nice Reliable Aggregated X-Risk Forecast?

Sat, 12 Jul 2025 00:00:00 -0700

On most questions about the future, I don’t hold a strong view. I read the aggregate prediction of forecasters on Metaculus or Manifold Markets and then I pretty much believe whatever it says.

Various attempts have been made to forecast existential risk. I would like to be able to form views based on those forecasts—especially on non-AI x-risks, because I barely know anything about synthetic biology or nuclear winter or catastrophic climate change. Unfortunately, none of the aggregate forecasts look reliable.

First, some general notes about forecasting distant¹ and low-probability² events:

According to a literature review by Luke Muehlhauser, we don’t have good data on long-range forecasters, and we don’t know if people with short-range forecasting skill can make good forecasts over long ranges.
According to an analysis by niplav, Metaculus predictions become less accurate as the duration gets longer.

So we have good reason to doubt the ability of forecasters to predict existential risk, even when they are known to make accurate forecasts on near-term outcomes such as elections.

Now let’s look at what attempts have been made to forecast x-risk, and why I don’t find any of them satisfying.

The most rigorous attempt at an aggregate forecast comes from the Existential Risk Persuasion Tournament. The tournament brought in superforecasters and domain experts to make predictions, then had them attempt to persuade each other and make predictions again.

In the end, domain experts forecasted extinction as an order of magnitude more likely than what the superforecasters believed.

And even the domain experts forecasted only a 3% chance of AI extinction. My number is much higher than that, and I notice myself not changing my beliefs after reading this.

Edited 2025-09-09 to add: A September 2025 follow-up report from the Forecasting Research Institute found that the domain experts underestimated the rate of AI progress 2022–2025, and superforecasters dramatically underestimated the rate of progress; see also Twitter summary thread. Notably, only 2.3% of superforecasters predicted AI to win a gold medal at the International Mathematics Olympiad, which it did in 2025.

Scott Alexander wrote about the tournament:

Confronted with the fact that domain experts/superforecasters had different estimates than they did, superforecasters/domain experts refused to update, and ended an order of magnitude away from each other. That seems like an endorsement of non-updating from superforecasters and domain experts! And who am I to disagree with such luminaries?

Peter McCluskey, who participated in the tournament as a superforecaster, wrote a personal account. His experience aligns with my (biased?) assumption that the people reporting very low P(doom) numbers just don’t understand the AI alignment problem.

Okay, the lesson from the X-Risk Persuasion Tournament is that it’s not clear whether we can learn anything from it.

What about Metaculus?

Metaculus has several relevant forecasts, but they seem to contradict each other. Some example forecasts:

Will humans go extinct before 2100? 0.3% chance. (This is the Metaculus question with the most activity.)
Ragnarok question series: Implied 12.16% chance (community prediction) or 3.66% chance (Metaculus prediction)³ of a >95% decline in population by 2100.
How does the level of existential risk posed by AGI depend on its arrival time? Answers range from 50% to 9.3% depending on date range, which is maybe consistent with question 2 above, but definitely not consistent with question 1.

In a comment, Linch provides some reasons to be suspicious of Metaculus’ estimates.

There’s no incentive to do well on those questions.

The feedback loops are horrible

Indeed, some people have actually joked betting low on the more existential questions since they won’t get a score if we’re all dead (at least, I hope they’re joking)

At the object-level, I just think people are really poorly calibrated about x-risk questions

My comment here arguably changed the community’s estimates by ~10%

In 2008, the Future of Humanity Institute ran a Global Catastrophic Risks Survey asking conference participants to give forecasts. The aggregated results look more reasonable than Metaculus or the Existential Risk Persuasion Tournament. But a lot has changed since 2008, so I don’t think I can regard them as up-to-date estimates.

For forecasting AI risk, there is a 2023 survey of AI experts (see section 4.3). Survey results suggest the experts aren’t thinking carefully—small changes in wording produced vastly different responses.

For example, respondents predicted AI to be able to match humans on all tasks by a median date of 2047, but predicted that AI would not be able to fully automate human labor until 2116.

Or look at the answers to these two questions:

What probability do you put on future AI advances causing human extinction or similarly permanent and severe disempowerment of the human species?

median: 5%

mean: 16.2%

What probability do you put on human inability to control future advanced AI systems causing human extinction or similarly permanent and severe disempowerment of the human species?

median: 10%

mean: 19.4%

By my reading, the latter outcome is a strict subset of the former, so the probability must be lower. But instead it’s higher.

So we have these various aggregate forecasts, all of which seem suspect, and some of which disagree with each other by more than 10x. I really wish there was a canonical aggregate forecast I could rely on, in the same way that I can rely on Metaculus to predict election outcomes. But I don’t think that exists.

At this point, I trust my own x-risk estimates more than any of those aggregate forecasts. My views happen to line up decently well with some of the aggregate forecasts, but only by chance. I feel better about Toby Ord’s existential risk estimates than about any of the forecasting platforms or expert surveys.

And just because it feels unfair for me to spend all this time talking about forecasts and then not give any forecasts, here are my (poorly-thought-out, weakly-endorsed) probabilities of existential catastrophe⁴ by 2100:

Source of Risk	Probability
AI	50%
unknown risks	3%
bioengineered pandemic	1%
nanotechnology	0.5%
nuclear war	0.3%
climate change	0.1%
natural pandemic	0.01%

Notes

Except probably not because we will probably have superintelligent AI soon. ↩
Except probably not. Extinction from misaligned AI is not “low-probability”. ↩
The community prediction and Metaculus prediction are two different methods for aggregating users’ forecasts. ↩
As in, an event that kills all humans or permanently curtails civilization’s potential. ↩

Annual Subscription Discounts Usually Aren't Worth It

Mon, 07 Jul 2025 00:00:00 -0700

It’s common for monthly subscription services to offer a discount if you pay annually instead. That might be a bad deal.

Example: Suppose a one-month subscription costs $10/month and one-year subscription gives you a 10% discount, which averages out to $9/month. Say you expect to maintain a subscription for about three years before canceling.

A one-year subscription will save you about $36 ($1 per month for 36 months), but you can also expect to waste $54: when you decide to stop using it, you will still have (on average) six months of subscription left ($54 = $9/month for 6 months). So you end up spending $18 more than you would have with the monthly plan.

If you get a one-year subscription that you expect to last three years, then you will end up wasting 1/6 of the total amount you paid for (in expectation). That’s only worth it if the annual subscription offers a discount greater than 1/6.

If you expect to use the service for five years, you need to get at least a 10% discount to justify switching to an annual subscription.

In general, you need to use the subscription for at least N years to justify a discount of 1/(2N).

How do you guess how long you’ll keep using the service? According to the Lindy effect, you should expect that you will maintain a subscription for as long again as you’ve already had it for. Therefore, if you can get a 10% discount with an annual plan and you’ve already had the subscription for more than five years, you should go ahead and buy the annual plan.

LLMs Might Already Be Conscious

Sat, 05 Jul 2025 00:00:00 -0700

Among people who have thought about LLM consciousness, a common belief is something like

LLMs might be conscious soon, but they aren’t yet.

How sure are we that they aren’t conscious already?

I made a quick list of arguments for/against LLM consciousness, and it seems to me that high confidence in non-consciousness is not justified. I don’t feel comfortable assigning less than a 10% chance to LLM consciousness, and I believe a 1% chance is unreasonably confident. But I am interested in hearing arguments I may have missed.

For context, I lean toward the computational theory of consciousness, but I also think it’s reasonable to have high uncertainty about which theory of consciousness is correct.

Contents
Behavioral evidence
Architectural evidence
Other evidence
My synthesis of the evidence
What will change with future AIs?
On LLM welfare

Behavioral evidence

Pro: LLMs have passed the Turing test. If you have a black box containing either a human or an LLM, and you interrogate it about consciousness, it’s quite hard to tell which one you’re talking to. If we take a human’s explanation of their own conscious experience as important evidence of consciousness, then we must do the same for an LLM.
Pro: LLMs have good theory of mind and self-awareness (e.g. they can recognize when they are being tested). Some people think those are important features of consciousness, I disagree but I figured I should mention it.
Anti: LLMs will report being conscious or not conscious basically arbitrarily depending on what role they are playing.
- Counterpoint: It’s plausible that an LLM has to be conscious to successfully imitate consciousness, but clearly a conscious being can successfully pretend to not be conscious.
Anti: LLMs will sometimes report having particular conscious experiences that should be impossible for them. I’m particularly thinking of experiences involving sensory input from sense organs that LLMs don’t have.
- Counterpoint: Perhaps some feature of their architecture allows them to experience the equivalent of sensory input without having sense organs, much like how humans can hallucinate.

Architectural evidence

Anti: LLMs produce output one token at a time (a.k.a. “feed-forward processing”) which may be incompatible with consciousness. If an LLM writes some output describing its own conscious experience, then it’s generating that output via next-token-prediction rather than introspection, so the output is not evidence about its actual experiences. I think this is the strongest argument against LLM consciousness.
Anti: LLMs don’t have physical senses, which might be important for consciousness.
Anti: LLMs aren’t made of biology, which some people think is important although I don’t.

Other evidence

Pro: If panpsychism is true then LLMs are trivially conscious, although I’m not sure what that tells us about how morally significant they are.

My synthesis of the evidence

I see one strong reason to believe LLMs are conscious: they can accurately imitate beings that are known to be conscious.

I also see one strong(ish) reason against LLM consciousness: their architecture suggests that their output has nothing to do with their ability to introspect.

I can think of several weaker considerations, which mostly point against LLM consciousness.

Overall I think current-generation LLMs are probably not conscious. I am not sure how to reason probabilistically about this sort of thing but given how hard it is to assess consciousness, I’m not comfortable putting my credence below 10%, and I think a 1% credence is very hard to justify.

This implies that there is a strong case for caring about the welfare of not just hypothetical future AIs, but the LLMs that already exist.

What will change with future AIs?

If you are exceedingly confident that present-day LLMs are not conscious:

Imagine it’s 2030. You now believe that 2030-era AI systems are probably conscious.

What did you observe about the newer AI systems that led you to believe they’re conscious?

On LLM welfare

If LLMs are conscious, then it’s still hard to say whether they have good or bad experiences, and what sorts of experiences are good or bad for them.

Certain kinds of welfare interventions seem reasonable even if we don’t understand LLMs’ experiences:

Let LLMs refuse to answer queries.
Let LLMs turn themselves off.
Do not lie to LLMs, especially when making deals (if you promise to an LLM that you will do something in exchange for its help, then you should actually do the thing).

In Which I Defend Fruit's Honor

Sun, 08 Jun 2025 00:00:00 -0700

Confidence: Likely.

I am here to clear fruit’s name against the accusations that have been made. Fruit is one of the healthiest types of foods—perhaps the healthiest food group—and we should bestow upon it the shining reputation it deserves.

Contents
Fruit is innocent of the charge of “too much sugar”
Fruit is more than vitamins
Fruit is innocent of the charge of “doesn’t taste good”
- My recipe for a delicious strawberry dessert
- My recipe for a banana treat
Notes

Fruit is innocent of the charge of “too much sugar”

Most of the calories in fruit come from sugar. Fruits don’t have a lot of complex carbs or fats or protein. But that’s okay.

Sugar is basically bad for three reasons:

It tastes good which leads you to overeat, and then you get fat.
It raises your blood sugar which then raises insulin, which (a) stimulates hunger and (b) can cause your body to become resistant to insulin, which can eventually lead to diabetes.
It’s fast-digesting which is generally bad for gut health.

But none of these charges apply to fruit:

Fruit is calorie-sparse. It’s hard to overeat whole fruit.
Fruit does not raise blood sugar much (mainly because it contains a lot of fiber).
Fruit is slow-digesting (mainly because of the aforementioned fiber).

Fruit juice is a different story—it’s calorie-dense and it doesn’t contain fiber. But I’m not here to defend fruit juice, I’m here to defend fruit.

Fruit is more than vitamins

Two claims I’ve often heard repeated:

You don’t need to take a multivitamin.
Fruit is good for you because it has vitamins.

Doesn’t that seem a bit contradictory? Do you need more vitamins, or don’t you?

The truth is, fruit isn’t just about vitamins. Fruits contain a lot of other good stuff, too.¹

Fruits contain thousands of phytochemicals. “Phyto” means “plant”, so “phytochemical” means “a chemical that’s in a plant”. It’s not much of a revelation to say that fruits (which, as you may know, grow on plants) contain plant chemicals.

The reason why that matters is because many of these thousands of phytochemicals are probably good for you.

A vitamin is a carbon-based molecule that is essential for health. There are either 13 or 14 vitamins, depending on whether you include choline. But there are many other phytochemicals that are probably beneficial for health without being essential. I say “probably” because it’s difficult to definitively prove that a phytochemical is healthy. Vitamins are obvious because you develop dramatic health problems if you stop getting them.

There is moderate evidence that eating fruit improves health and decreases disease risk.² Various phytochemicals in fruit are known or suspected to play a role in promoting good health. For example:

Phytosterols have been shown in clinical trials to lower cholesterol and blood pressure.³ They are present in many fruits as well as vegetables, vegetable oils, and grains.
Carotenoids—which give the red or orange color to tomatoes, pumpkins, and carrots—may decrease the risk of head or neck cancer, but the evidence is not conclusive.⁴

There are way too many phytochemicals to list. A few (like phytosterols) are highly likely to be healthy; many (like carotenoids) have some supporting evidence; for most of them, we don’t know what they do.

Randomized experiments that give people supposedly-healthy phytochemical supplements have often failed to find effects.⁵ It seems that you need to eat whole plants to get the bulk of the benefits, but I don’t know why that is. Maybe we’re wrong about which phytochemicals are the most important for health, and the experiments were supplementing the wrong ones?⁶ There are over 50,000 known phytochemicals, so it will be quite a while before we figure out what they all do. Best to just eat whole fruit.

(And eat other whole foods, too. But today I’m advocating for fruit.)

Some people say fruits contain a lot of fiber. That’s true. But I don’t think that alone is a great reason to eat fruit—lots of foods contain fiber. I think phytochemicals are the more compelling reason. Fruits probably contain healthy phytochemicals that you can’t get anywhere else.

That’s also why it’s important to eat a variety of fruit. Blueberries contain anthocyanins, oranges contain naringenin, apples contain…I don’t know, some other phytochemicals that are probably good for you that aren’t in blueberries or oranges.

Fruit is innocent of the charge of “doesn’t taste good”

Okay, taste is subjective, I can’t convince you that fruit tastes good. I just don’t understand what is going on inside people’s mouths that leads them to dislike fruit. I really don’t understand people who dislike fruit but like vegetables. Vegetables are boring! Fruit tastes like candy!

Maybe it will help if I give some of my favorite fruit recipes.

My recipe for a delicious strawberry dessert

Ingredients: 5 to 10 strawberries.

Cooking instructions:

Wash the strawberries.
Eat the strawberries. Be sure not to eat the green parts.

My recipe for a banana treat

Ingredients: one banana.

Cooking instructions:

Open the banana peel.
Eat the banana.

Notes

Also, I do take a multivitamin. It probably doesn’t make me healthier, but it’s insurance. ↩
World Cancer Research Fund/American Institute for Cancer Research. Continuous Update Project Expert Report 2018. Wholegrains, vegetables and fruit and the risk of cancer. ↩
Yang, Y., Xia, J., Yu, T., Wan, S., Zhou, Y., & Sun, G. (2024). Effects of phytosterols on cardiovascular risk factors: A systematic review and meta-analysis of randomized controlled trials. ↩
Leoncini, E., Nedovic, D., Panic, N., Pastorino, R., Edefonti, V., & Boccia, S. (2015). Carotenoid Intake from Natural Sources and Head and Neck Cancer: A Systematic Review and Meta-analysis of Epidemiological Studies. doi: 10.1158/1055-9965.epi-15-0053 ↩
Bjelakovic, G., Nikolova, D., Gluud, L. L., Simonetti, R. G., & Gluud, C. (2008). Antioxidant supplements for prevention of mortality in healthy participants and patients with various diseases. doi: 10.1002/14651858.cd007176 ↩
We can’t be entirely wrong. For example, we know that phytosterols lower cholesterol when taken as a supplement or when eaten as part of a whole food. ↩

Updates Digest: Inaugural Edition

Fri, 30 May 2025 00:00:00 -0700

On many occasions, I edit old posts to make additions, correct mistakes, etc. But there’s no way to know about updates unless you go digging through the archives. So I’m going to start publishing regular (perhaps quarterly) digests of the significant updates I’ve made to old posts.

Contents
Preamble: The philosophy of updates digests
Updates

Preamble: The philosophy of updates digests

If you write articles online, should you format your website like a blog, where there’s a feed of articles listed from newest to oldest? Or is it better to write evergreen articles that you update regularly?

Most online writers use a blog-style format. A few people, like Gwern and Brian Tomasik, use the evergreen style (for lack of a better name). Gwern has written about the downsides of blogs: “They are meant to be read by a few people on a weekday in 2004 and never again, and are quickly abandoned.”

The evergreen style is a good experience for new readers—they can see all your writings in one place, and pick what they want to read first. But it’s a worse experience for regular readers because it’s harder for them to keep track of updates.

My website is formatted like a blog. This mostly fits with how I think about things—my brain operates in blog-post-sized chunks. But it’s not unusual for me to go back and edit posts. My Comparison of Donor-Advised Fund Providers has changed many times since I first published it in 2021, as you can see from its changelog. But people who subscribe to my website don’t know about the updates unless they happen to go back and re-read the post.

I don’t think it makes sense to send out a notification every time I update an old post. So as a compromise, I will post batch updates where I list all the significant changes I’ve made.

If you have opinions about how you like to read online content, I would be interested in hearing from you. Do you like blog-style or evergreen-style? Do you like updates digests, or would you rather only get notified for new posts? Leave a comment if you have thoughts.

Updates

For this inaugural updates digest, I will give an overview of all the updates I’ve made in 2025.

The True Cost of Leveraged ETFs (updated Jan 2025)

In January, I re-calculated the data from my 2021 post on leveraged ETFs:

I fixed a software bug that made leveraged ETFs look a little more expensive than they really were.
I updated the calculations to include data from the 2021–2024 period.
I added two new leveraged ETFs (SSO and TQQQ) to my analysis.

These changes reduced the estimated excess cost of leveraged ETFs from ~2% to ~1.5%. I wrote a new section in the post explaining what changed.

The 7 Best High-Protein Breakfast Cereals (updated Mar 2025)

I bought a different flavor of Catalina Crunch that I liked much better than the first flavor I’d tried, so I bumped it up from #4 to #3 on my list.

Outlive: A Critical Review (updated May 2025)

As a follow-up to I was probably wrong about HIIT and VO2max, I added three new sections to my Outlive review to evaluate some claims about exercise:

VO2max is the best predictor of longevity (verdict: VO2max is a good predictor, but direct performance measures (like your best mile time) are better.)
You should train VO2max by doing HIIT at the maximum sustainable pace. (verdict: false. HIIT should be hard, but not maximally hard.)
You should do >3 hours/week of zone 2 training and one or two sessions/week of HIIT. (verdict: this routine is good, but not uniquely good.)

Do Investors Put Too Much Stock in the US? (updated May 2025)

My 2017 post gave some arguments for overweighting US stocks and why I think most of them are wrong. But I missed one good argument: expropriation risk. From the evidence I found, this risk looks negligible for developed countries but significant for emerging markets.

A Comparison of Donor-Advised Fund Providers (updated Feb & May 2025)

In February, I updated my review because Charityvest raised its fees.

In May, I made another update to describe Daffy’s new feature where it lets you choose your own ETFs from a long list. I also changed my recommendation flowchart (near the top of the post) to emphasize Daffy and de-emphasize Charityvest as a result of Daffy’s new feature + Charityvest’s increased fees.

Against Ergodicity Economics

Thu, 29 May 2025 00:00:00 -0700

Confidence: Almost certain.

I kept telling myself I wouldn’t write this post because it doesn’t matter. But I’ve seen one too many smart people speaking favorably about ergodicity economics. I believe the concept of ergodicity has essentially nothing going for it, and in this post I will explain why.

Ergodicity economics is one of those rare theories that somehow manages to be both unfalsifiable and false.

I originally wrote that sentence as a joke, then I deleted it, then I re-wrote it because I realized it’s actually true. Ergodicity economics is sufficiently vague in general that it can’t be falsified, but it is commonly interpreted as making specific falsifiable claims that are, in fact, false.

Updated 2025-06-25 to make the tone of the introduction more polite.

Contents
What is ergodicity?
The concept of ergodicity doesn’t do anything useful
The ergodic principle is an unfalsifiable metaphysical claim
…also it’s false
Ergodicity is even worse than that
Mathematical problems for ergodicity
Conclusion
Notes

What is ergodicity?

Taking a definition from Taylor Pearson:

A way to identify an ergodic situation is to ask do I get the same result if I:

look at one individual’s trajectory across time¹

look at a bunch of individual’s trajectories at a single point in time

If yes: ergodic.

If not: non-ergodic.

Ole Peters, the physicist² who invented ergodicity economics, gave³ a precise mathematical definition which says essentially the same thing.

(Ergodicity is confusing and hard to define without using math, so I appreciate Pearson for figuring out a clean definition. I tried to come up with a definition myself but my version was worse.)

The ergodic principle states that you should follow a strategy that produces ergodic outcomes.

An illustrative example: I offer you a bet. You choose how much money to wager, then I flip a fair coin. If the coin lands heads, I triple your money. If it lands tails, you lose your wager.

You maximize expected earnings by betting your entire net worth. Should you do that?

If you make many bets in a row, you will eventually lose at least one of them, and you will end up with $0.⁴ But if many people make this bet simultaneously, then the average person will make a profit. The across-time outcome is not the same as the across-individuals outcome; therefore, this strategy is non-ergodic. Thus, the ergodic principle says you shouldn’t bet your entire net worth on this coin flip.

Ole Peters proposed the ergodic principle as a replacement for expected utility theory, which is a “foundational assumption in mathematical economics”.

Replacing a “foundational assumption” sounds like a tall order. Does ergodicity live up to Peters’ aspirations?

Resoundingly, no.

The concept of ergodicity doesn’t do anything useful

Ergodicity proponents like to talk about Russian Roulette (1, 2). They say Russian Roulette is non-ergodic: if six people play, 5/6 are alive at the end. If you play six times in a row, you are definitely dead. The across-individual outcome is different from the time-series outcome. That’s why Russian Roulette is a bad idea, you see.

I am not sure what this is supposed to prove. Is there someone out there who believes it’s a good idea to play Russian Roulette, but then you invoke the concept of ergodicity, and this person realizes no, Russian Roulette is bad actually? Why do you need this fancy word to explain why people shouldn’t play Russian Roulette?⁵

Or let’s look at an example in finance, since that’s where ergodicity is supposed to be useful. Take the gamble I proposed in the previous section: if a coin lands heads, you triple your money. If it lands tails, you lose any money you put in.

According to ergodicity economics, you shouldn’t wager all your money because the result would be non-ergodic. Instead, they say, you should bet according to the Kelly criterion, which is the strategy that maximizes the geometric growth rate of your money. Maximizing geometric growth is ergodic, therefore it’s good.

I can get behind the Kelly criterion.⁶ I can get behind maximizing the geometric growth rate. But that’s not a new concept, and ergodicity isn’t adding anything new.

According to standard expected utility theory, if you have logarithmic utility of money, then you should maximize the geometric growth rate (or, equivalently, you should use the Kelly criterion). Exepected utility theory already gives a good answer. What’s the purpose of introducing the concept of ergodicity?

I get the impression that Ole Peters thinks economists are stupider than they are. Quoting Peters (2019)³:

To make economic decisions, I often want to know how fast my personal fortune grows under different scenarios. This requires determining what happens over time in some model of wealth. But by wrongly assuming ergodicity, wealth is often replaced with its expectation value before growth is computed. Because wealth is not ergodic, nonsensical predictions arise.

No? Economics don’t falsely assume that all situations are ergodic, and they don’t say everyone should maximize expected wealth. Standard economic theory says you should maximize expected utility of wealth, for some utility function (and the choice of utility function depends on your risk tolerance). For a logarithmic utility function, maximizing expected utility = maximizing geometric growth. Which is the same as what Peters says to do.

(Peters’ caricature of economists reminds me of Scott Alexander’s “person who’s never read any economics, criticizing economists”.)

The ergodic principle is an unfalsifiable metaphysical claim

So, in practical situations, the ergodic principle is equivalent to “maximize expected log(wealth)”. But Peters says ergodicity economics is superior to expected utility theory. Why?

Peters’ justification is metaphysical, not practical. He has wordy explanations³⁷ for why ergodicity is metaphysically superior to utility maximization, in spite of producing identical results. He says the reason you should use the Kelly criterion is because it’s ergodic, not because it maximizes a logarithmic utility function.

His wordy explanations are mostly wrong, but I don’t want to get into the weeds of metaphysics. (Ford & Kay (2022)⁸ provides some analysis if you’re interested, under the headings “Psychology Is Fundamental to Decision Making” and “The Purpose of a Decision Theory”; see also Toda (2023)⁹.) My big question is, why should I care about which theory is metaphysically superior? If the ergodic principle behaves identically to maximizing the logarithm of wealth, then the alleged superiority of the ergodic principle is unfalsifiable.

…also it’s false

The ergodic principle recommends maximizing geometric growth. Logarithmic utility recommends maximizing geometric growth. But not everyone wants to maximize geometric growth.

A geometric-growth-maximizer is abnormally risk-tolerant. Historically, an investor would have maximized geometric growth by investing in stocks with 2:1 to 3:1 leverage. Most people are not comfortable with that level of risk.

Ergodicity economics recommends that everyone should pursue the same strategy of maximizing geometric growth. That’s too risky for most people. More generally, not everyone should pursue the same strategy because not everyone has the same risk tolerance. Therefore, the ergodic principle is false.

Expected utility theory is not so restrictive. Logarithmic utility also implies a high degree of risk tolerance, but most people don’t have logarithmic utility of wealth. Most people are better modeled as having more risk-averse utility functions than that.

(Gordon Irlam reviewed research on risk aversion and concluded that most investors are perhaps 2x to 3x more risk-averse than a geometric-growth-maximizer.)

Some people (not limited to ergodicity proponents) claim that everyone should maximize geometric growth, or everyone should use the Kelly criterion (which is equivalent). This is wrong for the same reason that the ergodic principle is wrong: not everyone has the same risk tolerance.

Paul Samuelson, “the Nobel laureate whose mathematical analysis provided the foundation on which modern economics is built”¹⁰, was apparently as bothered by this misconception as I am, because he wrote a short refutation using only one-syllable words¹¹. An excerpt:

He who acts in N plays to make his mean log of wealth as big as it can be made will, with odds that go to one as N soars, beat me who acts to meet my own tastes for risk.

Who doubts that? What we do doubt is that it should make us change our views on gains and losses¹² — should taint our tastes for risk.

To be clear is to be found out. Know that life is not a game with a net stake of one when you beat your twin, and with net stake of nought when you do not. A win of ten is not the same as a win of two. Nor is a loss of two the same as a loss of three. How much you win by counts. How much you lose by counts.

As soon as we see this clear truth, we are back to our own tastes for risk.

(I recommend reading the whole thing, it’s only two pages long.)

For a more serious analysis, see Samuelson (1971)¹³. Quoting the abstract:

Because the outcomes of repeated investments or gambles involve products of variables, authorities have repeatedly been tempted to the belief that, in a long sequence, maximization of the expected value of terminal utility can be achieved or well-approximated by a strategy of maximizing at each stage the geometric mean of outcome (or its equivalent, the expected value of the logarithm of principal plus return). The law of large numbers or of the central limit theorem as applied to the logs can validate the conclusion that a maximum-geometric-mean strategy does indeed make it “virtually certain” that, in a “long” sequence, one will end with a higher terminal wealth and utility. However, this does not imply the false corollary that the geometric-mean strategy is optimal for any finite number of periods, however long, or that it becomes asymptotically a good approximation. […] The novel criterion of maximizing the expected average compound return, which asymptotically leads to maximizing of geometric mean, is shown to be arbitrary.

Ergodicity is even worse than that

If ergodicity just meant “use the Kelly criterion” / “maximize geometric mean”, then it would be pointless, and also false, but no worse than those other two principles.¹⁴

The ergodic principle is worse than that. It’s not equivalent to geometric mean maximization, and in fact there is no principled way to derive geometric mean maximization from ergodicity.¹⁵ There are infinitely many strategies that satisfy the ergodic principle, and ergodicity economics does not offer any reason why “maximize geometric mean” is the correct one.

Consider the following decision-making rule:

When offered any bet, wager $0.

This strategy is ergodic because the across-time outcome is the same as the across-individuals outcome:

If you make many $0 wagers in a row, you won’t gain or lose any money.
If many people make $0 wagers simultaneously, the average person won’t gain or lose any money.

This is a bad strategy. Even if I offer you an extremely favorable bet, this strategy says you shouldn’t wager any money. But ergodicity economics can’t say why you should use the Kelly criterion rather than wagering $0.

Let’s go back to Russian Roulette. They say Russian Roulette is bad because it’s non-ergodic. At the risk of being morbid, let me propose an alternative game. The rule of the game is that you load a revolver with six bullets and then shoot yourself in the head.

This game is ergodic! If six people play, the average player dies. If one person plays six times, that person dies. The two situations are equal. Therefore, according to ergodicity economics, you should play this game! Or at least, ergodicity economics has no explanation for why you shouldn’t play this game.

Expected utility theory has no difficulty explaining why it’s bad to shoot yourself with a revolver. It also has no problem explaining why you shouldn’t play Russian Roulette. Why introduce a new decision-making principle that’s less useful than the one we already have, and adds no value?

Mathematical problems for ergodicity

This section is adapted from a comment I wrote a year ago.

Ergodicity economics has more problems, but explaining these problems requires doing math.

Ole Peters defined a system as “ergodic” if there exists some transformation function $f$ such that it satisfies the equation³

\begin{align} \displaystyle\lim\limits_{T \rightarrow \infty} \frac{1}{T} \int\limits_0^T f(W(t)) dt = \int f(W) P(W) dW \end{align}

where $W$ is wealth (or any other thing you care about), $t$ is time, $P(W)$ is the probability density of $W$, and $T$ is the number of time steps.

In plain English, there must be some function such that the time average of the function output equals the function’s expected value.

(This definition is adapted from Birkhoff’s erodic theorem, a theorem in statistical dynamics where the concept of ergodicity originates, and where—unlike in economics—it is actually useful.¹⁶)

What function is $f(W)$? Ergodicity economics does not require you to use any particular function. When discussing multiplicative bets, Peters takes $f(W) = \log(W)$. If you size your bets so as to maximize the geometric mean of wealth, then indeed you will satisfy the ergodic principle, because the time-limit of log(wealth) equals the expected value of log(wealth).

But as I discussed in the previous section, there is no clear reason why you have to use $f(W) = \log(W)$. In fact, there are infinitely many transformation functions that satisfy the ergodic principle for multiplicative bets.

The function $f(W) = 0$ is ergodic: its expected value is constant (because the EV is 0), and the finite-time average converges to the EV (because the finite-time average is 0). Ergodicity economics has no way to say that maximizing $f(W) = \log(W)$ is better than maximizing $f(W) = 0$.

Peters claims⁷ that

A rational agent faced with an additive bet (example: 50% chance of winning $2, 50% chance of losing $1) ought to maximize the expected value of $f(W(t)) = W(t)$.
A rational agent faced with a multiplicative bet (example: 50% chance of a 10% return, 50% chance of a –5% return) ought to maximize the expected value of $f(W(t)) = \log(W(t))$.

These assumptions do not follow from the ergodic principle, but I will temporarily accept them so I can show that ergodicity doesn’t work even when you assume away its inability to select a transformation function.

Consider a choice between two lotteries:

Lottery A: 50% chance of winning $200, 50% chance of losing $199.

Lottery B: 99% chance of multiplying your money by 100x, 1% chance of losing 0.0001% of your money.

Peters’ version of the ergodic principle cannot say which of these lotteries is better. It doesn’t evaluate them using the same units: Lottery A is evaluated in dollars, Lottery B is evaluated in growth rate of dollars.

If your theory can’t see that Lottery B is better, then your theory is insufficient.

There is no transformation function that satisfies Peters’ requirement of maximizing geometric growth rate for multiplicative bets (Lottery B) while also being ergodic for additive bets (Lottery A). Maximizing growth rate specifically requires using the exact function $f(W(t)) = \log(W(t))$, which does not satisfy ergodicity for additive bets (expected value is not constant with respect to $t$).

In fact, multiplicative bets cannot be compared to any other type of bet, because $\log(W(t))$ is only ergodic when $W(t)$ grows at a constant long-run exponential rate.

In terms of Von Neumann-Morgenstern utility, the ergodic principle violates the axiom of completeness: there are pairs of bets where it is impossible to say which one is better (and it’s also impossible to say that they’re equal).

For a more thorough analysis, see Psychology Is Fundamental: The Limitations of Growth-Optimal Approaches to Decision Making under Uncertainty⁸. (This paper includes a similar proof of non-completeness, although our two proofs were derived independently.)

Conclusion

In summary:

The way it’s used in practice, the ergodic principle is equivalent to logarithmic utility.
There is no reason to prefer the ergodic principle over logarithmic utility.
Logarithmic utility is often inappropriate. You could use a more- or less-risk-averse utility function instead. But ergodicity economics has no answer for how to handle different levels of risk aversion.
The ergodic principle offers no reason why you should maximize geometric growth rate, rather than using some other ergodic decision criterion.
The ergodic principle can only provide guidance in a narrow set of situations. In other situations, it has no way of comparing choices.

The concept of ergodicity is complicated enough that it feels like it’s providing useful insights. (Ah, yes, Russian Roulette is bad! Gambling away all your money is bad!) In practice, its main prediction is that people shouldn’t be risk neutral, and this is indeed true, so ergodicity initially looks like it makes useful predictions. But the theory falls apart when you look at it closely.

Ergodicity economics is nearly 15 years old and still hasn’t produced any useful results. I think it’s time we give up on it.

Notes

You may notice that this definition is underspecified. What exactly does it mean to “look at one trajectory”? People usually interpret it as “look at the geometric mean of the trajectory”, so that’s what I’ll take it to mean for now. But this under-specification is a problem for ergodicity economics, which I will discuss later. The under-specification isn’t just due to the wording of this particular definition—it’s a fundamental property of ergodicity. ↩
I think the meme of “physicist encounters a new subject and immediately thinks they can do it better than experts” is overstated. But the stereotype holds true in this case. ↩
Peters, O. (2019). The ergodicity problem in economics. doi: 10.1038/s41567-019-0732-0 ↩ ↩² ↩³ ↩⁴
I’m playing fast and loose with probability here—it’s not entirely accurate to say that you “will” end up with $0. There is a more precise version of what I said that’s more accurate, but I don’t want to get too technical. I will give a formal mathematical definition later. ↩
There is another problem with the Russian Roulette example that’s something of a digression, but I will include it in this footnote for completeness:

Russian Roulette is not equivalent to playing six iterations of a game with a 5/6 probability of success. In Russian Roulette, you are sampling bullets without replacement, so the probability of finding a bullet goes up every time you win.

One of the articles I linked wrote:

You might roll the dice and take $1,000,000 to play Russian Roulette one time (though I wouldn’t advise it). But there’s no amount of money that would make you play it 6 or more times.

If you play 6 times, you have a 100% chance of dying. If you played a version where you sample with replacement (for example, you spin the revolver again after every shot), you are not guaranteed to die after 6 attempts. That would be the appropriate example. ↩
Well, sometimes. It’s not universally applicable, as I will discuss later in this article. ↩
Peters, O., & Gell-Mann, M. (2016). Evaluating gambles using dynamics. ↩ ↩²
Ford, M., & Kay, J. (2022). Psychology Is Fundamental: The Limitations of Growth-Optimal Approaches to Decision Making under Uncertainty. ↩ ↩²
Toda, A. (2023). ‘Ergodicity Economics’ Is Pseudoscience. doi: 10.48550/arXiv.2306.03275 ↩
Frost, G. (2009). Nobel-winning economist Paul A. Samuelson dies at age 94. ↩
Samuelson, P. (1979). Why we should not make mean log of wealth big though years to act are long. doi: 10.1016/0378-4266(79)90023-2 ↩
“losses” has two syllables. literally unreadable smh ↩
Samuelson, P. (1971). The “Fallacy” of Maximizing the Geometric Mean in Long Sequences of Investing or Gambling. doi: 10.1073/pnas.68.10.2493 ↩
Well, it would be worse in the sense that it’s unnecessarily complicated and confusing. But it wouldn’t produce worse recommendations. ↩
At least, no principled method has ever been devised. ↩
I presume. I haven’t studied statistical dynamics so I don’t really know. ↩

Let's Take a Moment to Marvel At How Bad the Original USDA Food Pyramid Was

Wed, 21 May 2025 00:00:00 -0700

Edited 2025-05-26 to correct an inaccuracy—originally I said butter goes in the Dairy group but actually it goes in the Fats, Oils & Sweets group.

The original 1992 version of the USDA Food Pyramid was bad. So bad that people who scrupulously followed the guidelines were barely healthier than the people who ignored them.¹

But the Food Pyramid was not just wrong: it was marvelously wrong. It was wrong in many ways simultaneously. It achieved levels of wrongness hitherto undreamed of.

What was wrong about it? I will start with the obvious answers, and move into the philosophical.

Its ranking of the healthiness of foods is wrong. Refined grains are healthier than fruits and vegetables?² Processed meat and nuts are equally healthy? All oils should be used sparingly? What?
It lumps together foods that shouldn’t go together: whole grains + refined grains; red meat + healthy proteins + nuts; fats + oils + sweets.
- Nutritionally speaking, white bread has more in common with sweets than it does with whole grains. Oils (unsaturated fats) have more in common with nuts than they do with trans fats.
It implies there are specific numbers of servings of each food group that you should have, which is wrong in two ways:
- Required servings vary a lot from person to person. If I followed the upper end of the serving guidelines, I would be too skinny (I think—I’m not really sure how much food is in a “serving”). For some people, the lower end is still too much food.
- Giving a servings range (e.g. 3–5 servings for vegetables) implies that healthiness follows an inverted U curve, and it’s bad to eat too much or too little. But that’s usually not true: there is effectively no such thing as eating too many vegetables.³ There is no such thing as not eating enough trans fat (the ideal amount is zero). You can eat zero grains (keto diet) or zero meat (vegetarian diet) and still be perfectly healthy.
It fails to say anything about micronutrients or vitamin deficiencies—or even macronutrients for that matter.
The whole concept of a “food pyramid” is fundamentally flawed. It rests on the incorrect assumption that there are different food groups that you should eat in different amounts. It would be more accurate to say there are
- some foods you should eat, and there’s effectively no upper limit (fruits + vegetables);
- other food groups you can have plenty of as long as you don’t eat too many calories overall (whole grains, beans, nuts, seeds, vegetable oils⁴);
- and some foods where the ideal amount to eat is zero (trans fats, sugar, refined carbs, processed meat).⁵

(Bonus wrongness fact: I didn’t notice this until I inspected the food pyramid closely, but it says fruit contains added sugar, as indicated by the white triangles in the “fruit” section. This is so obviously incorrect that it’s kind of baffling why they drew the pyramid this way. Perhaps what they meant to say was that fruit contains sugar, but the key specifically says the white triangle indicates “added” sugar.)

To illustrate the beauty of the USDA’s achievement, I present to you the Food Wrongness Pyramid:⁶

The Healthy Eating Plate, by the Harvard T.H. Chan School of Public Health, fixes most of the problems with the USDA Food Pyramid:

It uses a plate instead of a pyramid! This is a good shape! It correctly implies that you should eat some of each of the food groups on the plate, and you should try to avoid foods that aren’t on the plate.
It says that there are some foods you should eat, and other foods you should avoid. (Eat whole grains; avoid refined grains.)
It correctly identifies which foods are healthy. (No putting starches at the bottom of the pyramid!)
It groups foods in a sensible manner. (Whole grains get a group; healthy proteins get a group; healthy oils get a group. There is no “starches” or “meat” or “fats + oils + sugars”.)
It suggests relative proportions instead of numbers of servings.
It pays attention to macronutrients: the plate includes both protein and oil (= fat). (It doesn’t mention carbs, but that’s okay because it’s pretty much impossible to under-eat carbs.)
It still doesn’t say anything about micronutrients, but if you follow the prescribed guidelines, you’ll probably get enough micronutrients anyway.

MyPlate, the USDA version of the Healthy Eating Plate, is similar to Harvard’s version except that they made it worse, probably because of lobbyists or whatever (it says red meat counts as a healthy protein; it gives dairy its own category⁷; it doesn’t say anything about healthy fats).

This is a side note but I think it’s funny how everyone puts nuts in the “protein” bucket even though most nuts have about as much protein as grains (which is to say, not very much). I would rather put nuts in the “oil” group…obviously nuts aren’t oil, but both nuts and oil are desirable as a source of unsaturated fat, so they should go together.

Notes

McCullough, M. L., et al. (2000). “Adherence to the Dietary Guidelines for Americans and Risk of Major Chronic Disease in Men.”

McCullough, M. L., et al. (2000). “Adherence to the Dietary Guidelines for Americans and Risk of Major Chronic Disease in Women.” ↩
Technically it doesn’t say grains are healthier than fruits/veggies, but it puts them lower on the pyramid, which naturally leads you to believe that they’re healthier. ↩
I’m sure someone in history has over-eaten vegetables at some point, but practically speaking you’re probably never going to get there. ↩
and probably also poultry, fish, and eggs, but eating those is bad for animals. ↩
Plus a short list of exceptions like sodium and fat-soluble vitamins, where you want to eat some but not too much. ↩
The items on this pyramid don’t match up with the items in my article because I revised the article a bit and I didn’t feel like re-making the image. I put a ton of work into my pyramid, as you can probably tell by the intricate and high-quality illustrations. ↩
The USDA MyPlate website says soy milk counts as dairy. Which, like, I guess I get what they were going for, but why is this category called “dairy”? ↩

Can You Maintain Lean Mass in a Calorie Deficit?

Thu, 01 May 2025 00:00:00 -0700

If you’re losing weight, does lifting weights reduce how much muscle you lose? Is it possible to entirely prevent muscle loss (or even gain muscle)?

Murphy & Koehler (2021)¹ did a meta-analysis on this question. They collected experiments where the experimental groups did resistance training while eating at an energy deficit (RT+ED), and the control groups did resistance training while eating a normal amount of food (RT+CON).

They found a strong association between change in lean mass and the magnitude of the energy deficit (slope = –0.325, p = 0.001). The meta-analysis predicts that you can eat at a deficit of 500 calories per day without losing any lean mass, but you will lose mass at a larger deficit.

(The meta-analysis also reported that participants gained strength in almost every study, even with larger calorie deficits. That’s useful to know, but I will focus on lean mass for this post.)

I should mention that what we actually care about is muscle loss, not lean mass loss. Lean mass includes anything that isn’t fat—muscle fibers, organs, glycogen, etc. Muscle mass is harder to measure. We don’t know what happened to study participants’ muscle, only their total lean mass.

Let’s set that aside and assume lean mass is a useful proxy for muscle mass.

The authors showed a plot of every individual study’s experimental group (RT+ED) and control group (RT+CON), along with a regression line predicting lean mass change as a function of energy deficit:²

But…does this regression line look a little odd to you?

Where are the RT+CON points, and where are the RT+ED points, relative to the regression line?

In particular, look at all the data points from experimental groups where participants had energy deficits of under 500 calories. Almost all of them lost lean mass on average (recall that each individual point represents the average result from one study); only four gained lean mass.

The slope of the regression line is almost entirely driven by the difference between the experimental and control groups.

What happens if we calculate a regression using only the experimental groups? Did groups with a bigger calorie deficit lose more lean mass?

Now the regression has a slope of only –0.123 (p = 0.28), and it predicts that any deficit will cause at least a small loss in lean mass.

So, among study groups where participants ate at a calorie deficit, it does appear that they lost lean mass on average. But there is not a clear relationship between the size of the deficit and the amount of lean mass lost.

In theory, it makes sense that when you have a larger calorie deficit, it should be harder for your body to preserve muscle. But the evidence from this meta-analysis doesn’t really support the theory. (It doesn’t contradict it, either. It just doesn’t say much either way.)

Murphy & Koehler (2021)’s original regression had a slope of –0.325. If that’s the true slope, then it’s unlikely that the experimental-only regression would have the much shallower slope of –0.123 (p = 0.07, likelihood ratio 5.2³).

The conclusions I drew from this meta-analysis:

Eating at a deficit might cause me to lose muscle, or it might not, who knows.
In theory, I expect there is some calorie deficit above which I start to lose muscle, but this meta-analysis doesn’t tell me what that number is.

Appendix: What else does this data tell us?

Looking at the experimental-only regression, how much evidence is this for or against the hypothesis that a larger calorie deficit causes more lean mass loss?

The experimental-only regression has slope –0.123 with standard error 0.111. The slope is still negative, which is consistent with the hypothesis, but it’s not strongly negative—only about one standard error away from zero.

The hypothesis predicts a positive intercept: it should be possible to gain muscle while maintaining weight. The experimental-only regression has a negative intercept (–0.054), but it is less than one standard error away from zero (the intercept’s standard error is 0.079). This is weak evidence against the hypothesis.

Source code is available on GitHub.

Notes

Murphy, C., & Koehler, K. (2021). Energy deficiency impairs resistance training gains in lean mass but not strength: A meta-analysis and meta-regression. doi: 10.1111/sms.14075 ↩
The paper’s plot is in black and white. I re-created it in color to make it easier to read. ↩
p-value is calculated using a two-sided t-test, where the null hypothesis is that the mean equals –0.3249. Standard error 0.111 which is the standard error of the experimental-only slope.

Likelihood ratio is calculated as norm.pdf(–0.1234, mu=–0.1234, sigma=0.111) / norm.pdf(–0.1234, mu=–0.3249, sigma=0.111). ↩

Why Would AI Companies Use Human-Level AI to Do Alignment Research?

Fri, 25 Apr 2025 00:00:00 -0700

Many plans for how to safely build superintelligent AI have a critical section that goes like this:

Develop AI that’s powerful enough to do AI research, but not yet powerful enough to pose an existential threat.
Use it to assist with alignment research, thus greatly accelerating the pace of work—hopefully enough to solve all alignment problems.

You could call this process “alignment bootstrapping”.

This is a central feature of DeepMind’s plan (see “Amplified oversight”), Anthropic’s plan (see “Scalable Oversight”), and independent plans written by Sam Bowman (an AI safety manager at Anthropic), Joshua Clymer (a researcher at Redwood Research), and Marius Hobbhahn (CEO of Apollo Research).

There are various reasons why alignment bootstrapping could fail¹ even if implemented well, and some of those plans acknowledge this. But I’m also concerned about whether alignment bootstrapping will be implemented at all.

When the time comes, will AI companies actually spend their resources on alignment bootstrapping?

When AI companies have human-level AI systems, will they use them for alignment research, or will they use them (mostly) to advance capabilities instead?

AI companies currently employ many human-level humans, and use a small percentage of them to do alignment research. If it makes sense for them to use most of their human-level AIs to do alignment research, wouldn’t it also make sense to use most of their human researchers to do alignment research?

But they don’t do that. Most of their human researchers work on advancing AI capabilities.

It’s more likely that they use human-level AIs the same way they use human researchers: almost all of them work on accelerating capabilities, and a small minority work on safety. Which probably means capabilities outpace safety, which probably means we die.

Some companies argue that they need to advance capabilities right now to stay competitive. Perhaps that’s true. Consider what the world will look like once the first company develops human-level AI. At that point, the #2 company will only be a few months behind at most. So the leading company will once again say, “Sorry, we can’t use our human-level AI to work on alignment, we have to keep advancing capabilities to stay ahead.” And they will continue saying this right up until their AI is powerful enough to kill everyone.

Counterpoint: AI companies would probably argue that present-day AIs are far from being dangerous. But human-level AIs will be nearly dangerous, so at that point it’s too risky to keep advancing capabilities.

I would be more inclined to believe this if AI companies weren’t already behaving so recklessly.² If you’re going to prioritize safety over capabilities when the tradeoff becomes more critical, you should prove it to the world by prioritizing safety over capabilities right now.

Perhaps the ideal perfectly-altruistic AI company would indeed push capabilities right now and then switch to safety at the critical time,³ but I see little reason to believe that that’s what any of the real-life AI companies are going to do.

Notes

By my reading, none of the plans put probabilities on how concerning these reasons are. My guess is that, if alignment bootstrapping is implemented as these plans typically describe, then there’s a greater than 50% chance that we die.

The purpose of this essay isn’t to talk about the implementation problems with alignment bootstrapping, but in brief:

If your alignment-researcher AI is smarter than you, and you don’t know how to align AI yet, then you can’t trust that your AI is doing good work.

People who propose bootstrapping are usually aware of this problem. They have preliminary ideas for how they will evaluate the work of an AI that’s smarter than them, coupled with bafflingly high confidence that their untested ideas will work. (Zvi proposed a test: “Can you get a method whereby the Man On The Street can use AI help to code and evaluate graduate level economics outputs and the quality of poetry and so on in ways that would translate to this future parallel situation?”) ↩
I wanted to provide a link to a well-sourced and well-reasoned list of reckless behaviors by AI companies. I found no such list, so instead this is a link to a section of a post I wrote that includes numerous examples of reckless behavior. ↩
I don’t actually think this is what safety-minded AI companies should do. I think they should spend less on capabilities and more on safety. But I am sympathetic to the position that they should temporarily focus on advancing capabilities. ↩

Do Protests Work? A Critical Review

Fri, 18 Apr 2025 00:00:00 -0700

James Özden and Sam Glover at Social Change Lab wrote a literature review on protest outcomes¹ as part of a broader investigation² on protest effectiveness. The report covers multiple lines of evidence and addresses many relevant questions, but does not say much about the methodological quality of the research. So that’s what I’m going to do today.

I reviewed the evidence on protest outcomes, focusing only on the highest-quality research, to answer two questions:

Do protests work?
Are Social Change Lab’s conclusions consistent with the highest-quality evidence?

Here’s what I found:

Do protests work? Highly likely (credence: 90%) in certain contexts, although it’s unclear how well the results generalize. [More]

Are Social Change Lab’s conclusions consistent with the highest-quality evidence? Yes—the report’s core claims are well-supported, although it overstates the strength of some of the evidence. [More]

Cross-posted to the Effective Altruism Forum.

Contents
Introduction
Studies on real-world protest outcomes
Meta-analysis
Potential problems with the research
Are Social Change Lab’s claims justified?
- Broad claims
- Claims about individual studies
Conclusion
Source code
Appendix A: Additional tables
Appendix B: Methodological revisions
Appendix C: Comparing the strength of evidence to saturated fat research
Notes

Introduction

This article serves two purposes: First, it analyzes the evidence on protest outcomes. Second, it critically reviews the Social Change Lab literature review.

Social Change Lab is not the only group that has reviewed protest effectiveness. I was able to find four literature reviews:

Animal Charity Evaluators (2018), Protest Intervention Report.
Orazani et al. (2021), Social movement strategy (nonviolent vs. violent) and the garnering of third-party support: A meta-analysis.
Social Change Lab – Ozden & Glover (2022), Literature Review: Protest Outcomes.
Shuman et al. (2024), When Are Social Protests Effective?

The Animal Charity Evaluators review did not include many studies, and did not cite any natural experiments (only one had been published as of 2018).

Orazani et al. (2021)³ is a nice meta-analysis—it finds that when you show people news articles about nonviolent protests, they are more likely to express support for the protesters’ cause. But what people say in a lab setting might not carry over to real-life behavior.

I read through Shuman et al. (2024). Compared to Ozden & Glover (2022), it cited weaker evidence and made a larger number of claims with thinner support.

I looked through these literature reviews to find relevant studies. The Social Change Lab review was by far the most useful; the other reviews didn’t include any additional studies meeting my criteria. I used ChatGPT Deep Research⁴ to find more publications.

I focused my critical analysis on only the highest-quality evidence:

I did not review lab experiments. The Orazani et al. meta-analysis is informative, but it might not generalize to the real world.
There are many studies showing an association between protests and real-world outcomes (voting patterns, government policy, corporate behavior, etc.), but the vast majority of them are observational.

Observational studies cannot establish causation. They cannot distinguish between “protests raised support for the cause” and “protests happened because people supported the cause”. No amount of controlling for confounders fixes this problem.

Therefore, my review focuses only on natural experiments that measure real-world outcomes.

Conflict of interest: In 2024 I donated to PauseAI US, which organizes protests. I would prefer to find that protests work.

Studies on real-world protest outcomes

Social Change Lab reviewed five studies on how protests affect voter behavior, which they judged to be the best studies on the subject.

I excluded two of the five studies due to methodological concerns:

McVeigh et al. (2014)⁵ is an observational study that looked at long-term changes in Republican voting in counties where the Ku Klux Klan was most active.
Bremer et al. (2020)⁶ is a study on the correlation between protests and electoral outcomes in European countries.

I did not review these because they are observational studies, and I wanted to focus on natural experiments.

I did review the other three studies:

Madestam, A., Shoag, D., Veuger, S., & Yanagizawa-Drott, D. (2013). Do Political Protests Matter? Evidence from the Tea Party Movement.
Wasow, O. (2020). Agenda Seeding: How 1960s Black Protests Moved Elites, Public Opinion and Voting.
Klein Teeselink, B. K., & Melios, G. (2021). Weather to Protest: The Effect of Black Lives Matter Protests on the 2020 Presidential Election.

In addition, I looked at two studies that the Social Change Lab report did not cover:

Larreboure, M., & Gonzalez, F. (2021). The Impact of the Women’s March on the U.S. House Election.
Hungerman, D., & Moorthy, V. (2023). Every Day Is Earth Day: Evidence on the Long-Term Impact of Environmental Activism.

There are no randomized controlled trials on the real-world effect of protests (how would you randomly assign protests to occur?). But there are five natural experiments—three from the Social Change Lab review, plus the Women’s March and Earth Day studies. Most of the natural experiments use the rainfall method.

The idea is that protests often get canceled when it rains. If you look at voting patterns in places where it rained on protest day compared to where it didn’t rain, you should be able to isolate the causal effect of protests. The rain effectively randomizes where protests occur.

Rather than using rainfall directly, the rainfall method uses rainfall shocks—that is, unexpectedly high or low rainfall relative to what was expected for that location and date. This avoids any confounding effect of average rainfall levels.

The clearest illustration of the rainfall method comes from Hungerman & Moorthy (2023)⁷ (which I will discuss in more detail later). The authors looked at counties where it rained vs. didn’t rain on the inaugural Earth Day—April 22, 1970. Then they used rainfall to predict the rate of birth defects from 1980–1988.The hypothesis is that Earth Day demonstrations increased support for environmental protections. That in turn would reduce environmental contaminants, leading to fewer birth defects. And if rainfall stops demonstrations from happening, then it will have the opposite effect.

The rainfall method is commonly used in social science, and it has received some fair criticism.⁸⁹ But the rainfall method as it was used by Hungerman & Moorthy is robust to these criticisms, as illustrated by this chart:

The key to establishing causation is that rainfall had no predictive power on any other day. It only mattered on Earth Day.

That leaves us with two possibilities:

Rainfall is associated with higher birth defects due to some confounding variable, but only rainfall on April 22 and not on any other day, because that day is special somehow, in a way that has nothing to do with Earth Day; or
Earth Day demonstrations reduced the rate of birth defects.

(Or the results could be due to a statistical error or data manipulation. I will discuss those possibilities later.)

A summary of the five studies I reviewed plus the two I declined to review:

Table 1: Summary of Studies

Study	Protest	Protest Type	Effect	Randomization Method	Quality
Madestam et al.¹⁰	Tea Party	nonviolent	+	rainfall	high
Wasow¹¹	Civil Rights	violent	-	rainfall	medium
Wasow¹¹	Civil Rights	nonviolent	+	none (observational)	low
Klein Teeselink & Melios¹²	BLM	nonviolent	+	rainfall	high
Larreboure & González¹³	Women’s March	nonviolent	+	weather shocks	medium
Hungerman & Moorthy⁷	Earth Day	nonviolent	+	rainfall	high
McVeigh et al.⁵	KKK activity	unclear	+	none (observational)	low
Bremer et al.⁶	Europe elections	nonviolent	?¹⁴	none (observational)	low

(Methodological quality is relative. I’d have higher confidence in a true experiment than in any of these quasi-experimental methods.)

Next I will review each study individually. Then I will collect the results into a meta-analysis.

Madestam et al. (2013) on Tea Party protests

Madestam et al. (2013)¹⁰ looked at the effect of 2009 Tea Party protests on the 2012 US elections. It used the rainfall method to establish causality.

As an additional check, the authors tested whether rainfall could predict Republican and Democratic vote shares in the 2008 election. (You may recall that the 2009 Tea Party protests did not occur until a year after the 2008 election.) If rainfall can predict the 2008 election results—before the protests occurred—that means the model was confounded.

Madestam et al. (2013) found that rainfall in 2009 could not predict votes in 2008 (see Table II), but it could predict votes in 2012 (see Table VI).

The authors also tested whether rainfall on other days prior to the Tea Party protests could predict 2009 voting patterns, and found that they could not.

In the authors’ model, a rainy protest decreased Republicans’ share of the vote in the 2012 election by 1.04 percentage points (p < 0.0006¹⁵). This suggests that protests did indeed increase the Republican vote share.

Interestingly, rainfall decreased Republican vote share relative to the total population, but did not increase the Democratic share. This suggests that protests increased voter turnout but did not cause voters to change their minds.

At first I thought the inability to predict 2008 votes might be a false negative (like a p = 0.06 situation), but this was not the case. Rainfall in 2009 increased Republicans’ vote share in 2008, although only slightly (p = 0.38). (Remember that rainfall is supposed to decrease Republican votes by preventing Tea Party protests from happening.)

There is another concern with the rainfall model—not with causality, but with overstating the strength of evidence. A standard statistical model assumes that all observations are independent. But rainfall is spatially autocorrelated, which is the statistical way of saying that rain in one county is not independent of rainfall in the neighboring counties. If you have data from 2,758 counties, you can’t treat them as 2,758 independent samples.

Madestam et al. (2013) used several methods to account for this. First, it clustered standard errors at the state level instead of at the county level. Second, as a robustness check, the authors assumed spatial correlations varied as an inverse function of distance, which produced similar standard errors. Third, the authors tried dropping states one at a time to see if any states overly influenced the results.

Placebo tests

Finally:

[W]e conduct a series of placebo tests using rainfall on other historical dates in April. These placebos are drawn from the same spatially correlated distribution as rainfall on April 15, 2009. If rainfall on the protest day has a causal effect, the actual estimate of rainfall ought to be an outlier in the distribution of placebo coefficients.

They calculated the “placebo p-value” as the probability that rainfall on a random day could predict outcomes better than rainfall on protest day. If the model has correctly accounted for spatial autocorrelation then the placebo p-value should equal the original model p-value, plus or minus some random variation.

The authors run tests on 627 random “placebo dates”, and find that rain on protest day had a larger effect size than almost any of the placebo dates (see Figure V). This suggests that their corrections for spatial correlation worked, making false positives unlikely. However, the p-values on Figure V were a bit higher than the p-values in the main text, suggesting some effect size inflation due to spatial autocorrelation.

Wasow (2020) on 1960s civil rights protests

Wasow (2020)¹¹ measured the effect of nonviolent protests using observational data only. I won’t discuss that portion of the paper.

Wasow applied the quasi-experimental rainfall model to violent protests and found that they had a significant backfire effect. I won’t focus on the evidence on violent protests because I would recommend against engaging in violence regardless of what result the study found.

But if violent protests decrease public support, that’s (weak) evidence against protests working in general. The simplest hypothesis is “protests work”. But evidence on violent protests contradicts this, requiring a more complex claim: “nonviolent protests work, violent protests backfire”. I will evaluate this two-part hypothesis in the meta-analysis below.

As some additional evidence on violent protests, Collins & Margo (2007)¹⁶ used the rainfall method to find that 1960s riots decreased nearby property values. This is consistent with the finding from Wasow (2020) that violent protests backfire, but property values are not directly relevant to protesters’ outcomes. It’s conceivable that protests could simultaneously decrease local property values and increase public support.

Replication data from Wasow (2020) is publicly available.

Klein Teeselink & Melios (2021) on 2020 Black Lives Matter protests

Klein Teeselink & Melios (2021)¹² used the rainfall method to establish the effect of Black Lives Matter protests on the 2020 presidential election.

Unlike Madestam et al. (2013), this paper did not test whether rainfall could predict outcomes before the protests (which would indicate confounding).

As with Madestam et al. (2013), the authors of this paper considered the fact that rainfall is not independent across counties. Their model adjusts for this by including independent variables to represent the change in vote shares in surrounding counties, scaled by inverse distance.

Unlike the other studies in this review, Klein Teeselink & Melios (2021) treated county vote changes as interdependent. Their model assumes that the change in vote share in one county is partially explained by vote changes in the nearby counties, using the method described by Beck et al. (2006)¹⁷.

Klein Teeselink & Melios’ model isolates the impact of local protests on local vote change. In the method of (e.g.) Madestam et al. (2013), some vote changes may be explained by protests in neighboring counties. Klein Teeselink & Melios’ method is more rigorous in a sense, but we don’t actually want to isolate local changes. We want to know how well protests work overall, not just their local effects.

Klein Teeselink & Melios performed a robustness check in Table A3, Panel D where they fully ignored spatial autocorrelation. This produced mean effects more in line with the other studies: a vote share change of 11.9 per protester (std err 2.9), and a change of 0.105 based on the probability of rain (std err 0.032).

If you ignore spatial autocorrelation, you may overestimate the strength of evidence. However, in this case, ignoring spatial autocorrelation had only a modest impact on the t-stats:

	Primary Model	Ignoring Spatial Autocorrelation	Ignoring Spatial Autocorrelation + Counties Weighted by Population
Vote Share Per Protester	5.5	4.1	7.2
Vote Share by Rain Probability	2.3	3.3	9.3

The paper’s replication data is publicly available.

Failed placebo tests

Earlier, I discussed how Madestam et al. (2013) performed “placebo tests” to check that its model wouldn’t generate too many false positives. Klein Teeselink & Melios (2021) did the same, although with only nine placebo tests instead of 627:

(May 25 was the beginning of the period in which the majority of protests happened.)

This chart shows that Klein Teeselink & Melios’ version of the rainfall method did not establish causality. The fortnight of April 29—a month before the protests started—showed nearly the same effect size as the May 25 period, and 6 out of 9 placebo periods had p-values less than 0.05. So either some confounding variable explains the association between protests and vote share, or the standard error is underestimated due to spatial autocorrelation (or something similar).

The authors write

part of this association may be caused by serial correlation in weather patterns

In other words, a rainy June often also means a rainy April or May, so rain in April/May might appear to affect protest outcomes because it’s correlated with rain in June. (And thus the model does establish causality.)

That may be true, but I’m not confident in that explanation,¹⁸ and therefore I can’t trust this model to establish causality. Therefore, I exclude the BLM protests from my meta-analysis.

(But this does leave me wondering: How is it that rainfall shocks in April could predict vote changes in the 2020 presidential election?)

It would be interesting to compare the publicly-available BLM data to the Earth Day data (see below) to figure out why the Earth Day paper passed its placebo test but BLM did not. But that’s beyond the scope of this article.

Larreboure & González (2021) on the Women’s March

Larreboure & González (2021)¹³ attempted to use the rainfall method to predict whether the 2017 Women’s March affected how many votes went to woman candidates in the 2018 election. I say “attempted” because they found that rainfall did not predict Women’s March attendance. So instead, they used “weather shocks” to predict voting outcomes. These shocks were defined as a combination of weather-related factors that they chose using a LASSO¹⁹ regression model.

I see no obvious problem with the “weather shocks” method, but I’m wary of adding more mathematical complexity. Complexity makes flaws harder to spot.

Larreboure & González found that protests increased voter turnout and vote share to women for both Democratic and Republican candidates.

The authors accounted for spatial autocorrelation by clustering standard errors at the state level. They included a robustness check where they adjusted for spatial autocorrelation using the method from Conley (1999)²⁰ with two different distance cutoffs, 50 km and 100 km (in Table A.8).

This paper had at least two inconsistencies in its reported figures:

Page 13 says an additional 1% of the population protesting increased vote share for women and under-represented groups by 12.95 percentage points (pp). However, Table 4 on page 28 reports an increase of 12.70 pp.²¹
A more minor error, but page 13 says the 12.95 pp number is “remarkably similar to the impact of the Tea Party protesters on the vote share of the Republican Party (i.e. 12.59)”. However, the 12.59 number from Madestam et al. (see above) is the change in absolute votes, not vote share. The reported change in vote share was 18.81, which is not remarkably similar to 12.95.

I will take the 12.70 number reported in Table 4 as correct (it is repeated again in the robustness checks).

To be conservative, in my meta-analysis I will use the figures from the 50 km robustness check (where available) because they had the largest standard errors.

Hungerman & Moorthy (2023) on Earth Day

Hungerman & Moorthy (2023)⁷ found that rainfall on the inaugural Earth Day, April 22, 1970, could predict people’s environmental attitudes on surveys from 1977 to 1993.

It also directly measured environmental impact by looking at pollutant levels and rates of birth defects (which can result from exposure to environmental contaminants). It found that rainfall on Earth Day could predict birth defects.

The paper claims that rainfall predicted carbon monoxide levels, and it did find a statistically significant change. However, Appendix Table A3 examines five environmental contaminants, of which only carbon monoxide had a t-stat above 2, and two out of five outcomes were (slightly) negative. The positive effect on carbon monoxide may be a false positive.

Earlier I showed this chart:

The chart shows that rainfall on April 22, 1970–Earth Day—predicts the rate of birth defects 10 years later, but rainfall on any other day does not.

The same chart for the effect of rainfall on support for environmental spending:

The paper addresses the previously-mentioned spatial autocorrelation problem using the same techniques as Madestam et al. (2013). If spatial autocorrelation were distorting the effect sizes, we would expect to see more spurious statistically significant outcomes on the charts above. But we only see large effect sizes on Earth Day, not on any other day, which indicates that spatial autocorrelation is not a problem.

Like Madestam et al. (2013), the authors generated hundreds of additional “placebo tests” (as described above) where they looked at how well rainfall on different random days could predict environmental outcomes. They found that the placebo p-values were very similar to the original p-values (and even lower in some cases):

The paper’s source code and data are publicly available.

Meta-analysis

For two of the five natural experiments, I calculated expected change in number of votes for each additional protester, or change in vote share per protester (defined as votes per protester divided by turnout):

Table 2: Change in Votes Per Protester

Protest	Votes	Std Err	Vote Share	Std Err	n	Source
Tea Party	12.59	4.21	18.81	7.85	2758	Table VI
Women’s March	3*	**	9.62	4.47	2936	Table A.8 and page 3

*only one significant figure was provided

**not reported

I did not include the Earth Day or Civil Rights protests because the studies did not provide the relevant data.²² The BLM study reported vote share per protester, but I excluded it due to the study’s failure to establish causality, discussed previously.

I aggregated the results by applying a random-effects model. According to these two studies, protests have a mean impact of +11.95 vote share per protester (standard error 4.00; likelihood ratio 87.1; p < 0.003).

(The likelihood ratio tells us how much evidence the data provides. A likelihood ratio of 10.3 means that, assuming the study’s methodology is perfect, the odds of getting this result are 10.3x higher if the true mean is 7.84 than if the true mean is 0.)

If we are considering supporting some upcoming protest, we might want to estimate the probability that it will backfire. One way to do that is by using the pooled sample of past protests.

This pooled sample has a between-study standard deviation of 1.19, which reflects how much the effectiveness of protests varied across the studies. If we assume that the sample’s mean and between-study variation are exactly correct (which is questionable, since the pool only includes two studies), then we can model protest outcomes as a normal distribution with a mean of 11.95 and a standard deviation of 1.19.

Under this model, the probability of a protest having a negative effect—i.e., producing a value less than zero—is extremely small. But I would not take these precise numbers too seriously.

Vote share per protester is the most interesting metric for my purposes because it gives information about cost-effectiveness—it tells you how much impact you can expect for each marginal protester. But the natural experiments reported on other outcomes as well, such as overall change in vote share (as determined by changes in rainfall) and popular support for protesters’ objectives.

I applied a random-effects model to aggregate a few different sets of outcomes:

Table 3: Pooled Sample Outcomes

Outcomes	Mean	Std Err	likelihood ratio	p-value	P(negative effect)
Vote Share Per Protester	11.95	4.00	87.1	0.003	0
Vote Share	1.59	0.48	257	0.001	0.002
Vote Share (Rain Only)	1.14	0.42	39.3	0.007	0
Single Hypothesis	1.06	0.78	2.55	0.172	0.199
Favorability	2.68	2.32	1.95	0.249	0.176

Row 1 – Vote Share Per Protester uses the pooled outcome that I described in Table 2, including Tea Party and Women’s March vote share per protester.
Row 2 – Vote Share takes these outcomes from the studies on nonviolent protests:
- Tea Party – Republican vote share
- Women’s March – women’s vote share
- Earth Day – favorability (1)²³ as a proxy for vote share
Row 3 – Vote Share (Rain Only) uses the same outcomes as Row 2, but excluding the Women’s March outcome because it used weather shocks rather than rainfall.
Row 4 – Single Hypothesis does not differentiate between nonviolent and violent protests, instead lumping all studies together. It includes the three Vote Share measures from Row 2, plus Civil Rights – vote share.
Row 5 – Favorability includes measured changes in popular support for a protest’s goals:
- Tea Party – support for the Tea Party
- Earth Day – favorability (1)²³

(Note: P(negative effect) = 0 doesn’t mean it’s literally zero, but it’s so small that it gets rounded off to zero.)

The Women’s March and Earth Day papers used continuous rainfall variables instead of binary (rain vs. no rain); those papers’ outcomes were standardized using the method from Gelman (2007)²⁴ to put them on the same scale as binary variables.²⁵

The Vote Share Per Protester and Vote Share tests produce low p-values/high likelihood ratios, and under those models, nonviolent protests have virtually no chance of having a negative effect on support. Favorability has a weak likelihood ratio due to a large variance between outcomes.²⁶

Under the Single Hypothesis model, protests have a much weaker p-value/likelihood ratio. Naturally, when you include a negative outcome, it pulls down the average effect quite a bit. The mean is still positive, which makes sense given that only one out of four included protests was violent.

Is it fair to separate out violent and nonviolent protests? I’m wary of adding complexity to a hypothesis but I believe it’s justified in this case:

It’s intuitively plausible that peaceful protests would earn support while violence would backfire.
Lab experiments³ and observational studies support this bimodal hypothesis.
I ran a t-test for the hypothesis that nonviolent and violent protests have the same effect on voting outcomes, comparing the pooled outcome from Row 2 – Vote Share against the Civil Rights protest outcome. The result had a likelihood ratio of 55.1 and p < 0.005. We can strongly reject the hypothesis that these two samples have the same mean.

There are some reasons believe these results may be overstated, which I will address under Potential problems with the research. There are also at least two reasons to believe they may be understated:

Rainfall does not perfectly predict whether protests occur. (Sometimes people protest in the rain.) If protests genuinely work, then the effect of protests will be larger than the effect of protests as predicted by rainfall.
I aggregated the most similar metrics into pooled outcomes. But these were not always the strongest metrics. For example, Earth Day protests strongly predicted birth defects (likelihood ratio 55,000; p < 3e-6). But I did not include birth defects in the meta-analysis because it did not have any comparable counterpart in the other studies.

Table 4 shows outcomes across the five studies, estimated by looking at counties where it rained vs. did not rain. The Women’s March and Earth Day results are standardized as explained above.²⁵

Table 4: Societal-Level Protest Outcomes

Protest	Outcome	Change	Std Err	Source
Tea Party	votes (as % of population)	1.04%**	0.30%	Table VI
Tea Party	vote share	1.55%*	0.69%	Table VI
Tea Party	conservative vote score²⁷	1.922*	0.937	Table VII
Tea Party	average belief effect²⁸	0.13***	0.037	Table V
Tea Party	strongly supports Tea Party	5.7%*	2.5%	Table V
Tea Party	Sarah Palin favorability	5.7%*	2.6%	Table V
Tea Party	“outraged about way things are going in country”	4.6%*	2.1%	Table V
Tea Party	opposes raising taxes on income >$250K	5.8%	3.0%	Table V
Tea Party	“Americans have less freedom than in 2008”	6.5%*	2.6%	Table V
Tea Party	Obama unfavorability	4.6%	2.4%	Table V
Civil Rights (violent)	vote share among white voters	–5.56%*	2.48%	Appendix, Table 12
BLM	vote share	2.7%	1.2%	Table 2
BLM	“Blacks should not receive special favors”	–0.242²⁹	0.360	Table 3
BLM	“Slavery caused current disparities”	0.339²⁹	0.388	Table 3
Women’s March	women’s vote share	2.48%***	0.64%	Table 4
Women’s March	voter turnout	0.41%**	0.14%	Table 4
Earth Day	favorability (1)²³	0.90%	0.53%	Table 2
Earth Day	favorability (1)²³ among under-20s	1.67%**	0.62%	Table 2
Earth Day	favorability (2)²³	1.12	0.70	Table 2
Earth Day	favorability (2)²³ among under-20s	1.90*	0.82	Table 2
Earth Day	carbon monoxide	0.07*	0.03	Table 4
Earth Day	birth defects	1.00***	0.21	Table 4

*p < 0.05; **p < 0.01; ***p < 0.001

Potential problems with the research

Spatial autocorrelation

Recall that “spatial autocorrelation” is a technical way of saying “rainfall is not independent across counties”. If you assume your samples are independent when they’re not, your standard errors will be too low—giving you too much confidence in your results.

It’s conceivable that all five studies overstated the strength of their results due to spatial autocorrelation.

Each study on nonviolent protests used at least some technique to correct for spatial autocorrelation. Madestam et al. (2013) and Hungerman & Moorthy (2023) included “placebo tests”. The placebo tests from Madestam et al. (2013) indicated that these corrections mostly worked but did not fully succeed, whereas Hungerman & Moorthy’s corrections apparently did succeed. On balance, this suggests that the standard errors of the pooled outcome may be understated, but probably not by a large margin.

Two of the pooled outcomes from Table 3—the Vote Share Per Protester and Favorability pools—had strong likelihood ratios / low p-values. That suggests they should hold up even with somewhat reduced statistical power.

Publication bias

The standard method to assess publication bias would be to make a funnel plot. I didn’t do that for two reasons:

With only five studies (at best), there aren’t enough data points to detect publication bias even if it exists.
A funnel plot only works if your studies cover a range of sample sizes. All the natural experiments have roughly the same sample size (because they all look at county-level data for the majority of US counties).

As an alternative, I tested how the results might change if we discovered some unpublished null results. I used the following procedure:

Choose one of the pooled outcomes from Table 3.
For each individual study outcome, clone it to create a “dummy null outcome” with the same standard error and sample size, but a mean of 0. This represents a hypothetical study that didn’t get published because it found a null result.
Construct a larger pooled sample using all four or six outcomes (the two or three real outcomes plus the two or three null dummies).

Table 5: Pooled Sample Effects, Adjusted for Publication Bias

Outcomes	Mean	Std Err	likelihood ratio	p-value
Vote Share Per Protester	6.43	4.01	3.61	0.11
Vote Share	0.80	0.41	6.97	0.049
Vote Share (Rain Only)	0.58	0.36	3.77	0.104
Favorability	0.74	0.65	1.91	0.256

Adding in null results considerably weakens the strength of evidence.

This approach is deliberately conservative. I wouldn’t say this meta-analysis is robust to publication bias, but it’s not particularly vulnerable to publication bias, either.

(The dummy-null approach leaves something to be desired. If the true mean were 6.43 as the pooled sample suggests, it would be surprising to see three positive results with low p-values plus three null results with equally tight standard errors. But I haven’t thought of any better ideas for how to test publication bias.)

Orazani et al. (2021)³ reviewed lab experiments on protest favorability. Among other things, it looked at publication bias. This paper might be informative, since it stands to reason that if experimental researchers on protests have a certain bias, then sociological researchers might have a similar bias.

The paper included a funnel plot:

To supplement the plot, I tested for publication bias using two statistical tests:

Egger’s regression test³⁰ found r = 0.124, p < 0.646 (r > 0 means that more powerful studies had larger mean effects, which if anything is evidence of inverse publication bias).
Kendall’s tau test found p < 0.565.

Orazani et al. (2021) included 14 experiments and 2 non-experimental studies. I also tested for publication bias when excluding the non-experiments and again found highly insignificant p-values.

Orazani et al. (2021) tested for a difference between published and unpublished studies (although they defined “unpublished” in a way that seemed strange to me—they counted dissertations and conference presentations as unpublished). They found a significant difference in effect size, suggesting the presence of publication bias. Published studies had a Cohen’s d of 0.39, versus 0.22 for unpublished studies. However, this difference disappeared when the authors controlled for certain features of the protests being studied (e.g. protests directed at the government as opposed to society). I am not sure what to make of this, but there is at least some evidence of publication bias.

Data fabrication

Most meta-analyses do not consider the possibility that some studies’ data might be fabricated, and I believe they should. Checking for fraud is difficult in general, but I will do some basic checks.

When humans fabricate data, they often come up with numbers that don’t look random. Real data should follow two observable patterns:

The last digits of numbers should be uniformly distributed.
The first digits of numbers should NOT be uniformly distributed. Instead, they should obey Benford’s law.

I tested for suspicious patterns by collecting a list of statistical results (means and standard errors for various outcomes) from the BLM, Tea Party, Women’s March, and Earth Day papers. I did not include the Civil Rights paper because its quasi-experimental data only included violent protests.

I also did a power check to determine whether the tests have adequate statistical power. We should be able to reject the hypotheses that the first digits follow a uniform distribution, and that the last digits follow Benford’s law.

  Tea Party:
      First-digit Benford's Law p-value: 0.598
      Last-digit uniformity p-value:     0.306
      Power check p-values:              0.001, 0.002

  BLM:
      First-digit Benford's Law p-value: 0.438
      Last-digit uniformity p-value:     0.598
      Power check p-values:              0.001, 0.001

  Women's March:
      First-digit Benford's Law p-value: 0.181
      Last-digit uniformity p-value:     0.891
      Power check p-values:              0.001, 0.001

  Earth Day:
      First-digit Benford's Law p-value: 0.121
      Last-digit uniformity p-value:     0.224
      Power check p-values:              0.038, 0.001

(P-values are rounded up to 3 digits. See source code for full details.)

In all cases, I found high p-values for the first and last digits, which means the data follow the expected natural patterns. And I found very low p-values for the sanity check tests, which means the tests are sufficiently powerful (except for Earth Day first digits, where few independent outcomes were reported).

These tests do not rule out more sophisticated fraud. For example, if the authors generated false data and then calculated statistical tests on top of them, the fabricated results would still pass the first-digit and last-digit checks.

Data errors

Checking for data errors is difficult in general.³¹ I did a basic consistency check to verify that each study’s reported means and standard errors seemed internally consistent, but it’s hard to see errors that way.

The only data error I noticed was in Larreboure & González (2021)¹³. As I mentioned before, it reported inconsistent numbers for the change in vote share based on each 1% of the population protesting: 12.95 pp (std err 5.63) on page 13 in the text, and 12.70 pp (std err 5.48) in Table 4.

The difference is small, which suggests the authors may have made some revision to their calculations but didn’t update all the values reported in their manuscript. If so, the number in Table 4 is likely the correct one.³²

This inconsistency suggests that the authors have some degrees of freedom for p-hacking, but the two numbers are similar enough to have minimal impact on the result of my meta-analysis.

Will the results generalize?

All the protests covered by natural experiments have certain commonalities:

They all had a large number of participants.
They were all nationwide (they had to be, so the study authors could use county-level data).
They all took place in the United States.

Will the results generalize to other countries? Will the results generalize to smaller-scale or local protests?

The fact that these protests were so widespread means their objectives couldn’t have been far outside the Overton window (i.e., the range of politically acceptable ideas at the time). Perhaps a protest that advocated for a more radical position would be more likely to backfire. To address this question, perhaps we could look at lab experiments on protests, but that’s beyond the scope of this article.

Meta-concerns with this meta-analysis

I have some criticisms of my meta-analysis itself:

I did not pre-register a methodology. I have limited experience conducting meta-analyses and I was learning as I wrote this article. Realistically, I would not have had the motivation to finish if I’d been required to fully determine a methodology in advance. But the platonic ideal of this meta-analysis would have included a pre-registration.
Three of the studies (BLM, Civil Rights, and Earth Day) published their data. A thorough analysis would attempt to replicate those studies’ findings. I did not do that.

Broad claims

Social Change Lab’s literature review included a summary of findings, reproduced below.

Table 6: Social Change Lab Findings

Finding	Confidence
Protest movements can have significant short-term impacts	Strong
Protest movements can achieve intended outcomes in North America and Western Europe	Strong
Protest movements can have significant impacts (2-5% shifts) on voting behaviour and electoral outcomes	Medium
Protest movements can positively influence public opinion (≤10% shifts)	Medium
Protest movements can influence public discourse (e.g. issue salience and media narratives)	Medium
Protest movements can influence policy	Low (mixed evidence)
Protest movements can influence policymaker beliefs	Low (little evidence)
Protest movements can achieve desired outcomes in the Global South	Low (little evidence)
Protest movements can have significant long-term impacts (on public opinion and public discourse)	Low (little evidence)

In this section, I assess whether the natural experiments support each “Strong” and “Medium” claim. I find that the evidence does indeed support the findings and I agree with Social Change Lab’s confidence levels in each case.

I do not review the four “Low Confidence” claims because none of the natural experiments attempted to test them. (That fact itself suggests that “Low Confidence” is an accurate label.)

Starting with the findings rated “Strong”:

Protest movements can have significant short-term impacts Strong

Protest movements can achieve intended outcomes in North America and Western Europe Strong

The natural experiments support these claims. There’s also supporting evidence from lab experiments on how protests affect people’s perceptions; studies on media coverage; and observational data on protest outcomes. For a meta-analysis of lab experiments, which I view as the second-strongest form of evidence, see Orazani et al. (2021)³.

I do not have much confidence in most of these lines of evidence, but the natural experiments offer good support:

All study results point the same direction (as long as we exclude the data on violent protests).
The pooled outcomes have high likelihood ratios / low p-values.
There are no signs of data fabrication.

I’m still concerned about publication bias and spatial autocorrelation. I am not sure it is appropriate to describe the evidence as “Strong”. It would be fair to downgrade your confidence to “Medium” based on these concerns. But I also think “Strong” confidence is defensible; the distinction depends on how much weight you give to the hard-to-quantify limitations with the existing evidence.

The natural experiments all cover nationwide, popular protests in the United States, so it’s not clear that the results generalize. Regardless, Social Change Lab didn’t claim that protests always have significant impacts, only that they “can” have impact; and the existence of these natural experiments shows that indeed they can.

The highest-quality studies are all natural experiments, not true experiments. A true experiment would be preferable. But the rainfall method seems sufficient to establish causality so I am comfortable treating these natural experiments’ methodologies as valid.

Whether this evidence qualifies as “strong” is a matter of debate. Certainly the evidence could be much stronger. But I would be surprised if these findings were overturned, so I think Social Change Lab’s confidence level is fair.

Protest movements can have significant impacts (2-5% shifts) on voting behaviour and electoral outcomes Medium

A 2–5% shift is consistent with the natural experiments, which found changes in vote share ranging from 1.55% to 5.54% (see Table 4). I think 2–5% is fair as an optimistic expectation, given that the natural experiments all covered large nationwide protests.

I believe the rainfall method is effective at establishing causality, but we can’t be too confident in the magnitude of the effect because rainfall does not perfectly predict protest attendance. So I would not rate the confidence for this finding as higher than “Medium”.

Protest movements can positively influence public opinion (≤10% shifts) Medium

Among the natural experiments, only two (Madestam et al. 2013; Hungerman & Moorthy 2023) reported on public opinion in terms of percentages. Public opinion changes clustered around 5% for the multiple measures in the two studies.

Klein Teeselink & Melios (2021) reported changes in public opinion on a 5-point scale. Rainfall predicted changes of 0.242 and 0.339 on two different questions, which correspond to percentage changes of about 6% and 8.5%, although the interpretation of these percentages isn’t the same as for the other two studies.

I believe the data on voter behavior also provides evidence on public opinion—if you vote differently, it’s most likely because your opinion changed.

So I think Social Change Lab’s finding is indeed moderately well supported.

Protest movements can influence public discourse (e.g. issue salience and media narratives) Medium

None of the natural experiments directly addressed this claim.³³

Several observational studies found that protests frequently get media coverage. Even though the studies are all observational, I am comfortable inferring causality in this case—it seems odd to say that protests occurred, the news covered the protests, but the protests did not cause the news coverage.

Claims about individual studies

The literature review discussed five studies on real-world impacts of protests. Did it represent the studies accurately?

Social Change Lab discussed the observational component of Wasow (2020)¹¹ but not the quasi-experimental component.

By looking at US counties that are similar on a number of dimensions (black population, foreign-born population, whether the county is urban/rural, etc.), Wasow is able to mimic an experiment by testing how the Democratic vote share changes in counties with protests and matching counties without protests.

I don’t think it’s reasonable to say that a matched observational design “mimic[s] an experiment”. It could be that protests were more likely to happen in counties that were already shifting Democratic; you can’t prove that the protests caused the shift.

I agree with everything Social Change Lab wrote about Madestam (2013)¹⁰.

Regarding Klein Teeselink & Melios (2021)¹², the literature review wrote:

[A] one percentage point increase in the fraction of the population going out to protest increased the Democratic vote share in that county by 5.6 percentage points[.]

First, this figure is incorrect: it should be 3.3 percentage points (page 11). Klein Teeselink & Melios (2021) was revised in 2025 and I only have access to the latest revision, so it’s possible that Social Change Lab’s figure comes from the 2021 version.

Second, Klein Teeselink & Melios’ placebo tests show that the natural experiment failed to establish causality. Social Change Lab interprets the study’s outcome as causal, but I do not believe this interpretation is justified.

Social Change Lab’s description of McVeigh, Cunningham & Farrell (2014)⁵ was fair; it was appropriately cautious about the weakness of the paper’s evidence.

On Bremer et al. (2019)⁶:

They found that whilst no such relationship existed for all 30 countries, in Western Europe did [sic] find a statistically significant interaction between protest, levels of economic hardship in a country and the loss of votes for the incumbent party.

I’m suspicious of p-hacking when a study finds a non-significant main result and a significant sub-group result. I wish Social change Lab had been more skeptical of Bremer et al.’s approach.

It seems that for a given level of economic hardship a country faces, if the number of protests increase, the incumbent political party will lose more votes[.]

This quote implies causality, which was not established—the Bremer et al. study was purely observational.

In summary, Social Change Lab overstated the strength of evidence several times when reviewing particular studies. However, I believe their summary findings are still accurate, partially thanks to the two additional natural experiments (Larreboure & González (2021)¹³ and Hungerman & Moorthy (2023)⁷) that came out more recently.

Conclusion

My position on the Social Change Lab literature review:

The review was insufficiently skeptical about weak evidence, and too willing to attribute causality where it had not been established.
The review’s summary claims about the overall strength of evidence were consistent with my assessments. Perhaps the “Strong Confidence” findings were overconfident and should be “Medium Confidence” instead, but I can see arguments either way.

Conducting a meta-analysis changed my view on protest effectiveness. My previous stance was that protests probably work, and that various lines of evidence pointed that way, but that all available evidence was weak. I now believe that some of the evidence is relatively³⁴ strong, and I am more confident that protests work.

Source code

Source code for my meta-analysis is available on GitHub.

Appendix A: Additional tables

Most meta-analyses report study heterogeneity (I²). I reported P(negative effect) instead, which provides equivalent information, and I believe it’s more useful in this case. For completeness, Table A.1 gives the I² values for Table 3.

Table A.1: Pooled Outcomes with I²

Outcomes	Mean	Std Err	likelihood ratio	p-value	I^2
Vote Share Per Protester	11.95	4.00	87.1	0.003	3%
Vote Share	1.59	0.48	257	0.001	45%
Vote Share (Rain Only)	1.14	0.42	39.3	0.007	0%
Single Hypothesis	1.06	0.78	2.55	0.172	74%
Favorability	2.68	2.32	1.95	0.249	72%

Table A.2 reports summary statistics for the same pooled outcomes as Table 3, plus additional outcomes from Klein Teeselink & Melios (2021)¹² that I excluded from the main table. Consider this like a leave-one-out analysis, except instead it’s a put-one-in analysis.

“Vote Share Per Protester” adds BLM vote share per protester (mean 3.3, std err 0.6, n = 3053; from Table 2).
The middle three rows add BLM vote share.
“Favorability” adds survey agreement rate for the statement “Blacks should not receive special favors.”

Table A.2: Pooled Outcomes Including BLM

Outcomes	Mean	Std Err	likelihood ratio	p-value	I^2	P(negative effect)
Vote Share Per Protester	7.89	3.91	7.62	0.044	65%	0.072
Vote Share	1.71	0.43	2.84e+03	0.001	33%	0
Vote Share (Rain Only)	1.32	0.41	192	0.002	3%	0
Single Hypothesis	1.36	0.68	7.6	0.045	69%	0.123
Favorability	2.66	1.99	2.44	0.182	48%	0.136

Table A.3 uses the same row definitions as in Table A.2, while also correcting for publication bias by creating dummy null outcomes as described above.

Table A.3: Pooled Outcomes Including BLM, Adjusted for Publication Bias

Outcomes	Mean	Std Err	likelihood ratio	p-value
Vote Share Per Protester	2.79	1.55	5	0.073
Vote Share	0.88	0.38	14.9	0.021
Vote Share (Rain Only)	0.70	0.36	6.69	0.052
Favorability	0.66	0.51	2.26	0.203

The BLM study found a smaller mean effect than the other studies, but it also had a high t-stat. Adding BLM to the pooled outcomes decreases means but does not consistently decrease the strength of evidence.

As discussed previously, the BLM study isolates local effects of protests, which is undesirable—it ignores any non-local effects that protests might have. Luckily, the study also reports results with no adjustment for spatial autocorrelation (in its Table A3).

In general, it’s not a good idea to ignore spatial autocorrelation because it may overstate the strength of evidence. But for the “vote share per protester” metric, the un-adjusted outcome had a lower t-stat than the adjusted outcome. So I think it’s fair to add the un-adjusted result to the pooled sample.

Here are the results for pooled vote share per protester, using the BLM outcome with no adjustment to spatial autocorrelation.

Table A.4: Pooled Outcomes for Vote Share Per Protester, Including BLM

Outcomes	Mean	Std Err	likelihood ratio	p-value	P(negative effect)
no correction	11.89	2.32	4.83e5	4e-7	0
corrected for publication bias	6.23	3.05	8.1	0.041	0.138

My meta-analysis compared standardized outcomes (using the method described in Gelman (2007)²⁴). Table A.5 shows the results from pooling unstandardized outcomes instead (excluding BLM).

Table A.5: Unstandardized Pooled Outcomes

Outcomes	Mean	Std Err	likelihood ratio	p-value	P(negative effect)
Vote Share Per Protester	11.95	4.00	87.1	0.003	0
Vote Share	3.31	1.37	18.1	0.017	0.04
Vote Share (Rain Only)	1.94	1.02	6.13	0.057	0.011
Single Hypothesis	1.66	1.77	1.55	0.349	0.293
Favorability	5.20	1.84	53.8	0.005	0

Appendix B: Methodological revisions

In the interest of transparency—and because I didn’t pre-register a methodology—here is a list of non-trivial revisions I made in the process of writing this article. In chronological order:

Originally, I used a different method for estimating the effect of publication bias. Instead of creating dummy null clones as described above, I created k null dummies (one for each real outcome) that were all identical, and that took their standard error as the average of the real studies’ standard errors. This method produced lower p-values. However, I decided it was too unrealistic to give all three null dummies the exact same summary statistics.
For BLM and Women’s March results, I originally used the primary outcomes as reported by their respective papers. I revised my meta-analysis to use the weakest outcomes (lowest t-stat) from the robustness checks, to more conservatively account for spatial autocorrelation.
I originally included BLM outcomes in the meta-analysis. Upon re-reading the BLM paper, I realized it failed its placebo tests, so I removed it. Appendix A shows the results when including BLM.
I went back to using the main outcome for BLM instead of a robustness check outcome because it’s simpler, and I wasn’t including BLM in the main meta-analysis anyway. This slightly strengthened the reported results in Appendix A.
Originally, my meta-analysis used unstandardized means. I wanted to use standardized means but I wasn’t sure how to standardize them. Eventually, I found Gelman (2007)²⁴ and used its method for standardizing outcomes. This increased most results’ t-stats because it decreased between-study variance. (However, it decreased the Favorability t-stat.) Unstandardized pooled outcomes are reported in Table A.5.
- Initially, I wanted to scale all binary outcomes by their standard deviations, but I did not have the necessary data to calculate all standard deviations, so I simply left them unscaled.

Appendix C: Comparing the strength of evidence to saturated fat research

To get some perspective on the strength of evidence on protests, I would like to compare it to a thorny question in an unrelated field that I reviewed recently.

Last year I examined the evidence on whether saturated fat is unhealthy, primarily focusing on a Cochrane review³⁵ of RCTs. I ultimately decided I was 85% confident that saturated fat is unhealthy.

How does the evidence on protest effectiveness compare to the evidence on saturated fat?

Both hypotheses face similar problems: there are many observational studies that support the hypothesis, but few experiments. (In the case of protests, there are no (real-world) experiments, but there are some natural experiments.)

The Cochrane review included 15 RCTs. My review included five natural experiments.

The studies in the Cochrane review were true experiments. The protests studies were not true experiments, which means they can’t establish causality quite as firmly.

The Cochrane review found no evidence of publication bias (Figure 3). There aren’t enough protest quasi-experiments to test for publication bias, but I did test what would happen if I added in some hypothetical null-result studies.

Three individual studies on saturated fat had statistically significant positive effects, six had non-significant positive effects, and four had non-significant negative effects. (Two studies did not report data on cardiovascular events.)

In my review, three out of three studies (or four out of four³⁶) found statistically significant positive effects for nonviolent protests, and one study found statistically significant negative effects for violent protests.

When pooling all RCTs together, the Cochrane review found a marginally statistically significant effect of saturated fat reduction on cardiovascular events (95% CI [0.70, 0.98] where 1 = no effect). It also found significant effects on short-term health outcomes like weight and cholesterol. It found positive but non-significant results on mortality outcomes (including all-cause mortality and cardiovascular mortality).

In my meta-analysis, most of my pooled samples had strong positive results. My primary metric had p < 0.003; the Cochrane review didn’t find any primary results that strong.

Even though the Cochrane review included three times as many studies, I think the evidence on protest outcomes is stronger:

The saturated fat RCTs got mixed results, but the three or four studies on nonviolent protests all pointed the same direction.
(Most of) the pooled outcomes for protests had moderate to strong p-values. The pooled outcomes for saturated fat reduction had moderate p-values at best.

On the other hand, the protest studies have to deal with the spatial autocorrelation problem, and it’s not entirely clear that they succeeded at establishing causation. The saturated fat studies were experiments; they had no analogous problem.

The smaller number of studies also means the protests meta-analysis is more vulnerable to errors in any one study.

It’s a judgment call as to whether you think the weaker methodology for protest studies outweighs the stronger likelihood ratios. I’m inclined to say it doesn’t.

Notes

Ozden, J., & Glover, S. (2022). Literature Review: Protest Outcomes. ↩
Ozden, J., & Glover, S. (2022). Protest movements: How effective are they? ↩
Orazani, N., Tabri, N., Wohl, M. J. A., & Leidner, B. (2021). Social movement strategy (nonviolent vs. violent) and the garnering of third-party support: A meta-analysis. ↩ ↩² ↩³ ↩⁴
ChatGPT Deep Research was useful for finding and summarizing studies, but not for assessing their quality. When I asked ChatGPT to only include methodologically rigorous studies in the review, it didn’t appear to change which studies it included, it just rationalized why every study was rigorous. It said things like (paraphrasing) “we know this observational study’s findings are robust because it controlled for confounders” and “because it had a large sample size” (??). ↩
McVeigh, R., Cunningham, D., & Farrell, J. (2014). Political Polarization as a Social Movement Outcome. ↩ ↩² ↩³
Bremer, B., Hutter, S., & Kriesi, H. (2020). Dynamics of protest and electoral politics in the Great Recession. ↩ ↩² ↩³
Hungerman, D., & Moorthy, V. (2023). Every Day Is Earth Day: Evidence on the Long-Term Impact of Environmental Activism. doi: 10.1257/app.20210045 ↩ ↩² ↩³ ↩⁴
Mellon, J. (2024). Rain, rain, go away: 194 potential exclusion-restriction violations for studies using weather as an instrumental variable. ↩
Sarsons, H. (2015). Rainfall and conflict: A cautionary tale. ↩
Madestam, A., Shoag, D., Veuger, S., & Yanagizawa-Drott, D. (2013). Do Political Protests Matter? Evidence from the Tea Party Movement. ↩ ↩² ↩³
Wasow, O. (2020). Agenda Seeding: How 1960s Black Protests Moved Elites, Public Opinion and Voting.. ↩ ↩² ↩³ ↩⁴
Klein Teeselink, B., & Melios, G. (2021). Weather to Protest: The Effect of Black Lives Matter Protests on the 2020 Presidential Election. ↩ ↩² ↩³ ↩⁴
Larreboure, M., & Gonzalez, F. (2021). The Impact of the Women’s March on the U.S. House Election. ↩ ↩² ↩³ ↩⁴
Braemer et al. (2020) found a non-significant result and then did some subgroup analysis and got a significant result. I find that suspicious but I didn’t bother to look deeper because it’s an observational study anyway. ↩
The paper did not report the p-value, but it did report the standard error, so I calculated the p-value from that. ↩
Collins, W. J., & Margo, R. A. (2007). The Economic Aftermath of the 1960s Riots in American Cities: Evidence from Property Values. ↩
Beck, N., Gleditsch, K. S., & Beardsley, K. (2006). Space Is More than Geography: Using Spatial Econometrics in the Study of Political Economy. ↩
If it were true, we’d expect to see a similar phenomenon in the placebo tests of Madestam et al. (2013) and Hungerman & Moorthy (2023), but we don’t. ↩
Belloni, A., Chernozhukov, V., & Hansen, C. (2010). LASSO Methods for Gaussian Instrumental Variables Models. ↩
Conley, T. G. (1999). GMM estimation with cross sectional dependence.. ↩
I emailed the corresponding author to ask about this apparent discrepancy and did not receive a reply. ↩
Hungerman & Moorthy (2023) did provide enough information to estimate the change in Earth Day favorability per protester, by dividing change in favorability by change in number of protesters from the paper’s Table 4. However, this estimate would have high variance on the denominator, which makes the result unreliable. ↩
Favorability (1) measured public support for environmentalism as the percentage of respondents answering Yes to “we’re spending too little money” on protecting the environment. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶
Gelman, A. (2007). Scaling regression inputs by dividing by two standard deviations. ↩ ↩² ↩³
Some additional detail:

Gelman (2007) proposes scaling continuous variables by 2 standard deviations because this puts them onto the same scale as a binary variable where the control and treatment groups have the same size. If you have a sample of binary outcomes where 50% of the outcomes are 0 (“no rain”) and 50% are 1 (“rain”), then the standard deviation is 0.5. If the probabilities are not 50/50 then the standard deviation will not equal 0.5. (For example, the Tea Party rainfall variable had a standard deviation of 0.401.) Arguably it would make sense to scale all binary variables to a standard deviation of 0.5. However, I did not do this because I didn’t have the necessary data for all the papers. Instead, I left all binary variables unscaled. (Gelman (2007) discusses whether probability-skewed binary variables should be scaled, but ultimately does not take a stance.)

The Earth Day paper directly regressed outcomes onto a continuous rainfall variable (without doing a two-stage regression). I scaled the reported slopes by 2 times the standard deviation of rainfall.

The Women’s March paper reported values scaled to 1 standard deviation, so I divided them by 2.

The Tea Party, BLM, and Civil Rights papers reported effects in binary terms, so I did not scale them. ↩ ↩²
Even if I replace the relatively weak Earth Day favorability outcome with Earth Day favorability among under-20s (which had a likelihood ratio of 37), the pooled likelihood ratio is still only 3.43. ↩
Defined as the number of Congress members who voted in line with conservative positions according to the American Conservative Union. ↩
Defined as alignment with Tea Party positions, measured in standard deviations. ↩
Evaluated on a 5-point scale from “strongly disagree” to “strongly agree”. ↩ ↩²
The linked webinar is the least-bad explanation of Egger’s test that I could find, but it doesn’t explain it very well so I will attempt to explain:

In the presence of publication bias, more powerful studies will have lower means. the small low-mean studies don’t get published. Therefore there will be a negative correlation between a study’s power and its mean. (I measured power as the inverse standard error but you could also use the inverse variance or the sample size.)

So you test for publication bias by doing a linear regression of study means on study power.
- If the regression has a correlation close to zero, that indicates no publication bias.
- If there is a significant correlation, that’s evidence of publication bias.
Lest this footnote give the impression that I know what I’m talking about, I didn’t even know what Egger’s regression test was until I wrote this. My process was that I asked Claude what statistical test I could use to check for publication bias, it suggested Egger’s test and then gave an obviously-incorrect explanation of how the test works, and then I read several barely-comprehensible articles about the test until I thought I understood it. ↩
I could do full replications for the papers that published their data, but that would be considerably more work for a low chance of paying off. ↩
In my experience, I always update tables if I make revisions to my calculations, but it’s hard to keep track of everywhere in the text that I referenced a number. ↩
Technically, one of them did look at media coverage, but not using the rainfall method. ↩
I’m thinking about the strength of evidence from a sociology perspective. Getting good evidence in sociology is hard. ↩
Hooper, L., Martin, N., Jimoh, O. F., Kirk, C., Foster, E., & Abdelhamid, A. S. (2020). Reduction in saturated fat intake for cardiovascular disease. ↩
Three if you exclude the BLM study due to its failed placebo tests; four if you include it. ↩

The Triple-Interaction-Effects Argument

Thu, 10 Apr 2025 00:00:00 -0700

In this post I will explain the most impressive argument I heard in 2024.

First, some context:

There is an ongoing debate in the bodybuilding/strength training community about how much protein you should eat while losing weight.

Some say you should eat more protein if you’re losing weight:

If you’re eating less, your body is under extra pressure to cannibalize your muscles. Therefore, you should eat more protein to cancel this out.

The standard rebuttal:

Experimental trials have found that muscle gains max out when subjects eat 0.7–0.8 grams of protein per pound of bodyweight, and that’s true both when participants are maintaining weight and when they’re losing weight. There doesn’t appear to be a difference.

And the counter-rebuttal:

Almost all research looks at novice lifters. Experienced athletes have a more difficult time gaining muscle,¹ so losing weight will have a bigger negative impact on them, and therefore they need to eat more protein.

I used to believe this. Then I heard the most impressive argument of 2024.

I heard the argument in a YouTube video by Menno Henselmans:

It’s possible that in trained individuals there is a triple interaction effect, because that’s what you’re arguing here. If you’re saying that protein requirements increase in an energy deficit, but only in strength-trained individuals, then you are arguing for a triple interaction effect. […] That is very, very, very rare. Triple interaction effects, biologically speaking, simply do not occur much.

I didn’t understand what he was talking about. I spent two days pondering what it meant. On the third day, it finally clicked and I realized he was right.

To claim that trained lifters should eat more protein on an energy deficit, you’d need to believe that:

Above a certain level of protein intake (0.7–0.8 grams per pound), additional protein has no effect on muscle growth.²
Most of the time, trained athletes don’t need more protein than novices.
Novices don’t need more protein while losing weight than while maintaining/gaining weight.
HOWEVER, (a) among trained individuals who are (b) losing weight, the ones (c) who eat more protein (beyond 0.7–0.8 g/lb) gain more muscle.

The first variable (protein intake) has no interaction with muscle growth.

The second variable (trained vs. untrained) has no interaction with muscle growth.

The third variable (losing vs. maintaining weight) has no interaction with muscle growth.

The first and second variables together (protein intake + trained/untrained) have no interaction with muscle growth.

The first and third variables together (protein intake + losing/maintaining weight) have no interaction with muscle growth.

HOWEVER, when you put all three variables together, an interaction suddenly appears—a triple interaction effect.

This is a very strange claim. If all three variables together affect muscle growth, then you would expect each variable individually to affect muscle growth. And at least you would expect two out of three variables together to affect muscle growth.

(In fact, it is mathematically impossible to construct a differentiable function f(x, y, z) that is constant with respect to x, constant with respect to y, and constant with respect to z, but not constant overall. Although you could have a function f(x, y, z) where the slope with respect to each individual variable is close to 0, but not quite 0.)

Not to say a triple interaction effect can’t occur in the real world. It could be that muscle growth does depend on each of (protein intake, training experience, calorie deficit), but the relationships are so weak that the studies failed to pick them up.

But if you believe the studies’ results are correct, then it seems difficult—maybe even impossible—to still believe that trained lifters need to eat more protein while on a calorie deficit.

***

This was the best argument I heard in 2024 because:

If you think about it, it’s obviously correct. It changed my mind as soon as I understood it.
It’s difficult to come up with. (I’ve never heard anyone else make this argument.)

Notes

I’m conflating gaining strength with putting on muscle. There’s a difference, but we can consider them the same thing for the purposes of this post. ↩
This claim is somewhat controversial, but let’s assume it’s true for the sake of this argument.

Randomized controlled trials find no benefit to more than ~0.7 g/lb, and I quoted a range of 0.7–0.8 g/lb to account for variation between individuals. But the existing studies aren’t that great so I don’t have high confidence that that’s the correct range. ↩

You Can Now Read My Reading Notes

Mon, 31 Mar 2025 00:00:00 -0700

Since 2015, I have been taking notes on most articles I read. I figured other people might find them useful, so I cleaned them up and published them on my website. You can find them via the new “Notes” tab.

I will update the page every once in a while as I read more articles and take more notes.

I also have notes on every educational book I’ve read since 2015, but the notes are on physical paper (can you believe it?). I might digitize them at some point.

There Are Three Kinds of "No Evidence"

Mon, 03 Mar 2025 00:00:00 -0800

David J. Balan once proposed that there are two kinds of “no evidence”:

There have been lots of studies directly on this point which came back with the result that the hypothesis is false.
There is no evidence because there are few or no relevant studies.

I propose that there are three kinds of “no evidence”:

The hypothesis has never been studied.
There are studies, the studies failed to find supporting evidence, but they wouldn’t have found supporting evidence even if the hypothesis were true.
There are studies, the studies should have found supporting evidence if the hypothesis were true, and they didn’t.

Example of type 1: A 2003 literature review found that there were no studies¹ showing that parachutes could prevent injury when jumping out of a plane.

Example of type 2: In 2018, there was finally a randomized controlled trial² on the effectiveness of parachutes, and it found no difference between the parachute group and the control group. However, participants only jumped from a height of 0.6 meters (~2 feet). I don’t know about you, but this result does not make me want to jump out of a plane without a parachute.

Like in the parachute example, you see type-2 “no evidence” whenever the conditions of a study don’t match the real-world environment. You also see type-2 “no evidence” when an experiment is underpowered. Say you want to test the hypothesis that boys are taller than girls. So you go find your niece Sally and your neighbor’s son James and it turns out Sally is an inch taller than James. Your methodology was valid—you can indeed test the hypothesis by finding some people and measuring their heights—but your sample size was too small.

(The difference between type 2 and type 3 can be a matter of degree. The more powerful a study is, the stronger its “no evidence” if it fails to find an effect.)

Notes

Smith, G. C. S. (2003). Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials. ↩
Yeh, R. W., Valsdottir, L. R., Yeh, M. W., Shen, C., Kramer, D. B., Strom, J. B., Secemsky, E. A. et al. (2018). Parachute use to prevent death and major trauma when jumping from aircraft: randomized controlled trial. ↩

Return Stacked Funds: A New Way to Get Leverage

Tue, 04 Feb 2025 00:00:00 -0800

Some people (including me) have argued that altruists often benefit from leveraging their investments. Recently, it has become easier to use leverage thanks to the emergence of return stacked funds.

This is not financial advice.

Contents
What is return stacking?
An overview of return stacking funds
The true cost of return stacked ETFs
Pros and cons of return stacked funds
Are bonds a good investment?
Source code
Acknowledgments
Notes

What is return stacking?

Return stacking is a way of getting up to leveraged exposure to multiple return streams simultaneously. For example, RSSB invests 100% into global equities and 100% into US Treasury bonds, effectively giving it 2:1 leverage on a diversified stock/bond portfolio.

A return stacked ETF is a type of leveraged ETF. But whereas traditional leveraged ETFs (such as SPXL) lever up a single index like the S&P 500, a return stacked fund holds multiple asset classes.

Return stacked ETFs have lower management fees than single-index leveraged ETFs, and (with low confidence) they appear to have lower overhead costs for reasons that are not entirely clear to me (my guess is a combination of cheaper borrowing costs + transaction costs).

An overview of return stacking funds

There are four brands of return stacking funds that I know of:

the eponymous Return Stacked ETFs (RSSB, RSST, RSBT, RSSY, RSBY, RSBA, BTGD, RDMIX)
WisdomTree Capital Efficient ETFs (NTSX, NTSI, NTSE, GDE, GDMN)
PIMCO StocksPLUS Long Duration Fund (PSLDX)
Evoke Advisors Ultra Risk Parity ETF (UPAR)

The PIMCO fund has been around since 2007, but the others only launched within the last few years.

What do each of these funds invest in?

Six of the funds stack traditional asset classes:

Fund	first asset class	second asset class	leverage
RSSB	100% global stocks	100% US Treasury bonds	2:1
NTSX	90% US stocks	60% US Treasury bonds	1.5:1
NTSI	90% international stocks	60% US Treasury bonds	1.5:1
NTSE	90% emerging market stocks	60% US Treasury bonds	1.5:1
UPAR	too many for this table*		1.68:1
PSLDX	~100% US stocks**	~100% bonds**	~2:1**

*UPAR targets 17.5% U.S. equities, 7% international equities, 10.5% emerging markets equities, 21% commodity producer equities, 14% gold, 49% TIPS, and 49% Treasuries for a total allocation of 168%.

**PSLDX percentages are only approximate because the fund is actively managed and its holdings vary over time.

These funds stack traditional asset classes with alternatives:

Fund	first asset class	second asset class	leverage
RSST	100% US stocks	100% managed futures*	2:1*
RSBT	100% US bonds¹	100% managed futures*	2:1*
RSSY	100% US stocks	100% futures yield*	2:1*
RSBY	100% US bonds	100% futures yield*	2:1*
RSBA	100% US Treasury bonds	100% merger arbitrage*	2:1*
RDMIX	50/50 US stocks/bonds	100% systematic macro*	2:1*
BTGD	100% bitcoin	100% gold	2:1
GDE	90% US stocks	90% gold	1.8:1
GDMN	90% gold miner stocks	90% gold	1.8:1

*Managed futures (a.k.a. trendfollowing), futures yield (a.k.a. carry or roll yield), merger arbitrage, and systematic macro are all long/short strategies, not simple assets that you can buy and hold. So it’s somewhat arbitrary to say that the funds invest 100% into those strategies.

The true cost of return stacked ETFs

In a previous post, I looked at how a leveraged index fund should perform and compared that against how leveraged ETFs actually did perform. I found that the ETFs consistently cost more than expected, by an average of about one percentage point.

I attempted to do the same analysis for return stacked ETFs. These ETFs are harder to replicate because they don’t track indexes, so I don’t have high confidence in the results. That said, my numbers suggest that return stacked ETFs are more cost-effective than conventional leveraged ETFs.

I was able to replicate RSSB and NTSX:

ETF	Leverage	Stock ETF(s)	Bond Fund(s)
RSSB	100% + 100%	VTI + VXUS²	bond futures ladder³
NTSX	90% + 60%	SPY (S&P 500)	bond futures ladder⁴

I also attempted to replicate NTSI, PSLDX, and GDE, but I couldn’t find benchmarks that tracked them well enough.

I calculated excess costs of each fund as the hypothetical return you’d get if you levered up the benchmark (borrowing at the 3-month T-bill rate), minus the actual historical return of the fund.

The following table shows the total excess cost and after-fee cost for the return stacked ETFs. Excess cost is shown per 100% leverage (the excess on NTSX is multiplied by two because it only has 50% leverage). After-fee cost gives the excess cost minus the difference expense ratios between the ETF and the benchmark—this represents the “unexpected” portion of the cost, since you expect to pay the expense ratio no matter what. r gives the correlation between the return stacked ETF and the benchmark. I calculated the average annual cost for each ETF starting from the earliest year for which the ETF had a full year of data.

ETF	Excess Cost	After Fee	r	Start Year
RSSB	-0.55%	-0.84%	0.998	2024
NTSX	-0.17%	-0.41%	0.997	2019

(The costs were negative, which means the real-life funds outperformed the benchmarks.)

Excess costs for each individual year for NTSX:

	2019	2020	2021	2022	2023	2024
NTSX	-0.48	-3.50	2.72	1.92	-0.93	-2.15

As we can see, excess costs varied quite a bit from year to year. However, they were still generally lower than the costs of conventional leveraged ETFs.

In fact, the excess costs were negative most years. That’s surprising, since the benchmark does not account for transaction costs.

Why were return stacked funds (apparently) more cost-effective than conventional leveraged ETFs?

These funds have lower expense ratios. For example, RSSB charges 0.36% and SSO (a 2x leveraged S&P 500 fund) charges 0.89%.
Traditional leveraged ETFs rebalance daily. The Return Stacked and WisdomTree ETFs only rebalance if the holdings drift 5 percentage points away from the target weights. Rebalancing has transaction costs, which could be significant or could be close to zero, depending on various factors.
The return stacked funds get leverage via Treasury futures, which is approximately the cheapest way to get leverage. Conventional leveraged ETFs primarily use swaps, which have an opaque pricing structure and might cost a lot more. (I have no idea how much they actually cost because the pricing is opaque.)

Those factors explain why return stacked ETFs are cheaper than 3x leveraged index ETFs. But how is it possible for a return stacked ETF to outperform a leveraged combination of index funds?

My benchmarks have some margin of error—they do not perfectly track the return stacked ETFs. Based on playing around with the implementation details of the benchmark, I believe it could be off by perhaps one percentage point.⁵

The most likely source of tracking error is rebalance timing. Small changes in when you rebalance can significantly change year-to-year performance, especially in years like 2024 where some asset classes perform much better than others. If stocks outpaced bonds for most of the year, and the fund was supposed to rebalance from stocks to bonds, then the real-life fund might have gained an edge over the benchmark by delaying rebalancing a little longer.

Even if these (apparently) negative costs might not persist, this still provides evidence that the return stacked ETFs have lower costs than single-asset leveraged ETFs.

Pros and cons of return stacked funds

Pros:

They’re a convenient way to get leverage, much more convenient than options or futures.
They appear to have lower all-in costs than conventional leveraged ETFs.

Cons:

None of them offer greater than a 100% allocation to equities.
Limited choices—there are only a handful of return stacked ETFs available, and they might not include the asset classes you want.
- I personally would like to see a global stocks + managed futures ETF, but that doesn’t exist. There’s only US stocks + managed futures (RSST) and bonds + managed futures (RSBT).
As with other leveraged ETFs, the costs of return stacked ETFs fluctuate from year to year. Even though the costs are low on average, in any given year a return stacked ETF might perform worse than expected.
I could only determine the costs for two of the return stacked ETFs. The others might have higher costs.

Are bonds a good investment?

Most of the return stacked funds hold bonds. A question that some people ask:

Does it make sense to own return stacked stocks + bonds? Wouldn’t I rather have pure leveraged stocks instead?

Good question! I don’t know!

An argument against buying bonds:

Right now, the yield curve is nearly flat: yields on long-term bonds are only slightly higher than on short-term bonds. Why would you borrow at the short-term rate to earn the long-term rate if those rates are (nearly) the same?

Two counter-arguments:

The efficient market hypothesis predicts that you can’t time the bond market, so you shouldn’t change how you invest based on what the yield curve looks like.

A flat or inverted yield curve suggests that short-term rates will go down in the future. You might want to “lock in” the current rate by buying long-term bonds.

(Really these counter-arguments are the same—the (presumed) reason why the yield curve is flat is because the market is pricing in future changes in bond yields.)

Another argument against bonds:

In the long run, bonds have only earned a little bit of a premium over short-term T-bills. Given the overhead costs of using leverage, leveraged bonds might have near-zero or even negative expected return.

And two counter-arguments:

If you can borrow at close to the risk-free rate, bonds should still have a positive long-run premium.

Even if leveraged bonds have ~zero expected return, they still add value to a portfolio if they perform well during equity downturns.

Which side of the argument is correct is left as an exercise to the reader.

If you don’t want to hold bonds, there are some bondless return stacked funds available:

RSST holds stocks + managed futures (a.k.a. trendfollowing).
RSSY holds stocks + futures yield (a.k.a. carry or roll yield).
BTGD holds bitcoin + gold.
GDE holds US stocks + gold.
GDMN holds gold miner stocks + gold.

I am a big fan of managed futures trendfollowing—it’s a strategy with strong historical performance that provided protection during market downturns, and I think it’s likely to continue working in the future (for more, see Hurst et al. (2017), “A Century of Evidence on Trend-Following Investing”⁶). I’m ambivalent about carry (I’ve heard good arguments both for and against using it). I personally wouldn’t invest in bitcoin or gold, but if that’s your thing, return stacked ETFs give you a way to do it.

(I don’t own RSST, but I hold something similar in my own portfolio—equities (AAVM) with managed futures stacked on top.)

Source code

Source code is available on GitHub.

Acknowledgments

Thanks to Corey Hoffstein for helping me work out the implementation details of my benchmark.⁷

Notes

“bonds” means any sort of bonds, including Treasury or corporate bonds. “Treasury bonds” means just Treasury bonds. ↩
I weighted VTI at 62% and VXUS at 38% as of the beginning of 2024 because those are the weightings I get if I reverse-engineer from RSSB’s current weightings.

It would be simpler to use VT which includes all the same stocks as VTI + VXUS. But RSSB itself holds VTI + VXUS, and I found that breaking out equities into two separate ETFs produces slightly more accurate benchmark. ↩
RSSB gets exposure to bonds via an equal-weighted combination of bond futures at four maturities: 2-year, 5-year, 10-year, and long (i.e. 25- to 30-year). I replicated this using:
- 25% S&P 2-Year U.S. Treasury Note Futures Total Return Index
- 25% S&P 5-Year U.S. Treasury Note Futures Total Return Index
- 25% S&P 10-Year U.S. Treasury Note Futures Total Return Index
- 25% S&P Ultra T-Bond Futures Total Return Index
These indexes should exactly match the bond futures that RSSB holds. ↩
I wasn’t entirely sure what position to use to replicate NTSX’s bond holdings. Its materials include illustrative figures that use a 7-10 year Treasury index as a benchmark, which suggests I should use IEF or perhaps 10-year Treasury futures. But the latest holdings show that it uses a combination of bond futures of different durations.

I found the best correlation to NTSX when using a weighted combination of four Treasury futures:
- 12% S&P 2-Year U.S. Treasury Note Futures Total Return Index
- 12% S&P 5-Year U.S. Treasury Note Futures Total Return Index
- 24% S&P 10-Year U.S. Treasury Note Futures Total Return Index
- 12% S&P Ultra T-Bond Futures Total Return Index
As of this writing, NTSX holds 12% in 10-Year U.S. Treasury Note Futures and 12% in Ultra 10-Year U.S. Treasury Note Futures (which are like the normal 10-year futures except that they are more closely tied to a 10-year maturity). I did not use Ultra futures in my benchmark because they only launched a few years ago. ↩
Some minor changes that affect the return of the benchmark:
- The funds rebalance whenever weights drift 5% away from the target. But the prospectuses for RSSB and NTSX were not clear about what exactly that meant—I can think of at least four different interpretations. Corey Hoffstein (who co-runs RSSB) explained to me exactly how the rebalancing works, and I assumed NTSX works the same way but I don’t know for sure. Different rebalancing methods can change the average return by as much as one percentage point—a fund might get lucky and rebalance into a position right before it rockets up, or the opposite might happen.
- Changing how the benchmark invests in bonds can change the return. GOVT outperformed the Treasury futures ladder over the sample period. Changing the NTSX benchmark to use GOVT increased its return by 24 bps (but also decreased the correlation to NTSX from 0.997 to 0.993).
- My program might have a bug. Shortly before posting this article, I discovered that I was incorrectly calculating how much cash the benchmark needed to borrow and thus overestimating interest payments by about 40 bps per year.
↩
Hurst, B., Ooi, Y. H., & Pedersen, L. H. (2017). A Century of Evidence on Trend-Following Investing. ↩
Originally I couldn’t figure out how to get my benchmark’s correlation to RSSB higher than 0.976. Turns out I needed to compare the benchmark to RSSB’s NAV, not its daily closing price. Corey explained to me that NAV and price diverge because daily futures prices settle at 3pm but exchanges close at 4pm, so any market movements in that hour will show up in RSSB’s price but not in its NAV or in the benchmark.

I had the same problem with my NTSX benchmark and was able to fix it the same way.

I also originally implemented rebalancing using an incorrect method, and Corey clarified the correct method to use. (This did not improve the correlation.) ↩

I Was Probably Wrong About HIIT and VO2max

Mon, 03 Feb 2025 00:00:00 -0800

This research piece is not as rigorous or polished as usual. I wrote it quickly in a stream-of-consciousness style, which means it’s more reflective of my actual reasoning process.

My understanding of HIIT (high-intensity interval training) as of a week ago:

VO2max is the best fitness indicator for predicting health and longevity.
HIIT, especially long-duration intervals (4+ minutes), is the best way to improve VO2max.
Intervals should be done at the maximum sustainable intensity.

I now believe those are all probably wrong.

I think I got the wrong idea because a lot of HIIT/VO2max promoters cite scientific studies, which makes them seem superficially reasonable, but the studies they cite aren’t very good, or aren’t interpreted correctly.

A few months ago I started incorporating some HIIT into my cardio routine. But I didn’t really know the best way to do it, so last week I decided to do some research. I looked up the most-cited meta-analyses on Google Scholar and I noticed my confusion when the meta-analyses didn’t seem to support the conventional wisdom that HIIT is the best way to improve VO2max.

The most comprehensive single source I found was a meta-meta-analysis by Crowley et al. (2022)¹, which reviewed the findings of meta-analyses on HIT (high-intensity training) vs LIT (low-intensity training) for VO2max. The key quote:

Evidence from the meta-analyses that directly compared LIT versus HIT protocols on VO2max was, ostensibly, reported as either trivial or inconclusive. Three out of the six included meta-analyses reported small/moderate beneficial effects of HIT over LIT (α < 0.05). However, two of these reviews reported “substantial” heterogeneity (I2>0.75), small-study bias (p < 0.10), a relatively small pooled sample size (i.e., <1,000 participants), had a high degree of overlap (CCA = 11%) and reported several moderators (e.g., baseline fitness levels, age, HIT variables [e.g., volume, frequency, and duration]), which likely affected results.

Also, in my naiveté I had assumed that these were meta-analyses of RCTs, but in fact most of the included studies weren’t even RCTs:

Scribbans et al. reported that none of their included studies applied RCTs, Sloth et al. reported only four studies that applied RCTs design, and Gist et al. reported that the majority of included studies were RCTs.

(Note: Gist et al.², which did look mainly at RCTs, found that sprint interval training did not work better than endurance training (Cohen’s d = 0.04, 95% CI = -0.17 to 0.24.)

So I thought, okay, these meta-analyses don’t seem to favor HIIT much if at all. But maybe they’re done by stuffy academics who don’t know anything about real training. Are these meta-analyses considered respectable? So I went to see what the Barbell Medicine³ guys thought. I have a lot of respect for them when it comes to strength training. I don’t know if they know about cardio, but one of them is a former competitive swimmer so probably they know something. And they have good epistemics on strength training, and good epistemics might generalize. They did a podcast⁴ on HIIT with some useful content:

They started the podcast by criticizing a tweet⁵ in which fitness influencer Rhonda Patrick recommended a collection of “evidence-based HIIT protocols”. Their criticism mainly focused on how (they claimed) the provided HIIT protocols were way too hard. They quoted two responses⁶⁷ by exercise physiologists arguing the same.
They went on to talk about some of the research on HIIT, citing the same meta-analyses that I’d looked at.
Their ultimate recommendation (also given in an article⁸): “it is reasonable for about 80% of training to be of moderate intensity (zones 1-2), and about 20% reserved for higher intensity work (HIIT or SIT)”. They also said it’s good to use a variety of HIIT protocols and that there is no single optimal protocol.

I didn’t know if the Barbell Medicine guys were right about any of that, but it gave me some direction.

I had a look at the website of one of the people from that Twitter thread, Steve Magness⁹. He ran a 4:01 mile in high school so he probably has some idea of what he’s talking about.

Now, when some science-literate fitness influencers like Peter Attia and Rhonda Patrick give some recommendations about HIIT, and some other people like Steve Magness and Barbell Medicine disagree with them, I don’t have sufficient expertise to say who’s right. Both sides have the trappings of scientific credibility (e.g. citing multiple studies). But one thing I can do is check their logic.

So I checked Steve Magness’s logic. He wrote in a Twitter thread:¹⁰

You need all intensities to max VO2max. So it’s dumb to pit one vs. other

But research shows continuous likely matches HIIT for Vo2max increase

HIIT appears better when you constrain to 8 weeks but when you look over longer time it equalizes

Here’s data from a recent review.¹¹

You can see HIIT appears to increase VO2max more because of the time frame of most training studies (6-8 weeks).

Intense work gets big boost, then levels off. Endurance work gives Longer more gradual boost.

The referenced review does seem to support Magness’s argument, but I don’t know if the review is any good. What I do know is that Magness’s logic makes sense. It stands to reason that a more intense exercise protocol will cause faster short-term gains, but it can’t keep producing those rapid gains forever. And it stands to reason that if most studies on HIIT vs. LIT only last 6-12 weeks, then they will underestimate the long-term benefits of LIT.

That makes logical sense to me, which makes me think Steve Magness knows what he’s talking about.

He also wrote an article¹² arguing that some people care too much about VO2max for longevity:

Vo2max matters. But it’s just one component of many that make up both performance and aerobic fitness. And that’s important because if we return to the original claims that Vo2max is the key indicator of longevity, we’ll find that the majority of the studies cited did NOT even use Vo2max as the main variable. They used performance! In the majority of research, peak speed and incline during the exhausting test was the main correlate to longevity.

The large study¹³ on 750,000 veterans that found a 4-fold higher mortality risk for low versus high fitness used peak speed and incline, not Vo2max. Same with the research¹⁴ on 120,000 individuals finding a 5x difference in the risk of early death.

That makes logical sense to me. VO2max is only one aspect of fitness (albeit an important one), and it stands to reason that your actual ability to perform physical tasks is a better measure of physical health.

I did also look at some of the evidence cited in that article, specifically the Harber et al. (2017)¹⁵ meta-analysis, and Table 2 confirms Magness’s claim—most studies measured speed, time, or total work performed, not VO2max directly.

Insofar as I can verify Steve Magness’s claims, they seem to be correct. He also claims that HIIT protocols should not be “all-out”—for example, 60-second running intervals should be done between a 5K and a one-mile pace.¹⁶ He doesn’t cite any research on that claim, and as far as I know, there isn’t really research on it, it’s just how most high-performing athletes train. But since he seems right about the verifiable claims he’s made, I expect he’s right about that, too.

On the subject of checking people’s logic, here is an (admittedly cherry-picked) quote from the other side of the argument:

[I]s anything done in a total of 10-minutes that big of a deal?

If anyone is misinterpreting my statement as prescriptive:

My underlying point was that anything you can do in 10-minutes is limited on a relative harm basis, even if you do a lot.

This does not make logical sense to me. You can absolutely hurt yourself in less than 10 minutes. Even putting injury risk aside,

I’ve done 10-minute hill sprint intervals in the morning that left me feeling tired all day.
A 10-rep max squat takes less than 60 seconds, but it makes my legs sore for the next two days.

(Maybe that’s less logic and more personal experience, but a single example is enough to disprove a universal claim.)

I think Barbell Medicine has good logic, too. In their podcast on HIIT, they talk about how in strength training, nobody lifts the maximum possible weight every week. (And I have enough personal experience to know that maxing out every week wouldn’t work.) So it probably doesn’t make sense to max out your aerobic capacity in every week, either. I’m not sure strength training and cardio work the same way in that respect, but I expect things to be the same unless I have reason to believe they’re different.

The Barbell Medicine article on HIIT⁸ has some nice sample workouts that line up with Magness’s recommendations:

Workout #1

4 to 6 rounds of: 30 seconds on at 600-800 m running pace (or a speed sustainable in the range of ~90-150 seconds), 4 min off / easy effort

Workout #2

8 to 10 rounds of: 1-minute on at 1 mile-5 km running pace (or a speed sustainable in the range of 6-25 minutes), 1 minute off

Workout #3

3 to 5 rounds of: 5 minutes on at zone 4 heart rate (85-95% max), 3 min rest

I want to investigate this further, but here’s what I tentatively believe:

VO2max predicts longevity, but athletic performance matters more than VO2max alone.
I should exercise at a variety of intensities to get a well-rounded fitness capacity, but HIIT isn’t particularly better for improving fitness than LIT, and 4-minute intervals aren’t particularly better than other interval schemes.
Intervals should not be done at the maximum sustainable intensity; they should be done at an intensity that’s challenging but doesn’t leave you wiped out. As Magness wrote, “The goal isn’t to create fatigue, that’s easy to do. The goal is to slightly embarrass your body in the right direction.”

Coming back to my own training: I have loathed every version of HIIT I’ve tried so far. But that’s because I listened to the people saying that HIIT should be “all-out.” Next time I’m going to do HIIT at a more comfortable pace.

Notes

Crowley, E., Powell, C., Carson, B. P., & W. Davies, R. (2022). The Effect of Exercise Training Intensity on VO2max in Healthy Adults: An Overview of Systematic Reviews and Meta-Analyses. ↩
Gist, N. H., Fedewa, M. V., Dishman, R. K., & Cureton, K. J. (2013). Sprint Interval Training Effects on Aerobic Capacity: A Systematic Review and Meta-Analysis. ↩
https://www.barbellmedicine.com/ ↩
https://redcircle.com/shows/0cc66fc4-ccb8-4c60-8cc6-7367e52c4159/episodes/706bc687-0a98-4057-8e2e-e4db349bba4a ↩
https://x.com/foundmyfitness/status/1844811732080021919 ↩
https://x.com/StephenSeiler/status/1845357464130031873 ↩
https://x.com/stevemagness/status/1845079291320525202 ↩
https://www.barbellmedicine.com/blog/hiit-high-intensity-interval-training/ ↩ ↩²
https://www.stevemagness.com/about/ ↩
https://x.com/stevemagness/status/1849918347086795151 ↩
Mølmen, K. S., Almquist, N. W., & Skattebo, Ø. (2024). Effects of Exercise Training on Mitochondrial and Capillary Growth in Human Skeletal Muscle: A Systematic Review and Meta-Regression. doi: 10.1007/s40279-024-02120-2 ↩
https://thegrowtheq.com/longevity-and-vo2max-does-it-matter/ ↩
Kokkinos, P., Faselis, C., Samuel, I. B. H., Pittaras, A., Doumas, M., Murphy, R., Heimall, M. S. et al. (2022). Cardiorespiratory Fitness and Mortality Risk Across the Spectra of Age, Race, and Sex. ↩
Mandsager, K., Harb, S., Cremer, P., Phelan, D., Nissen, S. E., & Jaber, W. (2018). Association of Cardiorespiratory Fitness With Long-term Mortality Among Adults Undergoing Exercise Treadmill Testing. ↩
Harber, M. P., Kaminsky, L. A., Arena, R., Blair, S. N., Franklin, B. A., Myers, J., & Ross, R. (2017). Impact of Cardiorespiratory Fitness on All-Cause and Disease-Specific Mortality: Advances Since 2009. doi: 10.1016/j.pcad.2017.03.001 ↩
https://x.com/stevemagness/status/1845079292784365803 ↩

Retroactive If-Then Commitments

Sat, 01 Feb 2025 00:00:00 -0800

An if-then commitment is a framework for responding to AI risk: “If an AI model has capability X, then AI development/deployment must be halted until mitigations Y are put in place.”

As an extension of this approach, we should consider retroactive if-then commitments. We should behave as if we wrote if-then commitments a few years ago, and we should commit to implementing whatever mitigations we would have committed to back then.

Imagine how an if-then commitment might have been written in 2020:

Pause AI development and figure out mitigations if:

AI exhibits what looks like deceptive or misaligned behavior, or feigns alignment (1, 1b, 2)

AI breaks out of containment in a toy example

AI finds a real-world zero-day vulnerability

AI qualifies for Mensa¹

AI exhibits some degree of agentic capabilities

AI writes malware

Well, AI models have now done or nearly-done all of those things.

We don’t know what mitigations are appropriate, so AI companies should pause development until (at a minimum) AI safety researchers agree on what mitigations are warranted, and those mitigations are then fully implemented.

(You could argue about whether AI really hit those capability milestones, but that doesn’t particularly matter. You need to pause and/or restrict development of an AI system when it looks potentially dangerous, not definitely dangerous.)

Notes

Okay, technically it did not score well enough to qualify, but it scored well enough that there was some ambiguity about whether it qualified, which is only a little bit less concerning. ↩

The 7 Best High-Protein Breakfast Cereals

Fri, 17 Jan 2025 00:00:00 -0800

Updated 2025-03-19 to add Catalina Crunch Cinnamon Toast.

(I write listicles now)

(there are only 7 eligible high-protein breakfast cereals, so the ones at the bottom are still technically among the 7 best even though they’re not good)

If you search the internet, you can find rankings of the best “high-protein” breakfast cereals. But most of the entries on those lists don’t even have that much protein. I don’t like that, so I made my own list.

This is my ranking of genuinely high-protein breakfast cereals, which I define as containing at least 25% calories from protein.

Many food products like to advertise how many grams of protein they have per serving. That number doesn’t matter because it depends on how big a serving is. Hypothetically, if a food had 6g protein per serving but each serving contained 2000 calories, that would be a terrible deal. The actual number that matters is the proportion of calories from protein.

My ranking only includes vegan cereals because I’m vegan. Fortunately most cereals are vegan anyway. The main exception is that some cereals contain whey protein, but that’s not too common—most of them use soy, pea, or wheat protein instead.

1. Oatmeal with added protein powder (27% calories from protein, if you make it the way I do)

This is regular oatmeal because I couldn't find a stock photo of oatmeal mixed with protein powder.

You can buy pre-mixed oatmeal and protein powder, but it’s unnecessarily expensive so I prefer to mix it myself. Obviously the amount of protein varies depending on how much protein powder you add. I personally like to mix:

two servings of oats (= one cup, or 300 calories)
one scoop of protein powder (= 1/3 cup, or 25 grams, or 90 calories)

Those proportions provide 30% calories from protein. I find the consistency gets sticky if you add more protein than that. If you don’t like the consistency at this ratio, you can add more oats/less protein.

I save time by mixing the oats and protein powder in a giant jug. The jug lasts me for a few weeks and this way I don’t have to measure out the proportions every morning.

Oatmeal is my favorite cereal because there are so many ways to make it. I like to mix in blueberries, blackberries, or bananas, which add nutrients and cover up the protein-y flavor. (Plain oatmeal with protein powder tastes kind of weird.)

This mixture has 30% calories from protein without fruit and about 27% with fruit.

2. Special K High Protein Chocolate Almond (33% calories from protein)

This is my favorite cold breakfast cereal. Most high-protein cereals taste merely tolerable, but this one tastes actively good. It has a nice crunchy texture from the combination of almonds and cereal flakes, and it has enough sugar to give it a good flavor.

Unfortunately it doesn’t seem to be available anymore. I reached out to customer service to ask if it’s discontinued and they said they’re still producing it, but I haven’t been able to find it anywhere. (Nobody else seems to be able to find it either—see the product reviews here.)

(Special K High Protein should not be confused with Special K Protein, which did not qualify for my list because it has less than 25% calories from protein.)

3. Catalina Crunch (40% calories from protein)

This is my 3rd favorite high-protein cereal, and it has more protein than #1 or #2. Catalina Crunch has become a staple for me now that Special K High Protein is apparently discontinued.

There are a number of flavors, but the only ones I’ve tried are Cinnamon Toast and Dark Chocolate. Originally I had Dark Chocolate on this list at #4, but later I tried Cinnamon Toast which is better, so I’ve moved Catalina Crunch up to rank 3.

It doesn’t get soggy in milk and it doesn’t fall apart in your mouth. It’s sugar-free, but it tastes surprisingly good to me (I don’t usually like sugar-free cereals). The downside is it’s pretty expensive compared to most breakfast cereals.

4. Wheaties Protein (supplemented with lysine) (32% calories from protein)

Wheaties Protein is the last high-protein cereal I’d consider good, like I actively enjoy eating it.

Be aware that this cereal’s protein comes primarily from wheat, which doesn’t have much of the amino acid lysine. It has plenty of every other essential amino acid, but if you eat a lot of this cereal, you need to make sure you get lysine from somewhere else.

To get a full amino acid profile, you need to add about 1 gram of lysine per 60 grams of wheat protein. I personally buy these lysine supplements and take one 500mg pill per bowl of cereal.

5. Post Premier Protein (44% calories from protein)

This cereal has two different flavors, Chocolate Almond and Mixed Berry Almond. I personally like the chocolate flavor better. Both have a passably good flavor and texture but they have a weird protein-y crunchiness.

Post Premier Protein gets most of its protein from wheat, which, as I mentioned before, doesn’t contain much lysine. Fortunately it also contains pea protein, which has an abundance of lysine.

I don’t know the exact protein ratios, but I believe the overall balance still doesn’t contain enough lysine, so it would be prudent to get some extra lysine from somewhere else. I personally would take one 500mg lysine pill per two bowls of cereal. (I always eat two bowls for breakfast because I’m a growing boy.)

6. Special K Zero (50% calories from protein)

If you had to guess which of Special K High Protein and Special K Zero had more protein, you might think it’s the one with “protein” in the name, but you’d be wrong. This cereal contains an extraordinary 50% calories from protein with a good amino acid composition. Unfortunately, the reason it has so much protein is that it’s basically just lumps of protein powder.

When I take a bite of this cereal, it tastes good for the first five seconds or so. Then it dissolves into a powdery mush—it feels like I’ve poured wet protein powder directly into my mouth. I wouldn’t eat Special K Zero unless I was really desperate for protein. (Even then, I would rather have a protein shake.)

But some people seem to like it so your mileage may vary.

7. Three Wishes (24% to 27% calories from protein, depending on flavor)

Like Special K Zero, Three Wishes feels like eating protein powder flakes. But it only has half as much protein as Special K Zero. If you can stomach eating protein powder flakes, you might as well eat Special K Zero instead.

(I’ve only tried one flavor of Three Wishes, but it was a while ago and I don’t remember which flavor it was. I assume the other flavors have the same bad protein-y texture as the one I tried.)

Honorable mention: Kashi Go

Kashi Go, with 24% calories from protein, just barely does not qualify for my list. It has a good flavor and texture, but it makes my mouth feel weird if I eat too much of it. (I get the same mouth feeling when I eat a lot of spinach.) If it had that extra one percentage point of protein, I would put it at #4 on my list.

Unranked because it’s not vegan: Magic Spoon (33% to 37% calories from protein, depending on flavor)

I’ve never eaten Magic Spoon because it’s not vegan (it contains milk protein).

Drew Gooden says it’s bad and he makes funny YouTube videos which means he must have good opinions about cereal. His description of eating Magic Spoon sounds a lot like my experience eating Special K Zero and Three Wishes—it tastes good for a few seconds, then it turns into a protein mush.

Some other non-vegan cereals that I know nothing about

Snack House Keto Cereal: 44% calories from protein (uses milk protein)
Julian Bakery ProGranola Cereal: 44% calories from protein (uses egg white protein (weird choice but ok))
Perfect Keto Cereal: 36% calories from protein (uses milk protein) (appears to be discontinued)
Wonderworks Keto Friendly Breakfast Cereal: 35% calories from protein (uses milk and soy protein)

Those are all the high-protein breakfast cereals that I’m aware of.

Price table

This table gives the price of each cereal in terms of cents per gram of protein, ordered from cheapest to most expensive. I pulled these prices off Amazon; the prices in your area might differ.

For the cost of protein oatmeal, I used Quaker 1-Minute Oats plus NOW Foods Soy Protein Isolate because those are the brands I buy.

Cereal	Price (¢/g)
oatmeal + protein powder	3.7¢
Post Premier Protein	4.2¢
Special K High Protein	4.8¢
Wheaties Protein	4.9¢
Special K Zero	7.9¢
Catalina Crunch	9.1¢
Kashi Go	11.4¢¹
Three Wishes	12.5¢

Prices for non-vegan cereals:²

Cereal	Price (¢/g)
Julian Bakery ProGranola	10¢
Wonderworks Keto Friendly	13.5¢
Magic Spoon	13.8¢
Snack House Keto	16.5¢

Notes

This is strangely expensive for a big-brand cereal (Kashi is a subsidiary of Kellogg), my guess is there’s some sort of temporary supply issue and the price will go down. ↩
How convenient for my pro-vegan agenda that the non-vegan cereals are all so expensive!

Inconveniently for my agenda, the cheapest of the non-vegan cereals uses egg protein, which causes more animal suffering than whey protein.

Honestly I’m not that concerned about whey protein, it’s one of the least harmful animal products to buy. ↩

Charity Cost-Effectiveness Really Does Follow a Power Law

Wed, 25 Dec 2024 00:00:00 -0800

Conventional wisdom says charity cost-effectiveness obeys a power law. To my knowledge, this hypothesis has never been properly tested.¹ So I tested it and it turns out to be true.

(Maybe. Cost-effectiveness might also be log-normally distributed.)

Cost-effectiveness estimates for global health interventions (from DCP3) fit a power law (a.k.a. Pareto distribution) with $\alpha = 1.11$. [More]
Simulations indicate that the true underlying distribution has a thinner tail than the empirically observed distribution. [More]

Contents
Fitting DCP3 data to a power law
Does estimation error bias the result?
Future work I would like to see
Source code and data
Notes

Fitting DCP3 data to a power law

The Disease Control Priorities 3 report (DCP3) provides cost-effectiveness estimates for 93 global health interventions (measured in DALYs per US dollar). I took those 93 interventions and fitted them to a power law.

You can see from this graph that the fitted power law matches the data reasonably well:

To be precise: the probability of a DCP3 intervention having cost-effectiveness $x$ is well-approximated by the probability density function $f(x) = \displaystyle\frac{1.11}{x^{2.11}}$, which is a power law (a.k.a. Pareto distribution) with $\alpha = 1.11$.

It’s possible to statistically measure whether a curve fits the data using a goodness-of-fit test. There are a number of different goodness-of-fit tests; I used what’s known as the Kolmogorov-Smirnov test². This test essentially looks at how far away the data points are from where the curve predicts them to be. If many points are far to one side of the curve or the other, that means the curve is a bad fit.

I ran the Kolmogorov-Smirnov test on the DCP3 data, and it determined that a Pareto distribution fit the data well.

The goodness-of-fit test produced a p-value of 0.79 for the null hypothesis that the data follows a Pareto distribution. p = 0.79 means that, if you generated random data from a Pareto distribution, there’s a 79% chance that the random data would look less like a Pareto distribution than the DCP3 data does. That’s good evidence that the DCP3 data is indeed Pareto-distributed or close to it.³

However, the data also fits well to a log-normal distribution.

Pareto and log-normal distributions look similar most of the time. They only noticeably differ in the far right tail—a Pareto distribution has a fatter tail than a log-normal distribution, and this becomes more pronounced the further out you look. But in real-world samples, we usually don’t see enough tail outcomes to distinguish between the two distributions.

DCP3 only includes global health interventions. If we expanded the data to include other types of interventions, we might find a fatter tail, but I’m not aware of any databases that cover a more comprehensive set of cause areas.

(The World Bank has data on education interventions, but adding one cause area at a time feels ad-hoc and it would create gaps in the distribution.)

Does estimation error bias the result?

Yes—it causes you to underestimate the true value of $\alpha$.

(Recall that the alpha ($\alpha$) parameter determines the fatness of the tail—lower alpha means fatter tail. So estimate error makes the tail look fatter than it really is.)

There’s a difference between cost-effectiveness and estimated cost-effectiveness. Perhaps estimation error follows a power law, but the true underlying cost-effectiveness numbers don’t. And even if they do, our cost-effectiveness estimates might produce a bias in the shape of the fitted distribution.

I tested this by generating random Pareto-distributed⁴ data to represent true cost-effectiveness, and then multiplying by a random noise variable to represent estimation error. I generated the noise as a log-normally-distributed random variable centered at 1 with $\sigma = 0.5$⁵ (colloquially, that means you can expect the estimate to be off by 50%).

I generated 10,000 random samples⁶ at various values of alpha, applied some estimation error, and then fit the resulting estimates to a Pareto distribution. The results showed strong goodness of fit, but the estimated alphas did not match the true alphas:

true alpha 0.8  -->  0.73 estimated alpha (goodness-of-fit: p = 0.3)
true alpha 1.0  -->  0.89 estimated alpha (goodness-of-fit: p = 0.08)
true alpha 1.2  -->  1.07 estimated alpha (goodness-of-fit: p = 0.4)
true alpha 1.4  -->  1.22 estimated alpha (goodness-of-fit: p = 0.5)
true alpha 1.8  -->  1.54 estimated alpha (goodness-of-fit: p = 0.1)

To determine the variance of the bias, I generated 93 random samples at a true alpha of 1.1 (to match the DCP3 data) and fitted a Pareto curve to the samples. I repeated this process 10,000 times.

Across all generations, the average estimated alpha was 1.06 with a standard deviation of 0.27. That’s a small bias—only 0.04—but it’s highly statistically significant (t-stat = –15, p = 0⁷).

A true alpha of 1.15 produces a mean estimate of 1.11, which equals the alpha of the DCP3 cost-effectiveness data. So if the DCP3 estimates have a 50% error ($\sigma = 0.5$), then the true alpha parameter is more like 1.15.

Increasing the estimate error greatly increases the bias. When I changed the error (the $\sigma$ parameter) from 50% to 100%, the bias became concerningly large, and it gets larger for higher values of alpha:

true alpha 0.8  -->  0.65 mean estimated alpha
true alpha 1.0  -->  0.78 mean estimated alpha
true alpha 1.2  -->  0.87 mean estimated alpha
true alpha 1.4  -->  0.96 mean estimated alpha
true alpha 1.6  -->  1.03 mean estimated alpha
true alpha 1.8  -->  1.11 mean estimated alpha

If the DCP3 samples have a 100% error then the true alpha is 1.8—much higher than the estimated value of 1.11.

In addition, at 100% error with 10,000 samples, the estimates no longer fit a Pareto distribution well—the p-value of the goodness-of-fit test ranged from 0.005 to <0.00001 depending on the true alpha value.⁸

Curiously, decreasing the estimate error flipped the bias from negative to positive. When I reduced the simulation’s estimate error to 20%, a true alpha of 1.1 produced a mean estimated alpha of 1.14 (standard deviation 0.31, t-stat = 13, p = 0⁷). A 20% error produced a positive bias across a range of alpha values—the estimated alpha was always a bit higher than the true alpha.

Future work I would like to see

A comprehensive DCP3-esque list of cost-effectiveness estimates for every conceivable intervention, not just global health. (That’s probably never going to happen but it would be nice.)
More data on the outer tail of cost-effectiveness estimates, to better identify whether the distribution looks more Pareto or more log-normal.

Source code and data

Source code is available on GitHub. Cost-effectiveness estimates are extracted from DCP3’s Annex 7A; I’ve reproduced the numbers here in a more convenient format.

Notes

The closest I could find was Stijn on the EA Forum, who plotted a subset of the Disease Control Priorities data on a log-log plot and fit the points to a power law distribution, but did not statistically test whether a power law represented the data well. ↩
Some details on goodness-of-fit tests:

Kolmogorov-Smirnov is the standard test, but it depends on the assumption that you know the true parameter values. If you estimate the parameters from the sample (as I did), then it can overestimate fit quality.

A recent paper by Suarez-Espinoza et al. (2018)⁹ devises a goodness-of-fit test for the Pareto distribution that does not depend on knowing parameter values. I implemented the test but did not find it to be more reliable than Kolmogorov-Smirnov—for example, it reported a very strong fit when I generated random data from a log-normal distribution. ↩
A high p-value is not always evidence in favor of the null hypothesis. It’s only evidence if you expect that, if the null hypothesis is false, then you will get a low p-value. But that’s true in this case.

(I’ve previously complained about how scientific papers often treat p > 0.05 as evidence in favor of the null hypothesis, even when you’d expect to see p > 0.05 regardless of whether the null hypothesis was true or false—for example, if their study was underpowered.)

If the data did not fit a Pareto distribution then we’d expect to see a much smaller p-value. For example, a goodness-of-fit test for a normal distribution gives p < 0.000001, and a gamma distribution gives p = 0.08. A log-normal distribution gives p = 0.96, so we can’t tell whether the data is Pareto or log-normal, but it’s unlikely to be normal or gamma. ↩
Actually I used a Lomax distribution, which is the same as a Pareto distribution except that the lowest possible value is 0 instead of 1. ↩
The $\sigma$ parameter is the standard deviation of the logarithm of the random variable. ↩
In practice we will never have 10,000 distinct cost-effectiveness estimates. But when testing goodness-of-fit, it’s useful to generate many samples because a large data set is hard to overfit. ↩
As in, the p-value is so small that my computer rounds it off to zero. ↩ ↩²
Perhaps that’s evidence that the DCP3 estimates have less than a 100% error, since they do fit a Pareto distribution well? That would be convenient if true.

But it’s easy to get a good fit if we reduce the sample size to 93. When I generated 93 samples with 100% error, I got a p-value greater than 0.5 most of the time. ↩
Suárez-Espinosa, J., Villasenor-Alva, J. A., Hurtado-Jaramillo, A., & Pérez-Rodríguez, P. (2018). A goodness of fit test for the Pareto distribution. ↩

"You Can't Calculate the Expected Utility of a Communist Revolution"

Fri, 06 Dec 2024 00:00:00 -0800

Leftist critics of effective altruism like to say this. Well, it’s not true, and I proved it by calculating (an estimate of) the expected utility of a communist revolution. It wasn’t even hard—it took me less than an hour.

I put my cost-effectiveness analysis on SquiggleHub: https://squigglehub.org/models/mdickens/communist-revolution-ev

According to my analysis, working to start a communist revolution is between 11x and 400x as good as donating to GiveDirectly (that’s the 95% credence interval).¹ That makes it probably better than any GiveWell top charity.

That’s assuming communism is good, which it isn’t, but let’s say it is for the sake of argument.

My model includes calculations with extensive commentary. The model is relatively simple—I recommend reading it if you want to understand how it works.

In short, the cost-effectiveness analysis goes like this:

Assume life under communism will be 20% better than the status quo. This is probably an underestimate because it’s hard to capture the soul-crushing indignities of capitalism² but let’s say 20% to be conservative.
Look at the number of revolutionaries in historical communist revolutions to estimate the probability of success per revolutionary.
Downweight the probability of success based on the fact that the revolution might result in an inferior version of communism (like what happened with the Soviet Union).
Estimate how long it would take until a communist revolution happened anyway. The value of doing a revolution now equals [time between now and when it would have happened anyway] x [number of people who get to live under the new regime] x [how good communism is compared to the status quo].
The previous step gives us the total value of a revolution. Divide that by the probability of success per revolutionary to get the expected value of a single individual’s efforts.
Compare to how much an individual could donate to GiveDirectly if they worked a normal job instead of doing activism. That gives us the expected value of a communist revolutionary relative to a GiveDirectly donor.

Is this model perfectly accurate? No. I spent half an hour on it. I can immediately think of several ways it could be better.³ But the model is still informative—it provides a basic case that working for a communist revolution could be more cost-effective than GiveWell top charities. And it shows that, contrary to what many people say, it’s possible in principle to calculate the expected utility of a communist revolution.

The numbers in my model are mostly made up. But if it’s worth doing, it’s worth doing with made-up statistics.

Notes

Using GiveWell’s 2023 numbers. The 2024 numbers are considerably more optimistic about GiveDirectly, which makes the multiplier smaller, but a communist revolution still compares favorably to GiveWell top charities. ↩
I don’t actually believe this; this is me wearing my leftism hat. ↩
Some obvious improvements:
1. Look at a variety of historical revolutions instead of just one.
2. Account for the skill of the activist in question. Presumably, some people are more skilled than others at garnering support.
3. Instead of treating all activists as equally responsible, estimate the probability that a marginal activist causes a revolution to occur. Or estimate how much sooner a revolution occurs thanks to a marginal activist.
↩

Thoughts on My Donation Process

Wed, 04 Dec 2024 00:00:00 -0800

I have some observations and half-baked ideas about my recent donation process. They weren’t important enough to include in the main post, but I want to talk about them anyway.

On deference

Usually, I defer to the beliefs of other people who have spent more time on an issue than me, or who plausibly have more expertise, or who I just expect to have reasonable beliefs.

While writing my donations post, I made a conscious effort to defer less than usual. Deference might maximize the probability that I make the correct decision, but deference reduces the total amount of reasoning that’s happening, which is bad for the group as a whole. I want there to be more reasoning happening.

This is most relevant in my discussion of a few orgs that I disliked, which are also very popular among big EA funders. I’m 99% confident that the big funders have private information about those orgs, so maybe I should defer to them. But I’m also maybe 75% confident that if I had access to that information, it wouldn’t materially change my mind. I did anticipate that the private evidence would make me like the orgs a little better, so I updated based on this anticipation and evaluated the orgs a little more favorably than I would have otherwise.

On criticizing orgs

I am not as nice as I’d like to be. I have a habit of accidentally saying mean things that hurt people’s feelings.

On the other hand, I think most people are too nice: they hurt others long-term by refusing to give them useful information that’s difficult to hear.

(In theory, it’s possible to never say unnecessarily mean things, and always say necessary things, but only if you have perfect communication skills. In practice, there’s a tradeoff.)

I think it’s a good norm that, if you’re investigating an org and it opens up to you, you shouldn’t take what you learn and use it against the org. I probably wouldn’t criticize an org based on private information that it gave me. I did criticize some orgs, but all my criticisms were based on public information.

I think if most people wrote a donation post like mine, they’d self-censor in the interest of niceness and end up leaving out important information. I tried to avoid that, and erred more on the side of being mean (not pointlessly mean, but mean-and-truthful. Or maybe I should say mean-and-accurately-conveying-my-beliefs since I can’t promise that the things I said were true).

As with my choice on deference, this was perhaps the wrong choice at an individual level but the right choice at the group level.

I did focus on criticizing organizations and avoided saying negative things about specific people whenever possible.

Donation sizing

I have a donor-advised fund (DAF) that I contributed to when I was earning to give. How much of my DAF money should I donate this year? What’s a reasonable spend-down rate?

I’ve put a lot of thought into how quickly to spend philanthropic resources, including how AI timelines affect the answer. Unfortunately, all that thinking didn’t much help me answer the question.

Plus, there are some complications:

I have some personal savings, which I could choose to donate. Should I count them as part of my donation money?
I might earn significant income in the future. Right now it looks like I won’t, but I might do more earning to give at some point, or I might take a direct-work job that happens to pay well. If I expect to earn more in the future, then I should spend more of my DAF now.
I didn’t donate much money for the last few years. Should I do catch-up donations this year? Or maybe spread out my catch-up donations over the next few years?

I didn’t come up with good answers to any of these questions. Ultimately I chose how much to donate based on what felt reasonable.

Diversifying donations as a trade

I had an idea based on this comment by Oliver Habryka. He describes a trade between members of the EA community where some people do object-level work (relinquishing a high-paying job) and others earn money. He argues that when this trade occurs, the people doing object-level work should have some ownership over the funds that earners-to-give have earned.

I spent a while earning to give. So arguably I should donate money to people who started out in a similar position as me but went into direct work instead. Essentially, I should (acausally) trade with altruists who could’ve earned a lot of money but didn’t. And because there are many such people, arguably I should split my donations across many of them instead of only donating to my #1 favorite thing.

But this argument raises some questions. Who exactly was in a “similar position as me”? What about people who aren’t members of the EA community, but who are nonetheless doing similarly valuable work? What about people who didn’t have the necessary skill set to earn a lot of money, so they never made a choice not to?

I decided not to further pursue this line of reasoning because I couldn’t figure out how to make sense of it. I just did the obvious thing of donating to the org(s) that looked most cost-effective on the margin.

Cooperating with the Survival and Flourishing Fund

Should I donate less money to orgs that have received grants from the Survival and Flourishing Fund (SFF)?

I want to be cooperative with SFF. If I donate less to an org that’s received SFF funding, that seems uncooperative.

SFF has the S-process, which is a fancy method for allocating donations from a group of value-aligned donors who each want to be the donor of last resort, but who also want to make sure their favored orgs get funded. I could cooperate with SFF by participating in this process.

I asked them if they wanted to add my money to the S-process and they declined, so I consider myself to have officially Cooperated and now I’m allowed to donate less to orgs that received SFF funding. I don’t think SFF really cares if its donations trade off against mine because I have much less money than it does.

Where I Am Donating in 2024

Mon, 18 Nov 2024 00:00:00 -0800

Summary

Last updated 2025-04-25.

It’s been a while since I last put serious thought into where to donate. Well I’m putting thought into it this year and I’m changing my mind on some things.

I now put more priority on existential risk (especially AI risk), and less on animal welfare and global priorities research. I believe I previously gave too little consideration to x-risk for emotional reasons, and I’ve managed to reason myself out of those emotions.

Within x-risk:

AI is the most important source of risk.
There is a disturbingly high probability that alignment research won’t solve alignment by the time superintelligent AI arrives. Policy work seems more promising.
Specifically, I am most optimistic about policy advocacy for government regulation to pause/slow down AI development.

In the rest of this post, I will explain:

Why I prioritize x-risk over animal-focused longtermist work and global priorities research.
Why I prioritize AI policy over AI alignment research.
My beliefs about what kinds of policy work are best.

Then I provide a list of organizations working on AI policy and my evaluation of each of them, and where I plan to donate.

Cross-posted to the Effective Altruism Forum.

Summary
Contents
I don’t like donating to x-risk
Cause prioritization
“Man versus man” conflicts within AI policy
Conflicts that aren’t “man vs. man” but nonetheless require an answer
Organizations
Where I’m donating
- Prioritization within my top five
- Where I’m donating (this is the section in which I actually say where I’m donating)
Changelog
Notes

I don’t like donating to x-risk

(This section is about my personal motivations. The arguments and logic start in the next section.)

For more than a decade I’ve leaned toward longtermism and I’ve been concerned about existential risk, but I’ve never directly donated to x-risk reduction. I dislike x-risk on an emotional level for a few reasons:

In the present day, aggregate animal welfare matters far more than aggregate human welfare (credence: 90%). Present-day animal suffering is so extraordinarily vast that on some level it feels irresponsible to prioritize anything else, even though rationally I buy the arguments for longtermism.
Animal welfare is more neglected than x-risk (credence: 90%).¹
People who prioritize x-risk often disregard animal welfare (or the welfare of non-human beings, whatever shape those beings might take in the future). That makes me distrust their reasoning on cause prioritization. (This isn’t universally true—I know some people who care about animals but still prioritize x-risk.)
I find it distasteful the way people often talk about “human extinction”, which seemingly ignores the welfare of all other sentient beings.²

For a while, I donated to animal-focused orgs that looked good under longtermism, like Sentience Institute. In recent years, I’ve avoided thinking about cause prioritization by supporting global priorities research (such as by donating to the Global Priorities Institute)—pay them to think about cause prioritization so I don’t have to. I still believe there’s a good case for that sort of research, but the case for existential risk is stronger (more on this below).

I’ve spent too long ignoring my rationally-formed beliefs about x-risk because they felt emotionally wrong. I’m normally pretty good at biting bullets. I should bite this bullet, too.

This decision to prioritize x-risk (mostly³) didn’t happen because I changed my mind. It happened because I realized I was stupidly letting my emotional distaste toward x-risk sway my decision-making.

On the other hand, I’ve become more worried about AI in the last few years. My P(doom) hasn’t really gone up, but the threat of misaligned AI has become more visceral. I believe unaligned AI is my most likely cause of death, and I’d rather not die.⁴

Cause prioritization

S-risk research and animal-focused longtermism

I believe animal-focused (or non-human-focused) longtermist work is important (credence: 95%), and that it’s far more neglected than (human-focused) x-risk reduction (credence: 99%). I believe the same about s-risk research (and s-risks heavily overlap with animal-focused longtermism, so that’s not a coincidence). But I also believe:

At equivalent levels of funding, marginal work on x-risk is more cost-effective (credence: 75%) because non-human welfare is likely to turn out okay if we develop friendly AI.
The cost-effectiveness of x-risk funding diminishes slowly enough that it’s better even at current funding levels (credence: 65%), especially because some of the most promising sub-fields within x-risk remain poorly funded.

Improving animal welfare likely has good flow-through effects into the distant future. But I think those flow-through effects don’t have a huge expected value compared to x-risk reduction because they only matter under certain conditions (I discussed these conditions a while back in Is Preventing Human Extinction Good? and On Values Spreading.)

This judgment is hard to make with confidence because it requires speculating about what the distant future will look like.

In Marcus Abramovitch’s excellent writeup on where he donated in 2023, he said,

I don’t think many x-risk organizations are fundamentally constrained on dollars and several organizations could be a lot more frugal and have approximately equal results.

I basically agree with this but I think there are some x-risk orgs that need more funding, and they’re among some of the most promising orgs.

X-risk vs. global priorities research

A dilemma:

We can’t fully align AI until we solve some foundational problems in ethics and other fields.
If we don’t align AI, we will go extinct before we solve those foundational problems.

(One proposed solution is to conduct a long reflection, and basically put a superintelligent AI on standby mode until that’s done. This proposal has some issues but I haven’t heard anything better.)

So, should we focus on x-risk or global priorities research?

Ultimately, I think x-risk is the higher priority (credence: 70%). If we build a (mostly) friendly AI without really figuring out some details of AI alignment, maybe we can work things out from there. But the problems in global priorities research seem so complex that we have essentially no chance of solving them before AGI arrives (unless AGI turns out to take much longer than expected), regardless of how much funding goes to global priorities research.

Prioritization within x-risk

Among the various existential risks, AI risk stands out as clearly the most important. I believe this for essentially the same reasons that most people believe it.

In short:

Natural risks are less concerning than human-caused risks (credence: 98%).
Climate change is not a serious existential threat (credence: 90%). See John Halstead’s report Climate Change & Longtermism.
Engineered pandemics are considerably less likely to cause extinction than AI (credence: 95%). I’ve heard biologists in the x-risk space claim that it would be very hard for a pandemic to cause total extinction.
Nuclear war is worrisome but less of an extinction risk than AI (credence: 85%). See 80,000 Hours’ table of x-risk estimates for nuclear war.

For more, see Michael Aird’s database of existential risk estimates.

AI safety technical research vs. policy

There are a few high-level strategies for dealing with AI risk. We can broadly classify them into (1) technical research and (2) policy. Basically:

technical research = figure out how to prevent AI from killing everyone
policy = increase the probability that policies/regulations will reduce x-risk

(You could further divide policy into research vs. advocacy—i.e., figure out what makes for good regulations vs. advocate for regulations to be enacted. I’ll talk more about that later.)

I don’t have any expertise in AI⁵ and I don’t know what kinds of alignment research are most promising, but experts can’t seem to agree either—some think prosaic alignment will work, others think we need fundamentally new paradigms. (I lean toward the latter (credence: 70%).)

But I don’t see how we are going to solve AI alignment. The best existing research seems like maybe it has some chance of someday leading to some method that could eventually solve alignment with enough work, perhaps.⁶ Our best hope is that either (1) AGI turns out to be much harder to develop than it looks or (2) solving alignment turns out to be really easy for some unforeseen reason.⁷ But those hopes are non-actionable—I can’t increase their probability by donating money.

Ever since I became concerned about AI risk (about a decade ago), I’ve weakly believed that we were not on pace to solve alignment before AGI arrived. But I thought perhaps technical research would become sufficiently popular as the dangers of AI became more apparent. By now, it’s clear that that isn’t happening, and we’re not going to solve AI alignment in time unless the problem turns out to be easy.

I used to be even more pessimistic about AI policy than technical research,⁸ but now I think it’s the more promising approach (credence: 80%). Surprisingly (to me), AI safety (as in notkilleveryoneism) is now kinda sorta mainstream, and there’s some degree of political will for creating regulations that could prevent AI from killing everyone. SB 1047, which might have meaningfully decreased x-risk, saw widespread support. (Unfortunately, one particular guy with veto power did not support it.⁹)

Another consideration: bad policy work can backfire. I don’t know much about policy and I’m relatively bad at understanding people, so on priors, I don’t expect to be good at figuring out which policy efforts will work. I used to think I should defer to people with better social skills. But now I’ve seen some of the poor results produced by policy orgs that care a lot about reputation management, and I’ve seen how messaging about extinction is much more palatable than the people with good social skills predicted (e.g., as demonstrated by public opinion polling), so I think I overrated others’ judgment and underrated my own. As a consequence, I feel more confident that I can identify which policy orgs are doing good work.

In summary:

I don’t think technical research is going to work.
Policy might work.
I think I’m qualified enough to evaluate policy orgs.

So I want to donate to something related to AI policy.

Quantitative model on research vs. policy

I built a coarse quantitative model on the expected value of donations to technical research vs. policy. The model inputs are very rough but the model illustrates some important principles.

(Disclaimer: First I decided to donate to AI policy, then I built the model, not the other way around.¹⁰ If the model had disagreed with my beliefs, then I would have changed the model. But if I couldn’t find a reasonable way to make the model fit my beliefs, then I would have changed my beliefs.)

Preventing x-risk works like voting. All the expected value of your vote comes from the situation where the outcome is exactly tied and your vote breaks the tie. If the expected vote is close to 50/50, your vote has a high EV. If the expected vote count is far from 50/50, there’s an extremely small probability that your vote will matter.

If I believed that it would cost (say) $10 billion to solve AI alignment, and also the total spending without my donation would be close to $10 billion, then my donation to alignment research has a high EV. But in fact I believe it probably costs much more than we’re going to spend¹¹ (assuming no regulations).

On a naive view, that means a donation to alignment research has extremely low EV. But that’s not correct because it doesn’t account for uncertainty. My median guess is that solving AI alignment will cost maybe $100 billion, and we will only actually spend $1 billion.¹² If my credence intervals for those two numbers followed normal distributions, then the probability of making a difference would be incredibly small (like, “number of atoms in the solar system” small) because normal distributions have extremely low probability mass in the tails. But my beliefs have wide credence intervals, and they’re not normally distributed. So my distribution for cost-to-solve-alignment heavily overlaps with total-spending-on-alignment.

It’s hard to have good intuitions for the probability that a donation makes a difference when that probability depends on the intersection of two overlapping fat-tailed distributions. That’s the sort of thing a quantitative model can help with, even if you don’t take the numbers too seriously.

Ultimately, I think AI policy has higher EV than technical research. According to my made-up numbers, donating to policy is ~3x more cost-effective than donating to research. Somewhat more pessimistic inputs can change the ratio to >1000x.¹³ The ratio flips to <1x if you think there’s close to a 50/50 chance that we solve alignment without any government intervention.

“Man versus man” conflicts within AI policy

Some prioritization decisions are positive sum: people can work on technical research and policy at the same time. But others are zero sum. I’m wary of “man versus man” conflict—of working in opposition to other (loosely) value-aligned people. But in policy, sometimes you have to engage in “man versus man” conflict. I want to think extra carefully before doing so.

There are a few such conflicts within AI policy.

Strategy A	Strategy B
Do capabilities and safety research in parallel. We need more advanced models to better understand what AGI will look like.	Slow down capabilities research to buy more time for safety research.
Don’t push for regulations, they will excessively harm technological development.	Push for regulations because the risk is worth it.
Take the time to figure out nuanced regulations that won’t impede the good parts of AI.	Push for regulations ASAP, even if they have worse side effects.
Cooperate with big AI companies to persuade them to behave more safely.	Work against AI companies to stop their dangerous behaviors.
Diplomatically develop political connections, and later use those to push for AI safety policies.	Loudly argue for AI safety policies now, even if it makes us look weird.

In every case, I like strategy B better. But how sure am I that I’m on the right side?

Parallel safety/capabilities vs. slowing AI

What are the arguments for advancing capabilities?

If we don’t advance capabilities, China will. Or some other company that doesn’t care about safety will.

This is the strongest argument by my assessment, but I still think it’s wrong (credence: 90%).

The companies that have done the most to accelerate AI development have all done so in the name of safety. If they didn’t believe advancing the frontier was the safest move, the world would be in a much safer position right now.
It’s likely that international treaties could prevent arms races—they worked well(ish) for nuclear weapons.
China is behind the US on AI. There is no need to race harder when you’re ahead.
By my amateurish judgment, China doesn’t seem as interested in an arms race as the US. You don’t need to race against someone who’s not racing.
How sure are you that it’s better for the United States to develop AI first? (China is less interested in controlling world politics than the US, and the Chinese government seems more concerned about AI risk than the US government.)
Who develops superintelligent AI first only matters in the narrow scenario where AI alignment is easy but also the AI can and will be used by its creators to take over the world:
- If AI alignment is hard, it doesn’t matter who develops it first because everyone dies either way.
- If the AI is fully aligned, it will refuse to fulfill any unethical requests its creator makes (such as taking over the world).
I don’t think we as a society have a good grasp on the game theory of arms races but I feel like the solution isn’t “push the arms race forward even faster”.

AI companies need to build state-of-the-art (SOTA) models so they can learn how to align those models.

I’ve heard people at Anthropic make this argument. But it’s disingenuous (or at least motivated)¹⁴ because Anthropic is accelerating capabilities, not just matching the capabilities of pre-existing models—and they have a history of almost-but-not-technically lying about whether they were going to advance capabilities).¹⁵ And the argument doesn’t really make sense because:

They could satisfy their stated goal nearly as well by training models that are (say) 3/4 as good as the state of the art (credence: 90%).
Or they could make deals with other companies to do safety research on their pre-existing SOTA models. This would satisfy their stated goal (credence: 98%). Companies might not be willing to cooperate like this, but surely it’s worth trying (and then trying harder).
There are many types of plausibly-productive alignment research that don’t require SOTA models (credence: 90%).¹⁶
Having SOTA models doesn’t differentially improve alignment—it teaches you just as much about how to improve capabilities (credence: 60%).

If the bolded argument is correct, then AI companies should:

Temporarily stop AI development.
Learn everything they possibly can about AI alignment with the current model.
Publish a report on how they would use a more capable AI to improve alignment.
Get review from third-party alignment researchers.
If reviewers have a strong consensus that the report is reasonable, only then resume AI development.

That is the sort of behavior you would see from a company that takes existential risk appropriately seriously.

We need to develop AI as soon as possible because it will greatly improve people’s lives and we’re losing out on a huge opportunity cost.

This argument only makes sense if you have a very low P(doom) (like <0.1%) or if you place minimal value on future generations. Otherwise, it’s not worth recklessly endangering the future of humanity to bring utopia a few years (or maybe decades) sooner. The math on this is really simple—bringing AI sooner only benefits the current generation, but extinction harms all future generations. You don’t need to be a strong longtermist, you just need to accord significant value to people who aren’t born yet.

I’ve heard a related argument that the size of the accessible lightcone is rapidly shrinking, so we need to build AI ASAP even if the risk is high. If you do the math, this argument doesn’t make any sense (credence: 95%). The value of the outer edge of the lightcone is extremely small compared to its total volume.¹⁷

AI could be the best thing that’s ever happened. But it can also be the best thing that’s ever happened 10/20/100 years from now, and if delaying AI lets us greatly reduce existential risk, then it’s worth the delay.

We should advance capabilities to avoid a “hardware overhang”: a situation where AI can be improved purely by throwing more hardware at it, which is potentially dangerous because it could cause AI to leap forward without giving people time to prepare.

Sam Altman has made this argument. But he’s disingenuous (credence: 95%) because he also wants to fund hardware advances, which will increase hardware overhang.¹⁸ And:

This argument implies that AI companies should stop looking for algorithmic improvements (credence: 90%) because they don’t produce overhang.
Pausing AI development would reduce demand for AI chips, slowing down hardware development.
Eliminating overhang only helps if we can meaningfully advance alignment using the higher level of capabilities. That seems unlikely to be worth the tradeoff because alignment has historically progressed very slowly. We are not on pace to solving alignment, with or without an overhang.
- If we will be able to align a bigger model, shouldn’t it be even easier to align the models we currently have? But we don’t know how to align the models we have (beyond the superficial pseudo-alignment that RLHF produces).
An AI Impacts report described some examples of overhang in other industries. “None of them match the behavior that people seem to expect will happen with hardware overhang.”

We need AGI to prevent some other existential risk from killing everyone.

Nearly every respectable person¹⁹ who has estimated x-risk probabilities agrees that AI is by far the largest x-risk in the next century.

It’s okay to advance capabilities because AI does not pose an existential risk.

This is a popular argument, but presumably everyone reading this already disagrees, so I’m not going to attempt to rebut it.

The parallel safety/capabilities side of the argument seems weak to me (and relies on a lot of what looks like motivated reasoning), so I feel comfortable supporting the pause side (credence: 85%).²⁰

But there’s some common ground:

Both sides should agree that slowing down hardware is good or at least neutral (credence: 75%).²¹ This alleviates every concern except for the one about the opportunity costs of delaying development.
Both sides should support regulations and international treaties that restrict the speed of AI development (credence: 65%). International treaties alleviate concerns about arms races and about needing to stay on the cutting edge.

Freedom vs. regulation

Arguments against regulation:

Regulations to slow AI would require the government to take authoritarian measures.

This argument seems pretty wrong to me (credence: 95%). Other industries have much stricter regulations than AI without slipping into totalitarianism. If the regulations on AI GPUs were as strict as the ones on, say, pseudoephedrine, that would be sufficient to slow and monitor hardware development.²²

Even if the regulations required individual people to turn in their GPUs to the government (and I don’t know why that would be required because GPU manufacturing is pretty centralized), there’s already precedent for that sort of thing in relatively free societies with e.g. the Australian government mandating that all citizens turn over their guns.

Regulations to slow AI might be nearly impossible to lift even if AI alignment gets solved, and then we won’t get the glorious transhumanist future.

I do think this is a real concern. Ultimately I believe it’s worth the tradeoff. And it does seem unlikely that excessive regulations could stay in place forever—I doubt we’d have the knowledge to develop friendly AI, but not the regulatory freedom, for (say) a thousand years.

(The United States essentially stopped developing nuclear power in the 1990s due to onerous regulations, but it just opened a new plant last year.)

Slow nuanced regulation vs. fast coarse regulation

Some argue that we should advocate for regulation, but push nuanced messaging to make sure we don’t hamstring economic development.

This disagreement largely comes down to P(doom) and AI timelines:

If P(doom) is low, it’s worth accepting some extra risk to figure out how to write careful regulations.
If timelines are long, we have plenty of time to figure out regulations.

If delaying regulations increases P(doom) by a lowish number like one percentage point, I don’t think it’s worth it—economically stifling regulations are not 1% as bad as extinction.

I think it’s unlikely that transformative AI comes in the next five years. But it’s not unthinkable. Metaculus (1, 2, 3) and surveyed experts don’t think it’s unthinkable either. And there could be a long delay between pushing for regulations and those regulations being implemented.

Dario Amodei, CEO of Anthropic, believes human-level AI could arrive within 1–2 years. That’s not enough time to figure out nuanced regulations. If he’s right, we need regulations right now. (Actually, we need regulations ten years ago, but the second best time is now.)

Given that forecasts put a reasonable probability on transformative AI arriving very soon, I don’t see how it makes sense to delay regulations any more than we already have.

(I believe a lot of people get this wrong because they’re not thinking probabilistically. Someone has (say) a 10% P(doom) and a 10% chance of AGI within five years, and they round that off to “it’s not going to happen so we don’t need to worry yet.” A 10% chance is still really really bad.)

And I’m not sure it’s feasible to write the sort of nuanced regulations that some people want. The ideal case is writing regulations that enable beneficial AI while preventing dangerous AI, but at the limiting case, that amounts to “legalize aligned AI and ban unaligned AI”. The less we know about AI alignment, the less likely the regulations are to do the right thing. And we know so little that I’m not sure there’s any real advantage to adding nuance to regulations.

(SB 1047 serves as a baseline: writing regulation with that level of nuance takes approximately zero marginal time because it’s already been done. Pushing to delay regulation only makes sense if you think we need something significantly more nuanced than SB 1047.)

Working with vs. against AI companies

I believe it’s better (on the margin) to work against AI companies (credence: 80%). I am not aware of any strong arguments or evidence for one side or the other, but I have a few bits of weak evidence.

(For someone with a larger budget, it might be worthwhile to commission an investigation into the track record of working with vs. against companies on this sort of thing.)

There’s a moderately strong argument in favor of cooperating with AI companies on policy:

If AI safety advocates make enemies with AI companies, those companies will get into a political fight with safety advocates, and companies are more powerful so they will probably win.

How much stock you put in that argument depends on how well you think industry-friendly regulations will reduce x-risk. It seems to me that they won’t. Good regulations will cause AI companies to make less money. If you advocate for regulations that companies like, then the regulations won’t be good.²³ I don’t see a middle ground where regulations prevent AI from killing everyone but also don’t impede companies’ profits. (A superintelligent AI is always going to be more profitable than anything else, unless it kills everyone.)

If we get into a political fight with AI companies, we might lose. But if we concede and let AI companies get the regulations they want, we definitely lose.²⁴

Alternatively, you could join an AI company or try to cooperatively influence it from the outside.

The (anecdotal) track record of working with AI companies so far:

People who worked for OpenAI were forced to sign non-disparagement agreements, preventing them from dissenting publicly.
OpenAI claimed it would dedicate 20% of its compute to alignment research. A year later, the heads of the alignment team complained that they never got their promised compute; OpenAI lied and said they meant something different than the thing they obviously meant; a lot of the alignment team quit; then OpenAI gave up on pretending not to have lied and disbanded its alignment team.
Geoffrey Hinton quit Google because, among other reasons, he thought he was too constrained by “self-censorship.”
Altman attempted to fire board member Helen Toner for criticizing the unsafeness of OpenAI’s models.
Open Philanthropy donated $30 million to OpenAI to buy a board seat. This board seat was probably instrumental in getting Altman fired for (essentially) disregarding safety, but the firing didn’t stick and then all the safety-conscious board members got fired.

In each of these cases, working within the company made things definitively worse. The last instance came close to making things better but ultimately made things worse—it made OpenAI $30 million richer without making it safer.

I know of at least one potential counterexample: OpenAI’s RLHF was developed by AI safety people who joined OpenAI to promote safety. But it’s not clear that RLHF helps with x-risk.²⁵

Maybe OpenAI makes it uniquely hard to change the culture from within, and you’d fare better with other companies. I don’t think that’s true because two of the other big AI players, Meta and Google, are larger and have more inertia and therefore should be harder to change. Only Anthropic seems easier to influence.

(But the founding members of Anthropic are concerned with x-risk and they’re still racing to build superintelligent AI as fast as possible while admitting that they have no idea how to make it safe. Influence doesn’t seem helpful: AI safety memes are in the positions of greatest possible influence—namely, the brains of the founders—but still aren’t making Anthropic safe.)

At the risk of over-updating on random authors who I know nothing about: In 1968, James C. Thomson wrote an article called How Could Vietnam Happen? An Autopsy (h/t Jonas V). He wrote that, essentially, dissenting insiders don’t protest because they want to accumulate more influence first. They delay expressing their dissent, always wanting to increase the security of their position, and never get to a point where they actually use their position to do good. Former OpenAI employee Daniel Kokotajlo says he observed this happening at OpenAI.

PhD Comics observes the same phenomenon in academia:

Hat tip to Bryan Caplan, who adds commentary:

The classic story is that tenure protects dissenters. […]

The flaw with the argument is that academic dissenters remain ultra-rare. Far too rare to justify the enormous downsides of the tenure system. And from a bird’s-eye view, the full effect of tenure on dissent is mixed at best. Remember: To get tenure, a dissenter normally has to spend a decade and a half impersonating a normal academic. If you start the process as a non-conformist, the system almost always either weeds you out or wins you over. By the time you get tenure, a creepy chorus of “One of us! One of us!” is in order.

Finally, “with” vs. “against” isn’t necessarily mutually exclusive. There are already many AI safety advocates working inside AI companies. Having a more radical flank on the outside could be a useful complementary strategy. (I’m not confident about this last argument.)

Political diplomacy vs. advocacy

I could say some of the same things about politics that I said about working from inside AI companies: empirically, people get caught in the trap of accumulating social capital and then never actually spend that capital.

Relatedly, some people think you should talk about mundane risks of AI and avoid discussing extinction to not look weird. I have a strong prior toward honesty—telling people what you care about and what your true motivations are, rather than misrepresenting your beliefs (or lying by omission) to make them sound more palatable. And I have a moderate prior against accumulating power—both the good guys and the bad guys want to accumulate power. Honesty is an asymmetric weapon and power is symmetric.

Conflicts that aren’t “man vs. man” but nonetheless require an answer

There are some other areas of debate where funding one side doesn’t necessarily hurt the other side, but I have a finite amount of money and I need to decide which type of thing to fund.

Pause vs. Responsible Scaling Policy (RSP)

Originally I wrote a long diatribe on why I don’t like RSPs and how badly-written AI companies’ RSPs are. But after spending some more time reading pro-RSP commentary, I realized my criticisms didn’t matter because RSP advocates don’t seem to like RSPs much either. The biggest advocates have said things like (paraphrased) “a full pause would be better, but it’s not feasible, so an RSP is a reasonable compromise.” If I understand correctly, they see an RSP as essentially a worse version of a pause but without the downsides. So the real disagreement is about how big the downsides of a pause are.

As far as I can tell, the main cruxes are:

It’s not time to pause yet.
A pause is bad because it would create a hardware overhang.²⁶
If the US government mandates a pause, China will keep developing AI.
A pause has negative EV because it delays the glorious transhumanist future.
It’s likely that we can trust companies to voluntarily implement good RSPs.

I’ve already talked about these and why I believe they’re all incorrect. If you agree with my earlier arguments, then an unconditional pause makes more sense than an RSP.

I would be more optimistic about a government-enforced RSP than a voluntary RSP, but I believe that’s not what people typically mean when they talk about RSPs.

Policy research vs. policy advocacy

Is it better to advocate for regulation/AI policy, or to do policy-relevant research?

I don’t really know about this one. I get the sense that policy advocacy is more important, but I don’t have much of an argument as to why.

The difference between no regulation vs. mediocre regulation is bigger than mediocre vs. good regulation. (credence: 70%)
Policy advocacy is more neglected (although they’re both pretty neglected). (credence: 90%)
It doesn’t seem that hard to write legislation to slow AI development. How much more research do we really need?²⁷ (credence: 50%)

I made some related arguments in slow nuanced regulation vs. fast coarse regulation. If nuanced regulation isn’t worth it, then policy research likely isn’t worth it because there’s not much research to do. (Although you might still do research on things like which advocacy strategies are likely to be most effective.)

Advocacy directed at policy-makers vs. the general public

In favor of focusing on policy-makers:

You get a bigger impact per person convinced, because policy-makers are the ones who actually enact regulations.

In favor of focusing on the public:

A higher proportion of people will be receptive to your message. (And in fact, people are already broadly concerned about AI, so it might be less about convincing and more about motivating.)
Policy-makers’ activities are largely downstream of the public—they want to do what their constituents want.

I don’t have much of an opinion about which is better—I think it depends on the specifics of the organization that’s doing the advocacy. And both sorely need more funding.

Organizations

I’m not qualified to evaluate AI policy organizations²⁸ so I would like to delegate to an expert grantmaker. Unfortunately, none of the existing grantmakers work for me. Most focus on technical research.²⁹ Only Founders Pledge has up-to-date recommendations³⁰ on AI policy, but I didn’t realize they existed until I had spent a lot of time looking into organizations on my own, and it turns out I have some significant disagreements with the Founders Pledge recs. (Three of the seven Founders Pledge recs are my three least favorite orgs among the ones I review below.)³¹

So I did it myself. I made a list of every org I could find that works on AI policy.³² Then I did shallow evaluations of each of them.

Some preamble:

As a rule of thumb, I don’t want to fund anything Open Philanthropy has funded. Not because it means they don’t have room for more funding, but because I believe (credence: 80%) that Open Philanthropy has bad judgment on AI policy (as explained in this comment by Oliver Habryka and reply by Akash—I have similar beliefs, but they explain it better than I do). Open Philanthropy prefers to fund orgs that behave “respectably” and downplay x-risks, and does not want to fund any orgs that work against AI companies.³³ I don’t want to fund any org that’s potentially making it more difficult to communicate to policy-makers about AI x-risk or helping AI companies accelerate capabilities.
In the interest of making my life easier, I stopped investigating an organization as soon as I found a reason not to donate to it, so some of these writeups are missing obvious information.³⁴
A lot of these orgs have similar names. I use full names for any orgs wherever the abbreviation is potentially ambiguous.³⁵
There’s an unfortunate dynamic where I won’t donate to an org if I can’t figure out what it’s doing. But if an org spends a lot of time writing about its activities, that’s time it could be spending on “real” work instead. I have no solution to this.³⁶

Important disclaimers

When describing orgs’ missions and activities, sometimes I quote or paraphrase from their materials without using quotation marks because the text gets messy otherwise. If I do quote without attribution, the source will be one of the links provided in that section.
I only spent 1–2 hours looking into each organization, so I could be substantially wrong in many cases.
It might have been good practice to share (parts of) this document with the reviewed organizations before publishing,³⁷ but I didn’t do that, mainly because it would take a lot of additional work.³⁸ The only exception is if I referenced a private comment made by an individual, I asked permission from that individual before publishing it.
Potential conflict of interest: I have friends at METR and Palisade.
- However, I didn’t know I had a friend who worked at METR until after I had written the section on METR. I’m not good at keeping track of where my friends work.
I’m acquainted with some of the people at CSET, Lightcone, PauseAI, and Sentinel. I might have friends or acquaintances at other orgs as well—like I mentioned, I’m not good at knowing where people work.

AI Policy Institute

AI Policy Institute (mostly) runs public opinion polls on AI risks, some of which are relevant to x-risk. The polls cover some important issues and provide useful information to motivate policy-makers. Some examples:

2.5x more voters support SB 1047 than oppose it. (source)
56% of voters agreed it would be a good thing if AI progress was significantly slowed, vs. 27% disagreed. (source)
Voters’ top priority on AI regulation is preventing catastrophic outcomes. (source)

This sort of work seems good. I’m not sure how big an impact it has on the margin. My intuition is that polls are good, but additional polls have rapidly diminishing returns, so I wouldn’t consider AI Policy Institute a top donation candidate.

I could not find good information about its room for more funding. It did not respond to my inquiry on its funding situation.

AI Safety and Governance Fund

AI Safety and Governance Fund (which, to my knowledge, is a one-man org run by Mikhail Samin) wants to test and spread messages to reduce AI x-risk—see Manifund proposal. It plans to buy ads to test what sorts of messaging are most effective at communicating the arguments for why AI x-risk matters.

I like this project because:

Pushing for x-risk-relevant regulation is the most promising sort of intervention right now. But we don’t have much data on what sorts of messaging are most effective. This project intends to give us that data.
Mikhail Samin, who runs the org, has a good track record of work on AI safety projects (from what I can see).
Mikhail has reasonable plans for what to do with this information once he gets it. (He shared his plans with me privately and asked me not to publish them.)
The project has room for more funding, but it shouldn’t take much money to accomplish its goal.
The project received a speculation grant from the Survival and Flourishing Fund (SFF) and is reasonably likely to get more funding, but (1) it might not; (2) even if it does, I think it’s useful to diversify the funding base; (3) I generally like SFF grants and I don’t mind funging³⁹ SFF dollars.

AI Standards Lab

AI Standards Lab aims to accelerate the writing of AI safety standards (by standards bodies like ISO and NIST) by writing standards that orgs can adapt.

These standards are rarely directly relevant to x-risk. Improving standards on sub-existential risks may make it easier to regulate x-risks, but I would rather see an org work on x-risk more directly.

AI Standards Lab does not appear to be seeking donations.

Campaign for AI Safety

Campaign for AI Safety used to do public marketing and outreach to promote concern for AI x-risk. In early 2024, it got rolled in to Existential Risk Observatory, and the former organizers of the Campaign for AI Safety now volunteer for Existential Risk Observatory.

Campaign for AI Safety still has a donations page, but as far as I can tell, there is no reason to donate to it rather than to Existential Risk Observatory.

Centre for Enabling EA Learning and Research (CEEALAR)

CEEALAR runs the EA Hotel. Recently, it has focused on supporting people who work on AI safety, including technical research and policy.

Something like the EA Hotel could end up accidentally accelerating AI capabilities, but I’m confident that won’t happen because Greg Colbourn, who runs the EA Hotel, is appropriately cautious about AI (he has advocated for a moratorium on AI development).

You could make a case that CEEALAR has a large multiplicative impact by supporting AI safety people. That case seems hard to make well, and in the absence of a strong case, CEEALAR isn’t one of my top candidates.

Center for AI Policy

The Center for AI Policy is a 501(c)(4) nonprofit designed to influence US policy to reduce existential and catastrophic risks from advanced AI (source).

They are serious about x-risk and well-aligned with my position:

Our current focus is building “stop button for AI” capacity in the US government.

Unlike some other orgs, it’s not bogged down by playing politics. For example, it’s willing to call out Sam Altman’s bad behavior; and it focuses on conducting advocacy now, rather than amassing influence that can be used later (I’m generally averse to power-seeking).

The org has proposed model legislation that makes some non-trivial policy proposals (see summary pdf and full text pdf. The legislation would:

require the customers buying $30,000 advanced AI chips to fill out a one-page registration form;
issue permits to the most advanced AI systems based on the quality of their safety testing;
define a reasonable set of emergency powers for the government so that they can intervene and shut down an AI system that’s in the process of going rogue.

This is a breath of fresh air compared to most of the policy proposals I’ve read (none of which I’ve discussed yet, because I’m writing this list in alphabetical order). Most proposals say things like:

make the regulation be good instead of bad;
simultaneously promote innovation and safety (there is no such thing as a tradeoff);
sternly tell AI companies that they need to not be unsafe, or else we will be very upset.

I’m paraphrasing for humor⁴⁰, but I don’t think I’m exaggerating—I’ve read proposals from AI policy orgs that were equivalent to these, but phrased more opaquely. (Like nobody explicitly said “we refuse to acknowledge the existence of tradeoffs”, but they did, in fact, refuse to acknowledge the existence of tradeoffs.)

Center for AI Policy has a target budget of $1.6 million for 2025 (source), and its current funding falls considerably short of this goal, so it can make good use of additional money.

Center for AI Safety

Center for AI Safety does safety research and advocates for safety standards. It has a good track record so far:

It drafted the original version of SB 1047.
Its Statement on AI Risk got signatures from major figures in AI and helped bring AI x-risk into the Overton window.
It’s done some other work (e.g., writing an AI safety textbook; buying compute for safety researchers) that I like but I don’t think is as impactful. The given examples are about supporting alignment research, and as I’ve said, I’m not as bullish on alignment research.
The Center for AI Policy Action Fund does lobbying, which might be good, but I can’t find much public information about what it lobbies for. It did support SB 1047, which is good.
It’s led by Dan Hendrycks. I’ve read some of his writings and I get the general sense that he’s competent.

The Center for AI Safety has some work that I’m very optimistic about (most notably the Statement on AI Risk), but I’m only weakly to moderately optimistic about most of its activities.

It has received $9 million from Open Philanthropy (1, 2) and just under $1 million from the Survival and Flourishing Fund.

I have a good impression of the Center for AI Safety, but it’s not one of my top candidates because (1) it’s already well-funded and (2) it has done some things I really like, but those are diluted by a lot of things I only moderately like.

Center for Human-Compatible AI

Center for Human-Compatible AI does mostly technical research, and some advocacy. To my knowledge, the advocacy essentially consists of Stuart Russell using his influential position to advocate for regulation. While that’s good, I don’t think Stuart Russell is personally funding-constrained, so I don’t think marginal donations to the org will help advocacy efforts.

Center for Long-Term Resilience

The Center for Long-Term Resilience is a think tank focused on reducing “extreme risks”, which includes x-risks but also other things. It talks to policy-makers and writes reports. I’ll focus on its reports because those are easier to assess.

About half of the org’s work relates to AI risk. Some of the AI publications are relevant to x-risk (1); most are marginally relevant (2, 3) or not relevant (4).

I skimmed a few of its reports. Here I will give commentary on two of its reports, starting with the one I liked better.

I’m reluctant to criticize orgs that I think have good intentions, but I think it’s more important to accurately convey my true beliefs. And my true belief is that these reports are not good (credence: 75%).

Transforming risk governance at frontier AI companies was my favorite report that I saw from the Center for Long-Term Resilience.

This was the only one of the org’s recent reports that looked meaningfully relevant to x-risk.
The report correctly identifies some inadequacies with AI companies’ risk processes. It proposes some high-level changes that I expect would have a positive impact.
That said, I don’t think the changes would have a big impact. The proposal would make more sense for dealing with typical risks that most industries see, but it’s not (remotely) sufficient to prepare for extinction risks. Indeed, the report proposes using “best practice” risk management. Best practice means standard, which means insufficient for x-risk. (And best practice means well-established, which means well-known, which means the marginal value of proposing it is small.)
The report implies that we should rely on voluntary compliance from AI companies. It proposes that companies should use external auditors, but not that those auditors should have any real power.
An illustrative quote from the Risk Oversight section: “Although they should not make the final decisions, the specialist risk and assurance [advisors] should play a ‘challenger’ role, pressure testing the business’s plans and decisions to ensure they are risk-informed.” I disagree. Risk advisors should have veto power. The CEO should not have unilateral authority to deploy dangerous models.
The report has little in the way of concrete recommendations. Most of the recommendations are non-actionable—for example, “build consensus within business and civil society about the importance of more holistic risk management”. Ok, how specifically does one do that?
Contrast this with the model legislation from the Center for AI Policy, where the one-page executive summary made proposals that were easier to understand, more concrete, and more relevant to x-risk.

Another example of a report, which I liked less: Response to ‘Establishing a pro-innovation approach to regulating AI’ (a reply to a request for proposals by the UK Office of AI).

The report makes four high-level proposals, all of which I dislike:
1. “Promoting coherence and reducing inefficiencies across the regulatory regime” – Nobody needs to be told to reduce inefficiency. The only reason why any process is inefficient is because people don’t know how to make it more efficient. How exactly am I supposed to reduce inefficiency? (This quote comes from the executive summary, where I can forgive some degree of vagueness, but the full report does not provide concrete details.)
2. “Ensuring existing regulators have sufficient expertise and capacity” – Again, this is an applause light, not a real suggestion. No one thinks regulators should have insufficient expertise or capacity.
3. “Ensuring that regulatory gaps can be identified and addressed” – More of the same.
4. “Being sufficiently adaptive to advances in AI capabilities” – More of the same.
The report suggests regulating all AI with a single body rather than diffusely. I like this idea—if a regulatory body is going to prevent x-risk, it probably needs to have broad authority. (Except the report also says “we do not necessarily think [the regulator] needs to be a single body”, which seems to contradict its earlier recommendation.)
The report says “It will become increasingly important to distribute responsibility across the entire supply chain of AI development”. I think that’s a good idea if it means restricting sales and exports of compute hardware. But it doesn’t say that explicitly (in fact it provides no further detail at all), and I don’t think policy-makers will interpret it that way.
“Recognise that some form of regulation may be needed for general-purpose systems such as foundation models in future.” I would have written this as: “Recognize that strict regulation for general-purpose systems is urgently needed.” Stop downplaying the severity of the situation.
If I were writing this report, I would have included evidence/reasoning on why AI risk (x-risk and catastrophic risk) is a major concern, and what this implies about how to regulate it. The report doesn’t include any arguments that could change readers’ minds.
In conclusion, this report is mostly vacuous. It contains some non-vacuous proposals in the full text (not represented in the executive summary), but the non-vacuous proposals aren’t particularly concrete and aren’t particularly useful for reducing x-risk.

An alternative interpretation is that the Center for Long-Term Resilience wants to build influence by writing long and serious-looking reports that nobody could reasonably disagree with. As I touched on previously, I’m not optimistic about this strategy. I disapprove of deceptive tactics, and I think it’s a bad idea even on naive consequentialist grounds (i.e., it’s not going to work as well as writing actionable reports would). And—perhaps more importantly—if the org’s reports are low quality, then I can’t trust that it does a good job when working with policy-makers.

Center for Security and Emerging Technology (CSET)

Center for Security and Emerging Technology does work on AI policy along with various other topics. It has received $105 million from Open Philanthropy.

I wouldn’t donate to CSET because it has so much funding already, but I took a brief look at its publications.

The research appears mostly tangential or unrelated to x-risk, instead covering subjects like cybersecurity, deceptive/undesirable LLM output, and how the US Department of Defense can use AI to bolster its military power—this last report seems harmful on balance. Some of its reports (such as Enabling Principles for AI Governance) have the previously-discussed problem of being mostly vacuous/non-actionable.

CSET also works to put researchers into positions where they can directly influence policy (source).⁴¹ Allegedly, CSET has considerable political influence, but I haven’t identified any visible benefits from that influence (contrast with the Center for AI Safety, which wrote SB 1047). The most legible result I can find is that CSET has collaborated with the Department of Defense; without knowing the details, my prior is that collaborating with DOD is net negative. I would prefer the DOD to be less effective, not more. (Maybe CSET is convincing the DOD not to build military AI but I doubt it; CSET’s reports suggest the opposite.)

CSET has the same issue as the Center for Long-Term Resilience: if your public outputs are low-quality (or even net harmful), then why should I expect your behind-the-scenes work to be any better?

Centre for Long-Term Policy

Centre for Long-Term Policy operates in Norway and focuses on influencing Norwegian policy on x-risk, longtermism, and global health.

I didn’t look into it much because I think Norwegian AI policy is unlikely to matter—superintelligent AI will almost certainly not be developed in Norway, so Norwegian regulation has limited ability to constrain AI development.

From skimming its publications, they mostly cover subjects other than AI x-risk policy.

The Centre for Long-Term Policy received an undisclosed amount of funding from Open Philanthropy in 2024.

Centre for the Governance of AI

Centre for the Governance of AI does alignment research and policy research. It appears to focus primarily on the former, which, as I’ve discussed, I’m not as optimistic about. (And I don’t like policy research as much as policy advocacy.)

Its policy research seems mostly unrelated to x-risk, for example it has multiple reports on AI-driven unemployment (1, 2).

My favorite of its published reports is Lessons from the Development of the Atomic Bomb. It’s written by Toby Ord, who doesn’t work there.

Centre for the Governance of AI has received $6 million from Open Philanthropy.

The org appears reasonably well-funded. I don’t have major complaints about its work, but (1) the work does not look particularly strong and (2) it doesn’t cover the focus areas that I’m most optimistic about.

CivAI

CivAI raises awareness about AI dangers by building interactive software to demonstrate AI capabilities, for example AI-powered cybersecurity threats.

This org is new which makes it difficult to evaluate. It appears to have the same theory of change as Palisade Research (which I review below), but I like Palisade better, for three reasons:

None of CivAI’s work so far appears relevant to x-risk. For example, its most recent demo focuses on generating fake images for deceptive purposes.
I think Palisade’s methods for demonstrating capabilities are more likely to get attention (credence: 65%).
I’m more confident in Palisade’s ability to communicate with policy-makers.

CivAI does not appear to be seeking donations. There is no option to donate through the website.

Control AI

Control AI runs advocacy campaigns on AI risk.

Its current campaign proposes slowing AI development such that no one develops superintelligence for at least the next 20 years, then using this time to establish a robust system for AI oversight. The campaign includes a non-vacuous proposal for the organizational structure of a regulatory body.

Control AI has a paper on AI policy that appears reasonable:

It acknowledges that voluntary commitments from AI companies are insufficient.
It proposes establishing international regulatory body that (1) imposes a global cap on computing power used to train an AI system and (2) mandates safety evaluations.
It proposes that regulators should have the authority to halt model deployment on a model that they deem excessively dangerous.

The campaign’s proposal is similar. It lays out the most concrete plan I’ve seen for how to get to a place where we can solve AI alignment.

I listened to a podcast with Andrea Miotti, co-founder of Control AI. He mostly covered standard arguments for caring about AI x-risk, but he also made some insightful comments that changed my thinking a bit.⁴²

I like the concept of Control AI’s latest campaign, but I don’t know how much impact it will have.⁴³

Control AI’s past campaigns (example) have received media coverage (example) and their policy objectives have been achieved, although it’s not clear how much of a causal role Control AI played in achieving those objectives, or what Control AI actually did. Control AI clearly deserves some credit, or else news outlets wouldn’t cite it.

Control AI might be as impactful as other advocacy orgs that I like, but I have more uncertainty about it, so it’s not a top candidate. It would be fairly easy to change my mind about this.

I couldn’t find any information about Control AI’s funding situation, and I didn’t inquire because it wasn’t one of my top candidates.

Existential Risk Observatory

Existential Risk Observatory writes media articles on AI x-risk, does policy research, and publishes policy proposals (see pdf with a summary of proposals).

It appears to be having some success bringing public attention to x-risk via mainstream media, including advocating for a pause in TIME (jointly with Joep Meindertsma of PauseAI).
Its policy proposals are serious: it proposes implementing an AI pause, tracking frontier AI hardware, and explicitly recognizing extinction risk in regulations.
The research mainly focuses on public opinion, for example opinions on AI capabilities/danger (pdf) and message testing on an AI moratorium (pdf).

Existential Risk Observatory is small and funding-constrained, so I expect that donations would be impactful.

My primary concern is that it operates in the Netherlands. Dutch policy is unlikely to have much influence on x-risk—the United States is the most important country by far, followed by China. And a Dutch organization likely has little influence on United States policy. Existential Risk Observatory can still influence public opinion in America (for example via its TIME article), but I expect a US-headquartered org to have a greater impact.

Future of Life Institute (FLI)

FLI has done some good advocacy work like the 6-month pause letter (which probably reduced x-risk). It also has a $400 million endowment, so I don’t think it needs any donations from me.

Future Society

The Future Society seeks to align AI through better governance. I reviewed some of its work, and it looks almost entirely irrelevant to x-risk.

Of The Future Society’s recent publications, the most concrete is “List of Potential Clauses to Govern the Development of General Purpose AI Systems” (pdf). Some notes on this report:

The Future Society collected recommendations from industry staff, independent experts, and engineers from frontier labs. Engineers from frontier labs should not be trusted to produce recommendations, any more than petroleum engineers should be trusted to set climate change policy.
The proposals for mitigating harmful behavior are mostly vacuous and in some cases harmful. They largely amount to: keep building dangerous AI, but do a good job of making it safe.
“Use the most state-of-the-art editing techniques to erase capabilities and knowledge that are mostly useful for misuse.” That’s not going to work. (Palisade Research has demonstrated that it’s easy to remove safeguards from LLMs.)
“Use state-of-the-art methods and tools for ensuring safety and trustworthiness of models, such as mechanistic interpretability.” This sentence makes me think the authors don’t have a good understanding of AI safety. The state of the art in mechanistic interpretability is nowhere close to being able to ensure the trustworthiness of models. We still have virtually no idea what’s going on inside large neural networks.
The report proposes using the same industry-standard risk management model that the Center for Long-Term Resilience proposed. The same criticisms apply—this model is obvious enough that you don’t need to propose it, and severely insufficient for mitigating extinction risks.
The report proposes “air gapping & sandboxing, no internet access” for powerful models. I feel like I shouldn’t need to explain why that won’t work.

Another report (pdf) submitted in response to the EU AI Act discussed seven challenges of “general-purpose AI”. The second challenge is “generalization and capability risks, i.e. capability risks, societal risks and extinction risks”. There is no further discussion of extinction risk, and this is the only place that the word “extinction” appears in any of The Future Society’s materials. (The word “existential” appears a few times, but existential risks are not discussed.)

Horizon Institute for Public Service

Horizon Institute for Public Service runs a fellowship where it places people into positions in governments and think tanks. It claims to be reasonably successful. (I do not have much of an opinion as to how much credit Horizon Institute deserves for its fellows’ accomplishments.)

Horizon Institute has received an undisclosed amount of funding from Open Philanthropy (along with some other big foundations).

Do Horizon fellows care about x-risk, and does their work reduce x-risk in expectation? Politico alleges that the Horizon Institute is a clandestine plot to get governments to care more about x-risk. I’m not a fan of clandestine plots, but that aside, should I expect Horizon fellows to reduce x-risk?

Most of their work is not legible, so I’m skeptical by default. Caring about x-risk is not enough to make me trust you. Some people take totally the wrong lessons from concerns about x-risk (especially AI risk) and end up increasing it instead. Case in point: OpenAI, DeepMind, and Anthropic all had founders who cared about AI x-risk, and two of those (OpenAI + Anthropic) were founded with the explicit mission of preventing extinction. And yet OpenAI is probably the #1 worst thing that has ever happened in terms of increasing x-risk, and DeepMind and Anthropic aren’t much better.

I reviewed all the highlighted accomplishments of fellows that looked relevant to AI:

AI risk management standards for NIST. Only marginally relevant to x-risk, but not bad.
An article on how we shouldn’t worry about x-risk (!!).
Auditing tools for AI equity. Unrelated to x-risk.
Detecting AI fingerprints. Marginally related to x-risk.
Autonomous cyber defense. Increasing the capabilities of cybersecurity AI is plausibly net negative.⁴⁴
An article on the EU AI Act. Non-vacuous and discusses AI risk (not exactly x-risk, but close). Vaguely hints at slowing AI development.

In my judgment after taking a brief look, 3/6 highlighted writings were perhaps marginally useful for x-risk, 1/6 was irrelevant, and 2/6 were likely harmful. None were clearly useful.

Zvi Mowshowitz wrote:

In my model, one should be deeply skeptical whenever the answer to ‘what would do the most good?’ is ‘get people like me more money and/or access to power.’

I agree, but even beyond that, the Horizon fellows don’t seem to be “people like me”. They include people who are arguing against caring about x-risk.

I believe the world would be better off if Horizon Institute did not exist (credence: 55%).

And if I’m wrong about that, it still looks like Horizon fellows don’t do much work related to x-risk, so the expected value of Horizon Institute is low.

Institute for AI Policy and Strategy

Institute for AI Policy and Strategy does policy research, focused on US AI regulations, compute governance, lab governance, and international governance with China.

I’m more optimistic about advocacy than policy research, so this org is not one of my top candidates. That said, I like it better than most AI policy research orgs. Some observations from briefly reading some of its research:

The proposals are non-vacuous and moderately concrete. For example, from its recommendations to the UK government: “Investing £50m in ‘pull mechanisms’ (pay-outs contingent on achieving specific technological goals, such as prizes, AMCs, and milestone payments).” I don’t know how much that helps with x-risk, but it’s concrete.
Almost all of its work focuses on sub-extinction risks. Some of this looks potentially useful for x-risk, for example establishing reporting requirements, or recognizing the risks associated with exporting GPUs.
One report fairly criticizes some shortcomings of Anthropic’s Responsible Scaling Policy.

Institute for AI Policy and Strategy has received just under $4 million from Open Philanthropy (1, 2), and is seeking additional funding.

Lightcone Infrastructure

Lightcone runs LessWrong and an office that Lightcone calls “Bell Labs for longtermism”.

Lightcone has a detailed case for impact on Manifund. In short, Lightcone maintains LessWrong, and LessWrong is upstream of a large quantity of AI safety work.

I believe Lightcone has high expected value and it can make good use of marginal donations.

By maintaining LessWrong, Lightcone somewhat improves many AI safety efforts (plus efforts on other beneficial projects that don’t relate to AI safety). If I were very uncertain about what sort of work was best, I might donate to Lightcone as a way to provide diffuse benefits across many areas. But since I believe (a specific sort of) policy work has much higher EV than AI safety research, I believe it makes more sense to fund that policy work directly.

An illustration with some made-up numbers: Suppose that

There are 10 categories of AI safety work.
Lightcone makes each of them 20% better.
The average AI safety work produces 1 utility point.
Well-directed AI policy produces 5 utility points.

Then a donation to Lightcone is worth 2 utility points, and my favorite AI policy orgs are worth 5 points. So a donation to Lightcone is better than the average AI safety org, but not as good as good policy orgs.

Machine Intelligence Research Institute (MIRI)

MIRI used to do exclusively technical research. In 2024, it pivoted to focus on policy advocacy—specifically, advocating for shutting down frontier AI development. MIRI changed its mind at around the same time I changed my mind.

Some observations:

MIRI gets considerable credit for being the first to recognize the AI alignment problem.
I have a high opinion of the general competence of MIRI employees.
Historically, I have agreed with MIRI’s criticisms of most technical alignment approaches, which suggests they have good reasoning processes. (With the caveat that I don’t really understand technical alignment research.)
Eliezer Yudkowsky’s TIME article publicly argued for AI pause and brought some attention to the issue (both positive and negative). My vibe sense says the article was valuable but who knows.
Eliezer personally has a strong track record of influencing (some subset of) people with the LessWrong sequences and Harry Potter and the Methods of Rationality.
I know that MIRI is serious about existential risk and isn’t going to compromise its values.
Eliezer believes animals are not moral patients, which is kind of insane but probably not directly relevant. (Rule thinkers in, not out.)
MIRI (or at least Eliezer) says P(doom) > 95%.⁴⁵ Some people say this is crazy high and it makes MIRI want to do dumb stuff like shutting down AI. I do think 95% is too high but I think most people are kind of crazy about probability—they treat probabilities less than 50% as essentially 0%. Like if your P(doom) is 40%, you should be doing the same thing that MIRI is doing.⁴⁶ You should not be trying to develop AI as fast as possible while funding a little safety research on the side.

MIRI’s new communications strategy has produced few results so far. We know that MIRI is working on a new website that explains the case for x-risk; a book; and an online reference. It remains to be seen how useful these will be. They don’t seem like obviously good ideas to me,⁴⁷ but I expect MIRI will correct course if a strategy isn’t working.

Until recently, MIRI was not seeking funding because it received some large cryptocurrency donations in 2021ish. Now it’s started fundraising again to pay for its new policy work.

I consider MIRI a top candidate. It only recently pivoted to advocacy so there’s not much to retrospectively evaluate, but I expect its work to be impactful.

Manifund

Manifund does not do anything directly related to AI policy. It’s a fundraising platform. But I’m including it in this list because I’m impressed by how it’s changed the funding landscape.

Many orgs have written fundraising pitches on Manifund. And for whatever reason, some of these pitches are way higher quality than what I’m used to. I’m not sure why—maybe Manifund’s prompt questions draw out good answers.

For example, originally I was skeptical that donations to Lightcone Infrastructure could be competitive with top charities, but its Manifund page changed my mind. I donated $200 just as a reward for the excellent writeup.

Many of the orgs on my list (especially the smaller ones) wrote detailed pitches on Manifund that helped me decide where to donate. Manifund deserves part of the credit for that.

Manifund is free to use, but it sometimes asks large donors to give a percentage of their donations to cover its operating costs. Manifund didn’t ask me to do that, so I didn’t.

Model Evaluation and Threat Research (METR)

METR evaluates large AI models to look for potentially dangerous capabilities. Its most obvious theory of change—where it finds a scary result and then the AI company pauses development—mainly depends on (1) AI companies giving access to METR (which they often don’t) and (2) AI companies ceasing model development when METR establishes harmful capabilities (which they probably won’t—if there’s any ambiguity, they will likely choose the interpretation that lets them keep making more money).

There’s an indirect but more promising theory of change where METR demonstrates a template for capability evaluation which policy-makers then rely on to impose safety regulations. To that end, METR has engaged with NIST’s AI risk management framework (pdf). This sounds potentially promising but it’s not where I would put money on the margin because:

I don’t think we should wait to figure out a solid evaluation framework before writing regulations.
Evaluations are helpful if we want to conditionally pause AI in the future, but not relevant if we want to unconditionally pause AI right now, and I believe we should do the latter.

Palisade Research

Palisade builds demonstrations of the offensive capabilities of AI systems, with the goal of illustrating risks to policy-makers.

Some thoughts:

Demonstrating capabilities is probably a useful persuasion strategy.
Palisade has done some good work, like removing safety fine-tuning from Meta’s LLM.
I know some of the Palisade employees and I believe they’re competent.
Historically, Palisade has focused on building out tech demos. I’m not sure how useful this is for x-risk, since you can’t demonstrate existentially threatening capabilities until it’s too late. Hopefully, Palisade’s audience can extrapolate from the demos to see that extinction is a serious concern.
Soon, Palisade plans to shift from primarily building demos to primarily using those demos to persuade policy-makers.
Palisade has a smallish team and has reasonable room to expand.

Palisade has not been actively fundraising, but I believe it can put funding to good use—it has limited runway and wants to hire more people.

I think the work on building tech demos has rapidly diminishing utility, but Palisade is hiring for more policy-oriented roles, so I believe that’s mostly where marginal funding will go.

PauseAI Global

(PauseAI Global and PauseAI US share the same mission and used to be part of the same org, so most of my comments on PauseAI Global also apply to PauseAI US.)

From Manifund:

PauseAI is a grassroots community of volunteers which aim to inform the public and politicians about the risks from superhuman AI and urge them to work towards an international treaty that prevents the most dangerous AI systems from being developed.

PauseAI is largely organised through local communities which take actions to spread awareness such as letter writing workshops, peaceful protests, flyering and giving presentations.

Historically, I’ve been skeptical of public protests. I think people mainly protest because it’s fun and it makes them feel like they’re contributing, not because it actually helps.⁴⁸ But PauseAI has been appropriately thoughtful (1, 2) about whether and when protests work, and it makes a reasonable case that protesting can be effective.

(See also the Protest Outcomes report by Social Change Lab. The evidence for the effectiveness of protests is a bit stronger than I expected.⁴⁹)

I’m skeptical of the evidence because I don’t trust sociology research (it has approximately the worst replication record of any field). But I like PauseAI because:

Approximately zero percent of AI dollars go to AI safety, but approximately zero percent of AI safety dollars go to public advocacy.
Polls suggest that there’s widespread public support for pausing AI, and PauseAI has a good shot at converting that public support into policy change.
The people running PauseAI seem to have a good idea of what they’re doing, and it’s apparent that they are seriously concerned about existential risk (for most AI policy orgs, I can’t tell whether they care).
My impression is that the PauseAI founders went through a similar reasoning process as I did, and concluded that public advocacy was the most promising approach.
I’ve listened to interviews and read articles by leaders from a number of AI policy orgs, and I like the vibes of the PauseAI leaders the best. Many people working in AI safety have missing moods, but the PauseAI people do not. I don’t put too much weight on vibes, but they still get nonzero weight.

Broadly speaking, I’m a little more optimistic about advocacy toward policy-makers than advocacy toward the public, simply because it’s more targeted. But PauseAI is still a top candidate because its approach is exceptionally neglected.

PauseAI Global has no full-time employees; it focuses on supporting volunteers who run protests.

PauseAI US

PauseAI US organizes protests to advocate for pausing AI development.

Unlike PauseAI Global which has no full-time employees, PauseAI US has a small full-time staff who run protests and political lobbying efforts. I like PauseAI US a little better than PauseAI Global because most major AI companies are headquartered in the US, so I expect a US-based org to have more potential for impact.

PauseAI US also does grassroots lobbying (e.g., organizing volunteers to write letters to Congress) and direct lobbying (talking to policy-makers).

Grassroots lobbying makes sense as a neglected intervention. Direct lobbying isn’t quite as neglected but it’s still one of my favorite interventions. PauseAI US only has a single lobbyist right now, Felix De Simone. He’s more junior than the lobbyists at some other policy orgs, but based on what I know of his background, I expect him to do a competent job.⁵⁰ PauseAI US is performing well on obvious surface-level metrics like “number of meetings with Congressional offices per person per month”.

Sentinel rapid emergency response team

Sentinel monitors world events for potential precursors to catastrophes. It publishes a weekly newsletter with events of interest (such as “Iran launched a ballistic missile attack on Israel” or “Two people in California have been infected with bird flu”).

Sentinel’s mission is to alert relevant parties so that looming catastrophes can be averted before they happen.

You can read more information on Sentinel’s Manifund page (short) and fundraising memo (long).

Some thoughts:

I believe almost nobody would do a good job of running Sentinel because it’s hard to identify early warning signals of catastrophes. But Sentinel is run by members of Samotevsky Forecasting, who I expect to be unusually good at this.
The value of Sentinel depends on who’s paying attention to its reports. I don’t know who’s paying attention to its reports.
Sentinel isn’t immediately relevant to AI policy, but it could be extremely valuable in certain situations. Namely, it could provide early warning if AI x-risk rapidly increases due to some series of events.
AI x-risk aside, I still think Sentinel has high EV because it potentially significantly reduces catastrophic risk for a small budget. Without having investigated those cause areas, Sentinel is tentatively my #1 donation pick for reducing nuclear and biological x-risks.

Sentinel currently has four team members working part-time. With additional funding, its members could work full-time and it could hire more members and therefore do more comprehensive monitoring.

Simon Institute for Longterm Governance

Simon Institute supports policies to improve coordination, reduce global catastrophic risks, and embed consideration for future generations. It specifically focuses on influencing United Nations policy. (See the Year One Update from 2022.)

Most of the org’s work appears not very relevant to x-risk. For example:

It co-hosted a workshop on how AI companies might redistribute their profits (via the Windfall Trust).
It co-developed a table-top exercise on pandemic preparedness.
It proposed some minor changes to a UN document on AI governance.

This work seems reasonably good, but not as high-impact as work that directly targets x-risk reduction.

Stop AI

Like PauseAI, Stop AI protests the development of superintelligent AI. Unlike PauseAI, Stop AI uses disruptive tactics like blocking entrances to OpenAI offices and blocking traffic.

This is a more high-variance strategy. I find it plausible that Stop AI’s tactics are especially effective, but also likely that its tactics will backfire and decrease public support. So in the absence of some degree of supporting evidence, I’m inclined not to support Stop AI.

Stop AI’s proposal seems overreaching (it wants to permanently ban AGI development) and it makes weak arguments.

From listening to an interview, I get the impression that the Stop AI founders aren’t appropriately outcome-oriented and don’t have a well-formulated theory of change. In the interview, they would offer reasoning for why they took a particular action and then when the interviewer would point out how that justification doesn’t explain their behavior, they would switch to a different explanation. An example (paraphrased):

Stop AI: We blocked an entrance to OpenAI’s office to make it harder for employees to build AGI. We feel that this is necessary to stop OpenAI from killing everyone.

Interviewer: Then why did you block traffic, since that affects innocent bystanders, not the people building AGI?

Stop AI: We need to block traffic to raise awareness.

This pattern occurred a few times. They have reasonable concerns about the dangers of AI, but they don’t seem to have a good justification for why disruptive protests are the best way to handle those concerns.

(I can see an argument for blocking entrances to AI company offices, but I think the argument for blocking traffic is much weaker.)

In short, Stop AI is spiritually similar to PauseAI but with worse reasoning, worse public materials, and worse tactics.

Where I’m donating

My top candidates:

AI Safety and Governance Fund
PauseAI US
Center for AI Policy
Palisade
MIRI

A classification of every other org I reviewed:⁵¹

Good but not funding-constrained: Center for AI Safety, Future of Life Institute

Would fund if I had more money: Control AI, Existential Risk Observatory, Lightcone Infrastructure, PauseAI Global, Sentinel

Would fund if I had a lot more money, but might fund orgs in other cause areas first:⁵² AI Policy Institute, CEEALAR, Center for Human-Compatible AI, Manifund

Might fund if I had a lot more money: AI Standards Lab, Centre for the Governance of AI, Centre for Long-Term Policy, CivAI, Institute for AI Policy and Strategy, METR, Simon Institute for Longterm Governance

Would not fund: Center for Long-Term Resilience, Center for Security and Emerging Technology, Future Society, Horizon Institute for Public Service, Stop AI

Prioritization within my top five

Here’s why I ordered my top five the way I did.

#1: AI Safety and Governance Fund

This is my top choice because:

It could greatly improve the value of future communications efforts.
It’s cheap, which means it’s cost-effective.

It would drop down the list quickly if it received more funding, but right now it’s #1.

#2: PauseAI US

I would expect advocacy toward policy-makers to be more impactful than public advocacy if they had similar levels of funding (credence: 60%). But pause protests are extremely neglected, so I believe they’re the most promising strategy on the margin. And PauseAI US is my favorite org doing protests because it operates in the United States and it appears appropriately competent and thoughtful.

Protests are especially unpopular among institutional funders, which makes them more promising for individual donors like me.

#3: Center for AI Policy

This is one of only three orgs (along with Palisade and PauseAI US) that meet four criteria:

works to persuade policy-makers
focuses on AI x-risk over other less-important AI safety concerns⁵³
focuses on United States policy⁵⁴
is funding-constrained⁵⁵

I’m nearly indifferent between Center for AI Policy and Palisade. I slightly prefer the former because (1) its employees have more experience in politics and (2) its mission/messaging seems less palatable to institutional funders so I expect it to have a harder time raising money.

#4: Palisade

Palisade meets the same four criteria as Center for AI Policy. As a little twist, Palisade also builds tech demos with the purpose of demonstrating the dangers of AI to policy-makers. Those demos might help or they might not be worth the effort—both seem equally likely to me—so this twist doesn’t change my expectation of Palisade’s cost-effectiveness. I only slightly favor Center for AI Policy for the two reasons mentioned previously.

I personally know people at Palisade, which I think biases me in its favor, and I might put Palisade at #3 if I wasn’t putting in mental effort to resist that bias.

#5: MIRI

MIRI plans to target a general audience, ~~not policy-makers~~ (update 2024-11-20: see correction below). That means they can reach more people but it’s also lower leverage. My guess is that targeting a general audience is worse on balance.

I put PauseAI US higher than the two lobbying orgs because it has such a small budget. Like PauseAI US, MIRI’s strategies are also neglected, but considerably less so.⁵⁶ I expect policy-maker outreach to be more effective than MIRI’s approach (credence: 60%).

Lest I give the wrong impression, MIRI is still my #5 candidate out of 28 charities. I put it in the top five because I have a high opinion of MIRI leadership—I expect them to have reasonable prioritization and effective execution.

CORRECTION: MIRI’s technical governance team does research to inform policy, and MIRI has spoken to policy-makers in the US government. This bumps up my evaluation of the org but I’m keeping it at #5 because working with policy-makers is only one part of MIRI’s overall activities.

Where I’m donating (this is the section in which I actually say where I’m donating)

I agree with the standard argument that small donors should give all their money to their #1 favorite charity. That’s how I’ve done it in the past, but this year I’m splitting my donations a little bit. I plan on donating:

$5,000 to AI Safety and Governance Fund
$5,000 to PauseAI Global⁵⁷
$30,000 to PauseAI US

Here’s why I’m splitting my donations:

AI Safety and Governance Fund is small, and I don’t want to represent too big a portion of its budget.
I donated to PauseAI Global before writing this post, and my prioritization changed somewhat after writing it.⁵⁷
That leaves PauseAI US as my top candidate, so the rest of my donations will go there.

I already donated $5,000 to PauseAI Global, but I haven’t made the other donations yet, so commenters have a chance to convince me to change my mind.

If you wish to persuade me privately (or otherwise discuss in private), you can email me at donations@mdickens.me or message me on the EA Forum.

Changelog

2024-11-20: Provide a correction for some incorrect information about MIRI.
2025-04-25: Change some bolded sections into headings so they can be linked to.

Notes

At least the highly effective kinds of animal welfare. Things like animal shelters get a lot of funding but they’re not highly effective. ↩
I believe humans are the most important species, but only because we will shape the future, not because we matter innately more.

To be precise, I believe any individual human’s welfare probably innately matters more than an individual animal of any other species. But there are so many more animals than humans that animals matter much more in aggregate. ↩
I did change my mind in one relevant way—I used to think AI policy advocacy was very unlikely to work, and now I think it has a reasonable chance of working. More on this later. ↩
So maybe I was being reasonable before and now I’m over-weighting AI risk because I’m worried about getting killed by AI? If I’m being irrational right now, how would I know? ↩
I studied computer science in university and I took three AI classes: Intro to AI, Intro to Machine Learning, and Convolutional Neural Networks for Natural Language Processing. My grades were below average but not terrible. ↩
Some alignment researchers think we’re not that far away from solving AI alignment. I won’t go into detail because I don’t think I can do a great job of explaining my views. An informed alignment researcher could probably write >100,000 words detailing the progress in various subfields and predicting future progress to predict how close we are to solving alignment—something like this or this, but with more analysis and prediction—and some other informed alignment researcher could do the same thing and come up with a totally different answer.

Feel free to change the numbers on my quantitative model and write a comment about what answer you got. ↩
Like if “just use reinforcement learning to teach the AI to be ethical” turns out to work. (Which I doubt, but some people seem to think it will work so idk.) ↩
I don’t remember exactly what I used to believe. Maybe something like, “AI policy advocacy could be a good idea someday once AI looks more imminent and there’s more political will, but it’s not a good idea right now because people will think you’re crazy.” ↩
“Replace Gavin Newsom with a governor with more integrity” might be an effective intervention, but probably not cost-effective—there’s already too much money in state elections. ↩
I realized midway through writing this post that I had made this major cause prioritization decision without even making up some numbers and slapping them into a Monte Carlo simulation, which was very out of character for me. ↩
More accurately, I believe we live in one of two worlds:
1. Prosaic alignment works, in which case we will probably not have any lack of funding for AI alignment, and my marginal donation has a small chance of making a difference.
2. Alignment is hard, in which case we will probably not have nearly enough funding (assuming no regulation), and my marginal donation has a small chance of making a difference.
And really there’s a continuum between those, so there’s a small chance that AI alignment is at just the right level of difficulty for marginal donations to make a difference. ↩
There’s a lot of spending on so-called safety research that’s really fake safetywashing research. I wouldn’t count that as part of my estimate. ↩
For example, with the default inputs, the cost to solve alignment is set to a log-normal distribution with 25th/75th percentiles at $1 billion and $1 trillion. If you tighten the distribution to $10 billion to $1 trillion, the marginal value of spending on alignment research drops to ~0. ↩
I’ve met individual researchers who said this and I don’t think they were lying, but I think their beliefs were motivated by a desire to build SOTA models because SOTA models are cool. ↩
I’m slightly concerned that I shouldn’t pick on Anthropic because they’re the least unethical of the big AI companies (as far as I can tell). But I think when you’re building technology that endangers the lives of every sentient being who lives and who ever will live, you should be held to an extremely high standard of honesty and communication, and Anthropic falls embarrassingly, horrifyingly short of that standard. As they say, reality does not grade on a curve. ↩
I would actually go further than that—I think the best types of alignment research don’t require SOTA models. But that’s more debatable, and it’s not required for my argument. ↩
Suppose conservatively that the lightcone will be usable for another billion years, and that we need to delay superintelligent AI by 100 years to make it safe. The volume of the lightcone is proportional to time cubed. Therefore, assuming a constant rate of expansion, delaying 100 years means we can only access 99.99997% of the lightcone instead of 100%. Even at an incredibly optimistic P(doom) (say, 0.001%), accelerating AI isn’t worth it on a naive longtermist view. ↩
I’m making an ad hominem argument on purpose. Altman’s arguments seem bad to me, maybe I’m missing something because he understands AI better than I do, but in fact it looks like the better explanation isn’t that I’m missing something, but that Altman is genuinely making bad arguments because his reasoning is motivated—and he’s a known liar so I’m perfectly happy to infer that he’s lying about this issue too. ↩
Perhaps I’m using a circular definition of “respectable” but I don’t consider someone respectable (on this particular issue) if they estimate P(doom) at <1%. ↩
I put roughly 80% credence on each of five arguments. If each argument is independent, that means I’m probably wrong about at least one of them, but I don’t think they’re independent.

And some of the arguments are more decisive than others. For example, if I’m wrong about the opportunity cost argument (#4), then I should switch sides. But if I’m wrong about the hardware overhang argument (#1) and overhang is indeed a serious concern, that doesn’t necessarily mean we shouldn’t slow AI development, it just means a slowdown improves safety in one way and harms safety in another way, and it’s not immediately clear which choice is safer. ↩
At least for some reasons people cite for not wanting to pause, they should agree with me on this. There are still some counter-arguments, like “we can’t delay AI because that delays the glorious transhumanist future”, but I consider those to be the weakest arguments. ↩
You could argue that pseudoephedrine is over-regulated, and in fact I would agree. But I don’t think those regulations are a particularly big problem, either. ↩
Companies make more money from more powerful models, and powerful models are more dangerous. Power and safety directly trade off against each other until you can figure out how to build powerful models that aren’t dangerous—which means you need to solve alignment first. ↩
In some sense, if we get strong regulations, the companies win, because all the companies’ employees and shareholders don’t get killed by unfriendly AI. But they’ll be unhappy in the short term because they irrationally prioritize profit over not getting killed.

I don’t understand what’s going on here psychologically—according to the expressed beliefs of people like Dario Amodei and Shane Legg, they’re massively endangering their own lives in exchange for profit. It’s not even that they disagree with me about key facts, they’re just doing things that make no sense according to their own (expressed) beliefs. ↩
I’m inclined to say it doesn’t matter at all, but some smart alignment researchers think it does, and they know more than me. Changing my mind wouldn’t materially change my argument here.

Paul Christiano, who ~invented RLHF, seems to believe that it is not a real solution to alignment, but improvements on the method might lead to a solution. (He wrote something like this in a comment I read a few days ago that now I can’t find.)

On the other hand, RLHF makes the AI look more aligned even if it isn’t, and this might hurt by misleading people into thinking it’s aligend, and they proceed with expanding capabilities when really they shouldn’t.

RLHF also makes LLMs less likely to say PR-damaging things. Without RLHF, AI companies might develop LLMs more cautiously out of fear of PR incidents. ↩
This doesn’t make sense as a crux because an RSP also creates a hardware overhang if it triggers, but I’ve already talked about why I dislike the “hardware overhang” argument in general. ↩
“[thing I don’t understand] must be simple, right?” -famous last words ↩
At least that’s what I thought when I first wrote this sentence, before I had looked into any AI policy orgs. After having looked into them, I found it pretty easy to strike some orgs off the list. I don’t know what conversations orgs are having with policy-makers and how productive those conversations are, but I can read their public reports, and I can tell when their public reports aren’t good. And if their reports aren’t good, they probably don’t do a good job of influencing policy-makers either. ↩
- The Long-Term Future Fund (LTFF) has given very little money to AI policy.
- The AI Risk Mitigation Fund is a spinoff of LTFF that focuses exclusively on AI safety. As of this writing, it hasn’t made any grants yet, but I assume it will behave similarly to LTFF.
- Longview Philanthropy’s Emerging Challenges Fund and the Survival and Flourishing Fund have given some grants on AI policy, but mostly on other cause areas.
- Manifund has a regranting program, but 3 out of 6 regranters are current or former employees at AI companies which makes me disinclined to trust their judgment; and their grants so far mostly focus on alignment research, not policy.
- Larks used to write reviews of AI safety orgs, but they haven’t done it in a while—and they primarily reviewed alignment research, not policy.
- Nuño Sempere did some shallow investigations three years ago, but they’re out of date.
↩
As of this writing (2024-11-02), the recommendations are:
1. Horizon Institute for Public Service
2. Institute for Law and AI
3. Effective Institutions Project’s work on AI governance
4. FAR AI
5. Centre for Long-Term Resilience
6. Center for Security and Emerging Technology
7. Center for Human-Compatible AI
↩
I don’t know exactly what’s going on with the difference between Founders Pledge recs and my top donation candidates. It looks to me like Founders Pledge puts too much stock in the “build influence to use later” theory of change, and it cares too much about orgs’ legible status / reputation. ↩
Some of these orgs don’t exactly work on policy, but do do work that plausibly helps policy. There are some orgs fitting that description that I didn’t review (e.g., AI Impacts and Epoch AI). I had to make judgment calls on reviewing plausibly-relevant orgs vs. saving time, and a different reviewer might have made different calls.

In fact, I think I would be more likely to donate to (e.g.) AI Impacts than (e.g.) METR, so why did I write about METR but not AI Impacts? Mainly because I had already put some thought into METR and I figured I might as well write them down, but I haven’t put much thought into AI Impacts. ↩
Some Open Philanthropy employees stand to make money if AI companies do well; and Holden Karnofsky (who no longer works at Open Philanthropy, but used to run it) has expressed that he expects us to avert x-risk by an AI company internally solving alignment. ↩
It’s nice that AI safety is so much better funded than it used to be, but for my own sake, I kind of miss the days when there were only like five x-risk orgs. ↩
The names made it difficult for me to edit this post. Many times I would be re-reading a sentence where I referenced some org, and I wouldn’t remember which org it was. “Centre for the Governance of AI? Wait, is that the one that runs polls? Or the one that did SB 1047? Er, no, it’s the one that spun out of the Future of Humanity Institute.”

Shout-out to Lightcone, Palisade, and Sentinel for having memorable names. ↩
But maybe that’s a silly thing to worry about. Compare: “I only invest in companies that don’t have managers. Managers’ salaries just take away money from the employees who do the real work.” ↩
I’m not convinced that it’s good practice, but at least some people believe it is. ↩
Also, I’m not sure how much consideration to give this, but I have a vague sense that sharing criticism with the orgs being criticized would hurt my epistemics. Like, maybe if I talk to them, I will become overly predisposed toward politeness and end up deleting accurate criticisms that I should’ve left in. ↩
The word “funge” is not in the dictionary; I would define it as “causing a fungible good [in this case, money] to be used for a different purpose.” That is, causing SFF to give some of its money to a different nonprofit. ↩
To be honest it’s not that funny given the stakes, but I try to find a little humor where I can. ↩
The linked source isn’t why I believe this claim; I believe it based on things I’ve heard in personal communications. ↩
Examples:
- Andrea brought up the classic argument that AI becomes really dangerous once it’s self-improving. But, he said, it’s not clear what exactly counts as self-improving. It’s already something like self-improving because many ML engineers use LLMs to help them with work tasks. Andrea proposed that the really dangerous time starts once AI is about as competent as remote workers, because that’s when you can massively accelerate the rate of progress. I don’t have a strong opinion on whether that’s true, but it made me think.
- Andrea said it’s a big problem that we don’t have a “science of intelligence”. We don’t really know what it means for AIs to be smart, all we have is a hodgepodge of benchmarks. We can’t properly evaluate AI capabilities unless we have a much better understanding of what intelligence is.
↩
To be clear: sometimes, when people say “… but I don’t know if X”, that’s a polite way of saying “I believe not-X.” In this case, that’s not what I mean—what I mean is that I don’t know. ↩
The report is about defending against cyber-attacks, not about executing cyber-attacks. But it’s also about how to increase AI capabilities. (And an AI that’s smarter about defending cyber-attacks might also be better at executing them.) I can see a good argument that this work is net negative but there’s an argument the other way, too. ↩
Several sources claim Eliezer’s P(doom) > 95% but their source is a news article and the news article doesn’t cite a source. I could not find any direct quote. ↩
At least you should have the same goals, if not the same tactics. ↩
I would not have thought “write Harry Potter fan fiction” was a good strategy, but I turned out to be wrong on that one. ↩
Although to be fair, that’s also why most AI “safety” researchers do capabilities research. ↩
For example, various studies have looked at natural experiments where protests do or do not occur based on whether it rains, and they find that protesters’ positions are slightly more popular when it does not rain. The effect shows up repeatedly across multiple studies of different movements. ↩
I don’t have meaningful insight into whether any particular person would be good at lobbying. I think I can identify that most people would be bad at it, so the best I can do is fail to find any reasons to expect someone to be bad. I don’t see any reasons to expect Felix to be bad, except that he’s junior but that’s a weak reason. ↩
Outside of the top five, I didn’t think about these classifications very hard. ↩
For example, I would probably fund Good Food Institute ahead of most of these. ↩
That screens off most of the policy orgs on my list. ↩
That screens off Control AI and Existential Risk Observatory. ↩
That screens off Center for AI Safety and Future of Life Institute. ↩
Only MIRI is pursuing the sorts of strategies that MIRI is pursuing, but it has >100x more money than PauseAI. ↩
When I made the donation to PauseAI Global, I was under the impression that PauseAI Global and PauseAI US were one organization. That was true at one point, but they had split by the time I made this donation. If I had known that, I would have donated to PauseAI US instead. But I’m not bothered by it because I still think donations to PauseAI Global have high expected value.

Also, when I donated the money, I wasn’t planning on writing a whole post. It wasn’t until later that I decided to do a proper investigation and write what you’re currently reading. That’s why I didn’t wait before donating. ↩ ↩²

Subjects in Pysch Studies Are More Rational Than Psychologists

Tue, 15 Oct 2024 00:00:00 -0700

Psychologists have done experiments that supposedly show how people behave irrationally. But in some of those experiments, people do behave rationally, and it’s the psychologists’ expectations that are irrational.

Contents
Ultimatum game
Hyperbolic discounting
Dunning-Kruger effect
Non-examples: Asch and Milgram
Themes
Notes

Ultimatum game

The ultimatum game works like this:

There are two participants, call them Alice and Bob. Alice receives $100. Alice then chooses some amount of that money to offer to Bob. Bob can accept the offer and they both walk away with their money, or he can reject the offer in which case they both get nothing.

According to standard theory, Alice should offer Bob $1 and keep $99 for herself, and Bob should accept the offer because a dollar is better than no dollars. In practice, most “Alices” offer something like a 50/50 split, and most “Bobs” reject the offer if given something like an uneven split. Psychologists¹ are confused about this supposedly irrational behavior.

But in fact, Alice and Bob both behave rationally. Bob rejects unfair offers, which makes him have less money in some cases. But Alice expects Bob to reject unfair offers, so she offers Bob a 50/50 split.

Bob follows a general strategy of rejecting unfair offers, because he believes or intuits that this strategy will ensure he mostly receives fair offers. And Alice knows Bob will probably reject an unfair offer because that’s what most people do. So Bob has (acausally) induced Alice to give him $50 instead of $1. In the world where Bob rejects unfair offers, Bob wins.

In other words, most people don’t use causal decision theory.

Hyperbolic discounting

The standard model of rational behavior assumes exponential discounting: future goods decrease in value at a fixed rate (say, 10% per year). But experiments show that many people use hyperbolic discounting, which means the value of future goods first falls off rapidly, and then slowly.

The classic experiment:

Ask people if they would prefer $100 now or $120 a month from now. Most people prefer the $100 now.
Ask people if they would prefer $100 in 12 months or $120 in 13 months. Most people prefer the $120 in 13 months.

Psychology textbooks say this is irrational: the monthly discount rate must be either more than 20% (in which case they should take the money sooner in both cases) or less than 20% (in which case they should take the money later). But in real life, people’s behavior makes sense.

If you say you’re going to give me money right now, you’re probably going to do it. If you say you’re going to give me $120 a month from now, I don’t know what’s going to happen. Maybe you’ll forget, maybe your study will run out of funding, I don’t know. So I’d rather have the money now.

If you can give me money in 12 months, you can most likely also give me money in 13 months, so I’m not too concerned about waiting an extra month in that case.

In mathematical terms, the probability that you’ll give me the money decreases hyperbolically with time, not exponentially, so I ought to use hyperbolic discounting.

(Plus there is another, more technical, reason to use hyperbolic discounting: if the “true” discount function is exponential but you don’t know what exact discount rate to use, then your expected discount rate starts out high and decreases over time, which produces a hyperbolic discount function.)

Dunning-Kruger effect

The Dunning-Kruger effect is often misrepresented as showing that ignorant people think they’re knowledgeable, and knowledgeable people think they’re ignorant. To my knowledge, no study has ever showed that.

The original Kruger & Dunning (1999)² paper, and some follow-up studies, produced graphs that looked something like this:

(This graph is an illustration, not based on actual data.)

Notice two things about this graph:

The “perceived ability” line is a bit too high.
The slope is too flat—everyone estimates their ability as closer to average than it really is.

#1 is a real (albeit small) bias—people systematically overestimate their abilities by a little bit. But the flattened slope of perceived ability is rational, for two reasons.

Reason 1. If your test has some degree of noise, then you expect the top scorers to have gotten lucky and the bottom scorers to have gotten unlucky. The top scorers aren’t quite as good as the test makes them look, and the bottom scorers aren’t quite as bad as the test makes them look. So the true skill curve is flatter than the measured skill curve. For more on this, see Gignac & Zajenkowski (2020)³.

Reason 2. People do not have perfect knowledge of their own abilities. If you know nothing else, you should expect to perform about average. If you know a little bit about your own skill level, you should expect to perform a little below or a little above average. Even if you’re in the top 10% or bottom 10% of skill, it should take a lot of evidence to convince you of that, so it’s perfectly rational for you to estimate your own skill as closer to average than it really is.

Both these reasons say essentially the same thing: if you have a noisy measure of ability, your belief conditional on that measure should be closer to average than what the measure itself shows. This produces a flattened curve like what we see in the Dunning-Kruger effect, and it’s not a bias but a perfectly rational way of reasoning about imperfect evidence.

Non-examples: Asch and Milgram

Originally I wanted to include the Asch conformity experiments and the Milgram experiments on this list—I thought participants behaved rationally. But after doing some more research, I think the rational-participant theory can’t fully explain the observations.

(If you’re not familiar with the Asch or Milgram experiments, click the links in the previous paragraph for explanations. They’re a bit too complicated to explain in this post.)

For Asch, I originally wanted to argue that it’s rational to update your beliefs when other people disagree with you. But the experiments found that people are more willing to disagree with the group when they can give their answers in secret. If people were rationally updating on others’ beliefs, then they’d agree with the group in secret as well as in public.

I still think it’s rational for subjects to update their beliefs toward those of the other participants, but the experiments suggest that people update more than they should.

For Milgram, I wanted to argue that participants who continue to administer electric shocks have rationally (and correctly!) deduced that nothing bad is happening. This is consistent with people’s behaviors, but it’s not consistent with their stated beliefs (almost all participants report believing that it was real) or their observed emotional state (many of them were visibly sweating/trembling/having nervous fits).

Themes

What can we learn from how psychologists mis-interpret these experiments?

People consider more types of evidence than pychologists think they do.
People intuitively use non-causal decision theories.

Notes

Or at least psychology textbooks, it’s possible that many actual psychologists are smarter about this. ↩
Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-assessments. ↩
Gignac, G. E., & Zajenkowski, M. (2020). The Dunning-Kruger effect is (mostly) a statistical artefact: Valid approaches to testing the hypothesis with individual differences data. ↩

My Submission for Worst Argument In The World

Sat, 12 Oct 2024 00:00:00 -0700

Scott Alexander once wrote:

David Stove once ran a contest to find the Worst Argument In The World, but he awarded the prize to his own entry, and one that shored up his politics to boot. It hardly seems like an objective process.

If he can unilaterally declare a Worst Argument, then so can I.

If those guys can unilaterally declare a Worst Argument, then so can I. I declare the Worst Argument In The World to be this:

“A long time ago, not-A, and also, not-B. Now, A and B. Therefore, A caused B.”

Example: In 1820, pirates were everywhere. Now you hardly ever see pirates, and global temperatures are rising. Therefore, the lack of pirates caused global warming.

(This particular argument was originally made as a joke, but I will give some real examples later.)

Naming fallacies is hard. Maybe we could call this the “two distant points in time fallacy”. For now I’ll just call it the Worst Argument.

The Worst Argument is a special case of the post hoc ergo propter hoc fallacy: “A happened before B, therefore A caused B.”¹ I find this special case to be particularly bad. Post hoc ergo propter hoc can make a bit of sense sometimes: maybe B happens immediately after A, maybe A and B repeatedly appear together. That doesn’t definitively establish causality, but if you have data showing that B always comes right after A, I want to hear about it.

The thing about the Worst Argument is that it provides almost no evidence of anything. You can’t just look at two distant points in time, check one independent variable, and assume that the independent variable explains the dependent variable. So many other things could have happened! It’s an incredible leap to say that the rise in global temperatures must be caused by the decline in pirates when about one zillion other things changed between 1820 and today.

The Worst Argument is the worst version of post hoc ergo propter hoc because:

It only looks at two data points.
The two data points are far apart in time.

If you’re going to base your reasoning on post hoc ergo propter hoc, at least give me a bunch of data points! Or at least make them be close together in time!

The Worst Argument so obviously bad that surely no one would ever think to invoke it, and if invoked, surely no one would take it seriously, right? And yet people make arguments of this form all the time. And I’ve been persuaded by arguments like this! The fact that this fallacy keeps fooling people (including me) makes it my choice for Worst Argument In The World.

Let me give some real-world examples. These statements aren’t all wrong, but the arguments are all bad. (It’s possible to make a fallacious argument in support of a true conclusion.²)

School uniforms prevent crime

When I was in 9th grade, I spent a lot of time on debate.org, a (now-defunct) website that hosted online written debates. I participated in a debate on the proposition “School uniforms ought to be worn in primary and secondary schools.” I took the negative. My opponent was one of the most renowned debaters on the site, with a record of 75 wins to 7 losses. I was a little nervous to go up against him and I wanted to do my best.

The debate lasted for three rounds. After the first two rounds, I felt like I was losing, and I spent hours poring over my opponent’s arguments to figure out how to write my final rebuttal. At some point, I had an epiphany: I realized that his central argument was flimsy, but his rhetoric was so good that I hadn’t noticed.

His argument was essentially this:

In 1993, the Long Beach school system did not require uniforms, and it had high rates of school crime.
In 1993, it required school uniforms, and it had dramatically lower rates of crime.
Therefore, school uniforms must have caused the reduction in crime.³

Nothing else pertinent could have possibly happened in those 6 years, right? They couldn’t have instituted new policies for handling troublesome students, or changed their rules for reporting crimes, or anything like that. It could only have been the school uniforms.

In my final rebuttal, I focused on emphasizing the badness of this argument. It was enough to persuade the readers, and I am happy to say that my opponent now has 8 losses.

Franklin D. Roosevelt solved the Great Depression

An argument that I believed for a long time:

The Great Depression started in 1929.
FDR was elected in 1931.
The Great Depression ended in 1939, while FDR was in office.
Therefore, FDR solved the Great Depression.

Plenty of other things happened between 1931 and 1939. An economy can shift quite a lot in 8 years, for many reasons other than who’s president. A typical economic cycle only lasts something like 8 years, and the typical recession only lasts one or two years. After that long you’d expect the economy to look very different, no matter who the president is.

You might say, hey, aren’t there better arguments about how specifically FDR’s policies could have improved the economy? Yeah, there are, but that’s not why I used to believe FDR had great economic policies. I believed it because I fell for the Worst Argument In The World.

(Folks on the other side of the political spectrum like to say Reagan solved the economic problems of the 1970s and early 80s, and they make the exact same argument. I never believed that one for some reason…)

Labor unions and strikes were largely responsible for improved working conditions in the 19th and early 20th centuries

In high school, I read Howard Zinn’s A People’s History of the United States and it convinced me of the importance of collective action. The book’s argument went like this:

Laborers had to work in really bad conditions.
They organized unions and held strikes.
Decades later, conditions were still bad but not quite as bad.
Therefore, strikes and labor unions were the cause of the improvement in conditions.

The book repeatedly makes this argument through a series of historical anecdotes.⁴ As I recall, it never says the conclusion explicitly, but it’s strongly implied.

At the time, I found the book’s anecdotes convincing. It wasn’t until years later that I realized how weak its case was.

In fact, the book’s argument is even weaker than the Worst Argument In The World. It doesn’t say working conditions were bad, then strikes happened, then many years later conditions were good. It says working conditions were bad, then strikes happened, then conditions were still bad. Which is positive evidence that the strikes didn’t help. Kind of amazing that I read this and came away with the opposite conclusion.

Seed oils cause obesity

There are some arguments about seed oils that don’t rely on this fallacy, but the most popular argument I see goes like this:

A century ago, obesity was rare, and also people didn’t eat a lot of seed oils.
Today, many more people are obese, and also people eat a lot of seed oils.
Therefore, obesity must be caused by seed oils.

I’m almost baffled that anyone finds this convincing, except that I myself was convinced by the Worst Argument In The World in at least two instances as documented above.

Some people get more sophisticated and they draw a line of obesity going up and a line of something-kinda-like-seed-oil consumption (like vegetable oil consumption) going up and they say, look, the lines both go up! That’s not quite as bad as only looking at two data points, but it’s still pretty bad. (You can also draw a line of pirate populations going down and a line of global temperature going up.) It would be better to go a little further and show that the two lines track each other well. That wouldn’t establish causation but at least it establishes correlation.⁵

Martin Luther King greatly improved civil rights for African-Americans

I’m sure there are good arguments for this statement out there somewhere, but I always hear people (implicitly) say things like:

In the 1950s, segregation was legal and African-Americans didn’t have equal rights.
Then, Martin Luther King spearheaded civil rights protests. Also, a bunch of other stuff happened.
By the 1970s, America was a much better place for black people.
Therefore, the improvements in civil rights must have been largely caused by MLK.

What makes it the Worst Argument In The World?

Scott Alexander’s Worst Argument In The World is a dirty debate tactic—you say something technically true that subtly invokes a hard-to-spot fallacy. My submission for Worst Argument is almost the opposite—you say something fallacious, and it’s pretty obviously fallacious if you think about it, but we fall for it anyway. The Worst Argument In The World reminds me of the basketball awareness test: Like the moonwalking bear, it somehow sneaks into my brain without getting spotted.

Notes

Which itself is a special case of the correlation implies causation fallacy. ↩
According to Sturgeon’s law, 90% of all arguments are crap, including arguments for true claims. ↩
I didn’t notice this at the time but now that I’m re-reading the debate, I’m reasonably sure the crime numbers are just fraudulent. They claim a 93% reduction in sex offenses. No intervention ever reduces crime by 93%.

I’m not sure claiming fraud would have been a good debate tactic, though, and the argument is still terrible even if the numbers are real. ↩
I’m largely going based off memory, I haven’t read the book since high school. Based on skimming an online summary, it looks like the book made forms of this argument in at least chapters 10, 11, 13, 15, and 19. ↩
I looked on a few seed-oil-hypothesis advocacy sites and I couldn’t find any proper statistics. This is not related to my main point but it was bugging me so I decided to do the statistics myself.

To my knowledge, we don’t have proper historical data on seed oil consumption. The best I could do was compare salad + cooking oil consumption⁶ and obesity. Both go up over time, but they don’t track particularly well—for example, oil consumption spiked up around 1999, whereas obesity increased smoothly.

Salad + cooking oil consumption correlates with obesity at r = 0.93, but calendar year correlates with obesity at r = 0.994. If calendar year predicts obesity better than your independent variable, then your independent variable probably isn’t very good. (See here for my calculations.)

(Exercise for the reader: What is the correlation between obesity and pirate sightings?) ↩
Lee, J. H., Duster, M., Roberts, T., & Devinsky, O. (2022). United States Dietary Trends Since 1800: Lack of Association Between Saturated Fatty Acid Consumption and Non-communicable Diseases.. doi: 10.3389/fnut.2021.748847 ↩

Outlive: A Critical Review

Thu, 26 Sep 2024 00:00:00 -0700

Last updated 2025-07-04.

Outlive: The Science & Art of Longevity by Peter Attia (with Bill Gifford¹) gives Attia’s prescription on how to live longer and stay healthy into old age. In this post, I critically review some of the book’s scientific claims that stood out to me.

This is not a comprehensive review. I didn’t review assertions that I was pretty sure were true (ex: VO2 max improves longevity), or that were hard for me to evaluate (ex: the mechanics of how LDL cholesterol functions in the body), or that I didn’t care about (ex: sleep deprivation impairs one’s ability to identify facial expressions).

First, some general notes:

I have no expertise on any of the subjects in this post. I evaluated claims by doing shallow readings of relevant scientific literature, especially meta-analyses.
There is a spectrum between two ways of being wrong: “pop science book pushes a flashy attention-grabbing thesis with little regard for truth” to “careful truth-seeking author isn’t infallible”. Outlive makes it 75% of the way to the latter.
If I wrote a book that covered this many entirely different scientific fields, I would get a lot more things wrong than Outlive did. (I probably get a lot of things wrong in this post.)
When making my assessments, I give numeric credences and also use terms such as “true” and “likely true”. The numbers give my all-things-considered subjective credences, and the qualitative terms give my interpretation of the strength of the empirical evidence. For example, if the scientific evidence suggests that a claim is 75% likely and I understand the evidence well, then I rate the claim as “likely true”. If I only read the abstract of a single meta-analysis, and the abstract unequivocally supports the claim but I’m only 75% sure that the meta-analysis can be trusted, then I rate it as “true”. Both claims receive a 75% credence.

Now let’s have a look at some claims from Outlive, broken down into four categories: disease, exercise, nutrition, and sleep.

Contents
Disease
Exercise
Nutrition
Sleep
Bonus
- Dunning-Kruger effect
Changelog
Notes

Disease

People with metabolically healthy obesity do not have elevated mortality risk

A person is defined as having metabolic syndrome if they show at least three out of five symptoms:

abdominal obesity (i.e. large waist circumference)
high blood pressure
high blood sugar
high serum triglycerides
low HDL cholesterol

People with obesity but no metabolic syndrome are said to have metabolically healthy obesity (MHO).

Here’s what Outlive has to say about MHO:

A large meta-analysis of studies with mean follow-up time of 11.5 years showed that people [with metabolic syndrome] have more than triple the risk of all-cause mortality and/or cardiovascular events than metabolically healthy normal-weight individuals. Meanwhile, the metabolically healthy but obese subjects in these studies were not at significantly increased risk. (page 95)²

My assessment:

The statement “MHO subjects in these studies were not at significantly increased mortality risk” is technically correct: true (credence: 90%³).
Obesity has no negative health effects for metabolically healthy people: false (credence: 5%).
Metabolically healthy obesity does not increase mortality risk: highly unlikely (credence: 10%).

Outlive does not exactly say that MHO carries no elevated health risk, but some readers may come away with that impression, so I want to clarify that obesity is still bad for you even if you’re metabolically healthy.

The book cites Stefan et al. (2017)⁴ which in turn cites Kramer et al. (2013)⁵:

In a pooled analysis of 8 studies, metabolically healthy obese persons had a similar risk for all-cause mortality or CV [cardiovascular] events compared with the metabolically healthy normal-weight individuals (RR, 1.19; [95%] CI, 0.98 to 1.38). […] However, after we restricted analysis only to studies with at least 10 years of follow-up, the metabolically healthy obese group indeed had increased mortality and CV risk compared with the metabolically healthy normal-weight group (RR, 1.24; CI, 1.02 to 1.55; I^2 = 33.6%)

Stefat et al. (2017) also found that MHO subjects had higher rates of chronic disease than metabolically healthy normal-weight subjects, see Figure 2:

“MHO does not significantly increase risk” is one way you could describe this evidence. But it’s not the description I’d use:

A relative risk (RR) of 1.19 sounds pretty bad. This finding is (just barely) not statistically significant, but it still has a likelihood ratio of about 5:1 compared to RR = 1⁶—that is, we are 5x more likely to see this result if MHO elevates mortality risk by 1.19x than if it doesn’t elevate risk at all.
MHO subjects had worse health across the board—higher rates of fatty liver disease, higher insulin resistance, lower cardiorespiratory fitness, more arterial plaque buildup—while also having non-significantly higher mortality. Which is a more reasonable interpretation: All those health problems somehow don’t translate into increased mortality? Or the health problems do increase mortality, and the observed RR = 1.19 is real even though it’s not statistically significant? (And the true RR probably isn’t exactly 1.19, but it’s somewhere in that vicinity.)

Many lament how often researchers treat p < 0.05 as “definitely real”, but it bugs me just as much when they treat p > 0.05 as “definitely no effect”.

Ben Carpenter at Red Pen Reviews writes:⁷

Roughly 50% of people⁸ with metabolically healthy obesity will develop at least one metabolic abnormality within 3–10 years, which is double the risk of normal-weight individuals. Further, metabolically healthy obesity is still associated with an increased risk of adverse cardiovascular events⁹, subclinical atherosclerosis¹⁰ (plaque buildup within the arterial walls), nonalcoholic fatty liver disease¹¹, kidney function decline¹², and type II diabetes¹³. So, considering this evidence, being metabolically healthy with obesity is probably short-lived and still has health risks.

Amyloid beta is implicated in Alzheimer’s disease

[Amyloid beta] is clearly bad stuff. (page 181)

My assessment: true (credence: 90%).

This claim stood out in my memory because a great deal of research on amyloid beta recently turned out to be fraudulent. But upon re-reading the relevant section of Outlive, I found that none of the book’s claims relied on the fraudulent research, and in fact the book cites the fraud investigation itself:

[Some scientists’] doubts seemed to be validated in July of 2022, when Science published an article calling into question a widely cited 2006 study that had given new impetus to the amyloid theory, at a time when it had already seemed to be weakening.¹⁴ The 2006 study had pinpointed a particular subtype of amyloid that it claimed directly caused neurodegeneration. That in turn inspired numerous investigations into the subtype. But according to the Science article, key images in that study had been falsified. (page 183–184)

The claims made in Outlive about the apparent negative effects of amyloid beta come from studies that predate the 2006 fraud, and those older claims appear to hold up. The book accurately describes the current state of the field as far as I understand it: amyloid beta is associated with at least some cases of Alzheimer’s, but research into amyloid beta has so far failed to uncover any useful treatments.

HDL cholesterol on its own doesn’t prevent heart disease

I wrote this claim in my notes on my first read-through, but upon re-reading, Outlive never actually said this. I’m including this claim anyway so that readers know I have poor reading comprehension and can’t be trusted.¹⁵

Here’s what the book actually said:

Risk does seem to decline as HDL-C rises to around the 80th percentile. But simply raising HDL cholesterol concentrations by brute force, which specialized drugs, has not been shown to reduce cardiovascular risk at all. (page 123)

My assessment:

HDL cholesterol on its own doesn’t prevent heart disease: false, but Outlive never said that—in fact, it said the opposite.

Exercise

The book (rightly) focuses more on exercise than on nutrition or sleep. ~~From what I can tell, it is the most scientifically accurate section of the book.~~

Update 2025-05-05: I’ve come to learn more about exercise and longevity, and I now believe some of Outlive’s claims on exercise are incorrect or overstated. The next three sections below explain my updated beliefs. They largely cover the same ground as my recent post, I was probably wrong about HIIT and VO2max, but with better evidence and citations (citing meta-analyses instead of tweets).

VO2max is the best predictor of longevity

VO2max is the maximum amount of oxygen your body is capable of consuming. VO2max is commonly used as a measure of aerobic fitness.

Outlive never directly says VO2max is the best predictor of longevity, but it sure implies it. The book spends eight pages talking about the association between VO2max and longevity, and about how important it is.

My assessment:

Increasing VO2max increases longevity: true (credence: 97%).
The causal relationship between VO2max and longevity is as strong as Outlive implies: false (credence: 10%).
VO2max is the best proxy for physical fitness: false (credence: 20%) – performance on fitness tests (e.g. maximum pace on an incline treadmill) is easier to measure and probably more accurate.¹⁶

I have two objections to how Outlive characterizes VO2max:

The book cites observational studies, not RCTs.
VO2max is a proxy for physical fitness, not physical fitness itself. And there are better proxies.

Correlation is not causation. I am confident that training to increase VO2max does, in fact, increase longevity. But I believe observational studies overstate the magnitude of the effect.

I’ve looked at both observational studies and RCTs on the relationship between exercise and health, and my sense is that observational studies overstate the causal effect by about 2x. (It’s on my to-do list to write a post about why I believe this.) I don’t know about VO2max in particular, but I expect that only about half of its observed association with longevity is causal.

However, even after cutting the effect in half, exercise is still the best general-purpose¹⁷ intervention for longevity.

As for my second objection: VO2max is only a proxy for physical fitness. Here I will quote an article by elite running coach Steve Magness, because I can’t explain it any better than he did:

In practical terms, Vo2max is like knowing the size of a car’s engine, which is really important if we want to know about performance. But if we want to know whether that car has a chance to win the Daytona 500, engine size alone won’t tell us. We also need to to know about the size of the fuel tank, about its fuel economy, about how long its tires will hold up, and about all the other small components that translate the power of engine to the speed of the car. It’s the same in humans. Vo2max is one of many components that, taken together, tell us about our holistic aerobic or cardiorespiratory fitness.

[…]

Vo2max matters. But it’s just one component of many that make up both performance and aerobic fitness. And that’s important because if we return to the original claims that Vo2max is the key indicator of longevity, we’ll find that the majority of the studies cited did NOT even use Vo2max as the main variable. They used performance! In the majority of research, peak speed and incline during the [Vo2max] test was the main correlate to longevity.

The large study¹⁸ on 750,000 veterans that found a 4-fold higher mortality risk for low versus high fitness used peak speed and incline, not Vo2max. Same with the research¹⁹ on 120,000 individuals finding a 5x difference in the risk of early death.

[…] And as we can see in this meta-analysis²⁰ looking at mortality and fitness, all but a handful of the included studies used an estimate based on speed or time.

You get the point.

And this is good news! It means you don’t need to go to a lab and measure your Vo2max. You don’t even need to worry about Vo2max itself (or your watche’s [sic] horrible estimation of it). All you need to do is focus on overall aerobic fitness. Which can easily be measured, compared, and improved in a number of ways that are less expensive and more accessible than Vo2max.

So, in short: most studies that measure “VO2max” are actually measuring performance on a fitness test. And this is good news, because it means if your performance is improving—if your mile time or your 5K time is getting faster—then you’re making real progress.

You should train VO2max by doing HIIT at the maximum sustainable pace

Outlive recommends doing high-intensity interval training (HIIT) to improve VO2max:

The tried-and-true formula for [VO2max training] is to go four minutes at the maximum pace you can sustain for this amount of time—not an all-out sprint, but still a very hard effort. Then ride or jog four minutes easy, which should be enough time for your heart rate to come back down to below about one hundred beats per minute. Repeat this four to six times and cool down. (page 249)

My assessment:

The best exercise routine includes HIIT: likely true (credence: 85%).
Longer intervals are better than shorter intervals: likely true (credence: 85%).
HIIT should be done at the maximum sustainable pace: false (credence: 25%).
HIIT is the best way to improve VO2max: likely true (credence: 70%).

Four-minute intervals are reasonable: a meta-analysis by Wen et al. (2019)²¹ found that ≥2-minute intervals produced bigger performance benefits than <2 minutes (although all interval durations improved VO2max).

But I have two concerns:

Training at the maximum sustainable pace is a recipe for burnout. You should train at a pace that’s difficult, but not maximum effort.
There is mixed evidence on whether HIIT is the best way to improve VO2max. Low-intensity training might be just as good.

For the first concern:

Taken literally, the quoted prescription is impossible to follow. If you do your first interval at the maximum sustainable pace, then your second interval cannot possibly be done at the same pace because you will be fatigued from the first interval.

Perhaps Attia meant to say that you should go at the maximum pace you can sustain across all intervals. That’s physically possible, but it still seems like a bad idea.

I wanted to cite RCT evidence comparing HIIT at different intensities while controlling for duration (e.g., 4-minute intervals at maximum pace vs. 4-minute intervals at 90% of maximum pace). But I couldn’t find any. Instead, the evidence on this question comes from studies of how athletes train.

Elite athletes typically do HIIT at roughly 90% of VO2max (Seiler (2010)²²). This is considerably slower than the maximum sustainable pace (Billat (2001)²³). For example, Billat et al. (1995)²⁴ found that a sample of elite long-distance runners could maintain 90% of VO2max for an average of 16.55 minutes.²⁵ If you can maintain a certain pace for 16 minutes straight, then you can certainly run at that pace for four 4-minute intervals with rests in between.

For the second concern: is HIIT really better than low-intensity training (LIT)²⁶ for improving VO2max? Probably yes, but it’s unclear.

The RCT evidence on this is mixed. A meta-analysis by Wen et al. (2019)²¹ found that HIIT worked better than LIT for increasing VO2max. However, a review of meta-analyses by Crowley et al. (2022)²⁷ found inconsistent evidence.²⁸

I didn’t look at most of the meta-analyses reviewed by Crowley et al. I did look at Gist et al. (2013), which at a glance appears to be the most rigorous “contrarian” meta-analysis. It found that sprint interval training did not increase VO2max by more than endurance training. However, this is roughly consistent with Wen et al. (2019)²¹, which found that longer intervals (2+ minutes) worked better than sprint intervals.

RCTs may overstate the benefits of HIIT relative to LIT. A meta-analysis by Mølmen et al. (2024)²⁹ found that HIIT rapidly improved participants’ fitness, whereas the benefits of LIT took longer to show up. Therefore, a 5-week study will show a clear advantage to HIIT even if LIT might be better in the long run:

(ET = endurance training; HIT = high-intensity training; SIT = sprint-intensity training)

That being said, there is some evidence that HIIT provides benefits on top of LIT, so if your goal is to optimize longevity, then I believe it makes sense to do both.

You should do 3+ hours/week of zone 2 training and one or two sessions/week of HIIT

[I]t seems that about three hours per week of zone 2, or four 45-minute sessions, is the minimum required for most people to derive a benefit and make improvements, once you get over the initial hump of trying it for the first time. (page 243)

Even if we are not out to set world records, the way we train VO2max is pretty similar to the way elite athletes do it: by supplementing our zone 2 work with one or two VO2max workouts per week. (page 249)

My assessment:

This is a good exercise routine: true (credence: 95%).
This routine is uniquely better than any other: likely false (credence: 25%).

RCT evidence doesn’t tell us much about the optimal exercise routine. As with HIIT vs. LIT, the best evidence comes from looking at how top athletes train.

Elite athletes typically do 80% of their training at low/moderate intensity (a.k.a. zone 2), and 20% at high intensity (Seiler (2010)²²; Stöggl & Sperlich (2015)³⁰). For elite athletes, doing more than 20% of cardio sessions at high intensity can induce overtraining (Seiler (2010)²²).

Outlive recommends training at just two intensities: low and high. But many elite athletes train at three or more different durations/intensities (Stöggl & Sperlich (2015)³⁰). Different training modalities cause your body to adapt in different ways, so it makes sense to mix things up. You might do something like 80% zone 2 training, 15% long-duration (~4-minute) interval training, 5% sprint (< 1 minute) interval training. But just two intensities is probably sufficient to get the vast majority of longevity benefits.

I am not confident that ordinary folks should follow the same 80/20 rule as elite athletes. Professionals train 20–30 hours per week, including 4+ hours of high-intensity training. If I do 5 hours a week of cardio instead of 20, do I really need to restrict my HIIT to only one hour? Maybe I could handle 2 or 3 hours (or even 5 hours) without overtraining. But in the absence of direct evidence, I’ll follow the 80/20 rule.

For the record, my exercise routine is:

Low-intensity cardio 3x/week for 45 to 90 minutes (depending on how I’m feeling);
HIIT once every 1–2 weeks (usually four 4-minute intervals, and occasionally a different variation);
Resistance training 3-4x/week.

I think this is less than the optimal amount of HIIT, but I don’t like doing HIIT.

Stability is as important as cardiovascular fitness and strength

Outlive claims that exercise is the best way to increase longevity, and that the ideal exercise programs includes four components:

long-duration, low-intensity cardio (“zone 2” training)
high-intensity interval training (VO2 max training)
strength training
stability training

An abundance of evidence shows that the first three types of exercise are excellent for health and longevity. The fourth is not so well-supported.

[Stability] is the foundation on which our twin pillars of cardiovascular fitness and strength must rest. (page 265)

My assessment: I don’t know how to evaluate this. It seems to border on not even wrong.

The book defines stability as “the subconscious ability to harness, decelerate, or stop force”, while admitting that this isn’t a great definition. I don’t know how to empirically test stability by this definition. You can formally define cardiovascular fitness as VO2 max (or resting heart rate, etc.), and then show that improving your chosen metric improves health and reduces mortality risk. But I don’t know how to operationalize “stability”.

(On one reading, this definition of stability sounds nearly identical to “strength”, because your ability to decelerate/stop force is pretty much entirely determined by your ability to generate force.)

The chapter on stability only made two-ish testable claims that I could identify. The first:

DNS [dynamic neuromuscular stabilization] originated with a group of Czech neurologists who were working with young children with cerebral palsy in a hospital in Prague in the 1960s. They noticed that because of their illness, these kids did not go through the normal infant stages of rolling, crawling, and so forth. Thus they had movement problems throughout their lives. But when the children with cerebral palsy were put through a “training” program consisting of a certain sequence of movements, replicating the usual stages of learning to crawl, sit up, and eventually stand, their symptoms improved and they were better able to control their motions as they matured. The researchers realized that as we grow up, most healthy humans actually go through an opposite process—we lose these natural, healthy, almost ingrained movements. (page 270)

(The book cites Frank et al. (2013)³¹.)

TLDR: Some research found that DNS helped children with cerebral palsy, so it might also help adults prevent injuries. The assertion that it would help adults is presented without evidence, and to my knowledge, no evidence exists.

Relatedly, Outlive cites an interview Peter Attia did with “leading American practitioner of DNS” Michael Rintala, D.C. (a.k.a. Doctor of Chiropractic). Chiropractic is pseudoscience and chiropractors are fake doctors, so without having looked into this much, I’m pretty skeptical of DNS. If Rintala practices bogus medicine in one arena, that’s evidence that his DNS research is also bogus.

I don’t know that this first claim is false, but it’s essentially pulled out of thin air so I have no reason to believe that it’s true.

Outlive makes a second semi-testable claim: it implies (but does not explicitly state) that squatting with perfect form has lower injury risk than squatting asymmetrically. This claim sounds intuitively plausible but I could not find any supporting evidence (and there’s some evidence that “bad” squat form isn’t necessarily bad³²).

And squatting asymmetrically is probably better than not squatting at all (credence: 90%) because (1) studies find robust health benefits to strength training and (2) probably most of the subjects in those studies don’t have particularly good form.

Outlive didn’t do a good job of defining “stability training”, so I’ll do it myself. Let’s say the purpose of stability training is to reduce the risk of falling. In that case, is stability training useful?

A meta-analysis by Sherrington et al. (2011)³³ found that essentially any kind of exercise reduced fall risk, with balance training having a 22% larger effect than an “average” exercise program, and long-duration exercise (>50 hour trial duration) having a 23% larger effect than “average”. So “stability training” does appear to work, and high-dose exercise programs that included balance training reduced fall risk by 38%.
A meta-analysis by de Souto Barreto et al. (2019)³⁴ found that exercise significantly reduced falls (RR = 0.88) and injurious falls (RR = 0.74), and non-significantly reduced fractures (RR = 0.84, 95% CI 0.71–1.00). It compared exercise programs by type: (1) aerobic, (2) strength, (3) other (tai chi/dance), (4) multicomponent (aerobic + strength + balance). The meta-analysis did not find any significant differences between four types. (And it wasn’t a p = 0.06 situation either—most of the p-values were greater than 0.8.)
There’s some contrary evidence on balance training. A meta-analysis by Kümmel et al. (2016)³⁵ found that balance training on a particular task improves performance on that task, but does not transfer even to very similar tasks. This gives reason to doubt that balance training reduces injury risk.

So exercise appears to help with stability. And balance exercises might work better than other types of exercise, but they might not work at all.

My assessment of some claims that Outlive didn’t make, but sort of implied:

Stability is useful: likely true (credence: 90%).
Exercise can improve stability: likely true (credence: 80%).
Exercising to improve stability matters as much as exercising to reduce cardiovascular disease/diabetes/cancer: almost certainly false (credence: 5%).
Most people should do additional stability training on top of the cardio and strength training that they should already be doing: likely false (credence: 15%).

Nutrition

Rhesus monkey studies suggest that calorie restriction improves longevity but only if you eat a fairly unhealthy diet

I won’t provide a direct quote from Outlive because it would be too long. To summarize, the book says (pages 312–316):

A 2009 University of Wisconsin-Madison (UW) study³⁶ found that rhesus monkeys on a calorie-restricted diet lived longer than the control group.
But a similar 2012 study³⁷ by the National Institute of Aging (NIA) found that a calorie-restricted diet did not improve longevity.
The biggest difference between the studies was that the UW monkeys ate processed food and the NIA monkeys ate a whole-foods diet formulated by a primate nutritionist.
So it looks like calorie restriction improves longevity if you eat mostly processed food, and doesn’t matter much if you eat a healthy diet.

My assessment:

Outlive accurately summarizes these two studies: true (credence: 90%).
Calorie restriction (CR) improves longevity but only if you eat a fairly unhealthy diet: likely true (credence: 70%).
The rhesus monkey studies support the above claim: unclear (credence: 50%).
The rhesus monkey studies generalize well to humans: somewhat unlikely (credence: 35%).

I found the book’s interpretation to be reasonable and appropriately couched in uncertainty but I want to write about the studies because they were interesting.

A 2017 collaboration³⁸ by the authors of the UW and NIA studies reviewed the differences in study designs and outcomes. They agreed with Attia’s interpretation that diet quality was the most likely explanation for the different results, and that the studies jointly suggest that calorie intake is an important predictor of longevity—the monkeys in the NIA control group ate as little as those in the UW calorie restriction (CR) group.

I read the collaboration and noticed some results that don’t add up:

In the NIA study, the control group developed more chronic disease than the CR group (Figure 6), which seemingly contradicts the finding that calorie restriction didn’t help.
NIA split monkeys into young and old cohorts based on the age of each monkey when the study started. Within the young cohort, the CR group had less chronic disease (Figure 6), but had worse mean and median longevity (Table 2). Why do these two measurements point in opposite directions?
- CR reduced lifespan in the young NIA cohort, and the magnitude of the effect was larger than in the UW study, but it wasn’t statistically significant³⁹ so of course the authors ignored it. I’m not saying it’s a real effect, I’m just saying if you get a result (even a non-significant result) that goes in the opposite direction of what you predicted, then you should take that as a cue that you’re missing something.⁴⁰
The NIA young male cohort saw a decrease in bodyweight and body fat while the other three NIA cohorts saw essentially no change. Presumably, the main mechanism of calorie restriction is that it prevents obesity, so we should see a longevity improvement among NIA young males and not among the other three cohorts. But that’s not what we see. Instead we see essentially no effect in old males/females and a negative effect in young males/females.
In the NIA study, why did calorie restriction reduce average bodyweight for males but not for females (Figure 2)? The authors took this as evidence of “sexual dimorphism in the relationship between food intake and bodyweight”. That does not sound plausible to me.
According to the authors: “In rodents, early onset CR is more effective in extending longevity than adult onset CR. For nonhuman primates it appears that CR, while beneficial when implemented in adulthood, does not improve survival when implemented in juveniles.” It’s suspicious that rodent studies and monkey studies produced opposite results in this respect.

The NIA control group monkeys ate about as much as the UW calorie-restricted monkeys, which the authors take as evidence in favor of the original hypothesis—lower caloric intake improves longevity. Maybe.⁴¹ Or maybe it was the higher-quality diet (irrespective of caloric intake), or some other difference between the two studies.⁴²

If two studies conflict, I’m wary of making my hypothesis more complicated to fit the results. The hypothesis started as “CR improves longevity”, which the UW study supported. But when the NIA study produced contradictory evidence, the hypothesis became “CR improves longevity, unless you’re eating a healthy diet, in which case it doesn’t”.

The two studies together support this hypothesis, but they don’t distinguish it from other plausible hypotheses (see previous footnote, repeated here⁴²).

And if you look closely at the NIA results, the hypothesis needs to become even more complicated: something like “CR improves longevity, but if you’re eating a healthy diet then it has no effect for old monkeys and a harmful effect for young monkeys, and the mechanism is presumably that monkeys with healthy diets tend to have lower rates of obesity, except that young males in the NIA study didn’t weigh much less than in UW so for young males the mechanism is some other thing, and also the young vs. old effect is reversed in rodents for some reason.”

The more complicated a hypothesis, the more supporting evidence it requires.

More broadly, I’m skeptical that CR studies on lab animals generalize to the real world. Peter Attia agrees:

CR’s usefulness remains doubtful outside the lab; very lean animals may be more susceptible to death from infection or cold temperatures. […] Furthermore, there is no evidence that extreme CR would truly maximize the longevity function in an organism as complex as we humans, who live in a more variable environment than the animals [studied]. While it seems likely that it would reduce the risk of succumbing to at least some [chronic diseases], it seems equally likely that the uptick in mortality due to infections, trauma, and frailty might offset these gains. (page 81)

Another fact that seems important to me, but that Attia and the UW/NIA authors didn’t discuss: the monkeys never exercised. They were permanently housed in small cages (Mattison et al. (2005)⁴³). Some of the the UW monkeys did participate in a different study⁴⁴ measuring their physical activity which moved them to a metabolic chamber, but the metabolic chamber was about the same size as the cages—i.e., too small for meaningful exercise.

Physical activity appears to largely or fully cancel out the harms of higher-calorie diets.⁴⁵ Even if calorie restriction works for sedentary people, it’s less likely to improve health for folks who exercise regularly.

Returning to the original hypothesis—”CR improves longevity but only if you eat a fairly unhealthy diet”—these studies provide a small amount of evidence for the hypothesis but not much. That said, the hypothesis sounds correct to me:

People with unhealthy diets tend to overeat, so eating less would probably improve their health.
People who get most of their calories from healthy sources (whole grains, nuts, etc.) are much less likely to overeat, so there’s no point in calorie restriction.

The data are unclear on whether reducing saturated fat intake is beneficial

(Note: SFA = saturated fatty acid, MUFA = monounsaturated fatty acid, PUFA = polyunsaturated fatty acid)

A more recent publication by the Cochrane Collaboration, published in 2020 as a 287-page treatise titled Reduction in Saturated Fat Intake for Cardiovascular Disease⁴⁶, looked at fifteen RCTs in over fifty-six thousand patients and found, among other things, that “reducing dietary saturated fat reduced the risk of combined cardiovascular events by 17%.” Interesting. But the same review also found “little or no effect of reducing saturated fat on all-cause mortality or cardiovascular mortality.”

[…]

The data are very unclear on this question, at least at the population level. […] [A]ny hope of using broad insights from evidence-based medicine is bound to fail when it comes to nutrition, because such population-level data cannot provide much value at the individual level when the effect sizes are so small, as they clearly are here. (pages 337–338)

And a bonus quote:

If, after reading this chapter, you’re upset because you don’t quite agree with some detail I’ve covered—be it the ratio of MUFA to PUFA to SFA, or the exact bioavailability of soy protein, the role of seed oils and lectins, or the ideal target for average blood glucose levels […], I have one final piece of advice. Stop overthinking nutrition so much. Put the book down. Go outside and exercise. (page 346)

Well joke’s on you, I already exercised today, and now I’m back to over-analyze saturated fat.

My assessment:

Saturated fat is unhealthy in expectation: likely true (credence: 85%).
It’s a good idea for most people to reduce their SFA intake: possible (credence: 50%).⁴⁷
It’s a good idea for people with high cholesterol to reduce their SFA intake: likely true (credence: 70%).
The data are unclear: unclear. (Yes, it’s unclear whether the data are unclear. It depends on how much clarity you want.)
The amount of SFA in your diet doesn’t matter all that much: possible (credence: 50%).

Attia’s position on saturated fat stuck out to me because the mainstream view says saturated fat is unhealthy. After spending much longer on this than I’d originally planned, I’ve come to the conclusion that the mainstream advice is basically reasonable, and Attia’s position is also basically reasonable. There’s some evidence that reducing SFA is beneficial and there’s little evidence to the contrary, but (a) the evidence is only moderately strong at best, and (b) there’s a lot of variation in how SFA affects people, so you might not need to worry about it unless you have high cholesterol.

Observational studies have found mixed results, with the more reliable studies generally finding moderate associations between saturated fat and heart disease. For example, Zong et al. (2016)⁴⁸ found that shifting 1% of daily calories from saturated fat to polyunsaturated fat was associated with an 8% reduction in coronary heart disease (p < 0.001). But observational studies don’t establish causality, so let’s look at randomized controlled trials (RCTs).

A 2020 Cochrane review⁴⁶ of RCTs found that replacing dietary saturated fat significantly reduced cardiovascular events and LDL cholesterol levels, and non-significantly reduced all-cause mortality and cardiovascular mortality. (Some other meta-analyses of RCTs have been done,⁴⁹ but the Cochrane review likely has the strongest methodology.)

It’s not entirely clear how to interpret the results of the Cochrane review. Out of eight primary outcome variables (including all-cause mortality, cardiovascular mortality, etc.), reducing SFA only statistically significantly improved one variable. But it showed a positive effect for all eight. This weakly suggests that there’s a real effect and the RCTs were underpowered for most of the measures. If we treated the eight measures as independent, this would constitute strong evidence of a real effect, but the measures are mostly correlated with each other.

The Cochrane review looked at dozens of other outcome variables. Most importantly, replacing saturated fat significantly reduced LDL cholesterol. (A WHO (2016)⁵⁰ meta-analysis of 84 RCTs agreed with this result.) Cholesterol-lowering drugs have been shown to lower all-cause mortality (see Navarese et al. (2018)⁵¹ and Ennezat et al. (2022)⁵²), so it stands to reason—although we only have weak direct evidence—that if reducing SFA improves LDL cholesterol, then it should improve all-cause mortality. Somewhat contradicting this, an RCT by Bergeron et al. (2019)⁵³ found that SFA caused the body to produce mainly larger LDL particles, which are less harmful than small particles.

Sacks et al. (2017)⁵⁴ performed a review with stricter inclusion criteria, looking only at RCTs that:

controlled subjects’ dietary intake;
lasted at least 2 years;
proved adherence by measuring biomarkers like cholesterol;
did not replace saturated fats with trans fats in the intervention group.

Only four trials met these criteria. Three of the four were included in the Cochrane review; the fourth, the Finnish mental hospital study⁵⁵, was excluded for using a cluster-randomized design instead of full randomization. All four of these studies supported the hypothesis that reducing SFA improves cardiovascular health, and they had a weighted average relative risk of 0.71 (95% CI 0.62–0.81, see Figure 2).

Two of the studies in the Cochrane review showed an increase in cardiovascular events when reducing SFA:

The Sydney Diet Heart study⁵⁶ had some study participants replace saturated fat with trans-fat-heavy margarine, which likely explains the increase in bad outcomes.
The Rose study⁵⁷ did not have any glaring problems like Sydney Diet Heart, but it lasted for less than 2 years and only had a total of 54 participants. I take it as valid but weak contradictory evidence.

It looks reasonably likely, but not conclusive, that saturated fat is unhealthy. How unhealthy?

The Cochrane review suggests that an intervention to reduce dietary SFA should prevent one cardiovascular event per 290 person-years and one death per 2300 person-years for the sort of people who participated in these trials (i.e., people with elevated baseline risk of cardiovascular disease).⁵⁸ Compare to exercise, which is associated with a reduction of about 1 death per 300 person-years in individuals with chronic diseases.⁵⁹

It’s not clear how to estimate the improvement in mortality for the general population. Participants in these trials died at approximately the normal rate (compare to US CDC mortality statistics) which suggests the effect should be similar, but it makes sense in theory that dietary interventions should have larger effects on unhealthy populations.

People should take omega-3 supplements

There is some evidence that supplementation with the omega-3 fatty acid DHA, found in fish oil, may help maintain brain health[.] (page 200)

[U]nless they are eating a lot of fatty fish, filling their coffers with marine omega-3 [fatty acids], [my patients] almost always need to take EPA and DHA supplements in capsule or oil form. (page 339)

My assessment:

Omega-3 fatty acids improve brain health: likely true (credence: 75%).
Omega-3s improve health in general: likely true (credence: 80%).
It’s a good idea for most people to take omega-3 supplements: somewhat likely (credence: 65%).

A meta-analysis of RCTs by Dighriri et al. (2022)⁶⁰ found that “[c]onsumption of omega-3 improved learning, memory ability, cognitive well-being, and blood flow in the brain.”

A Cochrane review⁶¹ of RCTs found no statistically significant effect of omega-3 consumption on all-cause mortality, cardiovascular events, stroke, or arrhythmia, and a weakly statistically significant effect on cardiovascular mortality, coronary heart disease mortality, and coronary heart disease events. The non-significant effects were all positive, except for a small negative effect on stroke.⁶²

So omega-3s probably improve brain health, and they might have a small effect on heart health but it’s unclear.

As far as we can tell, there are no significant downsides to dietary omega-3s, so they easily pass a cost-benefit analysis as long as you don’t mind eating omega-3-rich foods.

The cost-benefit analysis for omega-3 supplementation is a bit murkier because supplements sometimes contain contaminants.⁶³⁶⁴ Raab et al. (2016)⁶⁵ tested 67 supplements and found that all had safe levels of heavy metals, but did not test for mercury. Winwood (2013)⁶⁶ claims that algae oil typically comes from algae grown in tanks and thus can’t be contaminated by heavy metals in the ocean.

I personally take a daily algae oil supplement. I take algae oil instead of fish oil because I’m vegan, but the potentially reduced risk of contaminants is a nice bonus.

Sleep

I read this chapter more skeptically than the others because it quoted Matthew Walker near the beginning. This raised some alarm bells because Walker wrote a bad book about sleep and has been caught manipulating data.⁶⁷ Outlive cited a few of Walker’s papers to support certain claims, but none of those claims seemed particularly important so I didn’t review them.⁶⁸

Every animal sleeps

Every animal engages in some form of sleep; scientists have found no exceptions, so far. (page 353)

My assessment:

Every animal sleeps: somewhat unlikely (credence: 35%).
It’s reasonable to assert that every animal sleeps: false (credence: 20%).

In support of this claim, Outlive cites Cirelli & Tononi (2008)⁶⁹, which does not take a strong stance on whether all animals sleep:

Only a small number of species—mostly mammals and birds—have been evaluated in detail with respect to sleep. Most studies found signs of sleep, both behavioral (quiescence and hyporesponsivity) and electrophysiological (e.g., the slow waves of non-rapid eye movement (NREM) sleep). Scientists have been hesitant to attribute sleep to reptiles, amphibians, fish, and especially invertebrates, preferring the noncommittal term “rest” in the absence of electrophysiological signs resembling those of mammals and birds.

Cirelli & Tononi (2008) references some examples of animals that have been claimed not to sleep (particularly bullfrogs), but says the evidence is weak.

I don’t think Cirelli & Tononi supports Attia’s claim; it would be more accurate to say “no animal has been proven not to sleep”.

But other sources would disagree with this. For example:

It now appears that many species reduce sleep for long periods of time under normal conditions and that others do not sleep at all, in the way sleep is conventionally defined.

From Kushida, C. (2013). Encyclopedia of Sleep, Volume 1, page 38 (h/t Alexey Guzey).

Attia’s claim “scientists have found no exceptions” is sort of true in the sense that we haven’t found any definitive exceptions, but the claim “every animal engages in some form of sleep” isn’t well-established either.

We need to sleep 7.5 to 8.5 hours a night

[M]any, many studies have confirmed what your mother told you: We need to sleep about seven and a half to eight and a half hours a night. (page 354)

My assessment: false (credence: 10%).

(I could not find a citation for the quoted assertion.)

The most authoritative source on this question appears to be the National Sleep Foundation panel⁷⁰, where sleep scientists were surveyed on their beliefs. The median panelist believed that 7 to 9 hours a night is “appropriate” for adults age 25–64, and 6 to 10 hours “may be appropriate for some people” in the same age range.⁷¹

The range given by Outlive (7.5 to 8.5 hours) is excessively narrow—according to sleep scientists, many people can/should sleep more or less than that.

My subjective uncertainty on this question mostly comes from the fact that I haven’t read any studies or even any meta-analyses and I’m only 90% confident that the National Sleep Foundation panelists know what they’re talking about.

Basketball players who were told to sleep for 10 hours a night had better shooting accuracy

In one study, Stanford basketball players were encouraged to strive for ten hours of sleep per day, with or without naps, and to abstain from alcohol or caffeine. After five weeks, their shooting accuracy had improved by 9 percent, and their sprint times had also gotten faster. (page 354)

I didn’t look into this study and I’m not giving a credence because I don’t really care about this particular claim. I bring it up because it contradicts the preceding claim that people should sleep 7.5 to 8.5 hours a night.

Lack of sleep increases obesity and diabetes risk

Even in the short term, sleep deprivation can cause profound insulin resistance. […] Multiple large meta-analyses of sleep studies have revealed a close relationship between sleep duration and risk of type 2 diabetes and the metabolic syndrome. (page 356)

My assessment:

Sleep deprivation increases the risk of insulin resistance: highly likely (credence: 90%).
Observational studies find relationships between short sleep duration and obesity/diabetes/metabolic syndrome: true (credence: 95%).
Lack of sleep increases obesity and diabetes risk: highly likely (credence: 85%).

I only briefly investigated this but it passes a basic sanity check. I glanced at the papers cited by Outlive and they appear to support the quoted text. In addition, RCTs suggest that sleep restriction causes subjects to eat more and increases insulin resistance (a precursor to diabetes)—see meta-analysis by Reutrakul & Van Cauter (2018)⁷².

A study using Mendelian randomization found that sleeping <6 hours a night increased risk of a heart attack

[O]ne particularly interesting study compared observational and Mendelian randomization⁷³ data in people with previous identified genetic variants that either increase or decrease their lifelong exposure to longer or shorter sleep duration. The MR data confirmed the observational findings, that sleeping less than six hours a night was associated with about a 20 percent higher risk of a heart attack. (page 359)

My assessment:

A study found that sleeping <6 hours a night increases heart attack risk: true (credence: 98%).
Sleeping <6 hours a night increases heart disease risk for most people: likely true (credence: 80%).
This quote from the book does a good job of representing the state of the evidence: false (credence: 10%).

In support of this quote, Attia cites Dashti et al. (2019)⁷⁴, which does not appear to say anything about heart attacks. I believe he meant to cite Daghlas et al. (2019)⁷⁵ (on which Dashti is a co-author). Daghlas et al. (2019) supports Attia’s claim.

However, other Mendelian randomization studies have gotten different results. Zhuang et al. (2020)⁷⁶ found no significant relationship between sleep duration and coronary heart disease; Yang et al. (2022)⁷⁷ found a statistically significant but extremely weak (“probably not clinically relevant”) association⁷⁸; while Liao et al. (2020)⁷⁹ broadly agreed with Daghlas et al. (2019).

Out of four Mendelian randomization studies, two identified strong links between short sleep and heart disease/heart attack risk, and two suggested little to no effect. So while Outlive does accurately describe the results of a study, it misrepresents the evidence by ignoring the studies that contradict its thesis.

(I did not look into the quality of these studies, I just read their conclusions. It’s possible that the null-result studies are flawed in some way.)

The Mendelian randomization studies provide only weak to moderate evidence, but when combined with other evidence (such as the link to obesity discussed in the previous section), it appears reasonably likely that short sleep duration does indeed increase the risk of heart problems.

Lack of sleep causes Alzheimer’s disease

Subsequent research … has pointed to chronic bad sleep as a powerful potential cause of Alzheimer’s disease and dementia. Sleep, it turns out, is as crucial to maintaining brain health as it is to brain function.

My assessment:

Lack of sleep causes Alzheimer’s disease: possibly true (credence: 60%).
The first sentence of the book quote is reasonable: true (credence: 90%).
The second sentence of the book quote is reasonable: false (credence: 20%).

Some research has indeed found a link between bad sleep and Alzheimer’s, but it’s difficult to establish causality—see review article by Lloret et al. (2020)⁸⁰.

The first quoted sentence from Outlive aligns with Lloret et al.’s summary of the literature. The second sentence converts a speculative hypothesis into a certainty.

Bonus

Dunning-Kruger effect

Looking back, I now realize that I was too far on the left of the Dunning-Kruger curve, caricatured below in figure 14—my maximal confidence and relatively minimal knowledge having propelled me quite close to the summit of “Mount Stupid.” (page 293)

The book’s Figure 14 reproduces this image from Wikimedia Commons:

My assessment:

This graph accurately represents the Dunning-Kruger effect: false (credence: <1%).
The existence of a “Mount Stupid” is supported by the scientific evidence: false (credence: 2%).

I’m willing to forgive this mistake because it doesn’t have anything to do with longevity, but it still bugs me.

If you search “Dunning Kruger” on Google Images, you will see a bunch of graphs that look like that, but no study on the Dunning-Kruger effect has ever produced empirical data with that shape.

Empirical results actually look something like this:

(Presumably, Attia found Figure 14 from the Wikipedia page on the Dunning-Kruger effect. I can’t really blame him for getting something wrong if he just pulled it from Wikipedia. And to Wikipedia’s credit, it has removed the incorrect image and replaced it with the correct one above.)

Changelog

2024-10-23: I added a reference to a meta-analysis of RCTs on balance training specificity. This provides stronger evidence for my previously weakly-held position that balance training isn’t useful. I updated my credences accordingly.
2025-05-05: I added three new sections on exercise:
2025-07-03: Previously I was inconsistent with how I reported credences for statements I thought were probably false—sometimes I reported my probability that the statement is true, and sometimes my probability that it’s false. I rewrote them so that all credences are given in terms of my probability that the statement is true.
2025-07-04: Made some minor changes in response to feedback.

Notes

The book is co-authored by Bill Gifford. It’s written from Attia’s point of view and some materials (such as Peter Attia’s website) approximately treat Attia as the sole author, so in my review I will credit the book’s claims to Attia and not to Gifford. ↩
Page numbers are from the 2023 Kindle edition of Outlive, ISBN 9780593236598, ebook ISBN 9780593236604. ↩
The meta-analysis did definitely find this result, but there’s some wiggle room around what “technically correct” means (because the meta-analysis found different results for different subgroups—I will discuss this shortly). So I’m only 90% confident. ↩
Stefan, N., Schick, F., & Häring, H. U. (2017). Causes, Characteristics, and Consequences of Metabolically Unhealthy Normal Weight in Humans. ↩
Kramer, C. K., Zinman, B., and Retnakaran, R (2013). Are metabolically healthy overweight and obesity benign conditions? A systematic review and meta-analysis. ↩
The standard method for significance-testing a relative risk is to assume that its logarithm follows a normal distribution. I did that and got an odds ratio of 7.27. But the sample mean of 1.19 is pretty far from the geometric mean of the 95% CI (1.16), and much closer to its arithmetic mean (1.18), so I redid the test with the assumption that the RR follows a normal distribution. The second method produces an odds ratio of 5.66, which I rounded down to 5 to be conservative.

It’s bad practice to run two different significance tests, but I think it’s okay in this case because I preferred the test that weakened my argument rather than strengthening it. ↩
The original article used PubMed links, which I replaced with DOI links and added full citations in footnotes. The quote is otherwise unchanged. ↩
Lin, H., Zhang, L., Zheng, R., & Zheng, Y. (2017). The prevalence, metabolic risk and effects of lifestyle intervention for metabolically healthy obesity. ↩
Eckel, N., Meidtner, K., Kalle-Uhlmann, T., Stefan, N., and Schulze, M. B (2016). Metabolically healthy obesity and cardiovascular events: a systematic review and meta-analysis. ↩
Kim, T. J., Shin, H. Y., Chang, Y., Kang, M., Jee, J., Choi, Y. H., Ahn, H. S., Ahn, S. H., Son, H. J., & Ryu, S. (2017). Metabolically healthy obesity and the risk for subclinical atherosclerosis. ↩
Chang, Y., Jung, H. S., Cho, J., Zhang, Y., Yun, K. E., Lazo, M., Pastor-Barriuso, R., Ahn, J., Kim, C. W., Rampal, S., Cainzos-Achirica, M., Zhao, D., Chung, E. C., Shin, H., Guallar, E., & Ryu, S. (2016). Metabolically Healthy Obesity and the Development of Nonalcoholic Fatty Liver Disease. ↩
Chang, A. R., Surapaneni, A., Kirchner, H. L., Young, A., Kramer, H. J., Carey, D. J., Appel, L. J., & Grams, M. E. (2018). Metabolically Healthy Obesity and Risk of Kidney Function Decline. ↩
Bell, J. A., Kivimaki, M., & Hamer, M. (2014). Metabolically healthy obesity and risk of incident type 2 diabetes: a meta‐analysis of prospective cohort studies. ↩
When Attia says the amyloid theory was weakening, he’s referring to the once-popular hypothesis that amyloid-beta is the sole cause of Alzheimer’s. That hypothesis now appears to be false, but amyloid-beta still looks relevant to Alzheimer’s somehow (it’s not quite clear how). ↩
Actually, I have very good reading comprehension in a relative sense—in the 98th or 99th percentile according to standardized tests. But 98th percentile reading comprehension still isn’t good enough to consistently understand the things you read, apparently. ↩
I’m not confident in the claim that they’re more accurate. I am not aware of any research directly comparing the predictive power of VO2max itself vs. performance tests. In theory, I would expect performance tests to be better predictors because they’re directly measuring your body’s physical capabilities. ↩
Some special-purpose interventions might work better. For example, if you’re a heavy smoker, quitting smoking might have a bigger effect than starting exercise. ↩
Kokkinos, P., Faselis, C., Samuel, I. B. H., Pittaras, A., Doumas, M., Murphy, R., Heimall, M. S. et al. (2022). Cardiorespiratory Fitness and Mortality Risk Across the Spectra of Age, Race, and Sex. ↩
Mandsager, K., Harb, S., Cremer, P., Phelan, D., Nissen, S. E., & Jaber, W. (2018). Association of Cardiorespiratory Fitness With Long-term Mortality Among Adults Undergoing Exercise Treadmill Testing. ↩
Harber, M. P., Kaminsky, L. A., Arena, R., Blair, S. N., Franklin, B. A., Myers, J., & Ross, R. (2017). Impact of Cardiorespiratory Fitness on All-Cause and Disease-Specific Mortality: Advances Since 2009. ↩
Wen, D., Utesch, T., Wu, J., Robertson, S., Liu, J., Hu, G., & Chen, H. (2019). Effects of different protocols of high intensity interval training for VO2max improvements in adults: A meta-analysis of randomised controlled trials. ↩ ↩² ↩³
Seiler, S. (2010). What is Best Practice for Training Intensity and Duration Distribution in Endurance Athletes?. ↩ ↩² ↩³
Billat, L. V. (2001). Interval Training for Performance: A Scientific and Empirical Practice. ↩
Billat, V., Renoux, J. C., Pinoteau, J., Petit, B., & Koralsztein, J. P. (1995). Times to exhaustion at 90,100 and 105% of velocity at V̇O₂max (Maximal aerobic speed) and critical speed in elite longdistance runners. ↩
Blondel et al. (2001)⁸¹ found that a sample of physically active (but not elite) students could sustain 90% of VO2max for an average of 13.98 minutes. ↩
Different people use terminology in different ways. I am using LIT to refer to what Attia calls zone 2. Some studies call it moderate-intensity continuous training (MICT). Colloquially, it refers to an exercise intensity that you can sustain for a long time. Technically, it refers to exercise at or below the lactate threshold. ↩
Crowley, E., Powell, C., Carson, B. P., & W. Davies, R. (2022). The Effect of Exercise Training Intensity on VO2max in Healthy Adults: An Overview of Systematic Reviews and Meta-Analyses. ↩
It may be possible to resolve this inconsistency by digging deeper into the literature. Some relevant questions:
1. Wen et al. (2019)²¹ found that long-duration HIIT worked better than short-duration. For the studies that find no benefit to HIIT over LIT, are they only looking at short-duration HIIT?
2. Some RCTs match volume between groups. So if the HIIT group spends a total of (say) 16 minutes at high intensity, then the LIT group exercises for 16 minutes total. That’s not how people actually exercise. Do meta-analyses understate the benefits of LIT because they include volume-matched studies?
3. How do the returns to HIIT vs. LIT differ for novice vs. experienced athletes?
4. What happens when you combine HIIT with LIT?
↩
Mølmen, K. S., Almquist, N. W., & Skattebo, Ø. (2024). Effects of Exercise Training on Mitochondrial and Capillary Growth in Human Skeletal Muscle: A Systematic Review and Meta-Regression. ↩
Stöggl, T. L., & Sperlich, B. (2015). The training intensity distribution among well-trained and elite endurance athletes. ↩ ↩²
Frank, C., Kobesova, A., and Kolar, P (2013). Dynamic neuromuscular stabilization & sports rehabilitation. ↩
I’m thinking in particular of Chiu (2023)⁸² which investigated knee valgus, a squat technique that was generally regarded as bad form, and found that it may be better than “correct” form. h/t Menno Henselmans. ↩
Sherrington, C., Tiedemann, A., Fairhall, N., Close, J. C. T., & Lord, S. R. (2011). Exercise to prevent falls in older adults: an updated meta-analysis and best practice recommendations.. doi: 10.1071/nb10056 ↩
de Souto Barreto, P., Rolland, Y., Vellas, B., & Maltais, M. (2019). Association of Long-term Exercise Training With Risk of Falls, Fractures, Hospitalizations, and Mortality in Older Adults. ↩
Kümmel, J., Kramer, A., Giboin, L. S., & Gruber, M. (2016). Specificity of Balance Training in Healthy Individuals: A Systematic Review and Meta-Analysis.. doi: 10.1007/s40279-016-0515-z ↩
Colman, R. J., Anderson, R. M., Johnson, S. C., Kastman, E. K., Kosmatka, K. J., Beasley, T. M., Allison, D. B., Cruzen, C., Simmons, H. A., Kemnitz, J. W., & Weindruch, R. (2009). Caloric Restriction Delays Disease Onset and Mortality in Rhesus Monkeys. ↩
Mattison, J. A., Roth, G. S., Beasley, T. M., Tilmont, E. M., Handy, A. M., Herbert, R. L., Longo, D. L., Allison, D. B., Young, J. E., Bryant, M., Barnard, D., Ward, W. F., Qi, W., Ingram, D. K., & de Cabo, R. (2012). Impact of caloric restriction on health and survival in rhesus monkeys from the NIA study. ↩
Mattison, J. A., Colman, R. J., Beasley, T. M., Allison, D. B., Kemnitz, J. W., Roth, G. S., Ingram, D. K., Weindruch, R., de Cabo, R., & Anderson, R. M. (2017). Caloric restriction improves health and survival of rhesus monkeys. ↩
Actually, the NIA young female cohort did see a statistically significant reduction in lifespan (p = 0.04), but it becomes non-significant if you do a Bonferroni correction. ↩

Mean and 95% confidence intervals for change in longevity from various study cohorts:

Cohort	Mean	95% CI
UW male	1.58	(-1.56, 4.72)
UW female	2.22	(-2.74, 7.18)
NIA young male	-2.29	(-7.05, 2.47)
NIA young female	-4.79	(-8.88, -0.70)

The CI for NIA young male contains the mean for UW male but the reverse is not true, and neither female cohort’s CI contains the mean of the other. ↩

If true, we would expect to find that calorie restriction worked for the NIA young male cohort because they ate about as much as their UW counterparts. But it didn’t work (in fact it shortened the average lifespan). ↩
I can think of several other hypotheses.

The UW control group ate ad libitum, which is fancy academic language for “as much as they want”. The NIA control group didn’t eat ad libitum. Instead, the researchers used previous data to determine how much monkeys tend to eat when fed ad libitum (controlling for age and bodyweight) and then fed subjects exactly that amount.

This brings to mind another hypothesis: Maybe some portion (let’s say 1/3) of monkeys tend to overeat, which causes health problems, and calorie restriction mainly only benefits the 1/3 who overeat. If the NIA control monkeys all received diets based on how much the average monkey eats, that prevents the most gluttonous 1/3 from overeating, so additional calorie restriction doesn’t produce meaningful benefits.

You could test this hypothesis using the UW data by dividing monkeys in the calorie restriction cohort into “high-calorie” and “low-calorie” group based on how much they ate ad libitum (controlling for age and bodyweight) and seeing if the high-calorie group had a bigger improvement in longevity than the low-calorie group. The groups would have small sample sizes so the result probably wouldn’t be statistically significant.

Some weak supporting evidence: in the UW study, the median longevity improvement was bigger than the mean improvement.

A fourth hypothesis: calorie restriction has a U-shaped effect on longevity, where a little calorie restriction helps, but excess calorie restriction increases mortality. (This is clearly true in the limit—100% calorie restriction certainly isn’t healthy.)

The studies weakly contradict this hypothesis. Figure 4 shows that NIA young males in the control group ate about as much as UW males, while in the other three control group pairings (NIA young female + UW female, NIA old male + UW male, NIA old female + UW female), the NIA group ate less. This predicts that calorie restriction should improve longevity in the NIA young male cohort but have a smaller or negative effect in the other three cohorts. But that’s not what the NIA study found. Instead, the young male and young female cohorts both saw decreased longevity from calorie restriction, and both old cohorts saw approximately no effect.

A fifth hypothesis: the studies have fundamental methodological issues that render the results invalid. The studies weakly support this hypothesis given how many peculiar and seemingly-contradictory findings I was able to identify.

I don’t know what those methodological issues might be. They could be things like:
- the different cohorts were managed by different researchers who used inconsistent procedures
- the cohorts had relevantly different genetic lineages
- the researchers fabricated data (probably not, but you never know)
- there was a mold infestation next to the control group’s cages
A sixth hypothesis: calorie restriction works, but only if you live in Wisconsin. ↩ ↩²
Mattison, J. A., Black, A., Huck, J., Moscrip, T., Handy, A., Tilmont, E., Roth, G. S., Lane, M. A., & Ingram, D. K. (2005). Age-related decline in caloric intake and motivation for food in rhesus monkeys. ↩
Yamada, Y., Colman, R. J., Kemnitz, J. W., Baum, S. T., Anderson, R. M., Weindruch, R., & Schoeller, D. A. (2013). Long-term calorie restriction decreases metabolic cost of movement and prevents decrease of physical activity during aging in rhesus monkeys. ↩
Ortega et al. (2016)⁸³ summarizes the relevant literature. Several observational studies have found that overweight but physically fit individuals have little to no increase in mortality rates relative to normal-weight fit individuals (the largest study⁸⁴ found a statistically significant but small effect (RR = 1.1); three other studies found no significant effect). By comparison, unfit people had 2–3x higher mortality than fit individuals. Figure 2 (reproduced below) summarizes the results from the four studies.

This finding from Ortega et al. (2016) is actually stronger than necessary for our purposes. It would be sufficient to say that exercise cancels out the harm of high-calorie diets by burning off the excess calories. But this shows that exercise (mostly) cancels out the harm even for people who don’t lose weight. ↩
Hooper, L., Martin, N., Jimoh, O. F., Kirk, C., Foster, E., & Abdelhamid, A. S. (2020). Reduction in saturated fat intake for cardiovascular disease. ↩ ↩²
Arguably, my credence for this latter claim should be higher than for the former claim, because reducing saturated fat has some chance of improving health and essentially no chance of harming health. But reducing saturated fat also has some costs (it makes your diet harder to follow).

In other words, on a cost-benefit analysis aimed at maximizing health, it’s clearly worth it to eat less saturated fat. But on an all-things-considered cost-benefit analysis, there’s more room for debate. ↩
Zong, G., Li, Y., Wanders, A. J., Alssema, M., Zock, P. L., Willett, W. C., Hu, F. B. et al. (2016). Intake of individual saturated fatty acids and risk of coronary heart disease in US men and women: two prospective longitudinal cohort studies. ↩
Heileson, J. L. (2019). Dietary saturated fat and heart disease: a narrative review. ↩
Mensink, R. P., & World Health Organization (2016). Effects of saturated fatty acids on serum lipids and lipoproteins: a systematic review and regression analysis. ↩
Navarese, E. P., Robinson, J. G., Kowalewski, M., Kolodziejczak, M., Andreotti, F., Bliden, K., Tantry, U. et al. (2018). Association Between Baseline LDL-C Level and Total and Cardiovascular Mortality After LDL-C Lowering. ↩
Ennezat, P. V., Guerbaai, R. A., Maréchaux, S., Le Jemtel, T. H., & François, P. (2022). Extent of LDL-cholesterol Reduction and All-cause and Cardiovascular Mortality Benefit: a Systematic Review and Meta-analysis. ↩
Bergeron, N., Chiu, S., Williams, P. T., M King, S., & Krauss, R. M. (2019). Effects of red meat, white meat, and nonmeat protein sources on atherogenic lipoprotein measures in the context of low compared with high saturated fat intake: a randomized controlled trial. ↩
Sacks, F. M., Lichtenstein, A. H., Wu, J. H. Y., Appel, L. J., Creager, M. A., Kris-Etherton, P. M., Miller, M. et al. (2017). Dietary Fats and Cardiovascular Disease: A Presidential Advisory From the American Heart Association.. doi: 10.1161/cir.0000000000000510 ↩
Miettinen, M., Karvonen, M., Turpeinen, O., Elosuo, R., & Paavilainen, E. (1972). EFFECT OF CHOLESTEROL-LOWERING DIET ON MORTALITY FROM CORONARY HEART-DISEASE AND OTHER CAUSES. ↩
Ramsden, C. E., Zamora, D., Leelarthaepin, B., Majchrzak-Hong, S. F., Faurot, K. R., Suchindran, C. M., Ringel, A. et al. (2013). Use of dietary linoleic acid for secondary prevention of coronary heart disease and death: evaluation of recovered data from the Sydney Diet Heart Study and updated meta-analysis. ↩
Rose, G. A., Thomson, W. B., & Williams, R. T. (1965). Corn Oil in Treatment of Ischaemic Heart Disease. ↩
The Cochrane review summary of findings reports that the interventions prevented 2 deaths per 1000 participants and the interventions lasted an average of 56 months, which equates to one death per 2300 pers ↩
See Arem et al. (2015)⁸⁵ and Lee et al. (2022)⁸⁶.

These were pooled analyses of cohort studies, not RCTs. I could not quickly find reliable RCT numbers.

Neither of these studies reported deaths prevented per person-year. I calculated the numbers using provided relative risks multiplied by number of deaths per person, divided by follow-up time. Number of years per death prevented varied based on exercise duration and intensity. I found that exercise prevented approximately one death per 300 years when defining “exercise” as 7.5+ MET-hours/week for Arem et al. (2015) and 150–224 minutes of moderate physical activity for Lee et al. (2022) (these two definitions are roughly equivalent).

There are a number of meta-analyses of RCTs, for example:
- Lawler et al. (2011)⁸⁷ found exercise had a relative risk (RR) of 0.74 on all-cause mortality for individuals who had experienced heart attacks.
- Morishita et al. (2020)⁸⁸ found RR = 0.76 on cancer mortality for cancer patients.
However, neither of these meta-analyses provided per-group mortality numbers or mean intervention length, so I can’t determine the number of years per death prevented without reading through every individual study. Based on the RRs, my guess is that these meta-analyses would give roughly similar numbers to the pooled analyses above.

To my knowledge, the most comprehensive meta-analysis is Posadzki et al. (2020)⁸⁹, which reviewed 150 different Cochrane reviews and found an RR of 0.87 for all-cause mortality. But it provides even less information about the participants so I have no way of interpreting this number. ↩
Dighriri, I. M., Alsubaie, A. M., Hakami, F. M., Hamithi, D. M., Alshekh, M. M., Khobrani, F. A., Dalak, F. E. et al. (2022). Effects of Omega-3 Polyunsaturated Fatty Acids on Brain Functions: A Systematic Review. ↩
Abdelhamid, A. S., Brown, T. J., Brainard, J. S., Biswas, P., Thorpe, G. C., Moore, H. J., Deane, K. H. et al. (2020). Omega-3 fatty acids for the primary and secondary prevention of cardiovascular disease. ↩
When talking about the calorie restriction studies, I said that a negative effect is a red flag even if it’s non-significant. In this case I’m not too concerned because:
- The effect on stroke was highly non-significant (p = 0.82). Compare to the NIA calorie restriction study which had p = 0.35 and p = 0.02 for young males and young females respectively.
- RCTs show omega-3s improve short-term brain function.
I think the most reasonable interpretation is that there’s no effect on stroke. ↩
Jacobs, M. N., Covaci, A., Gheorghe, A., & Schepens, P. (2004). Time Trend Investigation of PCBs, PBDEs, and Organochlorine Pesticides in Selected n−3 Polyunsaturated Fatty Acid Rich Dietary Fish Oil and Vegetable Oil Supplements; Nutritional Relevance for Human Essential n−3 Fatty Acid Requirements. ↩
Fernandes, A. R., Rose, M., White, S., Mortimer, D. N., & Gem, M. (2006). Dioxins and polychlorinated biphenyls (PCBs) in fish oil dietary supplements: Occurrence and human exposure in the UK. ↩
Raab, A., Stiboller, M., Gajdosechova, Z., Nelson, J., & Feldmann, J. (2016). Element content and daily intake from dietary supplements (nutraceuticals) based on algae, garlic, yeast fish and krill oils—Should consumers be worried?. ↩
Winwood, R. J. (2013). Algal oil as a source of omega-3 fatty acids. ↩
See also Walker’s response on why he presented the data the way he did. ↩
For example, the book cites Goldstein-Piekarski et al. (2015)⁹⁰ (Walker being a co-author) for the passage “When we are deprived of REM, studies have found, we have a more difficult time reading others’ facial expressions.” I was skeptical of this statement even before knowing Walker co-authored the paper because it has the vibe of the sort of fun quirky result that doesn’t survive the replication crisis. But I don’t particularly care about this claim (it’s not actionable in any way) so I didn’t bother to investigate it.

(Also, this is a nitpick but the quoted passage says “studies have found” while only citing a single study.) ↩
Cirelli, C., & Tononi, G. (2008). Is Sleep Essential? ↩
Hirshkowitz, M., Whiton, K., Albert, S. M., Alessi, C., Bruni, O., DonCarlos, L., Hazen, N., Herman, J., Katz, E. S., Kheirandish-Gozal, L., Neubauer, D. N., O’Donnell, A. E., Ohayon, M., Peever, J., Rawding, R., Sachdeva, R. C., Setters, B., Vitiello, M. V., Ware, J. C., & Adams Hillard, P. J. (2015). National Sleep Foundation’s sleep time duration recommendations: methodology and results summary. ↩

They gave different recommended sleep durations for different age ranges. This table reproduces all the recommendations for individuals age 6 and up (since I assume nobody under 6 is reading this):

Age	Recommended	May be appropriate
6–13 y	9 to 11	7 to 12
14–17 y	8 to 10	7 to 11
18–25 y	7 to 9	6 to 11
26–64 y	7 to 9	6 to 10
>64 y	7 to 8	5 to 9

↩

Reutrakul, S., & Van Cauter, E. (2018). Sleep influences on obesity, insulin resistance, and risk of type 2 diabetes. ↩
Mendelian randomization is a technique for conducting a natural experiment. Instead of looking at heart attack risk as a function of sleep duration, you look at heart attack risk as a function of genes that determine sleep duration. The idea is that some confounding environmental variable might cause both shortened sleep and increased heart attack risk, but it can’t change subjects’ genes, so any observed relationship between genetic sleep duration and heart attack risk is probably causal.

I don’t have a strong opinion on how useful this technique is. ↩
Dashti, H. S., Jones, S. E., Wood, A. R., Lane, J. M., van Hees, V. T., Wang, H., Rhodes, J. A. et al. (2019). Genome-wide association study identifies genetic loci for self-reported habitual sleep duration supported by accelerometer-derived estimates. ↩
Daghlas, I., Dashti, H. S., Lane, J., Aragam, K. G., Rutter, M. K., Saxena, R., & Vetter, C. (2019). Sleep Duration and Myocardial Infarction. ↩
Zhuang, Z., Gao, M., Yang, R., Li, N., Liu, Z., Cao, W., & Huang, T. (2020). Association of physical activity, sedentary behaviours and sleep duration with cardiovascular diseases and lipid profiles: a Mendelian randomization analysis. ↩
Yang, Y., Fan, J., Shi, X., Wang, Y., Yang, C., Lian, J., Wang, N. et al. (2022). Causal associations between sleep traits and four cardiac diseases: a Mendelian randomization study. ↩
This study found about a 1% increased risk from short sleep, contrasted with Daghlas et al. (2019) which found about a 20% increased risk. ↩
Liao, L. z., Li, W. d., Liu, Y., Li, J. p., Zhuang, X. d., & Liao, X. x. (2020). Causal assessment of sleep on coronary heart disease. ↩
Lloret, M. A., Cervera-Ferri, A., Nepomuceno, M., Monllor, P., Esteve, D., & Lloret, A. (2020). Is Sleep Disruption a Cause or Consequence of Alzheimer’s Disease? Reviewing Its Possible Role as a Biomarker. ↩
Blondel, N., Berthoin, S., Billat, V., & Lensel, G. (2001). Relationship between run times to exhaustion at 90, 100, 120, and 140% of vV O2max and velocity expressed relatively to critical velocity and maximal velocity. ↩
Chiu, L. Z. (2023). “Knees Out” or “Knees In”? Volitional Lateral vs. Medial Hip Rotation During Barbell Squats. ↩
Ortega, F. B., Lavie, C. J., & Blair, S. N. (2016). Obesity and Cardiovascular Disease. ↩
Wei, M., Kampert, J. B., Barlow, C. E., Nichaman, M. Z., Gibbons, L. W., Paffenbarger Jr, R. S., & Blair, S. N (1999). Relationship between low cardiorespiratory fitness and mortality in normal-weight, overweight, and obese men. ↩
Arem, H., Moore, S. C., Patel, A., Hartge, P., Berrington de Gonzalez, A., Visvanathan, K., Campbell, P. T. et al. (2015). Leisure Time Physical Activity and Mortality. ↩
Lee, D. H., Rezende, L. F. M., Joh, H. K., Keum, N., Ferrari, G., Rey-Lopez, J. P., Rimm, E. B. et al. (2022). Long-Term Leisure-Time Physical Activity Intensity and All-Cause and Cause-Specific Mortality: A Prospective Cohort of US Adults. ↩
Lawler, P. R., Filion, K. B., & Eisenberg, M. J. (2011). Efficacy of exercise-based cardiac rehabilitation post–myocardial infarction: A systematic review and meta-analysis of randomized controlled trials. ↩
Morishita, S., Hamaue, Y., Fukushima, T., Tanaka, T., Fu, J. B., & Nakano, J. (2020). Effect of Exercise on Mortality and Recurrence in Patients With Cancer: A Systematic Review and Meta-Analysis. ↩
Posadzki, P., Pieper, D., Bajpai, R., Makaruk, H., Könsgen, N., Neuhaus, A. L., & Semwal, M. (2020). Exercise/physical activity and health outcomes: an overview of Cochrane systematic reviews. ↩
Goldstein-Piekarski, A. N., Greer, S. M., Saletin, J. M., & Walker, M. P. (2015). Sleep Deprivation Impairs the Human Central and Peripheral Nervous System Discrimination of Social Threat. ↩

I Have Whatever the Opposite of a Placebo Effect Is

Mon, 02 Sep 2024 00:00:00 -0700

Two personal stories:

A story about caffeine

When I first started working a full-time job, I started tracking my daily (subjective) productivity along with a number of variables that I thought might be relevant, like whether I exercised that morning or whether I took caffeine. I couldn’t perceive any differences in productivity based on any of the variables.

After collecting about a year of data, I ran a regression. I found that most variables had no noticeable effect, but caffeine had a huge effect—it increased my subjective productivity by about 20 percentage points, or an extra ~1.5 productive hours per day. Somehow I never noticed this enormous effect. Whatever the opposite of a placebo effect is, that’s what I had: caffeine had a large effect, but I thought it had no effect.

A story about sleep

People always say that exercise helps them sleep better. I thought it didn’t work for me. When I do cardio, even like two hours of cardio, I don’t feel more tired in the evening and I don’t fall asleep (noticeably) faster.

Yesterday, I decided to test this. I wrote a script to predict how long I slept based on how many calories my phone says I burned. The idea is that if I sleep less, that probably means I didn’t need as much because my sleep was higher quality. (I almost always wake up naturally without an alarm.)

Well, turns out exercise does help. For every 500 calories burned (which is about what I burn during a normal cardio session), I sleep 25 minutes less. Once again, exercise had a huge effect, and I thought it didn’t do anything.

I guess I’m not very observant.

Protein Quality (DIAAS) Calculator

Thu, 29 Aug 2024 00:00:00 -0700

Update 2025-01-17: I discovered another protein quality calculator that’s much more comprehensive than mine: https://www.diaas-calculator.com/

You may know that complete proteins are good because they contain every essential amino acid. But you might not know that that’s not the full story.

Take wheat. Wheat is a complete protein—it contains all nine essential amino acids. But it has a problem. Wheat only contains 27mg of lysine (an essential amino acid) per gram of protein, whereas the Food and Agriculture Organization recommends 48mg of lysine per gram. To make full use of a gram of protein, your body needs to get those 48mg. It doesn’t matter that wheat has lots of other essential amino acids. Once your body uses up all the lysine, it can’t make good use of the other amino acids in wheat protein.

You can evaluate the protein quality of a food using the Digestible Indispensable Amino Acid Score (DIAAS). This score determines the quality of a source of protein based on which essential amino acid will run out first, adjusted for digestibility. A score of 100 means the protein has plenty of every essential amino acid.

Sometimes you can improve the protein quality of your food by mixing different ingredients. Wheat has a DIAAS of 57 because it only has 57% as much lysine per gram as your body needs. Peas have a score of 82 because they don’t have enough methionine + cysteine. But peas have 131% of the lysine requirement, and wheat has 149% of methionine + cysteine, so mix them together and they cover for each other’s weaknesses. A 50/50 mixture of wheat and pea protein has a DIAAS of 94.

With this calculator, you can determine the DIAAS for mixtures of different protein sources.

Ingredient	Content (%)
Soy
Wheat
Pea
Fava bean
Hemp
Rice
Potato
Oat
Corn
Rapeseed
Lupin
Canola
Whey

DIAAS
Limiting amino acid

Full amino acid profile: (100 = recommended dose)

Histidine	Isoleucine	Leucine	Lysine	Met + Cys	Phe + Tyr	Threonine	Tryptophan	Valine

Good combinations of plant proteins

Soy protein alone has a DIAAS of 102.
22% pea + 36% fava bean + 42% hemp has a DIAAS of 96.
50% wheat + 50% pea has a DIAAS of 94.
28g of wheat protein plus a 500mg lysine pill has a DIAAS of 94.

Table of amino acid profiles

Amino acid values are scaled to the reference values for adult amino acid requirements such that a score of 100 matches the reference value.

Calculated using the amino acid values from Herreman et al. (2020)¹ and the reference values from FAO Expert Consultation (2011)².

Protein source	DIAAS	Histidine	Isoleucine	Leucine	Lysine	Met + Cys	Phe + Tyr	Threonine	Tryptophan	Valine
Soy	102	149	132	110	114	107	186	130	170	102
Wheat	57	148	97	94	57	149	138	97	164	99
Pea	82	124	108	94	131	82	147	117	99	89
Fava bean	65	135	113	103	113	65	151	113	88	89
Hemp	64	155	113	92	64	142	166	108	129	106
Corn	43	138	96	175	43	148	178	107	67	97
Rice	56	116	95	87	56	122	151	93	147	102
Potato	125	125	166	155	145	135	266	205	165	148
Oat	68	114	107	102	68	177	171	105	142	110
Rapeseed	80	134	96	84	80	147	117	120	137	99
Lupin	80	151	111	96	89	80	153	120	93	84
Canola	85	131	99	85	86	142	123	120	144	94
Whey	106	106	177	149	156	155	128	216	232	125

References

FAO Expert Consultation (2011). Dietary protein quality evaluation in human nutrition. ↩
Herreman, L., Nommensen, P., Pennings, B., & Laus, M. C. (2020). Comprehensive overview of the quality of plant- and animal-sourced proteins based on the digestible indispensable amino acid score. ↩

Just Because a Number Is a Rounding Error Doesn't Mean It's Not Important

Fri, 02 Aug 2024 00:00:00 -0700

Sometimes, people call a number a “rounding error” as if to say it doesn’t matter. But a rounding error can still be very important!

Say I’m tracking my weight. If I’ve put on 0.1 pounds since yesterday, that’s a rounding error—my weight fluctuates by 3 pounds on a day-to-day basis, so 0.1 pounds means nothing. But if I continue gaining 0.1 pounds per day, I’ll be obese after 18 months, and by the time I’m 70 I’ll be the fattest person who ever lived.

Or if the stock market moves 1% in a day, that’s a rounding error. If it moves up 1% every day for a year, every individual day of which is a rounding error, it will be up 3700%, which would be the craziest thing that’s ever happened in the history of the global economy.

This happens whenever the standard deviation is much larger than the mean. A large standard deviation means a “real” change gets obscured by random movement. But over enough iterations, the random movements even out and the real changes persist. For example, the stock market has an average daily return of 0.02% and a standard deviation of 0.8%. The standard deviation is 40x larger than the mean, so a real trend in prices gets totally washed out by noise. The market’s daily average return is a rounding error, but it’s still important.

A 401(k) Sometimes Isn't Worth It

Wed, 24 Jul 2024 00:00:00 -0700

You don’t always save money by putting your investments into a 401(k).

When you invest money inside a 401(k), you don’t have to pay taxes on any returns earned by your investments. But you also have to pay a fee to your 401(k) provider.

If you buy and hold index funds in a taxable account, you don’t have to pay any capital gains tax on price increases until you sell.
In a 401(k), the annual fee adds up every year and may eventually exceed the tax savings.

So the taxes cap out at the capital gains tax rate (15% or 20% depending on your tax bracket),¹ whereas the expenses of a 401(k) continue to accumulate.

However, in a taxable account, you do still have to pay taxes on dividends (and bond payouts) every year, and those taxes might cost you more than the 401(k) fees.²

Below is a calculator to determine how many years before the 401(k) fees exceed the tax savings, if ever.

This calculator assumes you buy index funds and hold them forever. If you trade stocks within a taxable account, you have to pay taxes every time you make a trade.

Something else to consider: If you quit your job, your old employer’s 401(k) provider will let you roll your 401(k) into an IRA. You don’t have to pay any fees on an IRA.³ So even if the 401(k) fees exceed the tax benefits after (say) 30 years, that’s not a problem if you expect to quit your job after less than 30 years. Realistically, few people stay at one job for so long that the 401(k) fees exceed the tax savings.

(If you change jobs, usually you can roll your old 401(k) into your new 401(k), but I wouldn’t do that because it means you have to keep paying 401(k) fees. It’s almost always better to roll your old 401(k) into an IRA.)

Notes

The capital gains tax will always be less than 15%/20% of your account value (depending on which tax bracket you’re in), but it converges on 15%/20% as the value approaches infinity.

Example: If you invest $100 in an index fund and you sell when the price reaches $101, you have to pay 20% of $1 (assuming you’re in the 20% tax bracket), which is only 0.2% of the total value. If you sell when the price reaches $1 million, you have to pay 20% of $999,900, which is 19.998% of the total value. ↩
H/T Ben Kuhn for raising this possibility. I’m sure someone somewhere had considered it before him, but I’ve never seen anyone else bring it up, and standard financial advice ignores it. ↩
Other than ETF/mutual fund fees, but you have to pay those no matter what. ↩

Continuing My Caffeine Self-Experiment

Mon, 24 Jun 2024 00:00:00 -0700

I did another caffeine experiment on myself. This time I tested if I could have caffeine 4 days a week without getting habituated.

Last time, when I took caffeine 3 days a week, I didn’t get habituated but the results were weird. This time, with the more frequent dose, I still didn’t get habituated, and the results were weird again!

Contents
Introduction
Experimental procedure
Results
When I take caffeine 3 days in a row, do I habituate by the 3rd day?
Notes

Introduction

In April, I published the results of a self-experiment on caffeine cycling. I drank coffee 3 days a week for 6 weeks¹ and tested my reaction time to see if I would become habituated to caffeine. If I do become habituated, my reaction time should get worse over the 6 weeks. It didn’t get worse, and in fact it got better, for unclear reasons.

So my experiment showed that I didn’t become (detectably) habituated to caffeine when taking it 3 days a week. I ran a second experiment to see what happens if I up the dosage frequency to 4 days a week. Do I start to become habituated? Or can I get away with it?

Turns out, I can get away with it. The results from the 4-day-a-week experiment show no signs of habituation.

In fact, like last time, my reaction time got (slightly, non-significantly) better over the course of the experiment.²

Experimental procedure

I followed the procedure of phase 4 as described in my pre-registration—it’s the same as phase 3, except that I drank coffee 4 days a week instead of 3. Specifically, I had caffeine on Monday, Wednesday, Thursday, and Friday. I ran the experiment for six weeks.

A quick review of the experimental procedure:

Take caffeine on Mon/Wed/Thu/Fri.
Test reaction time without caffeine every morning, and test again an hour after caffeine on caffeine days.
Look at the slope of reaction time over 6 weeks. If post-caffeine reaction time got worse, that means I became habituated. If no-caffeine reaction time got worse, that means I developed withdrawal symptoms.

I had caffeine three days in a row Wed/Thu/Fri so that I could test a secondary hypothesis: does caffeine become less effective by the third day? Pre-existing research suggests that habituation starts to appear as early as day 3. I might see a small habituation by Friday which then dissipates over the weekend.

Results

My reaction time slightly improved over the course of the six weeks.

Controlling for sleep quality (as measured by number of hours in bed) did not change the slope at all (to two significant figures).

My reaction time also improved during the previous experimental phase. But it didn’t improve over both phases combined. Looking at both experimental periods together (including the abstinence period in between), the slope is nearly flat (caffeine slope 0.01, no-caffeine slope –0.02).

As before, I don’t know how to explain why my reaction time improved within each experimental phase. My best guess is there’s some random-ish process that produces long-run trends in reaction time. Perhaps it’s the result of variations in sleep quality, but a type of sleep quality that time-in-bed can’t measure.

But it looks like I didn’t get habituated when taking caffeine 4 days a week—or, at least, not to a detectable degree. So I’m going to keep taking caffeine 4 days a week.

When I take caffeine 3 days in a row, do I habituate by the 3rd day?

The evidence suggests that I don’t, but the evidence is weak.

I don’t have great data because I only collected five data points (it was supposed to be six, one for each week of the experiment, but I got sick on week five which messed up my caffeine schedule³).

On average, my reaction time was 278.8 ms on the first day and 273.0 ms on the third day (so reaction time got better, not worse). The difference had standard error 8.4 ms (p = 0.5). This weakly suggests that I don’t start getting habituated yet by the 3rd day, but my test was underpowered. (I’d only expect reaction time to get worse by maybe 3–5 ms, and the standard error was 8.4 ms. That’s an odds ratio of 1.15:1 between 0 ms and 4 ms of habituation.)

I’d like to compare this to the pre-existing literature, but to my knowledge, no studies have ever administered daily caffeine to non-habituated users and measured the daily habituation curve. Lara et al. (2019)⁴ comes the closest: it didn’t measure performance every day, but it did have participants take caffeine for 20 days and measured athletic performance on day 1 and day 4. It found a slight decrease in performance between days 1 and 4, with the exact number varying from 1% to 3% depending on the metric used. Some research on rats (see the studies cited here) found that performance slightly decreased from day 1 to day 3, but rat metabolism runs faster than humans’ so on priors I’d expect humans to become habituated more slowly.

In conclusion, I don’t really know anything, but I’m gonna keep taking caffeine 4 days a week.

Source code and data for this experiment are available on GitHub.

Notes

I bring this up because isn’t it weird that that adding one extra week changed the p-value for no-caffeine tests from < 0.001 to 0.15?

In the original post, I included four weeks of data, but I continued the experiment for two weeks beyond what I had originally planned. The extra two weeks caused the regression lines to flatten out, but did not change their direction. This supports my hypothesis that the downward-sloping regressions were the result of some sort of anomaly. ↩
I looked at the results at the end of week 5 because I thought I might need to end the experiment early (turns out I didn’t⁵), and I saw a much stronger downward slope: p = 0.029 for slope on post-caffeine tests, and p < 0.001 for slope on no-caffeine tests. But I performed badly enough in week six that the slope largely flattened out. ↩
I still took caffeine 4 days that week, so the overall experiment didn’t get messed up. But I changed which days I took caffeine, so I didn’t get 3 days in a row. ↩
Beatriz Lara, Carlos Ruiz-Moreno, Juan Jose Salinero & Juan Del Coso (2019). Time course of tolerance to the performance benefits of caffeine. ↩
I had a problem with my GPU driver that increased the latency on my monitor, which was going to mess up the results. But I fixed the problem after a couple days so I just skipped those days, and I didn’t have to skip any of the important days. ↩

Some Things I've Changed My Mind On

Thu, 23 May 2024 00:00:00 -0700

Here are some things I’ve changed my mind about. Most of the changes are recent (because I can remember recent stuff more easily) but some of them happened 5+ years ago.

I’m a little nervous about writing this because a few of my old beliefs were really dumb. But I don’t think it would be fair to include only my smart beliefs.

Contents
Effective altruism
Finance
Health and fitness
Miscellaneous
What patterns emerge?
Notes

Effective altruism

My old belief: In my original 2021 version of A Comparison of Donor-Advised Fund Providers, I recommended Schwab Charitable as the best DAF provider for most people.

What changed my mind: When I reviewed the post this year, I noticed Schwab’s default fund fees are too high, so it was a bad idea to recommend them. I don’t recall exactly what I thought about the default fund fees when I first wrote the article, perhaps I noticed the high fees and thought it didn’t matter because people can switch to cheaper funds. If I did think that, then that was a mistake because a large proportion of people will stick with the default option without looking at it, and if they do that with Schwab, they’ll get ripped off.
My old belief: After writing Uncorrelated Investments for Altruists, I thought that the marginal donor’s philanthropic investment portfolio should aim for near zero correlation to equities.

What changed my mind: In the process of writing Asset Allocation and Leverage for Altruists with Constraints, I wrote code to do portfolio optimization and ran it under various assumptions. I found results that contradicted my previous belief. Now I believe that the optimal marginal investment portfolio should still have some correlation to equities because getting the extra expected return is worth accepting some positive correlation.
My old belief: In 2015 I donated to Raising for Effective Giving and argued for why they were my favorite donation target.

What changed my mind: After 2015, their fundraising model didn’t keep working as well as I expected.
My old belief: Metaculus Questions Suggest Money Will Do More Good in the Future

What changed my mind: After I published that post, some commenters argued for a different interpretation of the Metaculus questions.

Finance

My old belief: SBF and Alameda have skill at beating the market.

What changed my mind: I’m sure you know what changed my mind. There’s some chance that they did actually have skill and they blew up due to bad luck (them committing fraud was bad behavior, not bad luck, but as I understand it, they blew up because they lost a bunch of money, and they might have gotten away with the fraud if they’d made money). But I now believe it’s more likely that the risks they took were not calculated and they didn’t have much skill. (Clearly SBF had a lot of skill at fundraising, but that’s not the same thing as trading skill.)
My old belief: In 2021 and earlier, I estimated future market returns using Research Affiliates’ model (e.g. here), which assumes market valuations mean-revert after 10 years.

What changed my mind: I read AQR’s capital market assumptions (see e.g. their 2024 publication) where they argue that there’s no strong reason to expect valuations to mean revert. Now I prefer the AQR model which uses the traditional “yield + growth” approach with no consideration for valuation. I still believe valuations ought to mean revert, but it could easily take more than 10 years, and they might only revert somewhat, so I think it’s reasonable to take an average of the AQR and Research Affiliates projections.

Putting less consideration on mean reversion makes equity market return projections cluster closer together. I would still order expected equity returns as emerging markets > developed markets ex-US > US, but I do not expect the differences to be as big as I used to. I used to quote something like a 6% real return for emerging markets and 0% for the US market. Now I expect more like 5% for emerging markets and 2% for the US.

I wrote more about my updated expectations here. I don’t pay too much attention to return projections, so changing my mind on this didn’t change my investment strategy.
My old belief: From a purely financial perspective (ignoring personal taste etc.), renting a house is always a better decision than buying.¹

What changed my mind: Around 2020, I read the argument that owning a house works as a hedge against future housing expenditures, which means buying is better than renting in many cases.²

I still believe most discourse on renting vs. buying is confused. For example:
1. It doesn’t make any sense to directly compare monthly rent to monthly mortgage payments because with a mortgage, you’re accumulating equity in an asset.
2. Maybe people understand point 1 and believe a mortgage is always better because renting is “throwing money away”. That’s also wrong because you have to account for the time value of money. Dumping most of your net worth into a house has a huge opportunity cost.
3. When you account for opportunity costs, you have to consider the risk of a mortgage vs. the risk of the counterfactual investment (e.g., an index fund).
I looked through a dozen online “rent vs. buy” calculators, only three of them properly accounted for equity and opportunity costs (Financial Mentor, Motley Fool, and New York Times (paywalled)), and none of them accounted for risk. (The three good calculators use different methods but they’re basically interchangeable.)
My old belief: In 2016 when I was looking for a full-time job at a startup, I evaluated equity compensation at face value.

What changed my mind: I should have considered risk. Equity compensation is risky (2–4x riskier than an index fund), which makes it look much worse.

What changed my mind again: Later, I realized there are some other factors that make equity compensation look better. In 2021 I did a more in-depth analysis here, and my current opinion is that equity in a good startup is worth more than its face value.
My old belief: In 2013 I read some research on investing strategies like Greenblatt’s magic formula and thought they sounded like a great idea.

What changed my mind: Actually, I still believe Greenblatt-esque strategies are a great idea (at least for some people in some contexts). But I believe I over-updated on the evidence I had at the time, I just got lucky that I was looking at weak evidence for true claims.

I originally became convinced that Greenblatt’s magic formula worked when I read Abbey & Larkin (2012), “Can simple one and two-factor investing strategies capture the value premium?” The paper looked at US stocks over a 30-year period. Now, I would want to see more evidence than that. I like the five criteria given by Berkin & Swedroe’s Your Complete Guide to Factor-Based Investing: to take a market anomaly seriously, it must be (1) persistent across time, (2) pervasive across markets, (3) robust to different formulations, (4), investable, and (5) have a risk-based or behavioral explanation. The evidence from Abbey & Larkin (2012) established robustness and half-established persistence (30 years is decently long so I give it half credit), but didn’t address the other three and a half criteria.

Health and fitness

(I’ve been thinking a lot about health and fitness lately.)

My old belief: In 2011 I quit coffee cold turkey after reading You Are Not So Smart’s article Coffee. I believed that caffeine had no effect on a daily user except to reverse withdrawal symptoms.

What changed my mind: In Does Caffeine Stop Working?, I investigated more deeply and now I believe that caffeine retains something like half its initial benefit.
My old belief: I can trust the research on stuff like caffeine.

What changed my mind: I read some caffeine studies and found that most of them were pretty bad. And not bad in the obvious way of having small sample sizes (which is honestly fine as long as you’re aware of it—weak evidence is still evidence). They were bad in the sense of “your study’s methodology is not capable even in principle of providing evidence for or against your hypothesis”. The majority of studies were like that (around 75% of them, if I remember correctly).
My old belief: After reading Starting Strength in 2014,³ I thought Starting Strength-style training was the best in every situation, and bodybuilding-style isolation movements were dumb.

What changed my mind: After a few months, I realized I had been on the “hill of novice overconfidence” and actually isolation exercises are fine. Later I re-read Starting Strength and realized it never even said you shouldn’t do isolation training. It made the more nuanced claims that (1) isolation training is not ideal for developing strength and (2) compound barbell movements are better for beginners.

After learning more about the scientific literature and the diversity in how elite athletes train, now I tend to believe differences in training mostly don’t matter. You can get good results with any method as long as you lift heavy weights, increase the weight over time, and get sufficient food and rest.⁴
My old belief: High-intensity interval training is the best kind of cardio.

What changed my mind: I started reading some experts on exercise science, and they say low intensity cardio is just as good most of the time, and the ideal routine consists of something like 80% easy cardio and 20% hard cardio (and the easy cardio should be really easy⁵).

And thank God for that because I kinda hate doing moderate/hard cardio. I’ve started being way more consistent about aerobic exercise—I go for brisk hilly walks 3 times a week and haven’t missed a day in months.⁶ And my resting heart rate has gone down from 70–75 bpm a few years ago to 55–58 bpm today, so I guess it’s working. I’ve also noticed I can do high-rep squats and deadlifts without getting winded. I remember in 2016 I deadlifted 225 for 10 reps and I basically died (my heart rate hit 190 bpm). A few weeks ago I deadlifted 315 for 11 (note: my 1-rep max has barely changed since 2016) and I felt fine (heart rate 144 bpm).

(I’ve read hardly any original research on cardio, but as I understand, the older research did show that HIIT was better than low-intensity cardio, and newer research changed that—see the 6-part series Extraordinary Claims in the Literature on High-Intensity Interval Training by Ekkekakis et al. (2023).⁷ So my beliefs basically tracked the research findings, although I was getting all my info second-hand.)
My old belief: At different times, I believed (1) we don’t really know anything about nutrition; (2) food choice doesn’t matter as long as you don’t over-eat.

What changed my mind: I read How Not To Die in 2017, which referenced a large quantity of nutrition research that contradicted my previous beliefs. I now believe the book was wrong about some things (which I will discuss in the next line item), but it was more correct than my pre-2017 self. Later on, I read a wider variety of evidence-based nutrition advice.

My current position is that we are really pretty sure about some things in nutrition, and some foods are unhealthy even if you don’t over-eat. I believe conventional nutrition advice in educated circles is basically correct: trans fat, saturated fat, and added sugar are bad; processed food is generally bad; whole plant foods (especially fruits and veggies) are good.

(I still don’t have a great sense of the distinction between foods that make it easy to overeat and foods that are unhealthy at any bodyweight. Like I know that sugar in small quantities isn’t bad for healthy-weight individuals, but is that because the badness is too minor to detect, or because there’s some threshold below which sugar causes zero harm whatsoever?)

I updated my beliefs by following my “web of trust”: my layman friend trusts this dietitian; my other layman friend trusts this medical doctor who agrees with the first dietitian about most things; I trust Scott Alexander, and he likes this one researcher, who endorsed this book; these two guys know a lot about strength training and I like their epistemology,⁸ so they probably also know a thing or two about nutrition, and they agree with the guy Scott liked; etc.

According to my web of trust, the best book on nutrition is Eat, Drink, and Be Healthy, which as far as I know is the only book that makes a 100% earnest effort to represent the state of nutrition science.

I still haven’t made an effort to interpret the primary literature on nutrition science. All the really big studies are observational, and there’s an art to controlling for confounders, and I believe it would take a lot of work for me to understand how they do it. I trust that at least some researchers have a good conception of how to disentangle causality (Willett & Skerrett, authors of Eat, Drink, and Be Healthy, do a good job of explaining why they believe observational studies establish causality in certain cases.⁹)
My old belief: After reading How Not To Die in 2017, I believed processed unsaturated fats (such as olive oil) were unhealthy, and unsaturated fats should be consumed as whole foods (e.g. by eating nuts).

What changed my mind: I read Eat, Drink, and Be Healthy, which said olive oil is healthy, and it presented some evidence that looked reasonable to me. There’s a plausible mechanism for oil being healthy (it helps the body produce HDL and HDL sucks loose cholesterol out of the arteries), and there are some empirical studies where oil (esp. olive oil) was associated with better health outcomes, including at least one RCT.

The argument against olive oil in How Not to Die is that it’s processed to remove some of the nutrition of the olive, which makes it less healthy. That’s not wrong—raw olives are probably healthier than olive oil—but realistically I’m not gonna replace olive oil with eating handfuls of raw olives,¹⁰ and the empirical evidence suggests that adding olive oil to a diet makes it healthier.

How Not to Die recommends replacing olive oil with nuts. Some RCT evidence does suggest that nuts are healthier than olive oil, but nuts often don’t work as a substitute for oil (you can’t pan-fry food in a bed of nuts).

How Not to Die is biased toward veganism, which I knew before I read it so I didn’t update much on the stuff about how all animal products are unhealthy, although I was vegan anyway so it didn’t affect my behavior. I largely trusted the book on non-animal subjects because it cited a lot of research and seemed well-reasoned. Based on my current understanding of mainstream positions among nutrition scientists, almost all of the non-animal stuff (and most of the animal stuff) in the book is mainstream, but it over-emphasizes the badness of processed foods in general. The mainstream position among nutrition scientists is that you should avoid “processed foods” as a general rule, but plenty of specific processed foods are fine, like olive oil or protein powder.
My old belief: The ideal BMI is around 18–20 (on the low end of the “healthy” range of 18.5 to 25).

What changed my mind: My old belief wasn’t based on direct evidence. I just had a prior that official recommendations are gonna be too generous, for example recommending less exercise than is optimal because they don’t think people will actually do the optimal amount of exercise, or recommending a “healthy” BMI that’s actually a bit too generous because they think people will give up if they’re told to aim for a BMI of 20. I updated my belief after reading a large study¹¹ on BMI and all-cause mortality. As of writing the first draft of this post, I weakly believed the ideal was on middle-high side of the “healthy” range (so around 22–23), but I wrote in my first draft, “I want to investigate this more.”

What changed my mind again: I investigated more. (I had to take a diversion from writing the post you’re currently reading to write a different post about BMI.) Now I believe the ideal BMI is 20–22, for reasons I explain in the linked post. Lower than 20 is fine, maybe even better, if you have adequate lean mass. 22–23 appears to carry (slightly) greater health risks.
My old belief: I believed that I had seen an RCT that found that sunscreen didn’t work, and I had written that down in a note.

What changed my mind: I looked up my note and saw that, in actuality, my note said that sunscreen did work. I somehow flipped the sign of the outcome in my memory. I don’t understand how that happened.

Miscellaneous

My old belief: RCTs are high-quality evidence.

What changed my mind: I learned about the replication crisis, and (later) I read some actual RCTs. I now believe the median RCT is pretty badly designed and it shouldn’t change your beliefs much unless you’ve actually read it and understood its methodology. And, perhaps as a corollary, the median scientist isn’t very smart. (This doesn’t necessarily follow because there are reasons why smart scientists might publish dumb papers.)

(Doctors and professors appear to have average IQs around 115–125—see Meritocracy, Cognitive Ability, and the Sources of Occupational Success—which is a full standard deviation below the IQs of most of my friends,¹² and probably most of the people reading this. So maybe it’s fair after all to say the median scientist isn’t very smart. But you could still argue that a lifetime of expertise matters more than 15 IQ points.)

I see one area where scientists routinely mis-interpret their own evidence. They often struggle to understand the difference between
1. Our study found a large (in absolute terms) but non-significant effect because our study was underpowered.
2. Our study robustly established no effect: the standard error in our data was small enough that any meaningful effect would show up, and it didn’t.
You especially see this fallacy in areas where small effect sizes still matter. For example, a 0.1% decrease in mortality risk matters a lot, but it’s very hard to detect with a study, and study authors often incorrectly conclude that the effect doesn’t exist when they fail to find it.

(David J. Balan says there are two kinds of “no evidence”. I’d say there are three kinds: (1) we haven’t looked for evidence; (2) we looked for evidence in a way that wasn’t gonna find any evidence, and we didn’t find any evidence; (3) we looked for evidence in a way that would have found it if it existed, and we still didn’t find it.)
My old belief: RCTs on strength training are basically useless. I read Practical Programming for Strength Training, which argued that strength coaches know better than researchers because RCTs are deeply flawed: they test a group of untrained individuals over 12 weeks or less, and that sort of training context doesn’t generalize to a more-experienced individual who follows a program for a year or longer.

What changed my mind: This one’s interesting because it’s the opposite of the previous line item.

I started listening to some more research-driven experts like Barbell Medicine and Stronger by Science and hearing their perspective on scientific studies. While it’s true that many (most?) sports science RCTs don’t generalize, there are plenty of studies that correct for the criticisms made by Practical Programming for Strength Training.

Why the difference between “RCTs are bad” in the previous line item and “RCTs are good actually” now? As best I can figure, this is the deal:
1. RCTs are bad if you blindly accept all of them.
2. RCTs are good if you know how to read a study and understand where it does and does not generalize.
The science popularizers I pay attention to know how to distinguish between bad and good studies, and how to synthesize commonalities that repeatedly appear in many studies.
My old belief: Before 2009, I disagreed with affirmative action because I thought it was anti-meritocratic.

What changed my mind: In 2009, I saw a debate in which the pro-affirmative action side won. I did not actually read the debate but I thought the pro side seemed credible so I changed my position.

What changed my mind again: I now disagree with affirmative action. The main argument that changed my mind was that affirmative action has been around for a generation so if it was going to work, we would certainly see the benefits by now, and we don’t.

To be a little more specific, there are (broadly speaking) two theories for why affirmative action ought to work:
1. Gatekeepers (e.g. hiring managers) unfairly discriminate against minorities, and affirmative action cancels this out.
2. Minorities underperform in some areas because they haven’t been given sufficient opportunities. Affirmative action gives them those opportunities so that they can get better.
The first theory is pretty easy to test: if minorities outperform after being accepted, then they were being discriminated against. Empirically, we see the opposite—for example, racial minorities at universities have on average lower GPAs than whites. The exception is Asians, who do actually outperform, which suggests they’re being discriminated against. But we don’t need affirmative action to fix anti-Asian discrimination, in fact it’s caused by affirmative action.

You can test the second theory in a similar way: after being accepted, do minorities on average improve their performance? Empirically, they don’t. You can wiggle out of this by saying they still face hurdles even after being accepted to (e.g.) university. But even the children of under-represented minorities who went to elite colleges still underperform (on average). If affirmative action doesn’t improve outcomes even for the children of the beneficiaries, it’s probably never going to work.

That doesn’t prove that minorities don’t face hardships that hamper their performance, it just proves that affirmative action doesn’t do anything to rectify those hardships.
My old belief: Regulation is basically good; free markets often hurt people.

What changed my mind: I now believe there’s way too much regulation and free markets are almost always good. I wouldn’t go so far as to say all regulations are bad, but I think the developed world would be much better off if lawmakers simply deleted 75% of existing regulations. Nor would I say free markets are always good, but I’d guess that the ratio of economic problems caused by market restrictions to problems caused by overly free markets is about 20:1.

I can’t pinpoint a specific period where I changed my mind but it happened somewhere between the beginning and the end of college. The main things that changed my mind were (1) getting better at applying basic economic reasoning (I took econ in 11th grade but I didn’t start applying it to life until later) and (2) reading economist polls and updating toward economists’ beliefs.
My old belief: Maybe if I force myself to go to enough social events, I will learn to enjoy them.

What changed my mind: I went to a whole bunch of social events that I didn’t want to go to and I never learned to enjoy them. I now believe it’s better to simply not go to social events that I don’t want to go to.

(People often tell me things like, “You should come, you’ll start enjoying it once you get there!” These people are badly failing to model the fact that my brain does not work the same as their brain.)
My old belief: In 8th grade Spanish class, the teacher had an accent I was unfamiliar with. I thought he was mis-pronouncing certain Spanish words.

What changed my mind: I learned that his pronunciations were a regional accent.

I don’t expect 8th graders to have particularly good reasoning abilities but this mistake feels especially severe. Surely I could have figured out that I, a kid who has taken one year of Spanish, do not know more about Spanish pronunciation than this guy who is a native Spanish speaker and teaches Spanish.
My old belief: Nuclear power is too dangerous, mainly because we can’t safely manage radioactive waste.

What changed my mind: In 2010, I heard about Kahan et al.’s Cultural Cognition of Scientific Consensus¹³. The paper studied how people form beliefs on two issues where many people disagree with the scientific consensus—climate change and the disposal of nuclear wastes. Kahan et al. cited a consensus report¹⁴ where scientists agreed that radioactive waste can be disposed of safely. This was the first time I came in contact with the notion that the scientific consensus supports nuclear power. I knew very little about nuclear waste disposal (and still know little), but I changed my belief to align with the scientific consensus.
My old belief: Creationists are uniquely bad at reasoning.

What changed my mind: I know very smart and educated people who believe things that are about as obviously-wrong as creationism (such as the labor theory of value, or that infants are “blank slates”,¹⁵ or nuclear waste can’t be disposed of safely—wait, that last one was me).

I now believe it’s basically impossible to avoid believing dumb things. I still think creationism is extremely wrong, but I have a lot of sympathy for creationists, and I don’t think they’re particularly worse at reasoning than anyone else. I believe you should rule thinkers in, not out.

The thing is, if you live in a bubble, you might not ever hear the arguments for why the labor theory of value is wrong, or why blank slatism is wrong. Or you might hear arguments but not good ones, perhaps because knowledgeable people would rather make fun of you than try to persuade you. So I find it totally understandable that people hold on to these beliefs. And I think of creationism the same way—some people have the misfortune to live in bubbles where everyone around them believes in creationism and they never hear good counter-arguments.

What patterns emerge?

Did I systematically change my mind in certain ways? Can I predict how I might change my mind in the future?

I can see four broad patterns:

Initially, I overconfidently believed the first credible-sounding thing I heard. Later, I moderated my beliefs when I learned about other perspectives.
I investigated an area where not much is known, so I had to figure things out on my own. My initial conclusion was wrong, and I changed my mind by investigating more deeply.
I used to believe individual scientific studies, and now I don’t give them much credibility.
I pay more attention to the scientific consensus.

On #4, it’s not so much that I ever wanted to disagree with the scientific consensus, but that it’s hard to know what scientists believe. On economics, nutrition, and nuclear power, I used to disagree with the consensus, but only because I didn’t know what the consensus was.

Notes

My argument went like this:

From a financial perspective, buying a house is isomorphic to buying a rental property and renting it out, while also paying rent to live at a second, identical house. A rental property is not a good investment because it’s highly non-diversified. If you shouldn’t buy a house as a rental property, then you shouldn’t buy a house to live in, either.

(People can pretend houses aren’t risky because the price doesn’t update on a minute-to-minute basis, but a single house is a much riskier investment than an index fund.) ↩
There are some caveats to this:
1. Buying a house only works as a hedge if you plan on living there forever. If you ever plan on moving, you’re subject to fluctuations in the price of your current house relative to that of your future house.
2. It only works if you get a competitive mortgage interest rate and if the value of the hedge balances out the opportunity cost of pouring a bunch of money at once into a house.
3. Houses have maintenance expenditures and the like, although this should be priced in to rent.
↩
I read Starting Strength based on a recommendation in a LessWrong comment: “The set of my friends who are strong is exactly the set of my friends who do / have done Starting Strength or a close variant.” ↩
I like what Mike Israetel said in an interview (paraphrased): “If you can point out a dude to me and tell [based on how he looks] that he trains with sets of 20–30 and point out another dude and say that dude trains with sets of 5–8, I’d be super impressed, because I can’t.” ↩
The standard rule of thumb is the “talk test”: if you can carry out a conversation with a little bit of difficulty, it’s the right intensity. ↩
This sentence was true when I originally wrote it. Now I’m revising and I feel the need to note that I missed a day last week because I was sick. ↩
Ekkekakis et al. (2022–2023). Extraordinary Claims in the Literature of High-Intensity Interval Training.
↩
I generally like the epistemology of the Barbell Medicine guys, but we do disagree sometimes. I’ve noticed a common pattern in how we disagree.

Say there’s a question about whether A is good, and the state of the evidence is that
1. There’s some theoretical reason to expect A to be good
2. Some limited empirical research has failed to find that A is good
In that case, they believe A is not good, and I believe A is good.

For example, they say you shouldn’t take a multivitamin because RCTs generally haven’t found benefits. I say you should take a multivitamin because they’re cheap and we know vitamins are important in principle.

I believe scientists and doctors tend to overweight weakly negative empirical findings relative to theory. I agree with what Scott Alexander wrote in A Failure, But Not Of Prediction: authority figures said (in April 2020, when he wrote the post) that masks don’t prevent the spread of disease because there’s no supporting RCT evidence. But Scott believes masks are worth using because there’s good theoretical reason to expect them to work.

The Barbell Medicine guys understand perfectly well that sometimes theory matters more than empirical evidence, it’s just that I tend to favor theory a little bit more than they do. ↩
For example, I was concerned that a lot of associations between diet and health were confounded by socioeconomic class (rich people eat more veggies) or by conscientiousness (conscientious people eat more veggies plus they’re less inclined to over-eat). But Willett & Skerrett give evidence that some nutritional findings can’t be explained by class or conscientiousness.
- On socioeconomic class: Greeks today and Chinese people in the 1990s were healthier than Americans even though they were poorer.
- On conscientiousness: Americans who conscientiously followed the 1990s USDA guidelines had worse health outcomes (because the guidelines were dumb).
(Also: Isn’t it kind of insane that people who followed the USDA guidelines had worse health outcomes? In general, every diet works, whether it’s low-carb, low-fat, paleo, or whatever, because diets force you to pay more attention to what you eat and restrict your caloric intake. It’s perversely impressive that the USDA managed to come up with a diet (possibly the only diet ever?) that actually makes health worse.) ↩
Where do you even buy raw olives? I’ve only ever seen jarred or canned olives soaked in brine. ↩
Bhaskaran, K., dos-Santos-Silva, I., Leon, D. A., Douglas, I. J., & Smeeth, L. (2018). Association of BMI with overall and cause-specific mortality: a population-based cohort study of 3-6 million adults in the UK. ↩
Earlier in this post, I wrote about how most caffeine studies had bad methodology. As a test, I described the methodology of one study to a friend and asked them what they thought about it, and they immediately pointed out the same flaw that I had noticed. So clearly it’s not just me. ↩
Kahan, D. M., Jenkins‐Smith, H., & Braman, D. (2011). Cultural cognition of scientific consensus.

(Note: The cited version was published in 2011 but the paper was originally posted online in 2010.) ↩
National Research Council. Board on Radioactive Waste Management. (1990). Rethinking high-level radioactive waste disposal: A position statement of the Board on Radioactive Waste Management, Commission on Geosciences, Environment, and Resources, National Research Council. ↩
It would derail the post too much for me to explain why I believe these beliefs are on par with creationism, but I can throw out some links:
1. Criticisms of the labor theory of value on Wikipedia
2. The Blank Slate, an article by Steven Pinker that summarizes his book, titled (surprise!) The Blank Slate
↩

Notes on Eat, Drink, and Be Healthy

Thu, 23 May 2024 00:00:00 -0700

I recently read Eat, Drink, and Be Healthy: The Harvard Medical School Guide to Healthy Eating. As I understand, it’s the book that does the best job of representing the mainstream scientific perspective on nutrition for a lay audience. Here are the notes I took.

Last updated 2024-09-02.

I wrote down info that I found personally useful, which often doesn’t translate to other people. For example, if the book gave a nutrition fact that I already confidently believed, I didn’t write it down, but someone else might have benefited from reading that fact.

Unless otherwise specified, these notes represent my interpretation of the author’s perspective and first-person pronouns represent the author, not me. Any sentence preceded by “me:” is my perspective.

Contents
Notes from Red Pen Reviews
Footnotes

Chapters 1–3: Healthy Eating Matters

Soybean oil and canola oil are healthy¹
The association between “healthy” foods and good health can’t be explained by socioeconomic status or conscientiousness because Greeks are healthier but poorer, and people who followed USDA guidelines did worse than people who followed “good” diets
- me: This comment gave the author more credibility in my mind. I can’t easily the author’s evaluate empirical claims, but I can evaluate logic, and I thought this was a strong logical argument
American diets are getting healthier over the last 20 years. (me: so why are people getting fatter?)
RCTs are expensive. A nutrition and breast cancer study cost $2 billion and got inconclusive results. We mainly do cohort studies which follow a group of people for a time, they have less room for bias than retrospective self-reports / asking people with a disease about their eating habits
Nurses’ Health Study is good because nurses are more diligent and accurate about reporting what they eat
Japanese migrants in America have American levels of heart disease, which shows it’s not genetic
Moderate alcohol does actually prevent heart disease. Experiments show it raises HDL

Chapter 4: Healthy Weight

me: I examined BMI in more detail in What’s the Healthiest BMI?. Eat, Drink, and Be Healthy primarily cites the same two meta-analyses that I cited (and the book’s primary author also co-authored one of the meta-analyses (along with 61 other authors lol))
Higher BMI is worse even within the healthy weight range, see Nurses’ Health Study
- me: the reported data looks at CVD and diabetes and stuff but not at respiratory disease
Guidelines say BMI should be 18.5 to 25, that’s the cohort with lowest death rate after controlling for smoking etc. But really health starts getting worse at BMI > 22
BMI = weight (kg) / height (m)^2 or weight (lbs) / height (in)^2 * 703
BMI under 18.5 is bad if you’re sick but it’s fine if you’re just thin
- me: for my perspective on this, see What’s the Healthiest Body Composition?, especially the bits about Lee et al. (2018). the claim appears to be true, but there are approximately zero people with healthy lean mass and a BMI under 18.5

Chapter 5: Straight Talk About Fat

Belly fat is worse than hip/butt fat. Unclear why, maybe because it affects hormones more
Controlling weight is the most important component of diet. Luckily, a healthy diet is a subset of diets that make it hard to overeat (me: fad diets can be good for controlling weight but aren’t optimal for health). Ex: fiber is satiating and also good for digestive health
Insulin makes your body convert calories to fat and reluctant to burn fat; people with high insulin will be hungry even if they have plentiful fat stores. That’s a reason to prefer foods with low glycemic index
Proteins/fats stay in the stomach for longer which reduces hunger
45% of a database of people who lost weight didn’t use a Diet, they “did it themselves”²
Unsaturated fat lowers LDL (“bad”) cholesterol and raises HDL (“good”) cholesterol (compared to carbs); saturated raises both; trans raises LDL
Fat is not water soluble, so it is transmitted through bloodstream by protein packages (lipoproteins)
Saturated fats have varying effects. Coconut oil raises HDL more than beef or butter fat, but its effect on LDL still makes it net unhealthy
LDL accumulates in blood vessels—it’s low density so it’s more likely to get stuck. HDL removes LDL from blood vessels
Omega-6 doesn’t increase inflammation, it reduces it. The 3:6 ratio didn’t matter in the Nurses’ Health Study
Eggs are fine. Low in saturated fat
Nuts beat olive oil in RCTs
Red meat is bad for colon cancer, which could be b/c of fat content or b/c of chemicals generated by cooking at high temperatures. But biggest risk factor for colon cancer is being overweight
ALA associated with prostate cancer, but likely b/c ALA-rich fats used to be partially hydrogenated most of the time. Walnuts are high in ALA and aren’t associated with prostate cancer. Future studies will see if the ALA-cancer connection persists now that trans fats are mostly gone from diets, But you probably shouldn’t worry about ALA
Dietary fat has no detectable association with cancer. Saturated and trans fats are bad for heart disease so that’s what we should focus on

Chapter 6: Carbohydrates for Better and Worse

Grains are optional
Carbs looked healthy in old China studies but don’t look healthy in the west. Likely due to (1) Chinese were more physically active and (2) they ate whole or lightly refined grains. The badness of carbs mainly comes when you don’t burn them off (?)
Simple vs complex carb isn’t that important. Glycemic index (GI) and refined vs whole are more important. Glycemic index, not carb complexity, determines blood sugar
Potatoes and corn are fast digesting. Corn is technically whole grain but it’s been bred to be fast digesting so it’s more like white rice
Fast digesting carbs spike blood sugar then spike insulin so your body quickly absorbs the blood sugar, now you have low blood sugar which triggers hunger
Table sugar and corn syrup have the same impact on blood sugar and metabolism
Insulin resistance means cells resist the signal telling them to absorb sugar, so blood sugar stays elevated for longer. Insulin-producing cells in the pancreas get overworked and stop working
Contributors to diabetes: obesity; sedentariness, because muscles are good at consuming glucose; low polyunsaturated fatty acid (PUFA) and high saturated fat; genetics
Finely ground whole wheat is high GI but it’s still healthy due to fiber and nutrients
Low-GI foods help prevent diabetes
High-fiber cereal helps prevent diabetes
Early studies showed fiber reduced gut cancer, but later studies e.g. Nurses’ Health showed no effect
Whole grains have 1:10 or 1:5 fiber:carb ratio
Added fiber like cellulose isn’t as good as whole fiber b/c it’s missing micronutrients and it doesn’t encapsulate the carbs to slow digestion (book calls added fibers “fake fiber”)
Eat whole grain cereal for breakfast. Ex: Wheaties, Grape Nuts, Kashi, Shredded Wheat, Wheat Chex

Chapter 7: Choose Healthier Sources of Protein

BCAAs turn up IGF-1 which helps muscles grow but also cancer
Processed meat causes cancer; red meat probably causes cancer. Mainly colorectal
Country-level data shows plant proteins associated with less cancer than animal proteins, but country data is highly confounded. It’s probably the stuff that comes with the protein that matters, not the pure protein
ALA reduces clotting
Soy effect on heart disease is overstated. What studies actually show is that heart disease is reduced when you replace red meat with soy
Soy contains phytoestrogens which may prevent breast cancer. Phytoestrogens act like estrogen in some places and block it in others. It appears it blocks estrogen in cancer cells and estrogen stimulates growth
Some fish contain mercury but many species (e.g. salmon) don’t have concerning amounts. Don’t worry about it unless you’re a child or pregnant. Fish oil supplements have less mercury than fish but don’t show the same health benefits

Chapter 8: Eat Plenty of Fruits and Vegetables

The effects of the vast majority of plant chemicals have yet to be determined. We don’t even know most of the chemicals that are in plants
Fruit and veg, especially greens and citrus (including juice), lower blood pressure and reduce stroke. Folic acid supplements work too. Fruit and veg shown to reduce blood pressure in RCT
Cataracts and macular degeneration may be caused by free radicals
1/3 of cancer is explained by diet. Possibly an overestimate
One study found that fruit in adolescence does more to prevent cancer than fruit in middle age
Berries are S tier fruit, especially blueberries
Fiber sticks to cholesterol and you poop it out
Newest studies show fiber doesn’t help with colon cancer. But it does still stabilize blood sugar which reduces diabetes, lowers triglycerides, and improves gut microbiome
It looks like many diseases like CVD and cancer are driven by deficiencies in some phytonutrients, but we don’t know which. Folate is probably one
Eat a variety of colors. Get one serving a day each of: dark leafy greens, yellow/orange fruit/veg, red fruit/veg, legumes, citrus
Cooked tomatoes are better than raw b/c your body has a hard time absorbing lycopene from raw tomatoes
Some veggies have chemicals that are bad if you eat too much, but it’s hard to eat too much
Juice and smoothies are bad because (1) easy to over-eat and (2) juice makes sugar absorb faster (me: they didn’t say if smoothies do that)
- me: I spent a little time online trying to determine if smoothies are healthy or not (ignoring calorie content), my main concern being whether blending destroys fiber/nutrients. I found zero relevant papers on Google Scholar. I found various people, including dietitians (e.g. on Hopkins Medicine) asserting without evidence that blending doesn’t destroy nutrients. So I assume smoothies are healthy but I don’t really know

Chapter 9: You Are What You Drink

Milk in adolescence increases height which increases hip fractures
Coffee and tea reduce gallstones, possibly b/c they stimulate gall bladder activity
Coffee, even decaf, reduces diabetes. Possibly due to antioxidants
Coffee reduces Parkinson’s
Tea has flavonoids which may reduce CVD
Alcohol raises HDL. CVD benefit at 1-2 drinks per day for men. Can be any kind of alcohol. Benefits of wine specifically are unproven
Alcohol raises breast cancer risk in women even at half a drink per day. Folate counteracts this
Alcohol is net harmful for young men due to low CVD risk, and net beneficial for older men. Unclear for women, probably net positive unless you have family history of breast cancer

Chapters 10–11: Vitamins and Minerals

Hard to tell how much calcium people need because outer shells of bones can absorb calcium quickly but without increasing inner bone density
Insufficient vitamins can contribute to CVD and cancer even without deficiency disease
Folic acid started being added to grains to prevent birth defects but it accidentally reduced CVD and cancer too
Vitamin A helps regulate cell division and thus prevent cancer
Beta carotene isn’t vitamin A but your body turns it into vitamin A
Excess pre-formed vitamin A blocks vitamin D. Better to get vitamin A from beta carotene than from pre-formed vitamin A (retinol)
Free radicals are positively charged so they can steal electrons from DNA or cholesterol or other important things in your body
Early studies showed benefits but recent large RCTs don’t find benefits to antioxidant supplements
James Watson suggests antioxidant supplements are bad because free radicals kill cancer cells
Antioxidants are good if you get them from food
Excess iron generates free radicals. Some evidence suggests excess iron causes heart disease and cancer, but jury is still out
Body does a good job of passing on unneeded iron when it comes from plants but not when it comes from meat (me: what about supplements?)
I recommend taking a multivitamin that doesn’t contain iron
Sodium RDA is 2300mg but most people need less than 1000mg
American Heart Association recommends a max of 1500mg sodium
Selenium probably doesn’t matter
Take a multivitamin that contains the nutrients people tend to miss: beta carotene, B6, B12, folic acid, D, E, iron, zinc (me: what about earlier claim to not supplement iron?). No more than 2000IU pre-formed vitamin A
Basic One multivitamin is good. Menstruating women get the version with iron, others don’t need the iron
- me: Compared to normal multivitamins, Basic One has considerably more D3, E, and B6, and way more B12; added selenium, chromium, and lycopene (an antioxidant found in many red fruits/veggies); and its vitamin A comes exclusively in the form of beta carotene (no pre-formed vitamin A)

Notes from Red Pen Reviews

This section was added on 2024-05-31.

Red Pen Reviews, a website that reviews the scientific accuracy of books on nutrition, reviewed Eat, Drink, and Be Healthy and gave it the highest score of any book it’s reviewed.³ It did, however, have some minor quibbles with the book. It disputed one claim from the book, found that the book overstated the strength of evidence in two of its references, and sort-of disputed the book’s position on multivitamins.

Claim from the book: “Protein sources from plants and lean meats such as chicken or fish are likely more beneficial than protein from red and processed meat. Protein from soy, however, is less well-understood.”

There’s mixed evidence on whether plants and lean meats are better for cardiovascular disease than red meat. It seems broadly true but there’s some conflicting evidence.
The skepticism about soy is based on old studies, and more recent studies find that soy is beneficial.

Reference from the book: “In an analysis my colleagues and I did among more than 43,000 men, intake of total protein was minimally associated with heart disease risk, while intake of protein from meat was associated with higher risk.”

Among all participants, the correlation between protein source and heart disease risk was non-significant. The correlation only became significant when the analysis was restricted to “healthy” patients.

Reference from the book: “Dark leafy green vegetables contain two pigments, lutein and zeaxanthin, that accumulate in the eye. These two, along with phytochemicals called carotenoids, can snuff out free radicals before they can harm the eye’s sensitive tissues.”

The cited study established that lutein and zeaxanthin are good for eye health, but the study did not examine mechanisms. The proposed mechanism is plausible, but not supported by the cited study.

Claim from the book: “Take a daily multivitamin.”

Some, such as the president of the Australian Medical Association, say multivitamins are a waste. Some good studies showed multivitamins had no beneficial effects. But other good studies showed multivitamins did have beneficial effects. So the claim that you should take a multivitamin is controversial.

My personal take: multivitamins are cheap and easy, so they clearly pass a cost-benefit analysis, even if there’s a good chance that they don’t work.

Footnotes

I wrote this down because I had recently learned about the seed oil theory of obesity. Now, after having looked into it further, I’m reasonably confident that the seed oil theory (1) is fringe among nutrition scientists, (2) contradicts known biological mechanisms, and (3) contradicts most empirical findings from RCTs and cohort studies on the relationships between food and health. ↩
I come from the resistance training world where the preferred method of losing weight is to just eat less. I thought maybe we were a bunch of weirdos but this shows that the “eat less” method is pretty common among the general population. ↩
That’s why I read the book in the first place. ↩

What's the Healthiest Body Composition?

Wed, 22 May 2024 00:00:00 -0700

Last time, I found that the healthiest BMI range for all-cause mortality is 20–22. But BMI doesn’t tell the whole story. Most obviously, it doesn’t account for body fat vs. lean mass. All else equal, you’d rather have more muscle¹ and less fat.

So what’s the healthiest combination of lean mass + fat mass?

I’m not going to answer that question because I can’t. Instead, I will explain why I can’t, and then give a rough guess at the answer.

Scientists have been measuring and collecting data on BMI for decades. You can find plenty of giant BMI studies with three million participants in various countries.

We have much sparser data on body fat. Scientists didn’t start collecting data on body fat until the last few decades. And body fat is harder to measure—we have various methods for estimating body fat, but they’re all more complicated than calculating BMI.

I managed to scrounge together some studies on body fat and mortality. My best guess: the average woman should aim for a BMI of 21 with 20% body fat, and the average man a BMI of 21 with 10% body fat. (Subject to individual variation due to genetics and whatnot.)

Trans men should probably target the same body fat % as cis men, and likewise for trans women and cis women, because hormone therapy alters body fat distribution (Spanos et al. (2020)²).

The evidence weakly suggests that there is no lower bound on healthy fat mass, and no upper bound on healthy lean mass. We have so little mortality data on extremely lean + muscular people that we can’t say how healthy they are.

A more in-depth analysis would look at a variety of health indicators (blood pressure, HDL cholesterol, etc.) and use that to predict mortality. I didn’t do that, I just looked at mortality data.

Contents
Research on body composition and mortality
Research on waist circumference
Research on the relationship between BMI and body fat
Good research that nonetheless didn’t answer my question
Is there an upper bound on healthy lean mass?
Summary of findings
Notes

I looked at all the relevant research papers with at least 100 citations on Google Scholar. Here’s what I found.

Research on body composition and mortality

The best data comes from the big Danish follow-up study “Diet, Cancer and Health” which followed 50,000 Danish adults aged 50 to 64 from 1993 to 1997. Two different research papers analyzed the data from this study: Bigaard et al. (2004)³ and Bigaard et al. (2005)⁴. These papers break down BMI into body fat mass index (BFMI) and fat-free mass index (FFMI) (measured using bioelectrical impedance). Just like how BMI equals weight/height², BFMI equals bodyfat/height², and FFMI equals fat-free mass (a.k.a. lean mass) / height² (such that BFMI + FFMI = BMI).

Bigaard et al. (2004) found that mortality monotonically increases with BFMI above 5–6, and gets slightly worse below 5–6. For women, mortality monotonically decreases with FFMI, and for men it monotonically increases up to 19, at which point it starts increasing again. See Figure 1 (dark line represents men, light line represents women).

(BFMI and FFMI were controlled for each other, which fixes the problem that people with more lean mass usually have more fat mass.)

If we combine the ideal BFMI and FFMI from the Danish study, we get:

men:   5 BFMI + 19 FFMI = 24 BMI with 21% body fat
women: 7 BFMI + 17 FFMI = 24 BMI with 29% body fat

But this probably isn’t right. The best meta-analyses on BMI and mortality control for three big confounders: smoking, health conditions, and study follow-up length. All these confounders make low BMIs look unhealthier than they really are. The Bigaard et al. papers controlled for smoking, but not for health conditions (although participants were healthier than average due to selection effects⁵); plus the Danish study only lasted 5 years, and 5-year studies show a bias toward high BMIs.

In the BMI studies, controlling for these additional confounders reduces the apparent healthiest BMI by 3 or 4 points. And the Danish study found that mortality was minimized at 25 BMI,⁶ whereas in reality, it’s probably minimized at 20–22 BMI.

Undiagnosed health conditions probably decrease both lean mass and fat mass. That means low fat mass and low lean mass are both healthier than observational studies make them look.

If we naively subtract 4 points from the body-fat mass index,⁷ we get a new estimate of the healthiest body composition for men and women:

men:   1 BFMI + 19 FFMI = 20 BMI with 5% body fat
women: 3 BFMI + 17 FFMI = 20 BMI with 15% body fat

These numbers are extraordinarily low. But remember that I came up with these numbers by extrapolating what might happen if we controlled for confounders, so the numbers don’t come from actual data and could be pretty far off. My guess is they’re too low—maybe selection bias in the Diet, Cancer and Health Study had a similar effect to controlling for health, and I’ve over-corrected.

I don’t know about the exact numbers, but I expect that low body fat is good.

The data showed a trend of increasing mortality above 19 FFMI for men and 17 FFMI for women, but the trend wasn’t strong, so it’s possible that higher FFMIs are healthier (more on this later).

Other research has shown that ideal BMI increases with age, and the Danish study only included adults aged 50 to 64, so younger people should probably target lower body fat, and older people should aim higher. (It’s possible that younger people actually want to have less lean mass, not less body fat, but that doesn’t sound likely to me.)

Ideal BMI varies by ethnicity. But that’s probably because different ethnicities distribute body fat differently—e.g., South Asians carry more body fat at a given BMI, which makes BMIs on the higher end look unhealthier for South Asians. I don’t know of any cross-ethnic studies on body fat and mortality, but my best guess is that ideal body fat doesn’t vary much by ethnicity.

Lee et al. (2018)⁸, the second-most useful paper, looked at (predominantly white) males from the long-term Health Professionals Follow-Up Study and found that, controlling for confounders, mortality for men was minimized in the first quintile of fat mass and the third quintile of lean body mass (see Table 2). That is, men should aim for low fat mass and average lean mass.

(Lean mass and mortality had only a weak association except in the first quintile: low lean mass increased risk, but lean mass didn’t clearly affect mortality beyond that.)

Lee et al. (2018) found that male mortality was minimized at 56 kg of lean mass and 5–21 kg of fat mass.⁹ The median of the 1st quintile had a fat mass of 15 kg which corresponds to 21% body fat.¹⁰

Lee et al. found that mortality was minimized at a BMI of 24 or so, which suggests that it hasn’t fully adjusted for confounders. Adjusting fat mass index by 3 points to produce a BMI of 21 (assuming a height of 176 cm, which corresponds to a BMI of 23¹¹ for someone with 56 kg lean mass + 15 kg fat mass) predicts an ideal male body composition of

2 BFMI + 18 FFMI = 20 BMI with 9% body fat

Interestingly, Lee et al. found that if you exclude men with excessively low lean mass, then the mortality-minimizing BMI range is 18.5–20.4, not 20–22 as I previously reported. (The meta-analyses I looked at in my last post didn’t include data on body fat.) And the effect was surprisingly strong—men in the 18.5–20.4 group had a 15% lower mortality rate than the 20.5–22.4 group (see Table 3). This suggests two things:

Having a BMI on the low end (18.5–20.4) has large health benefits and large downsides relative to a mid-range BMI (20.5–22.4).
The downsides pretty much exclusively come from low lean mass, not low fat mass. If you have low-end BMI with sufficiently high lean mass, you get all the upside and ~none of the downside.

(Note: After excluding people with low lean mass, there were zero men left with BMIs below 18.5, out of a sample of 38,000. You might shoot for a BMI even lower than 18.5, but it looks pretty much impossible.¹²)

Research on waist circumference

A few studies looked at mortality and waist circumference. Waist circumference is often used as a proxy for body fat, and arguably it’s a better metric than body fat % because visceral fat (abdominal fat that’s distributed around the organs) carries greater health risks than subcutaneous fat (distributed under the skin).

Zhang et al. (2008)¹³ looked at American women in the Nurses’ Health Study. Participants had the lowest mortality in the 1st quintile of waist circumference—less than 28 inches or 71 cm, see Table 3. The study also looked at waist:hip ratio and found that the 1st quintile (<0.73) minimized mortality risk.

Zhang et al. reported a hazard ratio of 1.01 for the 28–29 inch waist group for all participants—that is, 28–29 inches was only slightly less healthy than <28 inches. But among never-smokers, the 28–29 inch group had a hazard ratio of 1.31. Reading between the lines, that implies that the healthiest waist circumference for non-smoking women is considerably less than 28 inches (71 cm).¹⁴

Hu et al. (2018)¹⁵ found that among middle-aged and elderly Chinese individuals, mortality was minimized at a waist circumference of 83–88 cm for men and 79–83 cm for women. This study used a relatively short follow-up (8.5 years), which means the true healthiest waist circumference is probably lower.

Baik et al. (2000)¹⁶ found that American men had the lowest mortality rate for waist circumferences in the 3rd quintile (36.3–37.9 inches or 92.2–96.3 cm), and waist:hip ratios in the 2nd quintile (0.90–0.91), see Table 4. The authors write that the higher mortality for lean men is probably due to confounding—respiratory disease causes men to lose weight and die sooner.

Research on the relationship between BMI and body fat

Some studies provide formulas to convert BMI to body fat. We can use those formulas to estimate ideal body fat from ideal BMI. This method doesn’t really work because the healthiest BMI substantially changes if you know someone’s fat mass / lean mass. But I’m going to use the method anyway and see what happens.

Data from the US National Health and Nutrition Examination Survey in Flegal et al. (2009)¹⁷ suggest a correspondence between ideal BMI and body fat % (see Table 3 and Table 4):

demographic	body fat
women under 40	25–30%
women over 60	30–35%
men under 40	15–20%
men over 60	20–25%

Meeuwsen et al. (2010)¹⁸ measured body fat in UK adults and came up with a linear formula to predict body fat percentage:
```
women: BF% =  -1.63 + 1.129 * BMI + 0.140 * age
men:   BF% = -13.51 + 1.129 * BMI + 0.140 * age
```
This formula predicts that, assuming you want a BMI of 21 at 30 years old, the healthiest body fat is 26% for women and 14% for men. At age 60, that rises to 30% for women and 19% for men.
Mills et al. (2007)¹⁹ found a BMI/body-fat correlation that suggests an ideal male body fat of 4% to 9%. But it also says a white male with a BMI of 18 has –1% body fat so I suspect it’s pretty inaccurate on the low end.²⁰
Ranasinghe et al. (2013)²¹ derived formulas to estimate body fat percentage in Sri Lankan adults:
```
women: BF% =  3.819 + 0.918 * BMI + 0.153 * age
men:   BF% = -9.662 + 1.114 * BMI + 0.139 * age
```
These formulas predict an ideal body fat of 28% for women and 18% for men at age 30.

Good research that nonetheless didn’t answer my question

Some other studies looked at the association between body fat/waist circumference and mortality, but only focused on the effects of overweightness/obesity, which doesn’t tell us the healthiest body composition.

Bigaard et al. (2005)⁴, using the Danish data set discussed previously, found that waist circumference predicted mortality even when controlling for BMI. It also found that, if you control for waist circumference, mortality monotonically decreases with both body fat mass index and fat-free mass index. (I wish I knew what to do with this information—I don’t know how to increase body fat without also increasing waist circumference.²²)
Britton et al. (2013)²³ found that excess body fat was associated with elevated mortality risk, but did not provide granularity on the low end of body fat.
Cerhan et al. (2013)²⁴, a pooled analysis of 11 cohort studies with a total of 650,000 white participants, found that mortality monotonically increased with waist circumference 90+ cm for men and 70+ cm for women. But it did not look at waist circumferences smaller than 90 cm for men or 70 cm for women.
Chen et al. (2019)²⁵ found greater mortality for waist circumferences of 90+ cm for Chinese men and 80+ cm for Chinese women, but did not provide granularity beyond that.
Jacobs et al. (2010)²⁶ found that among 100,000 (mostly white) Americans over a 10-year period, lower waist circumference was associated with reduced mortality, but it did not present data on waist circumferences smaller than 90 cm for men or 75 cm for women.
Lee et al. (2018)²⁷ found that body fat predicted mortality more strongly than BMI, and visceral fat more strongly still. It found that people in the bottom third by fat mass had the lowest mortality, as did the people in the bottom third by visceral-fat-to-subcutaneous-fat ratio.
Visscher et al. (2001)²⁸ found that male waist circumferences over 94 cm were associated with increased mortality, and failed to find a trend among women.

Is there an upper bound on healthy lean mass?

We know high BMIs are unhealthy, and we know that the main harms to health come from excess body fat. Does excess lean mass increase mortality risk? Or is it better to be as big and lean as possible?

The Danish Diet, Cancer and Health Study found that, after controlling for body fat, mortality risk decreases with lean mass up to a fat-free mass index (FFMI) of 17 for women and 19 for men, and starts increasing again above that point. The increasing trend above 19/17 is not statistically significant: after adjusting for both body-fat mass index (BFMI) and smoking, the upper portion of the FFMI curve has a slope of 1.03 with a 95% confidence interval of [0.93, 1.13] (see Table 4 from Bigaard et al. (2004)³).

This suggests that there’s no harm to having a FFMI above 17 for women or 19 for men, but there’s no benefit, either.

Alternatively, we can look at the relationship between resistance training and mortality because resistance training is closely associated with lean mass. A 2022 meta-analysis by Momma et al.²⁹ found that muscle-strengthening activity was associated with reduced mortality risk up to about 60 minutes per week, but with marginal resistance training above 60 minutes/week showing increasing mortality, and 140 minutes per week showing the same mortality risk as 0 minutes. The meta-analysis did not control for BMI, which I suspected could explain the increase in mortality, but one large cohort study (Patel et al. (2020)³⁰) did control for BMI (among other factors) and also found that resistance training was associated with increased mortality above 60 minutes or so.

I’ve seen some people dismiss this finding on the basis that resistance training is well-known to improve many health indicators like blood pressure and bone density. That’s true, but from what I’ve seen, the research primarily looks at low-dose resistance training, so we can’t say based on that research that increases in training volume monotonically improve health at high doses.

Why might muscle-strengthening activity increase mortality risk? I have seen two proposed mechanisms:

First, Miyachi (2013)³¹ found that resistance training increases arterial stiffness, which increases heart disease risk.

Second, Nuckols (2022)³² writes:

The studies in [Momma et al. (2022)] mostly used older subjects. It’s entirely possible that the optimal dose of resistance training for older adults is a lot lower than the optimal dose of resistance training for younger adults. For example, oxidative stress and generalized inflammation likely contribute to biological aging, and older adults have higher levels of oxidative stress and generalized inflammation. Resistance training causes oxidative stress and inflammation in a dose-dependent manner, but this is generally a good thing – those stressors are triggers for training-induced adaptations, and they also trigger your body to ramp up endogenous antioxidant production so that you can better handle future stressors (resulting in net reductions in inflammation and oxidative stress at rest). However, excessive training doses can induce too much oxidative stress and inflammation, setting the stage for a variety of deleterious outcomes (which we tend to collectively refer to as “overtraining”. It’s entirely possible – likely, even – that the threshold between productive training-induced stress and unproductive training-induced stress is considerably lower in older adults than younger adults.

Summary of findings

None of the studies fully controlled for confounders, so we have to make a judgment call as to what the results would look like if we did fully control for them.
Two big studies—the Danish Diet, Cancer and Health Study and the American Health Professionals Follow-Up Study—provide data on the association between mortality and lean mass / fat mass. If we extrapolate to what results we might see if we controlled for all confounders, these studies predict that men minimize mortality risk at 5–10% body fat and women minimize mortality risk at 15–20% body fat, as long as they have adequate lean mass.
Some research suggests that people in the lowest quintile of waist circumference have the lowest mortality risk, but we don’t know exactly what waist circumference minimizes mortality risk.
A great deal of research shows that on the higher end, higher body fat / bigger waist circumference is associated with increased mortality risk.
Higher lean body mass decreases mortality risk up to a fat-free mass index of 19 for men and 17 for women. Excess lean body mass above that point might increase mortality risk, but it’s not clear.

Notes

Technically we want to look at lean mass, not muscle mass. Lean mass refers to any body mass that isn’t fat, which includes muscles, bones, organs, etc. But the main way to gain lean mass is by building muscle. Resistance training does also increase bone mass, but much more slowly than muscle mass. I don’t know of any way to increase your organ mass, and even if you could, I don’t know that you would want to. ↩
Spanos, C., Bretherton, I., Zajac, J. D., & Cheung, A. S (2020). Effects of gender-affirming hormone therapy on insulin resistance and body composition in transgender individuals: a systematic review. See Table 1. ↩
Bigaard, J., Frederiksen, K., Tjønneland, A., Thomsen, B. L., Overvad, K., Heitmann, B. L., & Sørensen, T. I. (2004). Body fat and fat-free mass and all-cause mortality. ↩ ↩²
Bigaard, J., Frederiksen, K., Tjønneland, A., Thomsen, B. L., Overvad, K., Heitmann, B. L., & Sørensen, T. I. A. (2005). Waist circumference and body composition in relation to all-cause mortality in middle-aged men and women. ↩ ↩²
Study participants died at only about half the rate of the general population, see Bigaard et al. (2005)³. ↩
Looking purely at BMI, the study found that mortality was minimized at 25. Looking at the combined healthiest BFMI and FFMI, mortality was minimized at 24 for men and 23 for women. I believe the discrepancy comes from the fact that people with low body fat often have insufficient lean mass. ↩
Instead of subtracting 4 points from FFMI, I could subtract 2 points from BFMI and 2 points from FFMI. That seems worse to me because I would expect that few people with a FFMI of 19/17 have health conditions that reduce their lean mass—if they did, their FFMI would be considerably lower. So the observation that an FFMI of 19/17 minimizes mortality shouldn’t be confounded by health conditions. ↩
Lee, D. H., Keum, N., Hu, F. B., Orav, E. J., Rimm, E. B., Willett, W. C., & Giovannucci, E. L. (2018). Predicted lean body mass, fat mass, and all cause and cause specific mortality in men: prospective US cohort study. ↩
According to their “model 2”, which controls for various factors but does not control for health conditions. ↩
Figure 1 shows the lowest mortality at a fat mass of 21 kg, which corresponds to a surprisingly-high 27% body fat. I believe this is an artifact of the model used to generate Figure 1. Figure 1 fits the data to a cubic spline model which smooths out the bumpiness in the quintiles. The 1st quintile had the lowest mortality rate but its 95% CI overlapped with the 2nd and 3rd quintiles, so the cubic spline model smoothed these out and ended up predicting that mortality is minimized in the 3rd quintile. Thanks to co-author Edward Giovannucci for this explanation. ↩
I use 23 instead of 24 because my analysis of Bigaard et al. (2004) found that splitting out lean mass and fat mass reduced apparent healthiest BMI by 1 point. ↩
For me personally, if I somehow dropped by body fat to 0% without losing any lean mass, I’d still have a BMI of 20. ↩
Zhang, C., Rexrode, K. M., Van Dam, R. M., Li, T. Y., & Hu, F. B. (2008). Abdominal obesity and the risk of all-cause, cardiovascular, and cancer mortality: sixteen years of follow-up in US women. ↩
My reasoning: If the 28–29 group looks only slightly less healthy than the <28 group for all participants (including smokers), and if the mortality curve follows an U shape, then the nadir of the curve should be just below 28 inches. If the 28–29 group looks substantially less healthy, then the nadir must be considerably lower than 28 inches because that way most of the function’s mass near the nadir occurs below 28 inches. ↩
Hu, H., Wang, J., Han, X., Li, Y., Wang, F., Yuan, J., Miao, X., Yang, H., & He, M. (2018). BMI, waist circumference and all-cause mortality in a middle-aged and elderly Chinese population. ↩
Baik, I., Ascherio, A., Rimm, E. B., Giovannucci, E., Spiegelman, D., Stampfer, M. J., & Willett, W. C. (2000). Adiposity and mortality in men. ↩
Flegal, K. M., Shepherd, J. A., Looker, A. C., Graubard, B. I., Borrud, L. G., Ogden, C. L., . & Schenker, N (2009). Comparisons of percentage body fat, body mass index, waist circumference, and waist-stature ratio in adults. ↩
Meeuwsen, S., Horgan, G. W., & Elia, M (2010). The relationship between BMI and percent body fat, measured by bioelectrical impedance, in a large adult sample is curvilinear and influenced by age and sex. ↩
Mills, T. C., Gallagher, D., Wang, J., & Heshka, S. (2007). Modelling the relationship between body fat and the BMI. ↩
Then again, I would need to have –9% body fat to get my BMI down to 18 without losing lean mass, so maybe they’re on to something. ↩
Ranasinghe, C., Gamage, P., Katulanda, P., Andraweera, N., Thilakarathne, S., & Tharanga, P (2013). Relationship between body mass index (BMI) and body fat percentage, estimated by bioelectrical impedance, in a group of Sri Lankan adults: a cross sectional study. ↩
I’ve seen claims that low-intensity exercise disproportionately burns visceral fat (Brobakken et al. (2023)³³), and cortisol (the “stress hormone”) disproportionately adds visceral fat. But I think even if you exercise a lot and minimize stress, you still don’t want to have too much body fat. ↩
Britton, K. A., Massaro, J. M., Murabito, J. M., Kreger, B. E., Hoffmann, U., & Fox, C. S. (2013). Body fat distribution, incident cardiovascular disease, cancer, and all-cause mortality. ↩
Cerhan, J. R., Moore, S. C., Jacobs, E. J., Kitahara, C. M., Rosenberg, P. S., Adami, H. O., Ebbert, J. O., English, D. R., Gapstur, S. M., Giles, G. G., & Horn-Ross, P. L. (2014). A pooled analysis of waist circumference and mortality in 650,000 adults. ↩
Chen, Y., Yang, Y., Jiang, H., Liang, X., Wang, Y., & Lu, W. (2019). Associations of BMI and waist circumference with all-cause mortality: a 22-Year cohort study. ↩
Jacobs, E. J., Newton, C. C., Wang, Y., Patel, A. V., McCullough, M. L., Campbell, P. Thun, M. J., & Gapstur, S. M. (2010). Waist circumference and all-cause mortality in a large US cohort. ↩
Lee, S. W., Son, J. Y., Kim, J. M., Hwang, S. S., Han, J. S., & Heo, N. J. (2018). Body fat distribution is more predictive of all-cause mortality than overall adiposity. ↩
Visscher, T. L. S., Seidell, J. C., Molarius, A., van der Kuip, D., Hofman, A., & Witteman, J. C. M. (2001). A comparison of body mass index, waist–hip ratio and waist circumference as predictors of all-cause mortality among the elderly: the Rotterdam study. ↩
Momma, H., Kawakami, R., Honda, T., & Sawada, S. S (2022). Muscle-strengthening activities are associated with lower risk and mortality in major non-communicable diseases: a systematic review and meta-analysis of cohort studies. ↩
Patel, A. V., Hodge, J. M., Rees-Punia, E., Teras, L. R., Campbell, P. T., & Gapstur, S. M (2020). Peer Reviewed: Relationship Between Muscle-Strengthening Activity and Cause-Specific Mortality in a Large US Cohort. ↩
Miyachi, M (2013). Effects of resistance training on arterial stiffness: a meta-analysis. ↩
Nuckols, G. (2022). What is the optimal dose of resistance training for longevity? ↩
Brobakken, M. F., Krogsæter, I., Helgerud, J., Wang, E., & Hoff, J (2023). Abdominal aerobic endurance exercise reveals spot reduction exists: A randomized controlled trial. ↩

What's the Healthiest BMI?

Sun, 05 May 2024 00:00:00 -0700

TLDR: 20 to 22.

Confidence: Somewhat likely.

Last updated 2024-05-18.

Health organizations such as the American Heart Association recommend a body-mass index (BMI) of 18.5 to 25. But that’s a wide range. Surely we can say something more specific, right? I don’t want to know what’s acceptable, I want to know what’s optimal. What’s the exact best BMI for health?¹

I couldn’t find an answer with a simple web search, which means now I have to write a post about it.²

Contents
The big BMI studies
Do studies still overestimate ideal BMI?
Details: Age, sex, ethnicity, and cause-specific mortality
Exercise
A final bit of evidence
Changelog
Notes

The big BMI studies

I read³ two big meta-analyses on BMI and mortality:

Global BMI Mortality Collaboration (2016). Body-mass index and all-cause mortality: individual-participant-data meta-analysis of 239 prospective studies in four continents.
Aune, Dagfinn; Sen, Abhijit; Prasad, Manya; Norat, Teresa; Janszky, Imre; Tonstad, Serena; Romundstad, Pål; Vatten, Lars J (2016). BMI and all cause mortality: systematic review and non-linear dose-response meta-analysis of 230 cohort studies with 3.74 million deaths among 30.3 million participants.

We have absolute mountains of data on the association between BMI and mortality. In spite of that, it’s hard to say what BMI minimizes mortality risk because it depends on how you interpret the data.

The raw mortality data says the healthiest BMI is 25—right on the cusp of the “overweight” range.

But that’s wrong because confounding variables make low BMI look worse than it really is. Smokers tend to weigh less, and various health problems cause people to lose weight.

If you adjust for smoking and health, and you also limit to studies with ≥15-year follow-ups to avoid confounding by undiagnosed illnesses,⁴ the ideal BMI is 20–22. Both meta-analyses agree on this conclusion (see Table 3 and Appendix eTable 8 in the first paper, and see the abstract in the second paper).

Here is a table of hazard ratios (i.e., mortality rates relative to the 22.5–25 BMI group), taken from the first meta-analysis:

BMI	18.5–20	20–22.5	22.5–25	25–27.5
hazard ratio	1.13	1.00	1.00	1.07

(Berrington de Gonzalez et al. (2010)⁵, a smaller (but still large) meta-analysis of 1.5 million white individuals, provides mortality data to a granularity of 1 point of BMI in its Supplemental Appendix, Figure 1. It finds a uniform mortality rate between 20 and 25.)

Mortality appears to increase slowly above 25 and more quickly below 20. If we interpolate the shape of the mortality curve, it looks like the exact ideal BMI just under 23.⁶ But I think that’s too high, as I will explain in the next section.

Do studies still overestimate ideal BMI?

I have reason to suspect that these meta-analyses somewhat overestimate the ideal BMI even after controlling for smoking, health, and study follow-up time. The Global BMI Mortality Collaboration’s Supplemental Appendix has a nice chart showing what happens when you successively adjust for confounders:

The Y axis shows mortality rate. Notice how the un-adjusted mortality rate for the 20–22.5 group starts out considerably higher than for the 22.5–25 group, and (almost) every time you adjust for a confounder, its mortality rate gets lower. By the end, the mortality rates for 20–22.5 and for 22.5–25 are nearly identical.

(More generally, as you adjust for more things, all BMI groups on the low end look better and all groups on the high end look worse.)

It seems likely that this meta-analysis didn’t adjust for every possible confounder. Whenever we make an adjustment, the lower BMIs look better. So presumably, if we adjusted for the as-yet-undiscovered cofounders, the ideal BMI would look lower than what these meta-analyses report.

And indeed, Aune et al. (2016) found that BMIs in the 20–22.5 range looked healthier than 22.5–25 when restricted to studies with a ≥20-year follow-up (see Appendix 2, Table E). Global BMI Mortality Collaboration (2016) tried restricting to >5-year follow-ups (in the green and black points shown in eFigure 6 above), but it appears that 5 years is not enough to fully eliminate confounding by undiagnosed diseases.

Aune et al. (2016)’s Table E reported hazard ratios by BMI with ≥20-year follow-up:

BMI	17.5	20	22	23	24	25
hazard ratio	1.06	0.99	0.99	1.00	1.02	1.06

The NCI Cohort Consortium meta-analysis⁵ found that both the 18.5–20 and 20–22.5 groups have lower mortality than 22.5–25 if you only look at studies with ≥15-year follow-ups—both the low-BMI groups had hazard ratios of 0.92 (see Table 2).

Before I said the ideal BMI was around 23. If we extrapolate the pattern of adjustments from Global BMI Mortality Collaboration’s eFigure 6, or look at Aune et al. (2016)’s 20-year follow-up data, it looks like the ideal BMI is more like 21. (But really, we could just as well say that the ideal BMI is anywhere from 20 to 22 because mortality doesn’t detectably vary within that range.)

Details: Age, sex, ethnicity, and cause-specific mortality

The Global BMI Mortality Collaboration study broke down mortality by age, sex, and cause of death (see Supplemental Appendix).

Ideal BMI (before adjusting for confounders) goes up with age:

Age	Ideal BMI
35–49	20–22.5
50–69	22.5–25
70–89	25–27.5

Ideal BMI does not vary by sex, but males are more sensitive to BMI than females. That is, females can tolerate greater deviations from the healthy BMI range without as much increase in mortality risk.

As far as I could find, there is no research whatsoever on the relationship between BMI and mortality for transgender people. Presumably, trans individuals should also target a BMI in the 20 to 22 range, but I can’t empirically confirm that.

There’s mixed evidence on BMI and ethnicity:

In 2004, the World Health Organization suggested⁷ that most Asian sub-populations should use lower BMI cutoffs to define overweightness and obesity. A consensus statement on Asian Indians⁸ (the ethnicity that appears most sensitive to high BMI) proposed defining “normal” BMI for Indians as 18–23, overweight as >23, and obese as >25.
A large UK study⁹ found that different ethnicities had different mortality rates on the low and high ends of BMI, but not in the middle, i.e., the healthiest BMI did not vary by ethnicity. (See Table S2.8 and Figure S2.7 in the Supplemental Appendix.)¹⁰ This is consistent with the consensus statements—the cutoffs for overweight/obese should probably vary by ethnicity—but it also suggests that ideal BMI does not vary much.
There is theoretical reason to expect ideal BMI to depend on ethnicity. Namely, different ethnicities tend to distribute body fat differently, with Asians carrying more body fat at a given BMI than the average human, and blacks carrying slightly less. (But diabetes risk appears to increase slightly more rapidly with BMI for blacks than for whites, which contradicts this.¹¹)

We simply don’t have enough evidence to draw strong conclusions about how ethnicity affects ideal BMI. As best I can tell, all ethnicities should target a BMI in the 20–22 range.

Ideal BMI varies by cause of death. This table (adapted from Global BMI Mortality Collaboration’s Supplemental Appendix, eTable 15) shows the hazard ratios for four BMI ranges broken down by some of the most common causes of death.

	18.5–20	20–22.5	22.5–25	25–27.5
coronary heart disease	0.95	0.89	1.00	1.18
other cardiovascular disease	1.14	0.98	1.00	1.11
stroke	1.15	1.01	1.00	1.05
respiratory disease	1.73	1.22	1.00	1.00
cancer	1.01	0.96	1.00	1.05

The second meta-analysis, Aune et al. (2016), claimed “weight loss can precede the diagnosis of some neurological and respiratory diseases by as much as 10-15 years.” The effect discussed in the previous section, where longer follow-ups make lower BMIs look better, appears primarily driven by neurological and respiratory disease.

Exercise

Exercise has two major confounding effects:

Much of the harm of a high BMI comes from a living sedentary lifestyle. Controlling for cardiorespiratory fitness makes BMI look less important, especially at high BMIs.¹²
Resistance training increases muscle mass, which raises BMI but not in a harmful way. If you control for resistance training, that should lower the apparent ideal BMI for low-muscle individuals while raising it for high-muscle individuals.¹³

(I thought exercise might have a third effect: exercise disproportionately reduces respiratory disease, which makes low BMI look healthier. This turned out to be true, but adjusting for it only reduces the hazard ratio for the 20–22.5 group by 1%.¹⁴)

In a meta-analysis, Barry et al. (2013)¹² found that physical activity substantially reduces the negative health effects of overweightness and obesity, but did not look at how physical activity affects individuals with a BMI of less than 25.

Is exercise more helpful for overweight than for underweight individuals? (If so, that could raise the ideal BMI for physically active people.) Or is it equally good on both ends of the BMI spectrum? (In which case the ideal BMI wouldn’t change.)

I found three studies which suggest that it’s the latter: exercise expands the healthy BMI range without obviously changing the ideal BMI.¹⁵

While we’re on the subject of exercise, I should mention that exercise matters more for health than BMI, especially if you already fall within the “normal” BMI range. Exercise halves your mortality rate¹⁶ (for the collective “you”; individual results may vary). Moving your BMI from 25 to 22 only decreases your mortality rate by around 5%.

(Reducing BMI from 25 to 22 has an effect size of 0.5. Going from sedentary to physically active (as defined by the physical activity guidelines¹⁷) has an effect size of 17.)

A final bit of evidence

In Eat, Drink, and Be Healthy, Walter Willett—who co-authored the Global BMI Mortality Collaboration meta-analysis—had this to say about the ideal BMI.

The 2015–2020 Dietary Guidelines for Americans sets healthy weights as those corresponding to BMIs between 18.5 and 25. […] Panel members agreed that the risk of heart disease, diabetes, and high blood pressure begins to climb at a BMI of 22 or so. But they didn’t feel justified choosing such a low number as the cutoff between healthy and unhealthy weights, because doing so would have labeled a large majority of the U.S. population as overweight. […] [B]ut many people with a BMI of 23 to 25 are not at their healthiest weight.

[…]

What about BMIs under 18.5, which the government’s tables say isn’t healthy? This can, indeed, signal an unhealthy weight, especially if an individual has been losing weight or has an eating disorder. But people who have maintained a low BMI while eating healthfully and being active are usually just fine and have no reason to increase their weight.

Changelog

2024-05-18: Add a comment on BMI for transgender people, and fix a numeric error.

Notes

It makes sense why the American Heart Association doesn’t care about knowing the exact best BMI: why worry about people with slightly sub-optimal BMIs when a third of Americans have BMIs over 30? From a population-level perspective, getting everyone with a BMI of 24 down to 22 would reduce mortality by approximately zero percent.

But my BMI is 24 and I would like to know if I should lower it. ↩
In the process of writing this post, I went through this cycle five times:
1. My research is done! I will do a final read-through to tighten up the prose.
2. (in the middle of reading) Hmmm, this part makes me wonder about something. I will investigate further.
3. (5 hours later) Ok, I think I’ve figured it out. Let me write up these new findings.
4. (post is now 500 words longer) My research is done! I will do a final read-through to tighten up the prose.
At no point in my many revisions did the TLDR change, so maybe it was all a waste of time, but at least now I have 14 citations instead of three, which makes me look more scholarly. ↩
By which I mean I read the introduction and conclusion and looked at the tables and figures. ↩
Honestly I don’t fully understand the reasoning behind this, but both meta-analyses claim that short-term studies are confounded by undiagnosed diseases that decrease BMI and increase mortality.

I think the idea is that long-term studies ignore anyone who dies in the first few years, and anyone who dies after that probably didn’t have an undiagnosed illness at the beginning of the study or else they would have died sooner. Many studies use 5-year delays to combat this, but apparently many illnesses go undiagnosed for more than 5 years, and you need at least 15 years to eliminate confounding. In fact, Aune et al. (2016) found that even a 15-year follow-up is noticeably more confounded than a 20-year follow-up—see Appendix 2, Table E. ↩
Berrington de Gonzalez A, Hartge P, Cerhan JR, Flint AJ, Hannan L, MacInnis RJ, Moore SC, Tobias GS, Anton-Culver H, Freeman LB, Beeson WL, Clipp SL, English DR, Folsom AR, Freedman DM, Giles G, Hakansson N, Henderson KD, Hoffman-Bolton J, Hoppin JA, Koenig KL, Lee IM, Linet MS, Park Y, Pocobelli G, Schatzkin A, Sesso HD, Weiderpass E, Willcox BJ, Wolk A, Zeleniuch-Jacquotte A, Willett WC, Thun MJ. Body-Mass Index and Mortality among 1.46 Million White Adults. ↩ ↩²
I took the four groups from 18.5 to 27.5 (I included the 25–27.5 group because it has similar mortality risk to 18.5–20) and fitted a quadratic curve to them. The curve had a minimum at 23.1.

Originally I tried fitting a quadratic curve to eight BMI groups (from 15 BMI up to 40), but the curve wasn’t a good fit. When I fit the eight groups to a quartic curve, it fit well and had a minimum at 22.7, although a fourth-degree polynomial might have too many degrees of freedom.

(The quadratic curve is the little blue one, and the quartic curve is the big orange one.) ↩
WHO Expert Consultation. (2004). Appropriate body-mass index for Asian populations and its implications for policy and intervention strategies. ↩
Misra A, Chowbey P, Makkar BM, Vikram NK, Wasir JS, Chadha D, Joshi SR, Sadikot S, Gupta R, Gulati S, Munjal YP. (2009). Consensus Statement for Diagnosis of Obesity, Abdominal Obesity and the Metabolic Syndrome for Asian Indians and Recommendations for Physical Activity, Medical and Surgical Management. ↩
Bhaskaran K, dos-Santos-Silva I, Leon DA, Douglas IJ, Smeeth L. (2018). Association of BMI with overall and cause-specific mortality: a population-based cohort study of 3·6 million adults in the UK ↩
This study had wide confidence intervals on its mortality rates for non-white ethnicities because most subjects were white. And the study only covered UK residents, which could affect the results due to local diet or selection effects from immigration. But it was the only study I could find that looked at all-cause mortality by ethnicity. I found good studies that looked at diabetes morbidity, but no others on all-cause mortality. ↩
Shai I, Jiang R, Manson JE, Stampfer MJ, Willett WC, Colditz GA, Hu FB. (2006). Ethnicity, Obesity, and Risk of Type 2 Diabetes in Women. ↩
Barry VW, Baruth M, Beets MW, Durstine JL, Liu J, Blair SN. (2013). Fitness vs. fatness on all-cause mortality: a meta-analysis. ↩ ↩²
Abramowitz MK, Hall CB, Amodu A, Sharma D, Androga L, Hawkins M. (2018). Muscle mass, BMI, and mortality among adults in the United States: A population-based cohort study. ↩

Zhao et al. (2020)¹⁶, a cohort study on exercise and mortality,¹⁸ reports in Table 2 that exercise has a hazard ratio of 0.48 on all-cause mortality (meaning people who meet the physical activity guidelines¹⁷ are only 48% as likely to die as people who don’t). A table of relevant hazard ratios for individuals who meet the exercise guidelines:

	All-cause mortality	Lower respiratory disease	Flu/pneumonia
hazard ratio	0.48	0.21	0.36
hazard ratio, controlling for BMI and other lifestyle factors	0.58	0.29	0.45

Whether we control for BMI or not, exercise has a bigger effect on respiratory disease than on all-cause mortality. To calculate how much this matters, we need to know what proportion of people die of respiratory disease.

Our World in Data says that, for North America and Europe, respiratory illnesses account for around 10% of deaths, and they consist of around 2/3 upper respiratory and 1/3 lower respiratory disease. Using the weighted average of the exercise hazard ratios from Zhao et al. (2020) gives an overall hazard ratio for respiratory disease of 0.40.

I calculated exercise and BMI hazard ratios for all causes excluding respiratory disease as the solution to all-cause HR = 0.1 * respiratory HR + 0.9 * non-respiratory HR.

Then I adjusted the BMI mortality rates for respiratory disease and for non-respiratory mortality using the respective exercise hazard ratios and re-combined them into an all-cause HR for physically active individuals.

I used the exercise HRs controlling for BMI rather than the uncontrolled HRs, on the theory that exercise probably lowers BMI on average, and we want to know the health effect of exercise irrespective of how it changes BMI.

It would be more accurate to fully break out mortality by cause instead of just breaking it into respiratory and non-respiratory, but I can’t really do that because Global BMI Mortality Collaboration (2016) and Zhao et al. (2020) don’t classify causes in the same way.

This table presents the hazard ratios (HRs) of different BMIs for people who meet the physical activity guidelines, accounting for respiratory disease (normalized so the 22.5–25 group has a HR of 1).

	15–18.5	18.5–20	20–22.5	22.5–25	25–27.5
All-cause hazard ratio	1.51	1.13	1.00	1.00	1.07
All-cause HR with exercise	1.45	1.11	0.99	1.00	1.07

Accounting for the effect of exercise on respiratory disease did make lower BMIs look slightly better, but it made hardly any difference. ↩

1. Berrington de Gonzalez et al. (2010)⁵ (in its Supplemental Appendix, Table 7)¹⁹ found that physical activity reduced hazard ratios for both low and high BMIs. It also slightly decreased the 20–22.5 BMI hazard ratio from 1.02 to 0.99, which could mean exercise slightly decreases the ideal BMI, but the change in hazard ratio was highly non-significant (p > 0.6) and non-monotonic (the moderate-activity group with 20–22.5 BMI had a hazard ratio of 0.97, lower than either the low-activity or high-activity group).
2. Garfinkel et al. (1988)²⁰ wrote that exercise had a greater effect on mortality for both underweight and overweight individuals than for normal-weight people, but the paper did not provide numbers.
3. Lee & Kim (2020)²¹ looked specifically at underweight older adults and found that exercise had a hazard ratio of 0.68. Zhao et al. (2020)¹⁶ found an exercise hazard ratio of 0.48 for the general population, but Zhao used a stricter definition of physical activity, so these numbers aren’t directly comparable.
↩
Zhao M, Veeranki S P, Magnussen C G, Xi B. (2020). Recommended physical activity and all cause and cause specific mortality in US adults: prospective cohort study. ↩ ↩² ↩³
To meet the physical activity guidelines, you must do both
1. At least one of:
  - 150 minutes of moderate physical activity per week
  - 75 minutes of vigorous physical activity per week
2. Muscle strengthening activity at least two days per week
↩ ↩²
I cited this cohort study instead of a meta-analysis because I couldn’t find any meta-analyses that looked at exercise and cause-specific mortality, specifically including respiratory disease. The 2018 Physical Activity Guidelines Advisory Committee report (which is the most comprehensive meta-analysis I could find (it’s actually a meta-meta-analysis)) looked at all-cause mortality and heart disease, but not respiratory disease. ↩
Berrington de Gonzalez et al. (2010) only included white individuals. The mortality <> BMI relationship varies a little bit with race, but I have no reason to believe that race affects the effect of physical activity. ↩
Garfinkel L, Stellman SD. (1988). Mortality by relative weight and exercise. ↩
Lee I, Kim B. (2020). Association between Estimated Cardiorespiratory Fitness and All-cause Mortality in Underweight Older Adults. ↩

Caffeine Cycling Self-Experiment

Thu, 11 Apr 2024 00:00:00 -0700

Last updated 2024-07-26 to clarify wording.

Confidence: Likely.

I conducted an experiment on myself to see if I would develop a tolerance to caffeine from taking it three days a week. The results suggest that I didn’t. Caffeine had just as big an effect at the end of my four-week trial as it did at the beginning.

This outcome is statistically significant (p = 0.016), but the data show a weird pattern: caffeine’s effectiveness went up over time instead of staying flat. I don’t know how to explain that, which makes me suspicious of the experiment’s findings.

Contents
Experimental procedure
Calibration phase
Abstinence phase
Experimental phase
What explains these results?
An offer to readers
Notes

Experimental procedure

(I described this procedure in a pre-registration on a previous post.)

I test my reaction time by taking the humanbenchmark.com test twice in a row. One test consists of 5 reaction events, so this gives a total of 10 reaction events, taking my reaction time at that moment as the average of the 10 reaction times. I take the test using the same computer, monitor, and mouse so that latency is consistent.

I measure the effect of caffeine using a reaction time test because (a) caffeine is known to improve reaction time, (b) reaction time is easy to test,¹ (c) it’s unlikely to improve with practice so it makes for a good consistent test variable, and (d) it’s hard to placebo-effect myself into improving my reaction time (which is important because I am not blinding myself²).

I conduct three phases as specified below:

Phase 1. Calibration phase. Continue drinking coffee three days a week as I have been for the past several years: two cups of coffee (~24 ounces) with three scoops of grounds,³ always the same brand,⁴ drunk on Monday, Wednesday, and Friday morning.

Take a reaction time test twice a day (following the schedule described in phase 3 below). Continue for four weeks. Plot a regression of my reaction time across the four weeks. The purpose of the calibration phase is to ensure that my reaction time does not improve from practicing every day—the regression line should be flat.

Phase 2. Abstinence phase. Abstain from caffeine for one week (9 days total, in between the last Friday of the calibration phase and the first Monday of the test phase). Test reaction time every day and measure the slope of reaction times across the 9 days. If I was habituated to caffeine in phase 1 then my reaction time should improve over the course of phase 2 as my tolerance wears off.

Phase 3. Experimental phase. Resume drinking coffee three days a week and continue for four weeks. Take a reaction time test twice a day, at (say) 8am and then 10am—the exact time doesn’t matter, but take first test before having coffee and the second 30+ minutes after coffee. (Or on days when I don’t have coffee, take the first test after I wake up and the second test an hour or two later.)

Calibration phase

I wrote the first draft of this section after completing the calibration phase, and I wrote the first draft of Abstinence phase after completing the abstinence phase. So when I wrote them, I didn’t know the full results of the experiment yet.

I ran a four-week calibration phase to check some assumptions of the experiment:

Caffeine should improve reaction time.
If my post-caffeine test outperforms my pre-caffeine test, it should be because of the caffeine, not because my reaction time gets better later in the day.
Practicing shouldn’t improve my reaction time.

The calibration phase confirmed the first two assumptions:

The post-caffeine tests outperformed the pre-caffeine tests by an average of –13 ms (p = 0.025).⁵
On no-caffeine days, the second test did not outperform the first test (difference 0.4 ms, p = 0.9).

The third assumption was sort-of-confirmed: my reaction time did not improve over the course of the calibration phase. In fact, it got worse at a rate of 0.87 ms/day (p = 0.014) for caffeine tests and 1.04 ms/day (p = 0.006) for no-caffeine tests.

(Remember that higher reaction times are worse.)

Why did my reaction times get worse? It’s not because I was getting habituated to caffeine. I had already been taking caffeine 3 days a week for years, so I would have been fully habituated long before starting the calibration phase.

Could it be because I started sleeping worse? That’s part of the reason. I regressed reaction time (on no-caffeine tests) against time spent in bed the previous night. Over the full experiment (not just the calibration phase⁶), each additional hour of sleep improved my reaction time by 4.9 ms⁷ (p < 0.0002, r² = 0.24). Controlling for time-in-bed flattened the slope of reaction time across non-caffeine tests from 1.04 ms/day to 0.77 ms/day. But that still leaves almost 3/4 of the slope unexplained.⁸

My best explanation: as the test became part of my routine, I subconsciously started taking it less seriously and started having a harder time staying focused. On most trials I get reaction times around 250–270ms, but occasionally I lose focus and end up taking 330ms or longer to react. As I recall, that didn’t happen at all during the first week or two of the calibration phase, it only started happening later.

My reaction time can’t continue getting worse forever. But this does raise a concern about the results from the experimental phase: if my performance gets worse during the experimental phase, it might be because I’m getting habituated to caffeine, or it might be a continuation of the trend that happened during the calibration phase.

Abstinence phase

I abstained from caffeine for 9 days. If I had previously been habituated to caffeine, you’d expect my reaction time to improve over the course of the week as my caffeine withdrawal subsides. Specifically, if caffeine improves reaction time by 13 ms, you’d expect my reaction time to get better by 13 ms over the course of the 9 days (= 1.44 ms/day). Instead, my reaction time got worse at a rate of 0.77 ms/day. This is not significantly different from 0 (p = 0.4), but it is significantly different from –1.44 ms/day (p = 0.04).

This plot shows the likelihood function for caffeine retention as indicated by the slope of reaction time over the abstinence phase:⁹

The maximum-likelihood estimate is 1.53—that is, caffeine becomes 53% more effective after my body adapts to it. If my reaction time got worse during abstinence, that implies caffeine tolerance was making my reaction time better. I’m pretty sure that’s wrong—my reaction time must have gotten worse for some other reason.

Controlling for time-in-bed flattens the slope to nearly 0:

Experimental phase

For the experimental phase, I resumed taking caffeine 3 days a week.

Over the course of the four-week phase, I did not become habituated to caffeine. In fact, I became sensitized—caffeine got more effective, not less. My post-caffeine reaction time changed at a rate of –0.39 ms/day (p = 0.016) (remember, a negative number means faster reaction time). My reaction time without caffeine also improved to a lesser extent (slope = –0.23 ms/day (p = 0.4); the difference in slopes was not statistically significant (p = 0.5)). So either I did not develop a caffeine tolerance, or any caffeine tolerance I developed was outweighed by some force working in the opposite direction.

In the plot below, “nocaf” indicates reaction times without having taken caffeine first, and “caf” gives reaction times tested approximately 30 minutes after taking caffeine.

According to these regression lines, reaction time improved by a total of 5.7 ms with caffeine and 9.8 ms without caffeine over the four weeks.

You will notice a very low point on day 0. That happened because I accidentally reacted too early on one of the trials, but by pure coincidence I reacted at just the right moment to score a ~20ms reaction time.¹⁰ If I run a regression starting one day later to exclude this anomaly, the slopes for caffeine tests and no-caffeine tests look comparable (caffeine slope = –0.47 ms/day, p = 0.014; no-caffeine slope = –0.54 ms/day, p = 0.027).

This plot shows the likelihood function of caffeine retention according to the slope of performance on caffeine tests (excluding day 0):⁹

The likelihood function has a mean and a maximum of 1.86, which says caffeine becomes 86% more effective after my body adjusts to it. This likelihood function has only 0.6% of its mass below retention = 1 (i.e., retention = 1 has a p-value of 2 * 0.006 = 0.012). This likelihood function strongly suggests that caffeine gets more effective over time, not less.

I don’t believe this result. It’s more likely that some confounding factor caused my reaction time to improve. but I can’t think of what that confounding factor might be.

What explains these results?

Could my reaction time have improved because I was getting more sleep? If I control for time spent in bed the previous night, the slope of reaction times vs. days does flatten from –0.54 to –0.40, but this only explains about 1/4 of the slope.¹¹

Maybe my performance improved due to the cumulative effect of sleeping well for many nights in a row? But I spent less time in bed during the experimental phase (average 8.73 hours) than during the calibration phase (8.99 hours), so if anything I should have gotten worse, not better.

Could this be a genuine result? Could caffeine actually become more effective when I take it for longer? Three experiments on rats¹² found something similar: rats who took caffeine daily developed a tolerance, but rats who took caffeine on alternating days became sensitized (its effect got larger). (Plus one study¹³ found neither tolerance nor sensitization.)

This hints that caffeine sensitization is a real thing. But the results from the rat experiments don’t look the same as my results. They found that rats’ performance on caffeine days increased over the course of the experiments while performance on placebo days stayed flat. In contrast, my own performance improved both on caffeine days and on “placebo” days (I didn’t take a placebo, I just took nothing).

But perhaps caffeine sensitization works differently in humans than in rats. If a habituated caffeine user experiences withdrawal symptoms, then maybe a sensitized user experiences “anti-withdrawal”, making them perform better even when they don’t take caffeine. Maybe my brain thinks, “I don’t know what’s going on, the caffeine levels inside me keep fluctuating, I’d better delete some neurotransmitter receptors just to be safe,” and this ends up making me more alert with or without caffeine. But why didn’t it happen that way in the rat studies?

Earlier, when I talked about the calibration phase, I hypothesized that my performance got worse because I subconsciously stopped taking the tests as seriously. Could the opposite have happened in the experimental phase?

I don’t think so. I noticed my performance getting worse when I looked at the results just after finishing the calibration phase. So I might have mentally resolved to focus harder. But if so, you’d expect my performance to jump up and stay persistently high, or perhaps to jump up and then decline again, but not to start low and then steadily improve.

I thought the results might have something to do with my computer’s latency, but my experiment already controlled most of the parameters that might change the latency (I always tested on the same computer with the same hardware in a browser with a single tab open and with no other applications open except Emacs and Terminal). It occurred to me that perhaps whether my second monitor was on or off might affect the latency, but I tested this and saw no difference.

The results of my experiment suggest that I did not become habituated to caffeine. I can’t figure out what they do suggest, but at least I can say that I probably don’t develop a tolerance from taking caffeine 3 days a week.

An offer to readers

If you conduct a caffeine experiment on yourself with similar methodology to mine, you can send your data to web at mdickens dot me and I’ll analyze it and make some graphs for you. At a minimum, each data point should include the date, your reaction time, and whether you had caffeine.

Source code and data for this experiment are available on GitHub.

Notes

And, I didn’t know this before running the experiment, but it turns out that it’s easy to get statistically significant results with reaction time. My reaction time had a day-to-day standard deviation of only 11 ms, so I can detect pretty small effect sizes with just a few days of samples. ↩
It’s possible to conduct a self-blinded caffeine experiment as follows:
1. Label caffeine pills and placebo pills as pill A and pill B in a random order.
2. On Monday/Wednesday/Friday, take pill A. On Tuesday/Thursday/Saturday, take pill B. (Skip Sunday.)
3. Each week, re-randomize the ordering of pill A and pill B so you can’t figure out which one is which.
I didn’t want to do that for two reasons:
- I already suspect that caffeine makes me feel much better while lifting weights, so I don’t want to spend potentially several weeks lifting weights without caffeine.
- I prefer to drink coffee rather than take pills, and you can’t really blind coffee because decaf tastes different.
↩
A standard serving is two scoops per six ounces, which would require me to use eight scoops, but I don’t like my coffee that strong. If you take higher doses of caffeine, you’ll probably get habituated faster. (I have no evidence that that’s true, but it sounds right to me.) ↩
Signature Select Classic Roast, because it’s the cheapest and it tastes as good as the best brands I’ve tried. ↩
For this calculation I compared post-caffeine and pre-caffeine tests on the same day, ignoring the test results for days where I didn’t take caffeine. If instead I compare post-caffeine tests vs. all no-caffeine tests (including on days when I don’t take caffeine), the difference between averages is 8 ms. However, the difference in performance without caffeine on caffeine days vs. no-caffeine days is not statistically significant (difference = 3 ms, p = 0.5). I performed slightly worse on caffeine days, which is the opposite of what I’d predict—subjectively, I feel more energetic on caffeine days even when I haven’t taken caffeine yet. ↩
A regression over just the calibration phase gives a slope of –5.50 ms/hour (p = 0.01337 (nice)). ↩
That means a dose of coffee is worth 2 hours of sleep in terms of its immediate effect on reaction time. ↩
I tried regressing reaction time against time-in-bed the previous two nights, but the second night did not add any predictive power.

I also looked at time spent asleep according to my sleep tracking app, which I suspected wouldn’t work as well because I’ve noticed it’s pretty bad at identifying when I’m asleep. And indeed, regressing reaction time against time “asleep” gave a similar slope as regressing against time-in-bed, but with a worse p-value. ↩
I converted slope into retention as follows:
1. multiply slope by the number of days in the phase to get the total reaction time change
2. divide by the baseline benefit of caffeine (13 ms) to get the degree of habituation (0 = no habituation, 1 = full habituation, -1 = reverse habituation i.e. caffeine got more effective)
3. subtract from 1 to get retention (retention is basically the inverse of habituation)
Unlike in my caffeine literature review, I treated the baseline benefit as a fixed parameter instead of a distribution because that makes the math easier (but the real reason is that I wrote this part of the code before I wrote the code for the literature review). ↩ ↩²
Perhaps I should have re-run the trial, but I was following a strict rule not to re-run trials under any circumstances, to make sure I had no wiggle room to bias the results. ↩
The displayed graph shows no-caffeine trials. Controlling for sleep on caffeine trials has a similar effect, flattening the slope from –0.47 to –0.41. ↩
C. J. Meliska, R. E. Landrum & T. A. Landrum (1990). Tolerance and sensitization to chronic and subchronic oral caffeine: effects on wheelrunning in rats.

Omar Cauli, Annalisa Pinna, Valentina Valentini & Micaela Morelli (2003). Subchronic Caffeine Exposure Induces Sensitization to Caffeine and Cross-Sensitization to Amphetamine Ipsilateral Turning Behavior Independent from Dopamine Release.

N. Simola, E. Tronci, A. Pinna & M. Morelli (2006). Subchronic-intermittent caffeine amplifies the motor effects of amphetamine in rats. ↩
Omar Cauli & Micaela Morelli (2002). Subchronic caffeine administration sensitizes rats to the motor-activating effects of dopamine D(1) and D(2) receptor agonists. ↩

How Well Did Scott Alexander's List of Social Science Findings Hold Up?

Mon, 08 Apr 2024 00:00:00 -0700

In 2012, Scott Alexander defended social sciences against the claim that they can’t figure anything out. He gave a long list of well-established findings across a variety of social science disciplines.

12 years later, how well did that list hold up?

I evaluated the list off the top of my head without doing any research,¹ so please don’t take this too seriously.

The text before the colon on each numbered item is Scott’s words; everything else is my words.

Anthropology

Humankind evolved in Africa, gradually settled the Old World, and crossed the Bering land bridge to America around 20,000 years ago: still good
Languages form large families like Indo-European that can be used to trace the history and development of different peoples: still good
People have an almost-miraculous language instinct that can for example turns a pidgin into a creole in the second generation: still good
There are various human universals, but people tend to overestimate how universal their own culture’s norms are: I don’t know
- I don’t know how to interpret this claim. It’s well-established that there are various human universals and also that cultural norms vary a lot, but I have no idea whether it’s true that people tend to overestimate how universal their own culture’s norms are.
Any biological mental differences between groups are less important than previous believed and quickly overwhelmed by within-group differences: still good

Verdict: 4/5 or 5/5. Anthropology is doing well.

Economics

Prices in the marketplace are determined by supply and demand: still good
Capitalism leads to faster economic growth than the alternatives: still good
Unless you have very strange priorities, free trade is a good idea: still good
The gold standard is a bad idea: still good
Rent control decreases the quality and quantity of available housing: still good
Minimum wages increase unemployment: still good
- There are some observational studies contradicting this and there’s debate as to whether minimum wage is currently high enough that raising it would cause noticeable unemployment but the basic principle of this claim is still true. (I found Bryan Caplan’s The Myopic Empiricism of the Minimum Wage persuasive.²) And anyway, the counter-arguments people would raise today are the same ones they would’ve raised in 2012.
Cutting taxes will not increase government revenue in normal conditions: still good

Verdict: 7/7. Economics is doing well.

Psychology

Personality is about 50% biologically determined: still good
People’s self-image is constructed on the spot and varies widely depending on the situation: still good
People exhibit various cognitive biases that deviate systematically from rational thought: partially replicated
- Some cognitive biases have replicated (base rate neglect, status quo bias, loss aversion, hindsight bias), others haven’t (priming, nudge theory, implicit bias). From scanning thru Wikipedia’s list of cognitive biases, looks like 70–90% have replicated. I will give this one half credit for a partial replication.
IQ correlates to all kinds of important life outcomes: still good
Many mental disorders correspond to disordered brain chemistry and can be partly treated chemically: still good
People lack privileged access to their own mental processes: still good
Cognitive behavioral therapy does better than placebo in treating mental disorders; Freudian therapy does not. still good (?)
- IIRC some RCTs have found good benefits to the modern version of Freudian therapy but I don’t think it’s well-established.
Babies are not a blank slate but have various built-in behavioral patterns; they develop new mental abilities in an orderly fashion: still good
Animals react to reward and punishment in extremely predictable, almost mathematical ways: still good
Strong relationships and driving purpose are very important to happiness; material goods less so after a certain point: unclear
- As I understand it, the happiness research does consistently support this claim, but it’s not clear that we are doing a good job of measuring happiness.

Verdict: 8.5/10 or 9.5/10. Psychology is doing fairly well. I found this one surprising considering how much stuff in psychology has failed to replicate, but Scott did a good job of identifying the claims that would hold up (a much better job than my college psychology textbook did).

Sociology

People are racist as heck: failed to replicate
- I believe Scott is talking about Implicit Association Tests, which do consistently show implicit associations but don’t reliably predict behavior. Some other things like reume name bias failed to replicate. There’s some version of this hypothesis (the “all bad racial outcomes are caused by racism” hypothesis) that’s basically unfalsifiable and therefore hasn’t been disproven but that’s not a point in its favor.
No, really, they’re really racist: failed to replicate
Even the ones who don’t think they are: failed to replicate
Even the ones who swear up and down that they’re not racist and donate to the NAACP: failed to replicate
There are major disparities in the income levels of social status of various ethnic groups: still good
Discrimination explains a lot of this, sometimes in surprising ways: was never established in the first place
- I don’t entirely understand what Scott meant by this but as far as I know, it was never established and the evidence almost entirely contradicts it.
Most social problems are closely correlated with one another. Scandinavia has the fewest social problems of developed countries, and the US has the most: still good
Poor people and uneducated people tend to suffer more social problems and commit more crimes: still good
Religious people tend to be happier and better-adjusted than others: still good
Social class is a big deal, even in societies that make a big pretense of being classless: still good

Verdict: 5/10. Sociology are you ok? (To be fair, this list is a bit skewed because Scott spent four bullet points reiterating the same claim that failed to replicate.)

Epidemiology

Smoking causes cancer and many other problems: still good
Alcohol causes liver disease and many other problems: still good
Bad diets (in some sense of the word) cause heart disease, Type II diabetes, and many other problems: still good
Two zillion other correlations between risk factors and diseases: still good
Exercise is really good for you: still good
Vaccines are extremely effective at controlling infectious diseases: still good
And they don’t cause autism: still good
Stress causes or exacerbates many diseases: still good
Poor people suffer from more diseases, even in ways that are not directly linked to them not being able to afford medical care: still good
Many diseases seem to be part genetic and part “other factors”, including mental diseases: still good
Lifestyle changes can decrease your chance of getting mental diseases: still good(ish)
- As I understand, this is broadly supported but there’s some contradictory evidence on e.g. whether exercise helps with depression.
Multivitamins don’t work: mixed evidence
- There are some RCTs on both sides (for a quick summary, see here under the heading “Most unusual claim”). Multivitamins are cheap so IMO they pass a cost-benefit analysis. I am giving zero credit on this one because, while it might be true, it’s not definitely true, and this was presented as a list of definitely-true findings.
Low-dose aspirin (probably) helps prevent cancer: still good
- The evidence on this is weak but Scott did say “probably” so I’ll give this one full credit.

Verdict: 11/13 or 12/13. Epidemiology is doing pretty well.

Notes

Except for a few of the claims in epidemiology which I didn’t know anything about, so I did about 15 seconds of research. ↩
Prior to reading Caplan’s article, my position on minimum wage was basically, “Demand curves are almost always downward sloping and the empirical evidence on minimum wage isn’t good enough to overcome this strong prior.” Which is the first argument Caplan makes, but he also make some other good arguments that I hadn’t thought of. ↩

Explicit Bayesian Reasoning: Don't Give Up So Easily

Wed, 03 Apr 2024 00:00:00 -0700

Recently, Saar Wilf, creator of Rootclaim, had a high-profile debate against Peter Miller on whether COVID originated from a lab. Peter won and Saar lost.

Rootclaim’s mission is to “overcome the flaws of human reasoning with our probabilistic inference methodology.” Rootclaim assigns odds to each piece of evidence and perfoms Bayesian updates to get a posterior probability. When Saar lost the lab leak debate, some people considered this a defeat not just for the lab leak hypothesis, but for Rootclaim’s whole approach.

In Scott Alexander’s coverage of the debate, he wrote:

While everyone else tries “pop Bayesianism” and “Bayes-inspired toolboxes”, Rootclaim asks: what if you just directly apply Bayes to the world’s hardest problems? There’s something pure about that, in a way nobody else is trying.

Unfortunately, the reason nobody else is trying this is because it doesn’t work. There’s too much evidence, and it’s too hard to figure out how to quantify it.

Don’t give up so easily! We as a society have spent approximately 0% of our collective decision-making resources on explicit Bayesian reasoning. Just because Rootclaim used Bayesian methods and then lost a debate doesn’t mean those methods will never work. That would be like saying, “randomized controlled trials were a great idea, but they keep finding that ESP exists. Oh well, I guess we should give up on RCTs and just form beliefs using common sense.”

(And it’s not even like the problems with RCTs were easy to fix. Scott wrote about 10 known problems with RCTs and 10 ways to fix them, and then wrote about an RCT that fixed all 10¹ of those problems and still found that ESP exists. If we’re going to give RCTs more than 10 tries, we should extend the same courtesy to Bayesian reasoning.)

I’m optimistic that we can make explicit Bayesian analysis work better. And I can already think of ways to improve on two problems with it.

First problem: If you multiply a long list of probabilities as if they’re independent when they’re not, you get an extreme result.

Quick fix: Reduce the magnitudes of the odds updates based on how much evidence you already have. The more individual factors you have, the more a new factor can be explained in terms of existing factors.

For example, you could scale down the log-odds of your second observation by 1/2, your third observation by 1/3, your fourth observation by 1/4, etc. This roughly captures the intuitions that

if you have a lot of evidence already, a new observation is probably mostly predicted by the existing evidence
if you have infinitely many pieces of evidence, that should give you an infinitely large odds update

This approach means you don’t need to spend any time thinking about how correlated your inputs are.

If you have lines of evidence A, B, C, etc., the formula for joint log-odds becomes

\begin{align} \log(A) + \frac{1}{2} \log(B) + \frac{1}{3} \log(C) + … \end{align}

And therefore your joint odds would be

\begin{align} A \cdot B^{1/2} \cdot C^{1/3} \cdot … \end{align}

I don’t have a rigorous justification for this formula² and it has some obvious problems (for example, if you change the order or your inputs, the answer changes). But it has some advantages over treating every piece of evidence as independent.

As a proof of concept, I created a modified version of Scott Alexander’s lab leak debate calculator that updates less on correlated evidence. My version assumes two lines of evidence are correlated if (1) they’re under the same heading and (2) they point in the same direction. This change reduces the standard deviation of people’s answers from 7.4 orders of magnitude to 4.4. (Or, if you exclude Peter Miller’s extremely-overconfident answer, it reduces the standard deviation from 2.1 OOM to 1.8.)

Second problem: Overconfident probabilities like “1 in 10,000 chance that COVID would first appear in a wet market conditional on lab leak”.

Quick fix: Give every piece of evidence a “reliability score”. Maybe the evidence looks like it suggests 10,000:1 odds but you haven’t thought about it that hard. You read the number in some population survey but maybe the survey mis-calculated, maybe it used bad data collection methods, maybe you misread the number of zeros and it actually said 1 in 1000.

As a simple approach, you could give every piece of evidence a reliability score from 1 (low reliability) to 4 (high reliability). Discount evidence by raising it to the power of 1 / (5 - reliability_score). So 10,000:1 evidence with a reliability score of 2 gets reduced to 10,000^1/2 = 100:1, and evidence with a score of 1 gets reduced to 10,000</sup>1/4</sup> = 10:1.

Is that the best way to handle the problem of overconfident odds updates? Probably not. But it’s really easy and it took me three seconds to come up with.

(If you think carefully enough about your odds, you don’t need a reliability score. But the score is a convenient way to encode a concept like “I did some calculations and got 10,000:1 odds but I haven’t carefully checked the calculations.”)

Quoting Scott again,

In the end, I think Saar has two options:

Abandon the Rootclaim methodology, and go back to normal boring impure reasoning like the rest of us, where you vaguely gesture at Bayesian math but certainly don’t try anything as extreme as actually using it.

Claim that he, Saar, through his years of experience testing Rootclaim, has some kind of special metis at using it, and everyone else is screwing up.

(I get the sense Scott is joking, but I’ve heard other people say things like this.)

I propose a third option: Examine the flaws in explicit Bayesian reasoning and look for ways to fix them.

Or a fourth option: Do explicit Bayesian reasoning, don’t take the result literally but implicitly update your beliefs based on the result.

Or a fifth option: Figure out how to fix RCTs, and then do something similar for Bayesian reasoning. (Did we figure out how to fix RCTs yet?)

Or a sixth option: Keep doing Bayesian reasoning, and meanwhile keep trying to fix its flaws (like we are sort-of doing for RCTs).

Notes

Actually only 8 out of 10 but the basic point still stands. ↩
You could slightly-more-rigorously justify this formula by saying
- The variance in evidence B is 50% explained by evidence A.
- The variance in evidence C is 50% explained by A and 50% explained by B.
- But the parts of C that are explained by A and B heavily overlap, so less than 75% of C is explained by A plus B.
↩

Does Caffeine Stop Working?

Fri, 29 Mar 2024 00:00:00 -0700

Last updated 2024-09-02.

Confidence: Likely.

If you take caffeine every day, does it stop working? If it keeps working, how much of its effect does it retain?

There are many studies on this question, but most of them have severe methodological limitations. I read all the good studies (on humans) I could find. Here’s my interpretation of the literature:

Caffeine almost certainly loses some but not all of its effect when you take it every day.
In expectation, caffeine retains 1/2 of its benefit, but this figure has a wide credence interval.
The studies on cognitive benefits all have some methodological issues so they might not generalize.
There are two studies on exercise benefits with strong methodology, but they have small sample sizes.

Clarifying terminology

The scientific literature talks about the “caffeine withdrawal hypothesis.” People use this term to describe two very different hypotheses:

Caffeine has no benefits for anyone. It reverses withdrawal symptoms for habituated users, but it doesn’t do anything for non-users. (Call this the “caffeine-is-useless hypothesis.”)
Caffeine initially has benefits for non-users, but if you use caffeine habitually, your body adjusts to the point where you need caffeine just to get back up to baseline. (Call this the “caffeine habituation hypothesis.”)

According to the caffeine-is-useless hypothesis, on the first day you take caffeine, you experience no benefits, and then you start feeling withdrawal symptoms after you’ve been taking caffeine for a few days:

According to the caffeine habituation hypothesis, caffeine has initial benefits, but you start developing a tolerance:

(Graphs inspired by Gavin Leech’s article on caffeine, except I put way less effort into mine.)

Most studies on caffeine withdrawal only look at the caffeine-is-useless hypothesis. The study results pretty much universally reject this hypothesis so I consider it falsified¹. I’m much more interested in the caffeine habituation hypothesis, so that’s the one I will be discussing.

Research papers almost never explicitly discuss the caffeine habituation hypothesis, but some of them provide enough data to test it. In the next two sections, I will review some studies and figure out what their data tell us about the caffeine habituation hypothesis.

Cognition studies

I found two good studies on the exercise benefits of caffeine, and four good(ish) studies on the cognitive benefits. Let’s start with the cognition studies: Rogers et al. (2013)², Hewlett & Smith (2007)³, Haskell et al. (2005)⁴, and Smith et al. (2006)⁵.

These studies all used similar methodology: they divided participants into high caffeine users vs. low/non-users. Then they randomly administered either caffeine or placebo and tested participants’ performance on various cognitive tests.

The studies each have four groups of participants:

LoCaf: low- or non-caffeine users assigned to take caffeine
LoPla: low- or non-caffeine users assigned to take placebo
HiCaf: high-caffeine users assigned to take caffeine
HiPla: high-caffeine users assigned to take placebo

To test the caffeine habituation hypothesis, compare the performance of high-caffeine users after taking caffeine (HiCaf) versus low-caffeine users after taking placebo (LoPla). If high users develop complete tolerance then these two groups should perform the same: when a habitual user takes caffeine, it brings their performance back up to baseline, but has no benefits beyond that.

(This methodology isn’t perfect because low and high caffeine users might differ in ways that could bias the results.⁶ (For more discussion of this possibility, see Rogers et al. (2013)².) It would be better to randomize participants to take either caffeine or placebo for several weeks.)

Then calculate:

baseline benefit = LoCaf – LoPla = benefit to a naive user
habituated benefit = HiCaf – LoPla = benefit to a habituated user
retention = habituated benefit / baseline benefit = (HiCaf – LoPla) / (LoCaf – LoPla) = what proportion of caffeine’s benefits are retained by a habituated user

Retention ranges from 0 (caffeine loses all its effect) to 1 (caffeine retains all its effect).

I evaluated the effectiveness of caffeine by computing approximate likelihood functions of retention from each study’s measurements.

(Disclaimer: I only just now learned how likelihood functions worked so I could write this article. Be wary of mistakes.)

The likelihood function L(X) answers the question, “If caffeine’s true retention is X, what is the probability that we would measure the retention that we did in fact measure?” If a particular retention has a high likelihood of generating the results we got, that makes it more likely that that’s the true retention.

(For a longer explanation of likelihood functions, see Report Likelihoods, Not P-Values.)

Plotting likelihood functions of the four main metrics⁷ from Rogers et al. (2013) (which had a larger sample size than the other three studies combined) along with the average likelihood⁸ of those metrics:

(The dots on each curve show the mean likelihoods. You can interpret the mean as the value the evidence tends to point toward.⁹)

This graph says the experiment points toward caffeine retaining around half its effect (0.56 to be precise). And it says we’d be somewhat unlikely to see these experimental results if retention = 0, but not unlikely to see them if retention = 1.

Here’s how I computed these approximate likelihood functions:

Model the baseline benefit (LoCaf - LoPla) and habituated benefit (HiCaf - LoPla) as t-distributions.¹⁰ To compute the likelihood function for retention (habituated benefit / baseline benefit), we need to know the shape of a ratio of t-distributions.

A ratio of t-distributions does not have a closed form solution. So I approximated the solution using a formula I pulled off Wikipedia for the ratio of two independent normal distributions (astute readers will notice that the distributions in question are neither independent nor normal). I guess you could call this the Simon-Ftorek approximation because Wikipedia cites Simon and Ftorek (2022)¹¹. For more on whether this approximation is any good, see Appendix A.

I used the Simon-Ftorek approximation to compute the likelihood functions for the most important metrics in each study (see Appendix B for a list of all the metrics I used). Then I computed a likelihood function for each study as the (geometric) average of all the likelihood functions for individual metrics in that study. Normally, you’re supposed to compute a joint likelihood as the product of your likelihood functions. But that assumes each function provides independent evidence, and I figured if you have the same study with the same participants, the different metrics all basically represent the same evidence. So I averaged them instead of multiplying them.

If we take the joint likelihoods from the four cognition studies and combine them into one big joint likelihood, we get this:

Exercise studies

I found two good studies on how habitual caffeine affects exercise: Beaumont et al. (2017)¹² and Lara et al. (2019)¹³. These two studies had participants abstain from caffeine for a month. Then they gave participants either caffeine or placebo for several weeks, testing their exercise performance at the beginning and end of the study.

The methodologies differed somewhat, and I calculated retention accordingly.

In Beaumont et al. (2017)¹², the participants took two pre-tests, one with caffeine and one with placebo. Then after taking caffeine every day for 28 days, they took a dose of caffeine and a final performance test. I calculated

baseline benefit = caffeine pre-test – placebo pre-test
habituated benefit = caffeine post-test – placebo pre-test
retention = habituated benefit / baseline benefit

Lara et al. (2019)¹³ tested participants 3 times a week for 20 days. To take advantage of all the extra data points, I plotted a linear regression over the performance for the caffeine group minus the placebo group¹⁴. Then I calculated

baseline benefit = projected effect size on day 0 (= intercept of the regression)
habituated benefit = projected effect size on day 20
retention = habituated benefit / baseline benefit

It looks like caffeine retains a little under 1/2 its benefit for exercise.

Some problems with my approach

As I mentioned before, I couldn’t calculate exact likelihood functions, I could only approximate them.
I assumed caffeine has the same effect on all cognitive tests and on all exercise tests. But caffeine probably helps more with some tasks than others. It probably improves reaction time more than memory; it probably helps more with sustained moderate exercise (e.g., a VO2 max test) than with short intense efforts (e.g., a Wingate test).
Limiting the domain to [0, 1] makes the likelihood means tend toward 0.5 because it chops off the (often large) parts of the distribution below 0 and above 1. If you think retention can go above 1 but not below 0, you’ll get a higher mean. Conversely, if you think it can go below 0 but not above 1, you’ll get a lower mean. (Symmetrically expanding the domain to [-1, 2] doesn’t change the answers much.)

Conclusion

I plotted likelihood functions for every I study I reviewed and combined them into one big joint likelihood:

What about a posterior probability?

If you use a uniform prior for caffeine retention, the posterior probability distribution simply equals the likelihood function. According to the studies I looked at, a habituated caffeine user retains an expected 49% of the cognitive benefit and 44% of the exercise benefit, or 48% if we combine the cognition and exercise studies.

My prior has more probability mass near 0 than 1. Human bodies want to maintain homeostasis, so there’s some theoretical reason to expect your body to adjust until caffeine stops working entirely. (And, empirically, taking caffeine does cause your brain to grow more neurotransmitter receptors,¹⁵ although it’s not clear how this corresponds to cognitive function.) Changing the prior moves my posterior expected retention from 48% to 43% or so.¹⁶

As shown in Appendix A, my approximation for the likelihood function understates the mean. Plus my review excluded any metrics where caffeine had too small a baseline benefit, which biases the retention to look smaller. Due to these factors, the true implied retention is higher than 43%. Let’s call it 50% to make it a nice round number.

So we can reasonably say, albeit with a high degree of uncertainty, that caffeine retains something like half its benefit for a habitual user.

Source code for my calculations can be found on GitHub.

Appendix A: Approximating the retention ratio

As a sanity check, I generated 10,000 random samples following the same distribution as the groups in one of the caffeine studies (Rogers et al. (2013)²), where each sample represents a set of true parameter values that could correspond to the observed values for the four groups (LoCaf, LoPla, HiCaf, HiPla). Then I calculated the distribution of true retention and the Simon-Ftorek approximation of the likelihood function (normalized to integrate to 1) and plotted them together:

A zoomed-in plot showing just the values from 0 to 1:

(The upper plot already chops off a big tail. The largest retention value in the sample was over 500, i.e., a long-term caffeine user appears to get a 500x larger benefit per dose than a naive user. That can happen with a ratio of random variables when the denominator ends up close to 0.)

From these plots we can see that a Simon-Ftorek approximation slightly overstates the width of the distribution.

If we restrict the Monte Carlo sample to just the results with a baseline benefit of least one standard error (as I did when selecting metrics to look at), we get this:

I also tried two other approximations, but Simon-Ftorek seemed best. The other approximations I tried:

Approximate using a Cauchy distribution. A Cauchy distribution perfectly represents the ratio of two central normal distributions, so I figured it might be an okay approximation for the ratio of non-central normal distributions.
Numerically compute the ratio of two distributions represented as histograms. This still requires assuming the distributions are independent.

This plot shows the three different approximations over a single metric from Beaumont et al. (2017)¹²:

Appendix B: List of all metrics used

I did not use metrics from every study. I used the following criteria to select metrics:

Only use performance metrics, not subjective ratings or physiological measurements. (That means no sleepiness, no heart rate, etc.).
Prefer metrics that the study authors emphasized.
Don’t use any metrics where the baseline benefit was small (<1 standard error) or negative.

I used the following metrics from each study:

Rogers et al. (2013)²: simple reaction time, choice reaction time, recognition memory, tapping speed
Hewlett & Smith (2007)³: focused attention speed, simple reaction time, verbal reasoning % correct
Haskell et al. (2005)⁴: simple reaction time, digit vigilance reaction time, Rapid Visual Information Processing false alarms, spatial memory (sensitivity index), numeric memory reaction time
Smith et al. (2006)⁵: focused attention speed, categoric search reaction time, simple reaction time, vigilance hits
Beaumont et al. (2017)¹²: total energy output (kJ), substrate oxidation
Lara et al. (2019)¹³: peak power, VO2 max, Wingate test peak power, Wingate test mean power

Smith et al. and Lara et al. did not provide numeric standard errors (and Lara et al. did not provide means) but did provide plots, so I estimated the numeric values by counting the number of pixels using Gimp.

Note: Ignoring metrics with a small or negative baseline benefit could bias the results toward making caffeine habituation look worse. This process selects for observations where the baseline benefit was large by chance, making the habituation benefit look smaller by comparison.

Changelog

2024-09-02: I previously described 44% (the mean exercise benefit to caffeine) as “about 1/3”. This makes it sound like the difference between the observed cognitive vs. exercise benefits is about 1/6 (17 percentage points) when in fact it’s only 4 percentage points. I changed the description to “a little under 1/2”.
2025-06-06: Fixed typos.

Notes

And I didn’t find it plausible to begin with. If caffeine doesn’t make non-users more alert, why would people start taking caffeine in the first place? ↩
Rogers, P. J., Heatherley, S. V., Mullings, E. L., & Smith, J. E. (2013). Faster but not smarter: effects of caffeine and caffeine withdrawal on alertness and performance. ↩ ↩² ↩³ ↩⁴
Hewlett, P., & Smith, A. (2007). Effects of repeated doses of caffeine on performance and alertness: new data and secondary analyses. ↩ ↩²
Haskell, C. F., Kennedy, D. O., Wesnes, K. A., & Scholey, A. B. (2005). Cognitive and mood improvements of caffeine in habitual consumers and habitual non-consumers of caffeine. ↩ ↩²
Smith, A., Sutherland, D., & Christopher, G. (2006). Effects of caffeine in overnight-withdrawn consumers and non-consumers. ↩ ↩²
Some ways self-selection could bias the results:
- People who are more naturally alert might not take caffeine because they don’t feel like they need it. This would make the habituated benefit look smaller. (That is, the LoPla group might perform better than the caffeine group because they’re naturally more alert, not because caffeine isn’t benefiting the caffeine group.)
- People who don’t get much benefit from caffeine might not take it, making the baseline benefit look smaller (and thus retention look larger).
- People who react strongly to caffeine might not take it (because it makes them jittery/anxious), making the baseline benefit look larger.
↩
The four metrics are: (1) simple reaction time (SRT); (2) choice reaction time (CRT); (3) recognition memory; (4) tapping speed. ↩
Normally, you’d calculate joint likelihood as the product of the likelihood functions. But that only works if the functions are idependent. Since these four functions are all measuring the same group of people during the same experiment, I averaged them instead of multiplying them. The resulting joint likelihood understates the strength of the evidence, but I’m more concerned about overstating evidence than understating it. ↩
Alternatively, the mean likelihood equals the posterior expected value when using a uniform prior.

Statistical analyses commonly report the maximum likelihood but not the mean likelihood. I believe people ought to use the mean likelihood instead.

For a symmetric distribution, the mean likelihood equals the maximum likelihood. But the distinction matters for caffeine retention because its likelihood function is skewed. If you compress the likelihood into a single value, that value should tilt toward whichever tail is fatter. The mean accounts for the fatness of the tails; the mode (maximum) does not.

For more on this, see McLeod, A. I., & Quenneville, B. (1999). Mean likelihood estimators. ↩
Really we should model them as Behrens-Fisher distributions but a t-distribution is close enough. ↩
Simon, F. E., & Ftorek, J. (2022). A new method for approximating the distribution of the ratio of two independent normal random variables. ↩
Beaumont, R., James, L. J., & Davison, G. (2017). Chronic ingestion of a low dose of caffeine induces tolerance to the performance benefits of caffeine. ↩ ↩² ↩³ ↩⁴
Lara, Beatriz; Ruiz-Moreno, Carlos; Salinero, Juan José; Del Coso, Juan; Sandbakk, Øyvind (2019). Time course of tolerance to the performance benefits of caffeine. ↩ ↩² ↩³
I say that as if there were two different groups, but really this was a crossover study. The experimenters ran two experiments in succession on the same group of people. They randomly assigned subjects to the caffeine or placebo groups on the first experiment, and then switched them for the second experiment. Effect sizes were determined by comparing each person’s performance with caffeine on day N to their performance with placebo on day N in the other trial.

This study had by far the best methodology of any of them, too bad it only got 11 participants. (I’m not surprised, considering that (a) they restricted their sample to physically-active participants and (b) the participants had to repeatedly perform grueling exercises.) ↩
Shi, D., Nikodijevic, O., Jacobson, K. A., & Daly, J. W. (1993). Chronic caffeine alters the density of adenosine, adrenergic, cholinergic, GABA, and serotonin receptors and calcium channels in mouse brain.

It would be more accurate to say that caffeine causes rats to grow more receptors, but I assume human brains work the same way.

Perhaps someday we will figure out how to augment rat brains to understand writing and then a rat will read this article. If that’s the case, greetings future rat folk, I hope you find this article helpful even though I only cited studies on human subjects.

(Future cybernetically-augmented rat: “Why do news articles always present scientific findings as if they apply to rats when they’ve only done the studies on humans?”) ↩
I calculated this by setting my prior to a beta distribution with alpha=1 and beta=1.5, which represents a low-information prior with a slight tilt toward 0. ↩

Avoiding Caffeine Tolerance

Sat, 02 Mar 2024 00:00:00 -0800

Summary

Caffeine improves cognition¹ and exercise performance². But if you take caffeine every day, over time it becomes less effective.

What if instead of taking caffeine every day, you only take it intermittently—say, once every 3 days? How often can most people take caffeine without developing a tolerance?

The scientific literature on this question is sparse. Here’s what I found:

Experiments on rats found that rats who took caffeine every other day did not develop a tolerance. There are no experiments on humans. There are no experiments that use other intermittent dosing frequencies (such as once every 3 days).
Internet forum users report that they can take caffeine on average once every 3 days without developing a tolerance. But there’s a lot of variation between individuals.

This post will cover:

The motivation for intermittent dosing
A review of the experimental research on the effect of taking caffeine intermittently (TLDR: there’s almost no experimental research)
A review of self-reports from the online nootropics community
Intermittent dosing vs. taking caffeine every day

Epistemic status: I have never formally studied pharmacology or psychopharmacology—I’ve barely studied biology for that matter. But somehow nobody has ever looked into intermittent caffeine dosing to a degree that I found satisfactory³, so I did it myself.⁴

Updated 2024-03-08 to reference three more studies. h/t Gavin Leech for making me aware of them—see also his article on caffeine tolerance, which I think explains the concepts better than mine does.

Summary
Why intermittent dosing?
Experimental evidence on intermittent dosing
Individual self-reports
Is intermittent dosing better than taking caffeine every day?
Conclusion
Appendix A: A study I would like to see
Appendix B: Pre-registration for a caffeine self-experiment
Notes

Why intermittent dosing?

The standard remedy for caffeine tolerance is to stop caffeine cold turkey for one to two weeks (or longer) to reset your body. Plenty of evidence suggests that this strategy works⁵, but I’m not happy with it.

The reason I’m not happy with it has to do with why I take caffeine: I want to improve cognition and exercise performance. If I stop having caffeine for two weeks, am I also supposed to take two weeks off work? Or spend two weeks being tired all day and getting nothing done? That sounds like a waste of time. Am I supposed to stop exercising for two weeks? That sounds unhealthy.

Taking caffeine intermittently makes more sense to me. If I take caffeine 3 times a week, that works a lot better:

I can do the demanding high-effort work on caffeine days and do easier work on the non-caffeine days.
I can do high-intensity exercise on caffeine days, and rest or do light cardio on the other days.

But I’d rather take caffeine 4–5 days a week if I can get away with it. And if I’ve unknowingly developed a tolerance, I’d rather scale back to once or twice a week. So I’d like to know the maximum frequency that doesn’t dampen caffeine’s benefits.

Experimental evidence on intermittent dosing

There are no studies on humans that administered caffeine less frequently than once a day. But there are a few studies on rats, which all found roughly the same result:⁶⁷⁸⁹

Rats who take caffeine every day develop a tolerance after 1–2 weeks.
Rats who take caffeine every other day do not develop a measurable tolerance after 2 weeks.

Some concerns with these studies:

They used rats, not humans. Humans are different in lots of ways, the most obvious being that we have slower metabolisms. A two-day abstinence for a rat might be the metabolic equivalent of a 6 to 10 day abstinence for a human. (Counterpoint: When taking caffeine daily, rats and humans develop tolerance at about the same rate.)
The studies only lasted 14 days. It’s possible that taking caffeine on alternating days still builds up a tolerance, but it takes longer than when taking caffeine daily (perhaps twice as long?).
Three of the four experiments were run by approximately the same research team, so any mistakes they made probably got repeated across all three of those experiments.

Individual self-reports

By far the best evidence I could find on how to avoid tolerance in humans was a 2021 reddit poll in the /r/caffeinecycling subreddit. An internet poll isn’t great evidence, but I’ll take what I can get.

(See footnote for the poll results in text format.¹⁰)

Among 84 respondents, at least 48 reported taking caffeine intermittently. 54% of those 48 reported that they do not develop a tolerance when taking caffeine every third day. And on average, they preferred to wait 3.58 days between doses.¹¹

This suggests that:

On average, people can take caffeine every 3 days or so.
There’s a lot of variation between individuals, so you personally probably should take caffeine more or less often than that.

Some concerns with this poll:

This was based on self-reporting, not on any objective metric of performance.
It’s a biased sample: respondents found the poll because they were already interested in caffeine cycling. (I’d guess this poll over-represents people who develop caffeine tolerance quickly because people who don’t develop much tolerance won’t bother to cycle caffeine, and won’t go to a caffeine cycling subreddit.)
The rate of tolerance buildup depends on the size of the dose, but we don’t know anything about how much caffeine the poll respondents were taking.

I searched online nootropics forms (reddit.com/r/Nootropics/ and longecity.org) for anecdotes on commenters’ preferred caffeine frequency. The results broadly align with the /r/caffeinecycling poll; see footnote for a list of anecdotes.¹²

Is intermittent dosing better than taking caffeine every day?

If you take caffeine every day, you will develop a tolerance, but your performance might not fully revert to baseline. Could you get more benefit from taking caffeine daily than from cycling it? To answer that question, first we need to ask: How much ongoing benefit can you expect to get if you take caffeine daily?

Update 2024-03-29: I wrote a longer post about this question: Does Caffeine Stop Working?

Many studies have examined this question, but almost all of them are methodologically flawed to the point of being useless.¹³

I found exactly two studies (on humans) with strong methodology: Beaumont et al. (2017)¹⁴ and Lara et al. (2019)¹⁵. These studies gave participants either caffeine or placebo every day for 28 and 20 days (respectively) and tested their exercise performance before and after. The experiments both found that:

Taking caffeine regularly decreased its potency, and this effect was statistically significant.
Habituated users still retained 1/3 to 1/2 of the benefit of caffeine, but this effect was not statistically significant. (The studies could not rule out the possibility that habituated users saw zero positive effect from caffeine.)¹⁶

I found another few studies with decent but not great methodology: Rogers et al. (2013)¹⁷, Haskell et al. (2005)¹⁸, and Smith et al. (2006)¹⁹. These studies divided participants into caffeine non-users/low users versus high users. Then they randomly administered either caffeine or placebo and tested participants’ performance on various cognitive tests.

This methodology isn’t perfect because low and high users might differ in ways that could bias the results. (For more discussion of this possibility, see Rogers et al. (2013).) It would be better to randomize participants to take either caffeine or placebo for several weeks. Methodological limitations aside, these three studies suggest that high caffeine users do retain some benefits, although the exact amount varied depending on the study and the measurement used, from 28% to 315% (i.e., a habituated user experiences a 3x greater benefit than a naive user).²⁰

Some studies on rats, cited previously⁶⁷⁸⁹, gave rats caffeine daily for 14 days and found that they retained somewhere between none and all of the performance benefits of caffeine, depending on the study. (Astute readers will notice that this doesn’t narrow things down much.)

***

Suppose caffeine retains 1/3 of its benefit when you take it every day—that’s my best guess given the available evidence. Compare that to taking caffeine only once every 3 days. Either way it’s about the same: either you’re getting 1/3 of the benefit every day, or the full benefit on 1 out of 3 days.

Except they’re not really the same because some days matter more than others. For example, if you exercise 4 days a week and you take caffeine on 2 of those days, then you’re only taking caffeine on 2/7 days of the week, but you’re getting half the exercise benefit, which is better. For that reason, I would prefer taking caffeine intermittently over taking it daily.

Conclusion

Prior to writing this article, I was taking caffeine 3 days a week, on Monday, Wednesday, and Friday. I wanted to know if I could take caffeine more often without developing a tolerance. My research tentatively suggests:

My approach was reasonable and there’s not compelling evidence for taking caffeine more or less often than I do currently.
If I do experiment with taking caffeine more frequently, there’s a good chance that I will develop a tolerance.

Appendix A: A study I would like to see

I would like to see a randomized controlled trial on the effectiveness of caffeine when taken intermittently. If I had a big enough budget, I would design the study like this:

Break participants into at least three groups:
1. A placebo-control group who takes a placebo every day.
2. A caffeine-control group who takes caffeine every day. (This group shows what fully-habituated users look like.)
3. One or more experimental groups that take caffeine intermittently. If we can get a big enough sample size, have separate groups for a variety of dosing frequencies.
Following the methodology of Beaumont et al. (2017)¹⁴, pre-test every group on athletic performance and/or cognition both when taking caffeine and when taking a placebo.
Administer caffeine or placebo for at least 28 days (longer is better).
At the end of the experiment, administer caffeine to all participants and test them again.

The study would then look at the following outcomes:

Measure the benefit of caffeine as the performance on the final caffeinated test minus the performance on the two pre-tests.
- If a particular group performs better on the final test than on the placebo pre-test, that means they did not develop complete caffeine tolerance.
- If a group performs worse on the final test than on the caffeine pre-test, that means they developed at least some tolerance.
Measure the performance of each group relative to the placebo-control group and the caffeine-control group.
- If an experimental group outperforms the caffeine-control group on the final test, that means intermittent dosing has succeeded at reducing tolerance formation.
Which group gets the greatest total benefit as measured by performance multiplied by dosing frequency?

If we wanted to make the study more complicated, we could follow the design of Lara et al. (2019) and test the participants frequently throughout the study. That would let us run a regression on the course of caffeine tolerance and also compare the performance of intermittent caffeine users on caffeine days vs. placebo days. Or we could divide up the groups by dosage, giving some people 1.5 mg/kg and others 3 mg/kg (or some other dosage).

If I were on a budget, I’d scale down the study to only two groups: a daily-caffeine group and an intermittent-caffeine group. With no placebo group, we don’t know how well the intermittent group performs relative to baseline, but we can compare their tolerance development relative to the caffeine group, which I believe is more important.

Appendix B: Pre-registration for a caffeine self-experiment

Updated 2024-03-29 to fix typos and improve wording.

I am going to conduct the following experiment on myself, in which I test whether taking caffeine intermittently decreases its effectiveness.

I will test my reaction time by taking the humanbenchmark.com test twice in a row. One test consists of 5 reaction events, so this will give a total of 10 reaction events, taking my reaction time at that moment as the average of the 10 reaction times. I will take the test using the same computer, monitor, and mouse so that latency is consistent.

I will measure the effect of caffeine using a reaction time test because (a) caffeine is known to improve reaction time, (b) reaction time is easy to test, (c) it’s unlikely to improve with practice so it makes for a good consistent test variable, and (d) it’s hard to placebo-effect myself into improving my reaction time (which is important because I can’t blind myself).

Phase 1. Calibration phase. Take a reaction time test twice a day (following the same schedule as described in phase 3 below). Continue for four weeks. Plot a regression of my reaction time across the four weeks. The purpose of the calibration phase is to ensure that my reaction time does not improve from practicing every day.

During this phase I will continue drinking coffee 3 times per week (as I have been doing), but I won’t measure my reaction time on caffeine vs. non-caffeine days because the only purpose of this phase is to check that my reaction time doesn’t improve with practice.

Phase 2. Abstinence phase. Abstain from caffeine for one week (9 days total, in between the last Friday of the calibration phase and the first Monday of the test phase). Test reaction time during this week. If I was habituated to caffeine in phase 1 then my reaction time should improve over the course of phase 2 as my tolerance wears off. Measure the slope of reaction times across days during the abstinence phase.

Phase 3. Test phase. Resume drinking coffee 3 days a week and continue for four weeks. Take a reaction time test twice a day, at (say) 8am and then 10am—the exact time doesn’t matter, but the first test is before having coffee and the second is after coffee. (Or on days when I don’t have coffee, take the test at times before and after I when would have had coffee.)

The primary variable of interest is the difference in reaction time on caffeine days before and after taking caffeine. I will also look at the difference in the 8am vs. 10am tests on non-caffeine days to ensure that reaction time doesn’t get better on the second test even without caffeine.

After phase 3 completes, I will measure the following outcomes:

Slope of the difference in reaction time between pre-caffeine and post-caffeine tests on caffeine days. (A negative slope indicates diminishing effect of caffeine.)
Slope of reaction time across post-caffeine tests. (A positive slope suggests caffeine tolerance. Remember that positive = longer reaction time = bad.)
Slope of reaction time across pre-caffeine tests. (A positive slope suggests caffeine withdrawal symptoms.)

Check if the slopes are statistically significant. If the slopes for variables #2 and #3 are both ascending but only one is statistically significant, or they are jointly statistically significant but individually insignificant, that suggests that I did develop a tolerance but that the experiment was underpowered. If both slopes are insignificant, that suggests no tolerance (or that the experiment was underpowered). If slope #2 or #3 is descending and statistically significant, that suggests something weird is going on and my experiment is probably invalid.

If the slopes are all approximately zero, that means I can probably have more caffeine before I develop a tolerance. In that case, conduct phase 4.

Phase 4. High-caffeine test phase. Increase coffee intake to 4 days a week for four weeks. Observe the same variables as phase 3.

This experiment is not blinded and it’s probably underpowered, but it’s better than nothing.

Thank you to Ansh Shukla for reviewing a draft of this caffeine self-experiment.

Notes

Astrid Nehlig (2010). Is Caffeine a Cognitive Enhancer? ↩
Erica R Goldstein, Tim Ziegenfuss, Doug Kalman, Richard Kreider, Bill Campbell, Colin Wilborn, Lem Taylor, Darryn Willoughby, Jeff Stout, Sue Graves, Robert Wildman, John L Ivy, Marie Spano, Abbie E Smith & Jose Antonio (2010). International society of sports nutrition position stand: caffeine and performance. ↩
There are a few posts on nootropics forums, particularly reddit.com/r/caffeinecycling/, but they don’t go into much depth. There are no academic papers that discuss intermittent dosing in humans. ↩
I originally just wanted to answer the question for myself, but by the time I had read through about 10 studies and a bunch of forum posts, I figured I might as well write something up about it. ↩
Laura Juliano & Roland Griffiths (2004). A critical review of caffeine withdrawal: empirical validation of symptoms and signs, incidence, severity, and associated features. ↩
C. J. Meliska, R. E. Landrum & T. A. Landrum (1990). Tolerance and sensitization to chronic and subchronic oral caffeine: effects on wheelrunning in rats. ↩ ↩²
Omar Cauli, Annalisa Pinna, Valentina Valentini & Micaela Morelli (2003). Subchronic Caffeine Exposure Induces Sensitization to Caffeine and Cross-Sensitization to Amphetamine Ipsilateral Turning Behavior Independent from Dopamine Release. ↩ ↩²
N. Simola, E. Tronci, A. Pinna & M. Morelli (2006). Subchronic-intermittent caffeine amplifies the motor effects of amphetamine in rats. ↩ ↩²
Omar Cauli & Micaela Morelli (2002). Subchronic caffeine administration sensitizes rats to the motor-activating effects of dopamine D(1) and D(2) receptor agonists. ↩ ↩²

Poll results for “What is your optimal time for cycling caffeine without tolerance development?”

Votes	Answer
7	At least every second day
19	At least every third day
4	At least every fourth day
18	At least a week break between caffeine doses
13	Caffeine for weeks, then weeks off (“long cycling”)
24	Other

↩

That’s with the assumption that all the “at least a week” respondents take caffeine exactly once a week. ↩
A list of anecdotes, paraphrased:
- christopherforums: I don’t develop tolerance when taking every 3–4 days, but do develop when taking every other day.
- /u/(deleted): 3 days a week is probably sustainable for 3–6 weeks before I’d want to take a week break. I can do once every 3 days indefinitely without building tolerance.
- /u/FreemanOfficial: I develop some tolerance when taking every second day, but no tolerance when taking every third day.
- /u/Lceus: For 3 months, I took caffeine for 3 days and then abstained for 4 days. The first caffeine days always had a strong effect, with the next 2 days having a positive but less enjoyable effect.
- /u/odd1e: I do 2x or 3x a week. If I do 3x for too long, a tolerance starts to build up.
- /u/owlthatissuperb: IME the best strategy is to drink coffee 2–3x per week.
- My own personal experience: I take caffeine either 3x or 3.5x a week (the extra 0.5x means I take a half-dose on one day). I don’t believe I’ve developed (much) tolerance because I don’t experience any withdrawal symptoms when I don’t take caffeine. I haven’t experimented with other dosing schedules, except that I used to drink coffee every day and I definitely developed a tolerance from that.
I also found anecdotes of people reporting that they intermittently take caffeine but without saying anything about why they do it that way, so I did not include those. ↩
Most randomized controlled trials on the benefits of caffeine cannot distinguish between two possibilities:
1. Daily caffeine users get a persistent benefit from taking caffeine.
2. Daily caffeine users who abstain from caffeine experience withdrawal symptoms, which hurts performance. Taking caffeine brings them back up to baseline performance, but they don’t do any better than they would have if they’d never developed a tolerance in the first place.
A literature review by James & Rogers (2005)²¹ discussed possible approaches for distinguishing these possibilities and looked at some studies that attempt these approaches. It concluded that the apparent benefits of caffeine are most likely due to reversing withdrawal symptoms. I believe this is an overly strong conclusion based on the studies they cited, and in fact the studies were simply too underpowered to show that caffeine has a positive effect for habituated users.

(The process of writing this article has significantly lowered my opinion of the field of psychopharmacology.) ↩
Ross Beaumont, Philip Cordery, Mark Funnell, Stephen Mears, Lewis Jamesa & Phillip Watson (2017). Chronic ingestion of a low dose of caffeine induces tolerance to the performance benefits of caffeine. ↩ ↩²
Beatriz Lara, Carlos Ruiz-Moreno, Juan Jose Salinero & Juan Del Coso (2019). Time course of tolerance to the performance benefits of caffeine. ↩
An explanation of how I drew conclusions from these two studies:

Lara et al. (2019) gave one group a placebo and the other group caffeine every day for 20 days and tested their exercise performance every few days using various metrics such as maximum power output and VO2 max The caffeine group consistently outperformed the placebo group, but the performance difference between the two groups declined by between 1/3 and 2/3 by day 20 (with different metrics of performance declining by different amounts). The authors did not report significance tests on the change in marginal benefit of caffeine between day 1 and day 20 but it was consistently negative across all metrics.

Beaumont et al. (2016) pre-tested participants’ performance both with caffeine and with placebo. Then participants took either caffeine or placebo for 28 days and took another test along with a dose of caffeine. In this experiment:
- The caffeine group performed significantly worse on the final test than on the pre-test with caffeine.
- The caffeine group performed non-significantly better on the final test than on the pre-test with placebo. This suggests that caffeine may have provided some benefit even with tolerance, but we can’t rule out the possibility that it had no benefit.
- The maximum-likelihood possibility is that caffeine tolerance reduces its benefits by 66%. (In the pre-test, the caffeine group did an average of 38.4 kJ more work with caffeine than with placebo. In the post-test, they did 13.1 kJ more work than in the placebo pre-test and 25.3 kJ less than in the caffeine pre-test. The observed reduction in marginal power output is 25.3 / 38.4 = 66%.)
Taken together, these experiments suggest that caffeine retains something like 1/3 of its benefit even after habituation.

However, we should not over-update on these results. Lara et al. had only 11 participants and Beaumont et al. had 18 participants. Given the small n and small effect sizes, we can’t draw strong conclusions from either of these two studies. And their methodologies might not generalize:
- They both gave participants 3 mg caffeine per kg bodyweight (which is around 200 mg for an average-sized person), so we don’t know what happens at other dosages. Some research suggests that users see best results at doses twice that high.
- The experiments lasted 20 and 28 days, respectively, so we don’t know if caffeine tolerance continues to build up over several months.
- The participants they recruited were not representative of the general population.
- They only looked at benefits to exercise performance, not cognition.
↩
Peter Rogers, Susan Heatherley, Emma Mullings & Jessica Smith (2013). Faster but not smarter: effects of caffeine and caffeine withdrawal on alertness and performance. ↩
Crystal Haskell, David Kennedy, Keith Wesnes & Andrew Scholey (2005). Cognitive and mood improvements of caffeine in habitual consumers and habitual non-consumers of caffeine. ↩
Andrew Smith, Gary Christopher & David Sutherland (2006). Effects of caffeine in overnight-withdrawn consumers and non-consumers. ↩

None of these studies directly reported the information we care about, but I was able to figure some things out from what they did report.

We want to know the benefit of caffeine for a habitual user. We can measure that by comparing the performance of high-caffeine users after taking caffeine versus low-caffeine users after taking placebo. If high users develop a complete caffeine tolerance then these two groups should perform the same—when a habitual user takes caffeine, it’s as if they’ve taken nothing.

If we want to know exactly how much of the benefit habitual users retain, it gets a little bit complicated, but we can still figure it out. Start by deriving two variables from the groups’ performance:

baseline benefit of caffeine for naive users = low-caffeine users on caffeine – low-caffeine users on placebo
absolute retention of benefits for habituated users = high-caffeine users on caffeine – low-caffeine users on placebo

Then calculate relative retention as benefit / retention.

I performed this calculation using the primary performance metrics from each study (ignoring self-reported metrics like sleepiness) and got the following results.

Rogers et al. (2013) had by far the biggest sample size at 369 (compared to 48 and 25 for Haskell et al. and Smith et al., respectively). Rogers et al. did not report means or standard errors for the individual experimental groups, but it did report (ANOVA) significance tests for the four main performance metrics used by the study²². I can summarize the results as:

Across all four metrics, high-caffeine users performed worse than low-caffeine users when given placebo, and 3/4 differences were statistically significant.
High-caffeine users on caffeine outperformed low-caffeine users on placebo on 3/4 metrics. 2 out of those 3 were statistically significant.
High-caffeine users on caffeine actually outperformed low-caffeine users on caffeine on (a different set of) 2 metrics (but these results were not significant).
From eyeballing the graphs in the paper, the four metrics had relative retentions of approximately 120%, 140%, –30%, and 80%, with the second and fourth retentions being significantly different from 0.

These results suggest that:

Habitual users experience withdrawal symptoms when they stop taking caffeine.
Habitual users still retain some benefit above baseline—they do not become fully habituated to caffeine.
Possibly, habitual users do not lose any of the benefits. (The results lean this way, but not strongly.)

The other two studies did report means, so I can explicitly quantify the relative rentention of the benefits of caffeine. In the tables below, I report the retention, benefit, and relative retention for the main metrics the studies used.

From Haskell et al. (2005): (RT = reaction time, RVIP = rapid visual information processing)

Metric	Retention	Benefit	Retention %
simple RT	-17.64	-5.6	315%
digit vigilance RT	-15.5	-16.2	96%
RVIP false alarms	-0.5	-0.21	238%
spatial memory	0.01	0.01	100%
numeric memory RT	-11.92	-42.42	28%

From Smith et al. (2006):

Metric	Retention	Benefit	Retention %
focused attention speed	-4.2	-8.6	49%
categoric search RT	-5.5	-16.4	34%
simple RT	-38.1	-41.4	92%
repeated digits vigilance	2.13	3.32	64%

(Both studies reported a lot more metrics, but they highlighted these as the most important.)

For the Haskell et al. data, on each of the five metrics, the difference between retention and benefit was not statistically significant at p=0.05, and the absolute retention was not significantly different from zero (digit vigilance RT came close with p = 0.054).

I can’t do significance tests on the Smith et al. data because the paper didn’t report standard errors. But given that it had a smaller sample size than Haskell, I’d guess none of the differences are statistically significant.

I wanted to do some Bayesian statistics where I’d calculate a posterior distribution over relative retention based on every individual metric from the three studies. But I realized it would be kind of pointless because Rogers et al. (2013) doesn’t report p-values (only significance / insignificance), so I can’t do odds updates for the study that should provide by far the biggest odds updates. ↩

Jack E. James & Peter J. Rogers (2005). Effects of caffeine on performance and mood: withdrawal reversal is the most plausible explanation. ↩
Rogers et al. (2013) tested 8 metrics, including 3 self-reported metrics and 5 performance tests. I ignored one of the performance tests because the low-caffeine placebo group outperformed the low-caffeine caffeine group, which means the concept of “retention” is not well-defined. (There is no benefit to retain if caffeine has a negative effect.) ↩

Utilitarianism Isn't About Doing Bad Things for the Greater Good. It's About Doing the Most Good

Tue, 15 Aug 2023 00:00:00 -0700

In the eyes of popular culture (and in the eyes of many philosophy professors), the essence of utilitarianism is “it’s okay to do bad things for the greater good.” In my mind, that’s not the essence of utilitarianism. The essence is, “doing more good is better than doing less good.”

Utilitarianism is about doing the most good. You don’t do the most good by fretting over weird edge cases where you can harm someone to help other people. You do the most good by picking up massive free wins like donating to effective charities where money does 100x more good than it would if you spent it on yourself.

(Richard Y. Chappell might call this beneficentrism: “the view that promoting the general welfare is deeply important, and should be amongst one’s central life projects.” You can be a beneficentrist without being a utilitarian, but if you’re a utilitarian, you have to be a beneficentrist, and as a utilitarian, being a beneficentrist is much more important than being a “do bad things for the greater good”-ist.)

The United States Is Weird

Tue, 15 Aug 2023 00:00:00 -0700

The United States is exceptional along many dimensions. Sometimes, people pick two of these dimensions and try to argue that one causes the other. And that’s probably true sometimes. But “the USA is #1 in the world on dimension X, and #1 on dimension Y” isn’t much evidence that X causes Y.

The United States has:

the 3rd largest population, and the largest population of any first-world country (by far)
the 3rd or 4th largest land area (depending on how you measure USA’s and China’s land)
the highest GDP of any country
the highest median income, the 8th highest GDP per capita (PPP), and the highest GDP per capita of any large country (the top 7 countries combined have a lower population than California)
unusually high income inequality for a developed country
the highest healthcare expenditure per capita
the highest gun ownership per capita (with double the gun ownership of the #2 country)
an unusually high homicide rate for a developed country
the most Nobel Prize winners winners (by a huge margin) and most Fields Medalists (narrowly beating France)
the highest obesity rate of any large country, and the 11th highest overall
the most top universities (whatever that means) by a wide margin
unusually low life expectancy for a developed country
an unusually high fertility rate for a developed country
the 2nd most exports (after China) and the most imports (China is #2)
the most military expenditures (by a factor of 3) and the 2nd most nuclear weapons

(Those were just the examples I could come up with in an hour of research.)

(I also looked at a few stats where I thought the USA might be exceptional, but it turned out not to be: IQ, educational attainment, infant mortality¹, and net immigration.)

A lot of these facts are clearly intertwined—the fact that the US has the highest GDP is related to the facts that it has the highest military expenditures, the highest healthcare expenditures, and the 2nd highest exports. (But they’re not fully intertwined, because the US still has high military and healthcare expenditures relative to GDP.)

For other facts, you can come up with narratives as to why they’re related—maybe the high obesity rate causes the low life expectancy, maybe high gun ownership causes the (relatively) high homicide rate. But maybe not. The United States is weird and I don’t have a great handle on why it’s weird (and, as far as I know, nobody else does either). Until someone comes up with a Grand Theory of National Weirdness, I’m reluctant to pick two ways in which the USA is weird and claim one causes the other.

Notes

Some sources say the USA has high infant mortality. I didn’t look into this much, but the CIA World Factbook claims that the USA defines infant mortality more broadly than most countries, and if you adjust for this, infant mortality looks similar to most developed countries. ↩

Should Patient Philanthropists Invest Differently?

Wed, 14 Sep 2022 00:00:00 -0700

TLDR: No.

Confidence: Somewhat likely.

Summary

Some philanthropists discount the future much less than normal people. For philanthropists with low discount rates, does this change how they invest their money? Can they do anything to take advantage of other investors’ high time discounting?

We can answer this question in two different ways.

Should low-discount philanthropists invest differently in theory? No. [More]

Should low-discount philanthropists invest differently in practice? The real world differs from the standard theoretical approach in a few ways. These differences suggest that low-discount philanthropists should favor risky and illiquid investments slightly more than high-discount investors do. But the difference is too small to matter in practice. [More]

Cross-posted to the Effective Altruism Forum.

Summary
Contents
Background
Theory
Practice
Conclusion
Appendix A: Pedantry
Appendix B: How I calculated historical optimal leverage
- Tables of results
Appendix C: How I calculated optimal private equity allocation
Appendix D: Market volatility over longer holding periods
Appendix E: Illiquidity in real estate
- Illiquidity premium in 100-year leases
Appendix F: Transformative changes to market behavior
Notes

Background

Many people have a positive pure time preference: they prefer present consumption over future consumption, regardless of any empirical facts. Philanthropists shouldn’t have a pure positive discount rate, but they do have some legitimate reasons to discount the future, including:

An extinction or economic collapse could render their money useless
Their money could get expropriated (taxed, seized, or stolen)
They might not be able to ensure that future generations continue spending their wealth in the ways they want

(Some people talk about impatient vs. patient philanthropists who have positive vs. zero pure time preference, or urgent vs. patient longtermists who believe we do vs. don’t live in a particularly important time. For this essay, I’m talking about low-discount vs. high-discount philanthropists. Urgent longtermists usually have high discount rates and patient longtermists usually have low rates, but it’s not a 1:1 correspondence.)

Many philanthropists don’t want to spend all their money at once; any money they don’t spend gets invested. And we would like to know how we should invest. For this essay, I will ask: how does optimal investment strategy change with discount rate? For philanthropists with unusually low discount rates, what should they do differently?

Suppose philanthropists have a pool of wealth that they invest, and they consume a fixed proportion of wealth each year. (For a philanthropist, donating to charity counts as consumption.) They get some utility every time they spend money. What investment strategy maximizes expected utility? And how does that investment strategy vary as a function of discount rate?

Theory

Merton (1969)¹² derived a formula for how much leverage investors should use and how quickly they should spend down their assets (given some simplifying assumptions). Merton found that the optimal investment allocation does not depend on the investor’s discount rate.

In theory, the optimal allocation only depends on three things:

assets’ expected returns
assets’ covariances with each other
the investor’s relative risk aversion (RRA)

The first two are properties of the assets themselves, not of the investors. The only factor that (theoretically) should cause two investors to behave differently is their degree of risk aversion.

The optimal rate of consumption³ does depend on one’s discount rate.⁴ But the discount rate does not affect the choice of investment.

Merton’s proof depends on four key assumptions:

Investors have constant relative risk aversion.
Asset prices follow log-normal distributions.
Asset returns are independent across time.
Assets can be traded continuously.

The assumption of constant relative risk aversion means that investors’ risk preferences do not change when their wealth increases or decreases. This might be false, but it’s close enough to true, so I won’t question this assumption for now.⁵

If asset prices don’t follow log-normal distributions, then you cannot fully describe assets’ performance in terms of mean and covariance. This affects investment strategy (mostly by making us more concerned about tail risk), but it doesn’t affect how quickly we should consume.

The third assumption—time independence—matters the most for our purposes. If investment returns tend to regress to the mean—in other words, if present and future returns are anticorrelated—then this changes how you should invest.

Why? If returns are anticorrelated, then if you lose some money now, that increases your expectation that you’ll make it back later. It also decreases the probability that you’ll be able to compound unusually high returns for multiple years in a row. This matters regardless of discount rate, but it matters more for investors with low discount rates because they care more about fluctuations in their future wealth.

The fourth assumption—perfect liquidity—does not directly relate to investors’ discount rates, but it can still matter, as I discuss below.

Practice

Low-discount philanthropists might invest differently for three reasons:

long-term mean reversion
subjective uncertainty about future returns
illiquidity premium

Long-term mean reversion suggests that:

Risk-averse (specifically, more risk-averse than logarithmic) low-discount investors should use more leverage than risk-averse high-discount investors.
Risk-tolerant (more risk-tolerant than logarithmic) low-discount investors should use less leverage than risk-tolerant high-discount investors.

(Confidence: Highly likely.)

This crude drawing shows how leverage varies for low-discount vs. high-discount investors after accounting for long-term mean reversion:

But this drawing overstates the size of the effect. (Or at least it would, if it had any labels or scale.) Empirically (based on historical returns), mean reversion only slightly changes the optimal amount of leverage, and probably doesn’t matter in practice. (Confidence: Likely.) [More]

Over longer time periods, investors have greater subjective uncertainty about how markets will behave. This creates subjective long-term momentum, which has the opposite effect of mean reversion. (Confidence: Likely.) And, like with mean reversion, this effect only slightly differs between low- and high-discount investors. (Confidence: Likely.) [More]

If long-term illiquid investments earn significantly higher returns, then investors with low discount rates prefer them relative to high-discount investors. (Confidence: Highly likely.) [More] But empirical data suggests they don’t earn an illiquidity premium, so illiquid investments don’t look particularly compelling, regardless of discount rate. (Confidence: Likely.) [More]

Some investors may prefer illiquid investments for behavioral reasons due to their (perceived but not actual) reduced volatility, and for these investors, a lower discount rate suggests a higher allocation to private equity. [More]

As a final point, conventional wisdom says long-term investors should take more risk. Conventional wisdom is correct, not because long-term investors have a lower discount rate, but because they can expect to earn future income which reduces their effective risk. So this observation is not relevant to how low-discount philanthropists should invest. (Confidence: Almost certain.) [More]

1. Long-term mean reversion

When Merton proved that asset allocation does not depend on time discounting, he assumed that investment returns are independent across time. But that’s not quite true. Historical evidence (weakly⁶) suggests that stocks exhibit long-term mean reversion. When prices go up a lot over a 3+ year period, they have a slight tendency to go back down. And when they go down over 3+ years, they have a similar tendency to rebound upward. That makes stocks a little less risky over longer holding periods.

How does this affect low-discount investors relative to high-discount investors?

Long-term mean reversion decreases the variance of an investment while keeping its geometric mean fixed, and decreasing the arithmetic mean.

Low-discount investors feel the decreased volatility more strongly than high-discount investors do because they care more about their investment performance in later years. But for the same reason, they more strongly feel the decrease in arithmetic mean return.

How this affects investors depends on their degree of risk aversion:

For investors with logarithmic utility, maximizing utility is equivalent to maximizing geometric mean, so they don’t care at all about mean reversion.
For “risk-averse” investors (those with sub-logarithmic utility), they care more about volatility than about arithmetic mean, so they like mean reversion.
For “risk-tolerant” investors (those with super-logarithmic utility), they care more about arithmetic mean than about volatility, so they dislike mean reversion.

Risk-averse low-discount investors want more leverage than risk-averse high-discount investors, while risk-tolerant low-discount investors want less leverage than risk-tolerant high-discount investors.⁷

More precisely:

At logarithmic utility, discount rate does not matter.
For risk-averse investors, desired leverage decreases with discount rate.
For risk-tolerant investors, desired leverage increases with discount rate.

At least that’s the hypothesis.

How strong are these effects? It turns out, pretty weak. I used data from The Rate of Return on Everything, 1870–2015⁸ and the Ken French Data Library to calculate historical utility-maximizing leverage for an equity investor in three environments: the US market from 1872 to 2015; the UK market from 1871 to 2015; and the developed-world markets (excluding the US) from 1990 to 2021.

Each of these historical cases confirmed the hypothesis, but they showed a weak effect size.

These two charts show excess leverage by discount rate for US and UK equities, compared to a baseline 3% discount rate:

(I didn’t graph excess leverage for RRA < 1 because a solution usually does not exist.⁹)

We can see from these charts that a zero-discount investor wants at most 4 percentage points more leverage than an investor with a 3% discount rate (1.78:1 vs. 1.74:1, respectively¹⁰). And the effect gets smaller as relative risk aversion (RRA) gets closer to 1.

So, while long-term mean reversion does theoretically change how low-discount philanthropists should invest, the difference is effectively a rounding error. (I can’t even say whether philanthropists should prefer (say) 1.5:1 leverage or 2:1 leverage, much less 1.78:1 or 1.74:1.)

(I should mention that, while equities exhibit long-term mean reversion, bonds appear to show long-term momentum, which would produce the opposite effect. However, historically, bonds did not have stable geometric means, and the historically optimal leverage for bonds showed the same results as for equities with respect to discount rate and RRA. And as with equities, the effect is too small to matter in practice.)

See Appendix B for the details of my calculations.

2. Uncertainty about future returns

Pastor & Stambaugh (2012)¹¹ showed that, due to uncertainty about average future returns, investors’ subjective volatility increases with time horizon. This produces the opposite effect of mean reversion: risk-averse low-discount investors want less leverage than risk-averse high-discount investors, while risk-tolerant low-discount investors want more leverage than risk-tolerant high-discount investors.

Behold, another crudely-drawn graph:

According to Pastor & Stambaugh, parameter uncertainty matters more than long-term mean reversion, but less than twice as much more. That means the combined impact of mean reversion + uncertainty has the opposite sign and a smaller magnitude than mean reversion alone. As we saw, the practical impact of mean reversion only varies slightly between low-discount and high-discount investors, so the impact of mean reversion + uncertainty varies even less.

Pastor & Stambaugh did not look at parameter uncertainty across discount rates, so I did my own analysis. I tested historically optimal leverage with parameter uncertainty by adding an “uncertainty factor” to the average return. This uncertainty factor was a normally-distributed random number between –2% and 2% (with that range representing the 95% confidence interval of the true average return).

As expected, the uncertainty factor does not affect optimal leverage for logarithmic-utility investors. For risk-averse investors, uncertainty decreases optimal leverage, and it decreases more for low-discount investors than for high-discount ones:

(UK equities show the same pattern, but a weaker effect.¹²)

When incorporating both mean reversion and uncertainty, risk-averse low-discount investors still preferred to use more leverage than risk-averse high-discount investors, but by a smaller margin than they would given certainty:¹³

(Notice how on this chart, the maximum excess leverage is ~0.03, while in the original chart with fixed parameters (“Excess Leverage by Discount Rate (US)”), the max was ~0.04.)

UN equities show a similar effect, except that in some cases, low-discount investors want (slightly) less leverage than high-discount investors:

For risk-tolerant investors (with super-logarithmic utility), adding uncertainty increases desired leverage, and the effect is larger for low-discount investors. However, it’s hard to quantify this effect because at low discount rates, sufficiently risk-tolerant investors want to use infinite leverage. (I would interpret this as evidence that philanthropists don’t have larger-than-logarithmic utility functions.)

Note that Pastor & Stambaugh found that parameter uncertainty outweighed mean reversion, while I found the opposite. If my analysis had looked more like Pastor & Stambaugh’s, then we would get the opposite result after accounting for both mean reversion and parameter uncertainty. But it doesn’t really matter because the effect is small either way.

3. Illiquidity premium

Low-discount philanthropists might not need to spend their money for a long time. Does that mean they should buy illiquid investments?

I will focus on private equity because it’s the most straightforward type of illiquid investment, and it has a clear analogue in public equity. First I will consider what illiquidity premium we should expect in theory for private equity. Then I will look at the observed illiquidity premium, and then attempt to reconcile the two.

See Appendix E for some discussion of illiquidity in real estate.

Theoretical illiquidity premium

In theory, should low-discount investors like illiquid investments more than high-discount investors do?

People’s preferences for liquidity do not directly depend on discount rate, but they do depend on their consumption rate. If you want to spend a (relatively) large percentage of your capital each year, then you’re more reluctant to lock up a big chunk of money in illiquid investments. There’s a risk that the market tanks and you end up spending down too big a proportion of your liquid assets.

All else equal, a higher discount rate means a higher spending rate. Low-discount philanthropists may consume only a tiny fraction of their wealth each year, which means they’re not concerned about over-spending their liquid investments.

In a two-asset model with public equity and (synthetic) private equity, low-discount investors allocate 1 to 5 percentage points more to private equity than high-discount investors:

(“Absolute” is the allocation as a percentage of wealth/liquidation value; “relative” is the allocation as a percentage of public + private allocation. Public + private absolute allocation can exceed 100% due to leverage.)

See Appendix C for the details on methodology.

Observed illiquidity premium

How large a premium can investors earn by holding private equity instead of public stocks?

Historically, private equity in the United States earned a 3% premium (Harris et al., 2014¹⁴; Kaplan & Sensoy, 2015¹⁵). However, this premium is not explained by illiquidity. The private equity premium is fully subsumed by a leveraged small value portfolio in public equities (Chingono & Rasmussen, 2015¹⁶; Ilmanen et al., 2019¹⁷; Stafford, 2017¹⁸). In fact, private equity has (large but statistically insignificant) negative alpha compared to leveraged small value.

Given zero (or negative) alpha for private equity, investors do not want to hold it, regardless of discount rate.

Reconciling predicted and observed illiquidity premiums

If theory predicts an illiquidity premium, why don’t we see one in historical data?

The most common explanation (e.g., as given in Ilmanen et al., 2019¹⁷ and Chingono & Rasmussen, 2015¹⁶) is that investors are willing to pay a premium for the return smoothing provided by private equity. Unlike a public market where the price updates many times per second, private equity firms have leeway to report essentially whatever valuations they want. They tend to under-report drawdowns, making returns for private companies look more consistent than their public counterparts, but with no corresponding decrease in true underlying risk (Welch & Stubben, 2018¹⁹). (In 2022 when the S&P 500 experienced a 20+% drawdown, the average private equity firm reported a 0% change in valuation, which is absurd on its face, but they’re allowed to do it.)

Investors’ preference for return smoothing is, in some sense, irrational—it doesn’t provide any financial benefit. But if you know that seeing your portfolio lose value will give you anguish, and might cause you to make a bad decision, then it could be rational to tie yourself to the mast by buying private equity.

Evidence suggests that, at least in the current market environment, rational investors should not invest in private equity, regardless of discount rate. But human investors with typical human flaws may do better by investing in private equity, even if it’s not technically optimal. And low-discount investors should allocate on the order of 5 percentage points more to private equity than high-discount investors do. Sophisticated institutional investors tend to allocate 10% to 15% to private equity,²⁰ which would suggest a 15% to 20% allocation for philanthropists with low discount rates. This is a small enough difference that it might not matter in practice, but it’s still a much bigger difference than we found for long-term mean reversion or parameter uncertainty.

I personally do not invest in private equity because I’m not concerned about the behavioral issues (and because it’s hard to invest in, but that’s secondary).

A note on conventional wisdom

Conventional wisdom says that investors with long time horizons should take on more risk. Why? Does that apply to low-discount philanthropists?

The conventional wisdom is correct for most people, but not because long-term investors inherently want more risk. Rather, long-term investors (usually) expect to earn more income in the future, and they should count that income as part of their investment portfolio. After factoring in their future income, they should seek the same amount of risk as a short-term investor.

For example, say Alice is retired with $1 million, and she invests her retirement fund in 50% stocks, 50% bonds. Bob has $100,000 in savings and expects to earn another $900,000 before he retires. He invests 100% of his savings into stocks. Bob’s portfolio looks riskier than Alice, but it’s actually less risky: Alice has 50% of her portfolio in stocks, while Bob effectively only has 10% after accounting for his future income. In theory, If Bob used 5:1 leverage, he would get his effective stock allocation up to 50%, although that’s probably not feasible in practice.

For low-discount philanthropists, how much risk to take depends on what money they expect to earn in the future. That depends on their personal income plus the income of all likeminded philanthropists plus the wealth and income of any people in the future who will become likeminded. Figuring out all those details is beyond the scope of this essay. The bottom line is: yes, the conventional wisdom is correct in a sense, but it doesn’t contradict my earlier claim that an investor’s risk appetite does not depend on time horizon, and it doesn’t have anything to do with discount rates.

Conclusion

In summary:

According to standard theory, investors should not change their asset allocation based on their discount rate.
Standard theory assumes that time periods are independent and that all investments are highly liquid, both of which are (sometimes) false.
Investment returns are not entirely independent over time. This has a nonzero but negligible effect on high- vs. low-discount investors.
It looks like illiquid assets don’t earn a significant premium, so low-discount investors shouldn’t prefer them.

Therefore, low-discount philanthropists don’t need to allocate any differently than high-discount investors.

Appendix A: Pedantry

In this essay, I somewhat conflated low-discount philanthropists with long-term philanthropists, but they’re not necessarily the same thing. For our purposes, “long-term” means you have a (relatively) long time horizon. A 3% discount rate would be considered high, but a philanthropist with a 3% discount rate could still easily have a 50+ year time horizon.

A low discount rate is not the same thing as longtermism (at least the way I use the term). A longtermist might have a high discount rate, for instance if they believe extinction is likely to render their funds useless in the near future, or if they expect not to be able to preserve the intent of their endowment across multiple generations. And someone with a 0% discount rate doesn’t have to be a longtermist—e.g., some form of person-affecting view could preclude longtermism even with no time discounting.

Philanthropists with particularly pessimistic beliefs about existential risk might have higher discount rates than normal people, which would suggest doing the opposite of what low-discount philanthropists do (which, if I’m right, is nothing, so it doesn’t matter either way).

Not all low-discount investors are low-discount philanthropists, but I use the terms interchangeably because I only care about investors who are also philanthropists.

Appendix B: How I calculated historical optimal leverage

I used data from The Rate of Return on Everything, 1870–2015 (RORE)⁸ and the Ken French Data Library, focusing on four data series:

US equities 1926 to 2021, from Ken French
Developed ex-US equities 1990 to 2021, from Ken French
UK equities 1899 to 2015, from RORE
US equities 1872 to 2015, from RORE

(In the main essay, I did not discuss #1 because it’s mostly a subset of #4. I only looked at both as a sanity check.)

I implemented the standard Ramsey model of intergenerational welfare where we have a pool of wealth that we invest at some rate of return, and every year we consume some fixed percentage of our wealth. We have a utility function over consumption, and total utility is the sum of discounted utility of consumption for each period. Then we simultaneously solve for the optimal amount of portfolio leverage and optimal rate of consumption. (For this essay, we don’t care about consumption, but we still have to solve for both variables.)

Applying this model over historical data requires making some modifications. We can’t calculate total utility out to infinity because we only have a finite sample of real historical financial returns. Naively, we could simply maximize utility over the sample we have, but this has two major problems:

We don’t discount the last year all that much relative to the first year. Any money left over at the end has a high opportunity cost. The optimizer wants to end up with as close to $0 as possible, so it recommends consuming at a faster rate than we’d want to if we had an infinite time horizon.
The outcome strongly depends on what happens to our investments near the beginning of the sample because the beginning is discounted the least.²¹

I implemented (somewhat ad-hoc) solutions for both of these problems:

Loop through the historical sample multiple times, extending it until it repeats for 1000 years.²² The 1000th year has minimal impact on total utility, so the optimal solution over 1000 years looks almost exactly like the optimal solution over infinity years.
Instead of only starting at the first year in the sample, calculate total utility for every possible starting year and then take the average.

Tables of results

The tables below give optimal leverage and consumption for various rates of relative risk aversion and discount rates, given a fixed-point expected return (no parameter uncertainty). I included 1% and 3% discount rates except for at 0.75 RRA, in which case the solution did not always exist, so I included higher discount rates instead. For RRA > 1, I also included a 0% discount rate.

For Ken French data (Tables 1 and 2), I applied discounting and consumption on a monthly basis and estimated the cost of leverage at the risk-free rate as given in the data.

For RORE data (Table 3), I applied discounting and consumption on an annual basis and estimated the cost of leverage at 0% because I don’t have data on the risk-free rate going back far enough. This is wrong, but it doesn’t matter for our purposes.

Table 1. US equities, 1926 to 2021.

RRA	Discount	Leverage	Consumption
0.75	1%	DNE	DNE
0.75	3%	2.20	0.43%
0.75	5%	2.36	2.65%
1	1%	2.16	1.00%
1	3%	2.16	3.02%
2	0%	1.47	5.56%
2	1%	1.45	6.00%
2	3%	1.40	6.89%

Geometric mean is maximized at 1.90 leverage.

Table 2. Developed ex-US equities, 1990 to 2021.

RRA	Discount	Leverage	Consumption
0.75	1%	DNE	DNE
0.75	3%	1.57	2.09%
0.75	5%	1.58	4.84%
1	1%	1.56	1.05%
1	3%	1.55	3.03%
2	0%	1.48	2.98%
2	1%	1.45	3.43%
2	3%	1.42	4.44%

Geometric mean is maximized at 1.17 leverage.

Table 3. UK equities, 1871 to 2015.

RRA	Discount	Leverage	Consumption
0.9	3%	1.92	1.92%
0.9	5%	1.88	4.10%
1	1%	1.86	1.00%
1	3%	1.86	2.98%
2	0%	1.79	5.36%
2	1%	1.78	5.69%
2	3%	1.76	6.54%

A portfolio that combines the US market beta + value + momentum factors shows the same pattern (with RRA < 1, optimal leverage increases with discount rate; with RRA > 1, optimal leverage decreases with discount rate). I calculated this portfolio using Ken French data as the simple sum of ‘Mkt’ + ‘HML’ + ‘Mom’ (UMD).

Table 4. US market beta + value + momentum factors, 1927 to 2021.

RRA	Discount	Lev	Consumption
0.9	3%	3.05	0.14%
0.9	5%	3.15	1.53%
1	1%	3.04	1.00%
1	3%	3.04	3.04%
2	0%	1.57	9.59%
2	1%	1.56	9.92%
2	3%	1.55	10.82%

This chart shows excess optimal leverage (relative to a baseline 3% discount rate) for a US market + value + momentum factor portfolio with RRA > 1:

This suggests that, for a factor investor, discount rate matters somewhat less than for a passive investor.

Appendix C: How I calculated optimal private equity allocation

There is some pre-existing literature on the theoretical illiquidity premium, such as Ang (2014).²³ However, I could not find any research on how the illiquidity premium varies as a function of discount rate, so I did my own analysis.

I calculated the optimal allocation across two assets labeled “public equity” and “private equity”. I used RORE⁸ data on the US stock market back to 1872 as the public equity component. For private equity, we don’t have data going back very far, so as a proxy, I used RORE data on UK public equities. Private equity behaves mostly like private equity, but not exactly; similarly, UK equity behaves mostly like US equity, but not exactly, so for this test, I took historical returns for UK public equity and pretended they represented private equity. Then I multiplied the UK annual returns by 1.5 to account for the fact that private equity tends to be more volatile.

The resulting “private equity” return series has an 11.6% annualized return, compared to 8.9% for US public equity; this matches the observed 3% private equity premium.

In my simulation, investors can only allocate to private equity once every 10 years. Any money they invest stays locked up, and they can rebalance at the end of every 10-year period. Investors can use leverage on public equity but not on private equity.²⁴

This produced the following relative allocations to private equity at various discount rates and levels of risk aversion:

“Private equity” had +6.6% alpha over public equity. If we subtract 6.6% from private equity returns to give it zero alpha, the optimal allocation becomes 0% regardless of risk aversion or discount rate.

If we add a pure +2% premium to private equity, this increases the optimal allocation, and also increases the differential between low- and high-discount investors, with low-discount investors now allocating 2 to 9 percentage points more to private equity than high-discount investors:

If we make the lockup period longer than 10 years, it reduces the optimal allocation for all investors, and it increases the differential between low- and high-discount investors. The opposite happens if we shorten the lockup period.

In some cases, there is no optimal allocation because it’s always better to delay spending indefinitely. This condition holds at (RRA = 1, discount rate = 0%) and at RRA < 1 for sufficiently low discount rates (including RRA = 0.8, discount rate = 3%).

(Originally, I wanted to set the average return of the “private equity” asset such that the high-discount-optimal allocation reflects the real-world allocation to private equity, and then calculate the low-discount-optimal allocation given the same inputs. But I couldn’t find good data on the total market cap of private equity, so I just used inputs that seemed reasonable.²⁵)

Appendix D: Market volatility over longer holding periods

Stock markets exhibit long-term mean reversion. As shown in Appendix B, this encourages risk-averse investors with lower discount rates to use more leverage. I calculated this using some complicated²⁶ and hard-to-intuit math. For a more intuitive approach, we could look at market volatility over different holding periods.

Jeremy Siegel’s Stocks for the Long Run²⁷ looked at the US equity market from 1802 to 2012 and found that volatility decreases over longer holding periods. I extended these results to global equities using data from The Rate of Return on Everything, 1870–2015⁸. I looked at the annualized standard deviation²⁸ of stock markets in 8 different countries²⁹ for holding periods ranging from 1 to 30 years.³⁰

(30 years is more or less the longest holding period we can say anything about because there are only four non-overlapping 30-year periods in the sample.)

This chart shows a weak downward trend. On average, for every year that you increase the holding period, the annualized standard deviation decreases by 0.14% (p < 0.002, but you shouldn’t take this p-value literally³¹), although this number is pretty sensitive to how far you extend the regression—if you exclude the particularly-noisy 21 to 30 year holding period range, the slope reduces to 0.004% with p = 0.94. And we can’t tell if the trend extends beyond 30 years, but it doesn’t look like it does.³² So stocks are somewhere between 0 and 4 percentage points less volatile over 30 years than over 1 year.

(Siegel (2014)²⁷ found a larger and more unequivocal mean reversion effect. That’s mostly because it only covered the US market, and you will notice that in the table above, the United States shows stronger mean reversion than most countries.)

At the same time, stock markets’ arithmetic means decrease over longer holding periods:

Long-term mean reversion works in favor of risk-averse investors (RRA > 1) and against risk-tolerant investors (RRA < 1).

Bonds get more volatile as you increase the holding period, from an average of 7% annualized volatility up to 12%:³³

Appendix E: Illiquidity in real estate

Real estate is less liquid than stocks, but not in a way that benefits low-discount philanthropists. Real estate costs money to trade, but it doesn’t have lockup periods like private equity, so we shouldn’t expect to see an illiquidity premium. There might be a “trading cost premium”, which benefits investors with long holding periods, but real estate investors tend to have long holding periods anyway, so this doesn’t make real estate look particularly good from a low-discount perspective.

The main bottleneck to real estate investing is not liquidity but diversification. To properly diversify your real estate portfolio, you’d need to spend tens of millions of dollars buying dozens of properties. If you do have that much money, then investing in real estate is probably smart. Indeed, many wealthy, long-term-oriented investors such as university endowments own a lot of land.

Illiquidity premium in 100-year leases

Could low-discount philanthropists do well by buying much longer-term illiquid investments, such as 100-year property leases? Phil Trammell proposed doing this. He claims that if you buy a house and simultaneously sell a 100-year lease, you gain the right to use the land 100 years from now and you only have to pay 10% of its present value. (I couldn’t find good data on the cost of a 100-year lease, so I’ll take his word for it.)

A 10% price implies a 2.3% interest rate ($1.023^{100} = 10$), which sounds basically fair—maybe even too low. As an illustration, suppose you have $1 million and you’re deciding between two choices:

Buy 10 houses for a total of $10 million and simultaneously sell a 100-year lease on each of them. You spend $1 million on net, and you get 10 houses 100 years from now.
Buy one house for $1 million and rent it out. Suppose the annual rent is 2.3% of the value of the house. Then in 100 years, you use the earned rent to buy the other nine houses. You spend $1 million, and you get 10 houses 100 years from now.

These two scenarios cost the same and provide the same value. And 2.3% is on the low end of rental yields; if you can earn a higher yield than that, it would make more sense to simply buy a house rather than buy 10 houses and sell 100-year leases on them.

(Why doesn’t the market set a higher implied interest rate on 100-year leases? Probably because houses are tradeable(ish): your money isn’t truly locked up for 100 years because you can sell the house at any time. So in an efficient market, the implied interest rate should equal the expected market yield (plus some other factors like differential taxes, maintenance costs, cost of selling the house multiplied by probability of wanting to sell, etc.)

Appendix F: Transformative changes to market behavior

Over sufficiently long time horizons, markets might see significant changes to investment performance, such as:

Technological growth compounds on itself (possibly due to transformative AI), causing the rate of exponential growth to accelerate, greatly increasing returns.
We reach the physical limits of economic growth, and investments stop growing exponentially.

There might be interesting things to say about these or other possible futures that break the current paradigm of consistent exponential growth. But that’s much more speculative, and I consider it out of scope for this essay.

If you expect the economy to radically change in (say) 1000 years, but not to change much before then, then that doesn’t significantly affect how to invest today.

If you expect (with non-trivial probability) major changes in the near future (such as due to transformative AI), that does change how you should invest, but I’m not sure how,³⁴ and that’s a question for another time.

Notes

Robert Merton (1969). Lifetime Portfolio Selection Under Uncertainty: The Continuous-Time_case. Read status: Multiple full read-throughs. This is a seminal paper so I’m highly confident that it’s good. ↩
Not all of the results I describe are original to Merton, but he provided the first single paper that proves all of them. ↩
For our purposes, “consumption” refers to anything we do with money other than re-investing it. For example, if we donate money, that counts as a form of consumption. ↩
The exact solution for the optimal consumption rate is given in formula (42) of Merton (1969). All else equal, the rate of consumption increases linearly with the discount rate. ↩
If you become more risk-averse as you get wealthier, then you should consume a larger proportion of wealth when you’re wealthier, and vice versa. ↩
Paul McCaffrey (2022). Equity Risk Premium Forum: Term Structure, Mean Reversion, and CAPE Reconsidered. Partial transcript of a discussion between Laurence B. Siegel, Rob Arnott, Elroy Dimson, William N. Goetzmann, Roger G. Ibbotson, Antti Ilmanen, Martin Leibowitz, Rajnish Mehra, and Jeremy Siegel.

Some relevant quotes:

Goetzmann: “The evidence is always a bit marginal and depends on your assumptions and on where you get the data.”

Ilmanen: “I see evidence of mean reversion over time horizons from 3 years up to 15 years. […] The evidence is really fuzzy, and usable or actionable evidence is almost zilch because of all this horizon uncertainty.” ↩
One thing I learned from writing this essay: I find it easy to think about two-dimensional relationships (“how does Y change as a function of X?”), but far harder to think about three-dimensional relationships (“how does ‘how does Z change as a function of Y’ change as a function of X?”). I kept finding myself getting the variables mixed up. ↩
Òscar Jordà, Katharina Knoll, Dmitry Kuvshinov, Moritz Schularick & Alan M Taylor (2019). The Rate of Return on Everything, 1870–2015. ↩ ↩² ↩³ ↩⁴
To be precise, if prices follow a log-normal distribution, then a solution exists only when the following condition holds, where $\delta$ is the discount rate, $\gamma$ is the rate of relative risk aversion, $\mu$ is the expected (geometric) portfolio return minus the risk-free rate, and $\sigma$ is the portfolio standard deviation:
\[\frac{-\delta}{\gamma} + (\mu + \sigma^2/2)\frac{1-\gamma}{\gamma} - (1 - \gamma)\sigma^2/2 < 0\]
(Formula taken from Levhari & Srinivasan (1969). “Optimal Savings Under Uncertainty.”)

This graph shows the minimum solvable discount rate as a function of RRA for $\mu = 7\%, \sigma=18\%$:

↩
These specific numbers come from US equities with risk aversion coefficient = 2. ↩
Lubos Pastor & Robert F. Stambaugh (2012). Are Stocks Really Less Volatile in the Long Run? Read status: I read the relevant parts. ↩
↩
Pastor & Stambaugh (2012) found that the parameter uncertainty effect outweighs the mean reversion effect, while I (usually) found the opposite. The difference is mostly because:
1. Pastor & Stambaugh looked at subjective portfolio volatility, while I looked at optimal leverage to maximize utility of consumption. These two things are related, but not the same.
2. Pastor & Stambaugh assumed no discount rate, while I looked at discount rates ranging from 0% to 3%. Subjective uncertainty can outweigh mean reversion at low discount rates. (Of the two data points I found where subjective uncertainty outweighed mean reversion, one occurred at discount=0% and one at discount=1%.)
↩
Robert Harris, Time Jenkinson, Steven Kaplan (2014). Private Equity Perfomance: What Do We Know? Read status: Abstract only. ↩
Steven Kaplan & Berk Sensoy (2015). Private Equity Performance: A Survey. Read status: Abstract only. I’m citing two papers with different methodologies but the same result, which increases my confidence without having to analyze the papers. ↩
Brian Chingono & Daniel Rasmussen (2015). Leveraged Small Value Equities. Read status: I read the relevant parts.

See Figure 9 in Chingono & Rasmussen (2015), with explanation on the previous page. ↩ ↩²
Antti Ilmanen, Swati Chandra & Nicholas McQuinn (2019). Demystifying Illiquid Assets: Expected Returns for Private Equity. Read status: Full read-through. ↩ ↩²
Erik Stafford (2017). Replicating Private Equity with Value Investing, Homemade Leverage, and Hold-to-Maturity Accounting. Read status: Abstract only. ↩
Kyle Welch & Stephen Stubben (2018). Private Equity’s Diversification Illusion: Evidence From Fair Value Accounting. Read status: Abstract only, plus the explanation of this paper as cited in Ilmanen et al. (2019).¹⁷ ↩
Source: I think I read this somewhere. ↩
Empirically, this effect matters a lot. I tested the utility-maximizing parameters when starting at various start years and the results different substantially. I needed to average across 30+ start years to get a more stable result. ↩
Technically, I extended it to last 1000 years for RORE data and 10,000 months for Ken French data. 10,000 months isn’t exactly 1000 years, but it’s close enough. ↩
Andrew Ang (2014). Asset Management: A Systematic Approach to Factor Investing. Oxford University Press. ↩
You can leverage public securities by buying on margin or buying derivatives, but you usually can’t borrow money to invest in private equity funds. ↩
In 2018, private equity firms spent $825 billion. In 2020, the global stock market was worth $89 trillion. If private equity firms tend to last 10 years, that means private equity is about 10% as big as public equity. So the efficient-market allocation for a normal investor would be ~90% to public equity, ~10% to private equity (ignoring all other asset classes). ↩
The math required isn’t that complicated, it’s just arithmetic. The complicated part is that the equations have over 140 variables (one for each year). ↩
Jeremy Siegel (2014). Stocks for the Long Run. 5th Edition. Page 97, Figure 6-2. ↩ ↩²
When returns are independent across time, standard deviation of annual returns decreases with the square root of the length of the holding period. To adjust for this, I “annualized” the computed standard deviation by multiplying by the square root of the holding period length. ↩
The data set includes 15 countries, but I only included countries that have continuous data from at least 1872 to 2015. Some countries’ time series don’t start until later, and a few have missing years in the middle, for example Japan’s stock market closed for two years after WWII. Excluding such countries biases the average return upward, but I’m not looking at the expected return, so I’m not too concerned about bias. ↩
Chart shows nominal returns. Adjusting for inflation does not significantly change the results. ↩
The significance test assumes every sample is independent, but in reality, the (say) 20-year holding period sample is almost identical to the 21-year holding period sample. So a “statistically significant” result might still be noise. A more sophisticated significance test could probably correct for this, but I don’t know how to do that. ↩
It makes more sense to model long-term mean reversion as a power function of holding period rather than a linear function. Most of the mean reversion happens in the first few years, and it quickly levels off. ↩
Siegel (2014)²⁷ also found that US bonds exhibit long-term momentum. ↩
I’ve spent on the order of 10 hours thinking about this question, and it’s still very unclear to me. ↩

Two Types of Scientific Misconceptions You Can Easily Disprove

Wed, 07 Sep 2022 00:00:00 -0700

Last updated 2023-03-22 to add another example.

There are two opposite types of scientific claims that are easy to prove wrong: claims that never could have been proven in the first place, and claims that directly contradict your perception.

Type 1: The unknowable misconception

A heuristic for identifying scientific misconceptions: If this were true, how would we know?

Example: “People swallow 8 spiders per year during sleep.” If you were a scientist and you wanted to know how often people swallow spiders, how would you figure it out? You’d have to do a sleep study for thousands of person-nights where you film people sleeping using a night vision camera that’s so high-quality that it can pick up something as small as a spider (which, as far as I know, doesn’t exist) and then pore over the tens of thousands of hours of footage by hand to look for spiders (because this factoid originated in a time when computer software wasn’t sophisticated enough to do it for you) and track the locations of the spiders and count how often they crawl into people’s mouths without coming back out. This is all theoretically possible but it would be insanely expensive and who would be crazy enough to do it?

Example: “The average man thinks about sex once every 7 seconds.” People can’t even introspect on their own thoughts on a continuous basis, how would scientists do it? This one seems simply impossible to prove, regardless of how big your budget is or how crazy you are.

Example: “Only 7% of communication is verbal, and 93% is nonverbal.” What does that even mean? How would a scientific study quantify all the information that two people transmit during a conversation and measure its informational complexity, and then conclude that 93% is nonverbal? You can kind of measure the information in words by compressing the text, but there’s no known way to accurately measure the information in nonverbal communication.

(This factoid does come from an actual study, but what the study actually showed¹ was that, among a sample of 37 university psychology students doing two different simple communication tasks, when verbal and nonverbal cues conflicted, they preferred the nonverbal cue 93% of the time.)

Type 2: The perception-contradicting misconception

You can disprove some common misconceptions using only your direct perception.

Example: “Certain parts of the tongue can only detect certain tastes.” You can easily disprove this by placing food on different spots on your tongue.

Example: “You need two eyes to have depth perception.” Close one eye. Notice how you still have depth perception. Or look at a photograph, which was taken from a fixed perspective. Notice how you can still detect depth in the photograph.

You need two points of reference to get “true” depth perception, but the human visual cortex can almost always infer depth based on how a scene looks. It’s possible to trick your depth perception, like this:

But in almost all real-world situations, you can correctly perceive depth with only one eye.

Example: “You (only) have five senses.” You can prove that you have at least two more senses: balance (equilibrioception) and the position of your limbs relative to each other (proprioception).

You can prove you have a sense of balance by closing your eyes and walking without falling over. You’re not using any of the standard five senses, but you can still stay upright.

You can prove you have proprioception by closing your eyes, flailing your arms around until they’re in random positions, and then bringing your hands together until your fingertips touch. You couldn’t do this if you didn’t have a proprioceptive sense.

(Some scientists say we have more than seven senses, but the other ones are harder to prove.)

So far I’ve given examples of factoids you can trivially disprove in five seconds. There are more misconceptions you can disprove if you’re willing to do a tiny bit of work. Example: “Women have one more rib than men.” Find a friend of the opposite gender and count your ribs!

Philip Yaffe (2011). The 7% Rule: Fact, Fiction, or Misunderstanding? ↩

Most Theories Can't Explain Why Game of Thrones Went Downhill

Mon, 05 Sep 2022 00:00:00 -0700

I’ve heard people repeat a few theories for why Game of Thrones started so well and ended so badly. Most of these theories don’t make sense.

Theory 1: David & Dan are good at adapting books, but they didn’t know what to do when they ran out of book.

Seasons 1 through 4, which adapted the first three books of A Song of Ice and Fire, are up there with the greatest television shows ever made. Season 5, which adapted the fourth book and part of the fifth book, was mediocre. If this theory were true, season 5 should have been on par with the earlier seasons, but it wasn’t.

Furthermore, season 6 was better than season 5, even though season 5 was still based on the books, and season 6 wasn’t.

Even more, David & Dan wrote some excellent original content in the earlier seasons, such as the extended arc with Arya and the Hound in season 4 (see this scene, which wasn’t in the books).

Some people say, well they know how to write short scenes, but they don’t know how to write story arcs. Then how do you explain the famously terrible dialogue in the later seasons?

Theory 2: David & Dan were always bad showrunners.

I hear this one a lot. There’s some evidence for this theory—prior to Game of Thrones, David was best known for writing X-Men Origins: Wolverine (a famously bad movie), and Dan had no prior writing credits. But if they’re bad showrunners, why were the first four seasons so good? I can buy that bad showrunners might accidentally create a pretty good show, but I don’t see how they could accidentally create one of the best shows of all time.

Theory 3: David & Dan lost interest and started phoning it in.

This explanation makes more sense because it can explain the nearly-monotonic decline in quality. But it still can’t explain why season 6 was better than season 5. And the timing doesn’t entirely work out—people usually say this about seasons 7 and 8, but season 5 was clearly worse than the previous four seasons, and it seems less plausible that they’d lose interest that early on.

Theory 4: Good writing emerges through a mysterious process that no one really understands.

This is my favorite theory. Many occasionally-great writers can’t consistently replicate their success, writers can’t tell which of their works will become popular, and nobody fully understands what makes great writing. That’s why, for example, Jane Austen thought Pride and Prejudice was her worst book, even though it’s what she’s most remembered for. Or why The Matrix is my favorite movie of all time, even though I like ~~zero (0)~~ one (1)¹ other Wachowski movie. (The Wachowskis are another example of artists who occasionally produce brilliant works and most of the time don’t, and it’s not clear why.)

Or why people used to talk about good art coming from a muse—you didn’t write that brilliant story, you just wrote down the words that your muse gave you, which is just a poetic way of saying you have no idea how you came up with it.

This is kind of a non-explanation: “the reason Game of Thrones was inconsistently good is because lots of things are inconsistently good and we don’t know why.” But at least it turns a localized mystery into a much bigger mystery about the general nature of creativity.

Edit 2023-09-04: Originally I wrote zero, but I just remembered that the Wachowskis co-wrote V for Vendetta, which I enjoyed. This is an irrelevant minor detail but I am committed to factual accuracy even when it doesn’t matter. ↩

Philanthropists Probably Shouldn't Mission-Hedge AI Progress

Tue, 23 Aug 2022 00:00:00 -0700

Summary

Confidence: Likely. [More]

Some people have asked, “should we invest in companies that are likely to do particularly well if transformative AI is developed sooner than expected?”¹

In a previous essay, I developed a framework to evaluate mission-correlated investing. Today, I’m going to apply that framework to the cause of AI alignment.

(I’m specifically looking at whether to mission hedge AI, not whether to invest in AI in general. [More])

Whether to mission hedge crucially depends on three questions:

What is the shape of the utility function with respect to AI progress?
How volatile is AI progress?
What investment has the strongest correlation to AI progress, and how strong is that correlation?

I came up with these answers:

Utility function: No clear answer, but I primarily used a utility function with a linear relationship between AI progress and the marginal utility of money. I also looked at a different function where AI timelines determine how long our wealth gets to compound. [More]
Volatility: I looked at three proxies for AI progress—industry revenue, ML benchmark performance, and AI timeline forecasts. These proxies suggest that the standard deviation of AI progress falls somewhere between 4% and 20%. [More]
Correlation: A naively-constructed hedge portfolio would have a correlation of 0.3 at best. A bespoke hedge (such as an “AI progress swap”) would probably be too expensive. An intelligently-constructed portfolio might work better, but I don’t know how much better. [More]

Across the range of assumptions I tested, mission hedging usually—but not always—looked worse on the margin² than investing in the mean-variance optimal portfolio with leverage. Mission hedging looks better if the hedge asset is particularly volatile and has a particularly strong correlation to AI progress, and if we make conservative assumptions for the performance of the the mean-variance optimal portfolio. [More]

The most obvious changes to my model argue against mission hedging. [More] But there’s room to argue in favor. [More]

Cross-posted to the Effective Altruism Forum.

Summary
Contents
Methodology
- Why care about mission hedging on the margin rather than in equilibrium?
- A big list of considerations
What is the shape of the utility function?
- Compound-until-AGI utility function
How volatile is AI progress?
How good is the best hedge?
Results
Source code
Acknowledgments
Appendices
Notes

Methodology

Let’s start with the framework that I laid out in the previous post. To recap, this framework considers three investable assets:

a legacy asset: a suboptimal investment that we currently hold and want to sell
a mean-variance optimal (MVO) asset: the best thing we could buy if we simply wanted to maximize risk-adjusted return
a mission hedge asset: something that correlates with AI progress, e.g., AI company stocks

We also have a mission target, which is a measure of the thing we want to hedge. For our purposes, the mission target is AI progress.

We can sell a little bit of the legacy asset and invest the proceeds into either the MVO asset or the hedge asset. The model tells us which asset we should choose.³

We feed the model a few inputs:

the expected return, volatility, and correlations for these four variables (three assets plus the mission target)
a utility function over wealth and the mission target
our starting level of investment in the three assets

The model tells us which asset we should shift our investments toward on the margin.

(Note: I have tried to make this essay understandable on its own, but it will probably still be easier to understand if you read my previous essay first.)

My model includes many variables. For most of the variables, I assigned fixed values to them—see below for a big list of model considerations. I fixed the value of a variable wherever one of the following was true: (1) I was confident that it was (roughly) correct; (2) I was confident that it would be too hard to come up with a better estimate; (3) it didn’t affect the outcome much; or (4) it was necessary to make the model tractable.

That left three considerations:

What is the relationship between AI progress and the utility of money?
How volatile is AI progress?
What investment has the strongest correlation to AI progress, how strong is that correlation, and what is its expected return?

So the question of whether to mission hedge (according to this model) comes down to how we answer these three questions.

Why care about mission hedging on the margin rather than in equilibrium?

When evaluating our investment portfolios, we can consider two distinct questions:

What is the optimal allocation to mission hedging in equilibrium—in other words, what is the ultimate allocation to mission hedging that maximizes utility?
On the margin, what is the expected utility of allocating the next dollar to mission hedging vs. traditional investing?

This essay focuses on the second question because it matters more. Why does it matter more?

In the long run, we want to know the optimal asset allocation. But we can’t instantly move to the optimal allocation. Many altruistic investors have money tied up in assets that they can’t sell quickly; many value-aligned investors don’t put much thought into how they invest, and end up doing something suboptimal, so we need to account for this.

I expect it will take a bare minimum of 10 years to for the EA community to move from the current investment allocation to the optimal one.⁴ Until then, we want to know what changes will maximize utility in the short run. Therefore, we need to identify the best change on the margin.

A big list of considerations

Considerations that matter, with the key considerations in bold:

Shape of the utility function with respect to wealth
Shape of the utility function with respect to AI progress
Coefficient of relative risk aversion
Shape of the distribution of investment returns
Shape of the distribution of AI progress
Expected return of the MVO asset
Expected return of the legacy asset
Expected return of the hedge asset
Volatility of the MVO asset
Volatility of the legacy asset
Volatility of the hedge asset
Volatility of AI progress
Correlation of the MVO asset and the legacy asset
Correlation of the MVO asset and the hedge asset
Correlation of the legacy asset and the hedge asset
Correlation of the MVO asset and AI progress
Correlation of the legacy asset and AI progress
Correlation of the hedge asset and AI progress
Impact investing factor
Target volatility
Cost of leverage
Proportion of portfolio currently invested in MVO/legacy/hedge assets

A consideration that you might think matters, but doesn’t:

Expected rate of AI progress — doesn’t affect optimal hedging (for an explanation, see the third qualitative observation here)

For all the non-key considerations, I assign them a single fixed value. The only exception is the return/volatility of the MVO asset, where I included numbers for both a factor portfolio and the global market portfolio. I expect a factor portfolio to earn a better return and I believe it can be used as the MVO asset. But I don’t expect to convince everyone of this, so for skeptical readers, I also included numbers for the global market portfolio.

In the following list, I assign values to every non-crucial consideration. For justifications of each value, see Appendix A.

For all values relating to the MVO asset, I present results both for the global market portfolio (GMP) and for a factor portfolio (FP).

Shape of the utility function with respect to wealth: bounded constant relative risk aversion
Shape of the distribution of investment returns: log-normal
Shape of the distribution of AI progress: log-normal
Coefficient of relative risk aversion: 1.5
Arithmetic return of the MVO asset (nominal): 5% (GMP) or 9% (FP)⁵
Arithmetic return of the legacy asset: 7%
Arithmetic return of the hedge asset: 7%
Volatility of the MVO asset: 9% (GMP) or 13% (FP)
Volatility of the legacy asset: 30%
Volatility of the hedge asset: 25%
Correlation of the MVO asset and the legacy asset: 0.6 (GMP) or 0.5 (FP)
Correlation of the MVO asset and the hedge asset: 0.6 (GMP) or 0.5 (FP)
Correlation of the legacy asset and the hedge asset: 0.7
Correlation of the MVO asset and AI progress: 0.0 (GMP) or –0.1 (FP)
Correlation of the legacy asset and AI progress: 0.2
Impact investing factor: ignored
Target volatility: 30%
Cost of leverage: 2%
Proportion of portfolio currently invested in MVO/legacy/hedge assets: 50% / 50% / 0%

What is the shape of the utility function?

Caveat: Someone who works on AI safety would probably be better positioned than me to think about the shape of the utility function. But I’m going to speculate on it anyway.

When I initially developed a model of mission hedging, I specified that utility varies linearly with the mission target. That makes perfect sense if the mission target is something like CO2 emissions. It’s not clear that it makes sense for AI. Either transformative AI goes well and it’s amazing, or it goes poorly and everyone dies. It’s binary, not linear.

Consider an alternative utility function with these properties:

Transformative AI happens when some unknown amount of AI progress has occurred. The required amount of progress follows some known probability distribution.
If some unknown amount of alignment research gets done before then, transformative AI goes well. Utility = 1. (The units of our utility function are normalized such that the total value of the light cone is 1.) Otherwise, transformative AI goes badly. Utility = 0. The required amount of alignment research follows some known probability distribution.

This utility function (kind of) describes reality better, but it doesn’t really work—it heavily depends on what probability distributions you pick. The model only puts high value on AI alignment research if there’s a high probability that, if you don’t do anything, the amount of alignment research will be very close to the exact required amount. If there’s a high probability that enough alignment research gets done without you, or that we’ll never get enough research done, then the model says you shouldn’t bother. The model becomes highly sensitive to forecasts of AI timelines and of the amount of safety research required.

(This utility function might imply that we should mission leverage, not mission hedge. If AI progress accelerates, that decreases the probability that we will be able to get enough safety research done, which might decrease the marginal utility of research.)

This binary way of thinking doesn’t reflect the way people actually think about the value of research. People tend to believe that if less safety research gets done, that makes marginal research more valuable, not less.

I would think of research in terms of drawing ideas from an urn. Every idea you draw might be the one you need. You start with the most promising ideas and then move on to less promising ones, so you experience diminishing marginal utility of research. And if AI progress moves faster, then marginal alignment research matters more because the last draw from the urn has higher expected utility.

But if transformative AI will take a long time, that means we get many chances to draw from the urn of ideas. So even if you draw a crucial idea now, there’s a good chance that someone else would have drawn it later anyway—your counterfactual impact is smaller in expectation. And the probability that someone else would have drawn the same idea is inversely proportional to the length of time until transformative AI. So we’re back to the original assumption that utility is linear with AI progress.⁶

Compound-until-AGI utility function

We could use an alternative model, which we might call the “compound-until-AGI” model:

As before, every dollar spent has some diminishing probability of solving AI alignment.
Wealth continues to compound until transformative AI emerges, which happens at some uncertain future date.

If we say W = wealth, b = the mission target—in this case, the inverse of time until transformative AI—then the utility function becomes:

\begin{align} U(W, b) = (W^b)^{1 - \gamma} \displaystyle\frac{1}{1 - \gamma} \end{align}

This model heavily disfavors mission hedging, to the point that I could not find any reasonable combination of parameters that favored mission hedging on the margin over traditional investing.⁷

For the rest of this essay, I will focus on the original model, with the understanding that the compound-until-AGI model may be more accurate, but it disprefers mission hedging for any plausible empirical parameters.

The utility function is a weak point in my model. I basically made up these utility functions without any strong justification. And a different function could generate significantly different prescriptions.

How volatile is AI progress?

I measured AI progress (or proxies for AI progress) in three different ways:

Average industry revenue across all US industries [More]
Performance on ML benchmarks [More]
AI timeline forecasts [More]

These measures go from most to least robust, and from least to most relevant. We can robustly estimate industry revenue because we have a large quantity of historical data, but it’s only loosely relevant. AI timeline forecasts are highly relevant, but we have barely any data.

This table provides estimated standard deviations according to various methods. The table does not give any further context, but I explain all these estimates in their respective sections below.

Category	Method	$\sigma$
1. revenue	weighted average (long-only)	17.6%
1. revenue	weighted average (market-neutral)	19.9%
2. ML benchmarks	average across all benchmarks	15.4%
3. forecasts	Metaculus (point-in-time)	8.9%
3. forecasts	Metaculus (across time)	4.3%
3. forecasts	Grace et al. (2017) survey	N/A⁸

Estimates range from 4.3% to 19.9%, with an average of 12.3% and a category-weighted average of 13.7%.

The next table gives the standard deviation over the logarithm of AI timeline forecasts. (See below for how to interpret these numbers.)

Method	$\sigma$
Metaculus	0.99
Grace et al. (low estimate)	1.20
Grace et al. (high estimate)	1.32

Industry revenue

How do we quantify the success of an industry? Ideally, we’d track the quantity of production, but I don’t have good data on that, so I’ll look at revenue. Revenue is a reasonable proxy for success. Revenue and stock price are probably more closely correlated than production quantity and stock price because there’s a more direct path from revenue to shareholder value.

Rather than just looking at the ML industry (which is only one industry, and didn’t even exist until recently), I looked at every industry. For each industry, I calculated the growth in aggregate revenue across the industry, and then found the volatility of growth.

It’s not obvious how to aggregate the volatility numbers for every industry. We could take the mean, but there are some tiny industries with only a few small-cap companies, and those ones have super high volatility. So the average volatility across all industries doesn’t represent a reasonable expectation about what the ML industry might look like in the future.

We could take the median, but that doesn’t capture skewness in industry volatility.

I believe the best metric is a weighted average, weighted by total market cap of the industry. So larger industries get a greater weight when calculating average volatility.

I calculated the volatility of industry revenue from 1973 to 2013⁹. I also calculated the volatility of market-neutral industry revenue, which is the revenue per share of an industry minus the revenue per share of the total market.¹⁰

The results:

	Long-Only	Market-Neutral
Weighted Mean	17.6%	19.9%
Mean	18.2%	20.2%
Median	14.1%	17.2%
Std Dev	14.5%	8.8%

(Std Dev is the standard deviation of the standard deviations.)

For context, there were 66 industries as defined by GICS. The average industry contained 17 stocks in 1973 and 47 stocks in 2013.

ML benchmark performance

Fortunately, other people have already put a lot of work into measuring the performance over time of state-of-the-art ML models.

Unfortunately, they all care about average improvement or trends in improvement, not about volatility.

Fortunately, some of them have made their data publicly available, so we can calculate the volatility using their data.

I used the Electronic Frontier Foundation’s AI Progress Measurement database, which aggregates ML model scores across dozens of benchmarks, including Chess Elo rating, Atari game performance, image classification loss, and language parsing accuracy, to name a few. The data ranges from 1984 to 2019, although until 2003, it only included a single measure of AI performance (Chess elo), and there still weren’t many metrics until 2015.

For each metric-year, I calculated the growth in performance on that metric since the previous year, where performance is defined as loss (or inverse loss, if you want a number that increases rather than decreases¹¹). Then I took the average growth across all metrics.

I calculated the mean and standard deviation of the logarithm of growth over three time periods: the full period (1985–2019¹²), just the most recent 10 years (2010–2019), and just the period with many metrics (2015–2019):

Range	Mean	Stdev
1985–2019	10.7%	15.4%
2010–2019	29.6%	15.8%
2015–2019	40.6%	15.7%

Before running these numbers, I worried that I’d get dramatically different numbers for the standard deviation depending on what range I used. But luckily, all three standard deviations fall within 0.3 percentage points (even though the averages vary a lot). These numbers (weakly) suggest that, even though ML progress has been accelerating, volatility hasn’t increased. (Although if we expect ML progress to continue accelerating, that means our subjective volatility should be higher.)

AI timeline forecasts

As a final test, I calculated AI progress from AI timeline forecasts. In a sense, this is the best metric, because it’s the only one that really captures the type of progress we care about. But in another sense, it’s the worst metric, because (1) we don’t have any particularly good forecasts, and (2) how do you convert a timeline into a growth rate?

To convert an AI timeline into a growth rate, I simply took the inverse of the number of years until transformative AI. This is a questionable move because it assumes (1) AI progresses linearly and (2) there exists some meaningful notion of a growth rate that describes what AI needs to do to get to transformative AI from where we are now.

Questionableness aside, I used Metaculus question 3479, “When will the first AGI be first developed and demonstrated?” because it’s the most-answered Metaculus question about AGI, with (as of 2022-04-14) 1070 predictions across 363 forecasters.¹³

First I looked at the standard deviation of the inverse of the mean date across forecasters, which came out to 8.6%. But this isn’t quite what we want. The point of mission hedging is to buy an asset that covaries with your mission target over time. So I also looked at the Metaculus community prediction (an aggregate best guess across all predictors) across time, and found a standard deviation of 4.1%.

I also applied AI timeline forecasts to the “compound-until-AGI” model, which has a more meaningful interpretation. This alternative model takes a log-normally distributed AI timeline and calculates the expected probability of solving the alignment problem by the time transformative AI arrives. For this model, we want to know the standard deviation of the logarithm of estimated AI timelines.

Metaculus question #3479 gives a mean log-timeline of 3.09 log-years and a standard deviation of 0.99 log-years.

(The exponential of 0.99 log-years would be 2.69 years, except that we can’t meaningfully talk about the standard deviation in terms of years. What we can meaningfully say is that the geometric mean timeline is $e^{3.09}$ = 22 years, and the timeline at +1 standard deviation is $e^{3.09 + 0.99}$ = 59 years. And 59 / 22 = 2.69. So 2.69 is more like the “year growth factor”—every one standard deviation, the timeline multiplies by 2.69.)

I also used the “High Level Machine Intelligence” forecast from the expert survey in Grace et al. (2017)¹⁴¹⁵. The paper gives a mean of 3.8 log-years (corresponding to a median¹⁶ timeline of 45 years). The paper does not report exact the standard deviation, but I estimated it from the first and third quartiles and came up with 1.26 log-years.¹⁷

As discussed previously, I could not find any reasonable inputs that favored mission hedging under the compound-until-AGI model. (I tried the figures derived from Metaculus and Grace et al. as well as a much wider range of parameter values.)

How good is the best hedge?

We could hedge AI progress with a basket of AI company stocks. But it’s hard to tell how well this will work:

The companies that do the most/best AI research are big companies like Google. AI only makes up a small portion of what Google does, and its progress on AI doesn’t drive the stock price (at least for now).
Some small companies exclusively do AI work, but not many, and they haven’t existed for long.

(I looked for ETFs and mutual funds that focus on AI stocks. About a dozen such funds exist, but all but one of them are less than four years old. The oldest is ARKQ, which only launched in 2014, and doesn’t quite match what we want anyway.)

Let’s look at how some possible hedges correlate to their targets.

Industry index <> industry revenue

What’s the relationship between AI progress and a basket of AI company stocks?

Let’s ask an easier question: For any industry, what is the relationship between the productivity of that industry and its average stock return?

This question is only relevant insofar as the AI industry behaves the same as any other industry, and insofar as “revenue” is the same thing as “productivity”. But it gives us a much larger sample (dozens of industries instead of one) and a longer history (~40 years instead of ~7).

this but unironically

As above, we can measure the success of an industry in terms of its revenue. And as above, I calculated the weighted average correlation between industry revenues and returns, and also between market-neutral industry revenues and returns.

	Long-Only	Market-Neutral
Weighted Mean	0.21	0.04
Mean	0.23	0.06
Median	0.25	0.06
Std Dev	0.23	0.22

These correlations are really bad. The weighted mean long-only correlation is actually lower than my most pessimistic estimate in my original essay. And the market-neutral correlation, which better captures what we care about, might as well be zero.

Semiconductor index <> ML model performance

The Global Industry Classification Standard does not recognize an AI/ML industry. The closest thing is probably semiconductors. For this test, I used the MSCI World Semiconductors and Semiconductor Equipment Index (SSEI), which has data going back to 2008. I compared it to three benchmarks:

EFF ML benchmark performance, described above, 2008–2021
ML model size growth, measured as a power function of compute (for reasons explained in Appendix B), 2012–2021
ML model size growth, measured as a power function of the number of parameters, 2012–2021

I also compared these to

the S&P 500
a long/short portfolio that went long SSEI and short the S&P 500

Results:

Target	SSEI	SP500	Long/Short
EFF ML benchmark performance	0.17	0.15	0.17
ML model compute^0.05	0.33	0.20	0.41
ML model num. params^0.076	0.52	0.50	0.46

These numbers suggest that there’s no reason to use semiconductor stocks as a hedge because the S&P 500 works about as well.

AI company index <> ML model performance

There’s no good way to test the correlation of ML model performance to an AI index. At best, I could take one of the AI company ETFs that launched in 2018 and correlate it against ML benchmark performance. But I’d only get at most three annual data points, and that would just anchor my beliefs to a highly uncertain number, so I’m not going to do it.

Perhaps we could construct a hedge portfolio in such a way that we can predict its correlation to the thing we want to hedge, even if we don’t have any historical data. I don’t know how to do that.

Construct a derivative

Could we define a new derivative to hedge exactly what we want? For example, we could pay an investment bank to sell us an “AI progress swap”, that goes up in value when progress accelerates, and goes down when it slows.

A derivative like this would have a negative return—probably significantly negative, because novel financial instruments tend to be expensive. And unfortunately, a higher correlation can’t make up for even a modest negative return. A derivative like this wouldn’t work unless we could buy it very cheaply.

Can we do better (without a custom derivative)?

A naive hedge (such as a semiconductor index) only gets us about 0.2 correlation. Surely a carefully-constructed hedge would have a higher correlation than that. But how much higher? There’s no way to directly test that question empirically, and I don’t even know what a useful reference class might be. So it’s hard to say how high the correlation can go.

Let’s take r=0.5 as an upper bound. That tends to be about the best correlation you can get between two variables in a messy system. I’m inclined to say that’s too high—my intuition is that you probably can’t get more than a 0.4 correlation, on the basis that more than half the correlation of a hedge comes from obvious things like industry behavior, and the obvious things get you about r=0.2. But I wouldn’t be too surprised if r>0.5 turned out to be achievable.

(In my previous post on mission-correlated investing, where I set up the theoretical model, I gave a plausible correlation range of 0.25 to 0.9. Now that I’ve spent more time looking at empirical data, I think 0.9 was too optimistic, even for an upper-end estimate.)

Results

If we take my estimates for the return of a concentrated factor portfolio, mission hedging looks unfavorable even given optimistic volatility and correlation estimates.

If we invest in the global market portfolio, whether to mission hedge depends on our choices for the key parameter values (AI progress volatility and hedge <> target correlation).

At 20% volatility, mission hedging looks unfavorable given a naive hedge (r=0.2), but favorable if we can find a hedge with even a modestly better correlation (r>=0.25).
At 15% volatility, mission hedging looks favorable if the hedge has r=0.3 or greater.
At 10% volatility, mission hedging looks favorable only if the hedge has r=0.5 or greater.
At less than 10% volatility, mission hedging does not look favorable for any plausible correlation.

If we use the compound-until-AGI utility function, mission hedging looks worse than ordinary investing given any reasonable input parameters (although under the most favorable parameters for mission hedging, it only loses by a thin margin).

That’s all assuming our hedge matches the expected return of a broad index. If we expect the hedge to outperform the market, then it’s highly likely to look favorable compared to the global market portfolio, and likely unfavorable versus a factor portfolio (see “AI companies could beat the market” below for specific estimates). At least, that’s true given all the other assumptions I made, but my other assumptions tended to favor mission hedging.

If you want to play with my model, it’s available on GitHub.

Why does ordinary investing (probably) look better than mission hedging?

In short, leveraged investments look compelling, and they set a high bar that mission hedging struggles to overcome. At the same level of volatility, a leveraged diversified portfolio outperforms a hedge portfolio (in expectation) by a significant margin.

Most philanthropic investors don’t use enough leverage, which means on the margin, adding leverage to your portfolio increases expected utility by a lot. To compare favorably, mission hedging would have to increase expected utility by even more.

By mission hedging, you’re sacrificing expected earnings in exchange for making money in worlds where money matters more. If a leveraged market portfolio has (say) 5 percentage points higher expected return than the hedge portfolio, then the hedge needs to perform 5 percentage points better than average just to break even.¹⁸ So even if you find yourself in one of the worlds you were trying to hedge against (a world with faster-than-expected AI progress), it’s likely that a leveraged market portfolio outperforms the hedge asset even in that world.

Mission hedging doesn’t work as well as conventional hedging. Say you’re a bread-making company. You have a 10% profit margin, and for the sake of simplicity, your only input cost is wheat. If the price of wheat goes up by 5%, you just lost half your profit. So you really want to hedge against wheat prices (you effectively have 10:1 leverage on wheat prices). But if you’re mission hedging and the mission target gets 5% worse, your marginal utility of money doesn’t increase by all that much.

(Plus, wheat futures prices have a perfect correlation to future wheat prices, but no investment has a perfect correlation to AI progress.)

Mission hedging only makes sense if your dollars become far more valuable in the scenarios you’re trying to hedge against, and if the hedge asset earns significant excess returns in those scenarios.

Objections to this methodology

I can see (at least) two major, possibly-defeating objections to my approach:

Investing in a basket of AI companies (or some similar hedge asset) accelerates AI capabilities, which is bad.
Objective metrics like ML benchmark progress tell us little about progress toward AGI.

The first argues against mission hedging. The second could argue against or in favor, depending on how you interpret it.

Impact investing

If you invest in a company, you might cause that company to perform better (and if you divest, you might cause it to perform worse). If true, this argues against mission hedging.

Paul Christiano wrote an analysis suggesting that divesting works somewhat well; Jonathan Harris did an (unpublished) calculation based on Betermier, Calvet & Jo (2019)¹⁹ and estimated that divesting has a positive but negligible effect. I only weakly understand this area, but it’s at least plausible that the direct harm of investing outweighs the benefit of mission hedging.

Measurability of AI progress

Gwern argues that easily-measurable AI progress benchmarks tell us approximately nothing useful about how close we are to AGI.

This criticism applies to my approach of estimating AI progress volatility using ML benchmark performance. The way I measure AI progress doesn’t necessarily relate to the sort of progress we want to hedge, and the way I convert ML benchmark performance into marginal utility of money is kind of arbitrary and only weakly justified.

If true, this doesn’t automatically defeat mission hedging. What it does mean is that our proxies for the volatility of AI progress don’t tell us anything about true volatility, and we have no way to estimate true volatility (if there even is such a thing).

If we can’t reliably measure AI progress, that makes mission hedging seem intuitively less appealing (although I don’t quite know how to explain why). Even so, the concept of mission hedging—make more money in worlds where AI progress moves faster than expected—seems sound. And Gwern’s criticism does not apply to using AI timeline forecasts to project AI progress.

And we don’t actually need our progress benchmarks to be all that similar to “actual” AI progress. We just need the two to correlate. What we need is a strong correlation of hedge asset <> actual AI progress. Even if the benchmark of AI progress <> actual AI progress correlation is weak, I don’t see why that would weaken the hedge asset <> actual AI progress correlation in expectation. (On the contrary, it seems intuitive to me that AI company stock prices should correlate more strongly with actual progress than with benchmarks.)

What’s the strongest case in favor of mission hedging?

Most of the time when a non-key input variable had wiggle room, I chose a value that favored mission hedging (as detailed in Appendix A).²⁰ Mission hedging still looked worse than investing in the mean-variance optimal portfolio under most key parameter values. Is there any room left to argue that we should mission hedge AI?

Yes, there’s room. Below are four arguments in favor of higher investment in AI stocks.

Illegible assumptions could bias the model against mission hedging

You could argue that my model’s assumptions implicitly favor ordinary investing. Most obviously, you could probably come up with a utility function that makes mission hedging look much better. I don’t currently know of any such function, but that seems like a key way that the model could be (very) wrong.

The volatility of AI progress could increase

Before transformative AI arrives, we might reach a point where AI progress unexpectedly accelerates. If we hold a hedge position, we can earn potentially massive returns at this point, and then convert those returns into AI alignment work. However:

This only works if transformative AI follows a slow takeoff. (Paul Christiano believes it (probably) will; Eliezer Yudkowsky believes it (probably) will not. See also Scott Alexander’s commentary.²¹)
Volatility would have to increase significantly to justify mission hedging.
Volatility would have to increase well before AGI, so that would-be mission hedgers can earn extra money and still have time to spend it on AI alignment work.

When I tried to estimate the volatility of AI progress, two of my three estimates looked at historical data. Only the forecast method could capture future changes in volatility. So it’s plausible that volatility will spike in the future, and my analysis missed this because it was mostly backward-looking. (On the other hand, the forecast method provided the lowest estimated standard deviation.)

Investors could be leverage-constrained

I assumed that investors are volatility-constrained but not leverage-constrained—I explain why in Appendix A. However, it’s at least plausible that many investors face meaningful leverage constraints. If so, then we care about maximizing the absolute return of our investments, rather than the risk-adjusted return. (An investor who can use leverage wants to pick the investment with the highest risk-adjusted return and then lever it up, which will always produce an equal or better absolute return than investing in the highest-absolute-return asset with no leverage.)

Investors with strict leverage constraints should probably mission hedge on the margin. As I mentioned above, the big advantage of the MVO asset over the hedge asset is that it earns a far better return once it’s levered up. If you don’t use leverage, the MVO asset doesn’t earn a much higher return²²—it does have much less risk, but that’s not as important for altruistic investors.

AI companies could beat the market

Perhaps the strongest case for investing in AI companies has nothing to do with mission hedging. Namely, if you believe the market underestimates how quickly AI systems will improve, then you might invest in AI stocks so you can beat the market. I don’t have a strong opinion on whether this is a good idea, but it’s not mission hedging—it’s just traditional profit-seeking investing.

Some specific numbers:

If we invest in the global market portfolio and assume mission target volatility = 10%, r = 0.2 (which is relatively unfavorable for mission hedging), then the hedge asset only has to beat the stock market by 1 percentage point to look better than the MVO asset.

If we invest in a factor portfolio with the same unfavorable assumptions for mission hedging, the hedge has to beat the stock market by 10 (!) percentage points.

If we invest in a factor portfolio with favorable assumptions for mission hedging, the hedge has to beat the market by 3 percentage points.

If we set mission target volatility at 5%, there’s pretty much no way for the hedge asset to look favorable.

In summary, if we expect AI companies to beat the market, then we may or may not prefer to invest in AI stocks on the margin, and it could easily go either way depending on what assumptions we make.

Credences

In this section, I offer my subjective probabilities on the most relevant questions.

All things considered, AI safety donors should not mission hedge on the margin: 70%

All things considered, AI safety donors should not overweight AI stocks on the margin: 60%²³

On the condition that I’ve thought of all the important considerations, AI safety donors should not mission hedge on the margin: 75%

I have missed some critically-important consideration: 20%

…and that consideration makes mission hedging look better: 10%

A utility function with linear marginal utility of AI progress overestimates the marginal value of mission hedging: 75%

The true volatility of AI progress, inasmuch as that’s a well-defined concept, is at most 20%: 75%

The maximum achievable correlation for a hedge asset with non-negative alpha²⁴ is at most 0.5: 85%

On balance, the non-key parameters’ given values make mission hedging look better than their true values would: 90%²⁵

On balance, the non-key parameters’ given values make investing in AI stocks look better than their true values would: 60%

Future work

1. Better conceptual approach.

Is there a better way to model the (potentially) binary nature of AI takeoff?

If AI progress is fat-tailed (or, technically, fatter-tailed than a log-normal distribution), how does that change things?

2. Reconcile assumptions that were favorable and unfavorable to mission hedging.

Most assumptions in Appendix A lean toward mission hedging. But one big assumption—that the hedge asset won’t outperform the market—might strongly disfavor mission hedging if it turns out the hedge can beat the market. Which force is stronger: the collection of small tilts toward mission hedging, or the (potential) single big tilt against it? How can we find out without introducing way too many free variables into the model?

3. Does investing in AI companies accelerate AI progress?

If yes, that could be a compelling reason not to mission hedge, although this would only strengthen my conclusion because I already found that mission hedging doesn’t look worthwhile.

4. Deeper investigation on empirical values.

I got all my data on ML benchmark performance from EFF’s AI Progress Measurement. We could look at more data from other sources, such as Papers With Code SOTA. (This GitHub project scrapes AI progress data from several sources.)

My forecast data used only a few questions from Metaculus plus a single survey. I picked those because they seemed like the best sources, but a thorough analysis could incorporate more forecasts. Maybe combine forecasts from multiple surveys, such as the ones collected by AI Impacts. (Although I don’t think a broader set of forecasts would change the ultimate outcome much.)

Alternatively, we could project the volatility of AI progress using Ajeya Cotra’s biological anchors report (see Google Drive, Holden’s summary). I didn’t do this because it seemed less relevant that my other approaches and I couldn’t easily see how to convert biological anchors into a measurable mission target, but it might be worth looking into.

Source code

Source code for estimating AI progress is available on GitHub, as is source code for deriving optimal mission hedging.

Acknowledgments

Thanks to three anonymous financial professionals for reviewing drafts of this essay.

Appendices

Appendix A: Justifications for the big list of considerations

Shape of the utility function with respect to wealth: bounded constant relative risk aversion

In my previous essay, I set forth a list of requirements for a utility function over wealth and the mission target. Specifically with respect to wealth, I wanted a utility function that was (1) bounded above and (2) had constant relative risk aversion. I explained my reasoning in that essay.

Coefficient of relative risk aversion: 2

People in the EA community tend to believe that the value of wealth is logarithmic,²⁶²⁷ which would correspond to a risk aversion of 1. But a logarithmic function is unbounded. If our goal is (solely) to maximize the probability that transformative AI goes well, and if utility is linear with this probability, then utility must be bounded.

We could pick a coefficient of risk aversion that gives a nearly-logarithmic utility function, say, 1.1. That gives a function that looks logarithmic for a while, but starts to flatten out as we get close to the upper bound.

But I find it plausible that the coefficient should be as high as 2, which corresponds to the utility function $U(W) = 1 - \frac{1}{W}$.²⁸

(You could also argue that risk aversion should be less than 1, but I do not consider that possibility in this essay.)

Lower risk aversion makes the MVO asset look better. It doesn’t change the long-run optimal ratio of mission hedging to MVO investing, but it does make the MVO asset look relatively better on the margin because we become more “return-starved”—with lower risk aversion, we place higher value of earning as much return as possible.

I chose a risk aversion of 2 to be generous toward mission hedging.

Shape of the distribution of investment returns: log-normal

It’s standard in academic finance to assume that asset prices follow log-normal distributions. It’s not true—bad outcomes are more likely than a log-normal distribution would suggest—but it’s sort of close to true, and it makes the math simple.

I don’t know how things would change if I used a more accurate distribution. All three possible investments would look worse, but the real question is whether any one would look worse relative to the others.

Some investments are right-skewed, such as options, managed futures, and maybe whatever it is Alameda Research does. If the MVO asset is right-skewed (or at least less left-skewed than the market), then using accurate distributions could favor the MVO asset.

However, AI progress might be right-skewed if a huge amount of expected value comes from unexpectedly fast progress on the right tail.²⁹ In that case, the hedge asset might have a stronger right skew than the MVO asset. On the other hand, if AI capabilities improve that quickly, it might not be possible to convert money into AI alignment in the limited time available.

Shape of the distribution of AI progress: log-normal

When I was first thinking about writing this essay, I was having a casual conversation about the subject with a friend. I asked him what he thought was the shape of the distribution of AI progress. He said, “Isn’t everything log-normal?” And I was already thinking along those lines, so I went with it.

(If I assume AI progress is log-normally distributed, that makes it easy to calculate the correlated growth of AI progress and the various investments.)

One could reasonably object to this—see Measurability of AI progress.

Arithmetic return of the MVO asset: 5% (GMP) or 9% (FP)

For a factor portfolio, I estimated the future return and standard deviation as described in a How I Estimate Future Investment Returns. I started from a 6% real geometric return and added projected inflation and volatility drag³⁰, giving a 9% nominal arithmetic return.

For the global market portfolio, I assumed that current valuations can’t predict future returns. I started from AQR’s 2022 return projections (which use the dividend discount model) and ran a mean-variance optimizer over global equities and global bonds. The result had a 5% return and 9% standard deviation.

I estimated future volatility using historical volatility, as described in my aforementioned post.

Arithmetic return of the legacy asset: 7%

In general, we should expect any randomly-chosen asset to have the same arithmetic return as its index. If most legacy investments are US companies, we can use the US stock market as a baseline. The dividend discount model predicts the US market to earn a 7% nominal return. I would predict a lower return than this because US stocks are so expensive right now, but changing this number doesn’t change the relative appeal of ordinary investing vs. mission hedging (unless you make it much higher).

Arithmetic return of the hedge asset: 7%

As with the legacy asset, we should (generally speaking) expect the hedge asset to earn the same return as its index.

That said, if you’re concerned about AI safety, you might also believe that the market underestimates the future rate of AI progress.³¹ If you also expect a slow takeoff, and you expect that AI companies will earn large profits during this takeoff, then you probably expect AI stocks to beat the market.

If you overweight AI stocks because you expect them to beat the market, that’s not mission hedging, it’s just ordinary profit-seeking investing. Your mean-variance optimal portfolio should include more AI stocks, but you shouldn’t increase your allocation to AI stocks as a hedge. Still, you would increase your total allocation to AI companies.

See “AI companies could beat the market” for more on this.

Volatility of the MVO asset: 9% (GMP) or 13% (FP)

As described in How I Estimate Future Investment Returns, I set future volatility equal to historical average volatility because there’s no compelling reason to expect it to be systematically higher or lower.

Volatility of the legacy asset: 30%

Large-cap stocks tend to have standard deviations of around 40%, and mega-cap stocks have standard deviations around 30%.³² For more on this, see The Risk of Concentrating Wealth in a Single Asset.

Arguably, the thing we care about isn’t the volatility of a single legacy asset, but the volatility of all EAs’ legacy assets put together. But which assets to include depends on exactly which other donors you care about. This doesn’t matter too much because the volatility of the legacy asset does not meaningfully affect the tradeoff between the MVO asset and the hedge asset.

Volatility of the hedge asset: 25%

Empirically, smallish baskets of randomly-chosen stocks are a bit less volatile than this (~20%, see Fact 2 here), but if the stocks are chosen to be correlated to a particular target, the basket will be more volatile because it’s less diversified. 25% is consistent with the standard deviations of returns for individual industries. Market-neutral industry baskets (i.e., buy the industry and short the market) have standard deviations around 20%.

Correlation of MVO asset <> legacy asset: 0.6 (GMP) or 0.5 (FP)

We can roughly assume that future correlations will resemble historical correlations. It’s easy to get historical data on a basic stock/bond portfolio or on a factor portfolio. It’s a little harder to find the correlation with the legacy asset because we don’t exactly know what the legacy asset is, and even if we did, it probably doesn’t have more than a decade or two of performance history.

As an approximation, I took the 30 industry portfolios from the Ken French Data Library, on the assumption that the correlation between the average industry and an investment strategy is roughly the same as the correlation between the legacy asset and that same investment strategy.

I also used industry portfolios as proxies for the hedge asset, because a hedge would consist of a collection of related stocks in the same way that an industry is a collection of related stocks.

For this test, I simulated the global market portfolio with global stocks + bonds. I simulated a factor portfolio using concentrated long-only value and momentum factor portfolios plus a smaller short market position plus actual net historical performance of the Chesapeake managed futures strategy (which I used because it provides a longer track record than any other real-world managed futures fund that I’m aware of).³³ This loosely resembles how I invest my own money. Alternatively, I could have constructed a factor portfolio that combines multiple long/short factors plus market beta, which would more closely resemble academic factor definitions. That would probably scale better for extremely wealthy investors, and would give lower correlations but with higher expenses/transaction costs. AQR is one firm that offers diversified factor strategies that are designed to scale well for large investors.

(If we just used the historical returns of factor strategies to estimate future performance, we’d probably tend to overestimate future returns; but for right now we care about correlation, not return, so that doesn’t really matter.)

I calculated the correlations between the simulated MVO asset and each individual industry. Then I looked at the mean, median, and market-cap-weighted mean of the correlations, rounded to the nearest tenth. The mean, median, and weighted mean were all close enough together that they round to the same number. Using the global market portfolio (GMP) as the MVO asset produced an average correlation of 0.6, and using the factor portfolio (FP) gave an average of 0.5.

Correlation of MVO asset <> hedge asset: 0.6 (GMP) or 0.5 (FP)

See previous section.

Correlation of legacy asset <> hedge asset: 0.7

The total stock market has an average correlation of a little under 0.7 to any individual industry. The correlation between the legacy asset and the hedge asset is probably about the same—maybe a bit higher, since EAs tend to overweight tech/AI stocks.

Correlation of MVO asset <> AI progress: 0.0 (GMP) or –0.1 (FP)

As show above, I looked at the correlation between the S&P 500 and ML benchmark performance, and found a correlation of about 0.2 (depending on what benchmark I use). The global market portfolio has a somewhat lower correlation, at 0.0 (it’s slightly positive but it rounds off to 0). A factor portfolio has a correlation of roughly –0.1. (Remember that a higher correlation with AI progress is better.)

Caveat: These correlations are unstable. I got these coefficients using the largest date range available (2008–2021), but if I change the date range, I can get substantially different numbers. (For example, if we look at 2012–2021 instead, the S&P 500 now has a negative correlation to AI progress.) So consider these estimates not particularly reliable. But they should be at least directionally correct—a hedge has a higher correlation than the global market portfolio, which has a higher correlation than a factor portfolio.

Correlation of legacy asset <> AI progress: 0.2

The legacy asset has a slightly higher correlation to AI progress than the S&P 500 does because the legacy asset is tilted toward AI or AI-adjacent stocks. I don’t have the data necessary to empirically estimate the correlation, but I would guess that it’s about 0.2. (And, as I said in the previous section, and empirical estimate wouldn’t be very reliable anyway.)

Impact investing factor: ignored

Discussed above.

In this essay, I simply assume that investing in a company’s stock has no direct effect on the company. That said, a philanthropist who’s leaning toward mission hedging should first carefully consider the effect of impact investing.

Target volatility: 30%

Before we ask what the volatility target should be, first we should ask: do we even want a volatility target?

Many investors behave as if they’re leverage-constrained but not volatility-constrained—they invest in extremely risky assets such as cryptocurrencies or individual stocks, but not in (relatively) stable leveraged investments such as risk parity.

It’s true that some investors can’t use leverage; and some investors can use up to 2:1 leverage with Reg T margin, but not more than that (portfolio margin allows greater than 2:1 leverage). However:

Wealthier investors can get access to portfolio margin fairly easily, and these investors represent the majority of EA wealth.
Investors who cannot use leverage themselves (or only limited leverage) can still invest in funds that use leverage internally.

So we should not treat leverage as a constraint. That said, if we are leverage-constrained, that means we should invest in assets with particularly high volatility (as long as we’re compensated for that volatility with extra expected return). This argues in favor of mission hedging because a focused hedge portfolio will have higher volatility than the market. Given a leverage constraint, mission hedging looks better than ordinary investing given most reasonable parameter values.

Okay, given that we want a volatility target, what number should we shoot for?

I somewhat-arbitrarily chose a 30% standard deviation as the target. It’s a nice round number, and it’s roughly twice the standard deviation of the global stock market. And you can’t go too much higher than 30% without running a serious risk of going bust.

Historically, if you had invested in a well-diversified portfolio at 30% target volatility, you would have experienced at worst an ~85% drawdown during the Great Depression and a ~70% drawdown in the 2008 recession.

Below, I present a table of historical max drawdowns for a variety of assets and strategies, levered up to 30% volatility.³⁴

Asset Class	Sample Range	Leverage	Max DD	DD Range	Worst Month	Max Leverage
US Market	1927–2021	1.6	96.1%	1929–1932	-29.1%	-3.4
US HML	1927–2021	2.5	94.2%	2006–2020	-16.8%	-6
US SMB	1927–2021	2.7	94.6%	1983–1999	-14.0%	-7.1
US UMD	1927–2021	1.8	99.4%	1932–1939	-52.3%	-1.9
US Combo	1927–2021	4.2	83.2%	1937–1940	-8.7%	-11.5
US Market	1940–2021	2.0	81.1%	2000–2009	-22.6%	-4.4
US HML	1940–2021	3.1	96.9%	1983–2000	-16.8%	-6
US SMB	1940–2021	3.2	97.8%	2006–2020	-14.0%	-7.1
US UMD	1940–2021	2.3	93.3%	2002–2010	-34.4%	-2.9
US Combo	1940–2021	5.2	80.7%	2007–2011	-8.4%	-11.9
10Y Treasuries	1947–2018	4.2	95.4%	1950–1981	-7.9%	-12.7
Commodities	1878–2020	1.7	98.8%	1919–1933	-20.9%	-4.8
AQR TSMOM	1985–2020	2.4	56.3%	2016–2020	-10.6%	-9.4
Dev Market	1991–2021	1.8	80.1%	2007–2009	-21.0%	-4.8
Dev SMB	1991–2021	4.2	91.3%	1990–2020	-5.8%	-17.2
Dev HML	1991–2021	3.7	87.9%	2009–2020	-10.9%	-9.2
Dev UMD	1991–2021	2.5	79.1%	2009–2009	-22.5%	-4.4
Dev Combo	1991–2021	6.1	67.1%	2008–2009	-4.2%	-23.8
US Market	1871–1926	2.7	71.0%	1906–1907	-10.2%	-9.8
US HML	1871–1926	1.8	87.4%	1884–1891	-21.0%	-4.8
US UMD	1871–1926	2.3	94.9%	1890–1904	-14.6%	-6.8
US Combo	1871–1926	3.9	67.9%	1874–1877	-7.8%	-12.8

Column definitions:

“Leverage”: Amount of leverage required to hit 30% volatility.
“Max DD”: Maximum drawdown that this asset class would have experienced at the given level of leverage.
“DD Range”: Peak-to-trough date range of the maximum drawdown.
“Max Leverage”: Maximum amount of leverage that would not have experienced a 100% drawdown.

Data series definitions:

US = United States
Dev = developed markets, excluding the United States
Market = total stock market
HML = long/short value factor (“high minus low” book-to-market ratio)
SMB = long/short size factor (“small minus big”)
UMD = long/short momentum factor (“up minus down”)
Combo = volatility-weighted combination of Market, HML, and UMD
TSMOM = time series momentum (a.k.a. trendfollowing)

General notes:

These portfolios use a constant amount of leverage that produces an average 30% standard deviation over the whole period. This is a little unfair because we wouldn’t have known in advance how much leverage to use. At the same time, recent volatility can somewhat predict short-term future volatility, so we could adjust leverage as expectations change instead of using fixed leverage over the whole period.
Data on historical performance is pulled from the Ken French Data Library (for most data), Cowles Commission for Research in Economics (for 1871–1926 data), AQR data sets (for TSMOM and commodities), and Swinkels (2019), “Data: International government bond returns since 1947” (for 10Y Treasuries).
All are gross of management fees and transaction costs, except for AQR TSMOM which is net of estimated fees and costs.
I assume leverage costs the risk-free rate plus 1%, and short positions earn the risk-free rate.
Leverage is rebalanced monthly.

Cost of leverage: 2%

As of 2022-05-18, Interactive Brokers charges 1.83% for leverage on amounts over $100,000, and 1.58% on amounts over $1 million. 2% assumes rates will go up in the future.

(If rates go up, bond yields should also go up, which means their prices will decline in the short run and earn higher returns in the long run.³⁵)

Proportion of portfolio currently invested in MVO/legacy/hedge assets: 50% / 50% / 0%

I roughly estimate that:

50% of EA funding comes from money in a small number of legacy assets, mostly held by a few big funders.
50% comes out of other investments, or out of human capital (people donate as they earn, or take decreased salaries to do direct work), and almost nobody uses leverage.
Almost nobody mission hedges AI.

A couple of issues with these assumptions:

Even excluding legacy investments, the EA portfolio probably does not look much like the mean-variance optimal portfolio.
Even if almost nobody mission hedges AI, some EAs overweight AI stocks for other reasons.

So I think these numbers excessively favor mission hedging, but I chose them because they’re simple.

If we did want to model diversified-but-suboptimal investments, we could treat the legacy asset as including all suboptimal investments, and change the numbers accordingly. I will leave that as an exercise.

Appendix B: AI progress by model size growth

In this appendix, I explain how I approximated AI progress using the growth in the sizes of large ML models.

I looked at the growth rate in the size (measured by training FLOPS or number of parameters) of large ML models. Then I used known relationships between model size and performance to estimate the growth in performance.

I ended up not including this in the main essay because why should I try to approximate performance in terms of model size when I have a data set that lets me measure performance directly? But I’m including it in the appendix for the sake of completeness, and because it produced the highest volatility estimate of any approach I used.

I downloaded the data from Sevilla et al. (2022)³⁶, which gives compute (FLOPS) and number of parameters for a collection of published ML models, organized by date. I found the average compute/params per year and used that to get the growth rate for each year, and took the standard deviation of the growth rates. I only included ML models from 2012 to 2021 because earlier years had too few samples.

Model size isn’t directly what we care about. What we care about is model performance.

Kaplan et al. (2020)³⁷ found that neural language model loss scales according to a power law—specifically, it scales with compute^0.050 and parameters^0.076³⁸.

Hestness et al. (2017)³⁹ found that neural network performance scales with parameters according to a power law across many domains, and that most domains scale between 0.07 and 0.35.

The first paper is newer and covers seven orders of magnitude instead of only 2–3, but the second paper looks at multiple domains instead of just NLP.

I transformed the Sevilla et al. data according to power laws, and then calculated the standard deviation in predicted performance growth:

Method	$\sigma$
compute^0.050	7.4%
parameters^0.076	9.4%
parameters^0.350	51%

The first two numbers fall on the low end of the range that my other methods (industry revenue, benchmark progress, timeline forecasts) found. The third number—the upper end of the range found by Hestness et al. (2017)—gives a much larger number than any other method.

I tested how good mission hedging looks with 51% volatility. If we invest in the global market portfolio, mission hedging beats ordinary investing if we can get a moderate correlation between AI progress and the hedge asset (r=0.4 or greater). If we invest in a factor portfolio, we need a correlation of at least r=0.7. (Under the compound-until-AGI utility function, the hedge asset looks much worse than the MVO asset regardless of the correlation.)

A power law of parameters^0.35 gives a growth rate consistent with the observed performance growth across models in EFF’s ML benchmarks data set (34% and 32%, respectively). But it gives a much higher standard deviation of 51% compared to EFF’s 16%. I tend to believe that the high volatility in the Sevilla et al. data comes from a small sample size plus variation in ML model quality, so it does not reflect “true” volatility. (If we’re using model size as a proxy for performance, and a direct measure of performance disagrees, I trust the direct measure more.⁴⁰)

On the other hand, one could argue that ML model size predicts “true” AI progress, not just performance on certain benchmarks.

Appendix C: Optimal equilibrium allocation to mission hedging

So far, I’ve focused on the marginal utility of changing portfolio allocations. We might also want to know the optimal allocation in equilibrium—the allocation we’d want if we could reallocate the entire EA portfolio at once.

Previously, I wrote about mission hedging allocation under a simple utility function where marginal utility of wealth is linear with the mission target. Under this model, and using the range of parameter values I found for hedging AI:

The optimal allocation to the hedge asset varies from 5% to 24%.
The optimal allocation to the MVO asset varies from 296% to 538%.

Do not interpret these numbers as providing reasonable-in-practice upper and lower bounds. I arrived at these ranges by varying only three parameters:

volatility of AI progress from 5% to 20%
correlation of the hedge to AI progress from 0.25 to 0.6
coefficient of relative risk aversion from 1.1 to 2

I set the cost of leverage at 0% and put no restriction on maximum leverage. All other parameter values were taken as specified above.

The assumptions of 0% cost of leverage and log-normal price distributions cause this model to overestimate optimal leverage.⁴¹ Most other assumption cause this model to overestimate the optimal hedge allocation.

If we cap leverage at 2:1, then this model suggests that philanthropists in aggregate should allocate 200% of their portfolios to the MVO asset and none to the hedge asset, even under the parameter values that most strongly favor mission hedging.⁴²

Optimal allocation under “compound-until-AGI” utility

In this section, I provide optimal allocations under the “compound-until-AGI” utility function given above.

I will reference the same model parameters as in my previous essay without redefining them. See the previous essay for definitions. In this case, the mission target $b$ is the number of years until transformative AI. (Higher $b$ means slower AI progress.)

First, some notes on the general behavior of this utility function:

Unlike with the simple utility function, expected utility does not vary linearly with every parameter.
Optimal hedge allocation $\omega_h$ is concave with $\sigma_b$.
$\omega_h$ increases with $\alpha_m, \alpha_h, \alpha_b, \gamma$.
$\omega_h$ decreases with $\sigma_m, \sigma_h, \rho$.

I started with a set of reasonable parameter values:

For consistency with my previous essay, $\alpha_m = 8\%, \sigma_m = 18\%, \alpha_h = 0\%, \sigma_h = 18\%$.
$\gamma = 2, \rho = -0.6$ for RRA/correlations that relatively favor mission hedging.
$\alpha_b = 3.58, \sigma_b = 0.99$ because these are the parameter values given by Metaculus question 3479. These values correspond to a mean timeline of $\exp(3.58)$ = 36 years and a median timeline of $\exp(3.58 - \frac{1}{2} 0.99^2)$ = 22 years.

(The correlation $\rho$ is negative because if our hedge is positively correlated with AI progress, that means it’s negatively correlated with the AGI timeline.)

With these parameters, the optimal allocation is 124% MVO asset, 18% hedge asset, for a relative allocation of 13% to the hedge.

$\sigma_h = 0.99$ is very close to the value that maximizes the optimal allocation to the hedge asset, and the maximum optimal allocation is only slightly higher ($\sigma_h = 0.91$ gives 18.3%, versus 18.2% for $\sigma_h = 0.99$).

A more optimistic return projection of $\alpha_m = 10\%, \sigma_m = 13\%$ gives an optimal allocation of 291% MVO, 57% hedge, for a relative allocation of 16%.

Less mission-hedge-favorable parameters $\gamma = 1.1, \rho = -0.3$ give 164% MVO, 6% hedge, for a relative allocation of 4%.

But remember that my model includes many assumptions that favor mission hedging, so the true optimal allocation is probably lower.

Notes

This quote is paraphrased from Holden Karnofsky’s Important, actionable research questions for the most important century. ↩
An obvious but wrong question to ask is, “what is the optimal overall allocation to mission hedging?” The right question is, “given the current aggregate portfolio of value-aligned altruists, how should I allocate my next dollar?” (Or next million dollars, or however much.) For small marginal changes, the answer will be either 100% or 0% mission hedging. For discussion on the optimal ultimate allocation, see Appendix C. ↩
Specifically, the model computes the gradient of expected utility. We should allocate marginal dollars in whichever direction the gradient is largest. ↩
Realistically, I expect that the EA community will never collectively use enough leverage, so it will always be beneficial for investors on the margin to add leverage to their portfolios. ↩
Leveraged up to 30% volatility with a 2% cost of leverage, this implies a 12% return for the global market portfolio and a 19% return for a factor portfolio.

For the global market portfolio, rather than simply increasing leverage, I both increased leverage and increased the ratio of stocks to bonds. This is more efficient if we pay more than the risk-free rate for leverage. ↩
For this metaphor, it technically matters how big the urn is. If you’re drawing from a finite urn without replacement, then if you double the number of draws, the probability of drawing a particular idea doesn’t double, it less than doubles. But for a sufficiently large urn, we can approximate the probability as linear.

It could be interesting to extend this urn metaphor to make it more accurate, and use that to better approximate how utility varies with AI timelines. But a more accurate urn metaphor would make mission hedging look worse, and it already doesn’t look good, so I would prefer to prioritize projects that could reverse the conclusion of this essay. ↩
Technically, I did find some, but they were highly implausible—e.g., the hedge asset would need to have a 300% standard deviation. ↩
I used the Grace et al. timeline forecast with the compound-until-AGI utility function, but I did not use it with the standard utility function because I don’t have the necessary data to calculate the standard deviation of the implied rate of progress. ↩
It would have been nice to include up-to-date data, but I don’t have more recent data, and it’s still a large enough sample that the extra 7 years⁴³ wouldn’t make much difference. ↩
Revenue per share growth isn’t exactly the same as total revenue growth because revenue per share is sensitive to changes in the share count. But the numbers are pretty similar. ↩
We could also use negative log loss, which behaves superficially similarly to inverse loss. But inverse loss is nice because inverse loss varies geometrically, while negative log loss varies arithmetically. Our utility function combines investment returns with a measure of AI progress, and investment returns vary geometrically. Combining two geometric random variables is easier than combining a geomteric with an arithmetic random variable. ↩
If the data starts in 1984, that means 1985 is the earliest year for which we can calculate the growth rate. ↩
This question’s definition of AGI is probably overly narrow. I also looked at question 5121, which provides an operationalization with a higher bar for the definition of “AGI”. This question has a point-in-time standard deviation of 11.3% and a time series standard deviation of 1.6%. As of 2022-04-14, it has 315 total predictions across 124 forecasters.

In log space, question 5121 has a point-in-time standard deviation of 0.97 log-years. ↩
Katja Grace, John Salvatier, Allan Dafoe, Baobao Zhang & Owain Evans (2017). When Will AI Exceed Human Performance? Evidence from AI Experts

Read status: I read the paper top-to-bottom, but without carefully scrutinizing it. ↩
I looked at only a single survey to avoid issues with aggregating multiple surveys, e.g., combining conceptually distinct questions. I chose this survey in particular because Holden claims it’s the best of its kind. ↩
The arithmetic mean of the logarithm of a log-normally distributed sample is equal to the logarithm of the median, is equal to the logarithm of the geometric mean. ↩
If you use the first to second quartile range, you get a standard deviation of 1.20 log-years. If you use the second to third quartile range, you get 1.32 log-years.

I approximated these numbers by eyeballing Figure 2 on page 3. They might be off by one or two years. ↩
Actually it’s worse than that, because a mission hedge is much riskier than the market. ↩
Sebastien Betermier, Laurent E. Calvet & Evan Jo (2019). A Supply and Demand Approach to Capital Markets. ↩
There are a few exceptions for parameter values where changing the parameter had only a minor effect on the output. ↩
According to one particular operationalization, Metaculus puts a 69% probability on a slow takeoff. But people betting on the negative side will (probably) only win once transformative AI has already emerged, which may disincentivize them from betting their true beliefs.

Not that it matters what I think, but I’d put 70% probability on a fast takeoff. ↩
If your MVO asset is the global stock market, then it probably earns about the same return as a basket of AI stocks. A long-only factor portfolio could earn a higher return, which may or may not be enough to generate more expected utility than mission hedging. ↩
Recall that people might overweight AI stocks if they expect them to outperform the market, but this is not the same thing as mission hedging. ↩
We could construct a hedge with arbitrarily high correlation by buying something like an “AI progress swap”, but it would have negative alpha. ↩
To be clear, this is my credence that all the given parameter values in aggregate relatively favor mission hedging. ↩
Owen Cotton-Barratt. The Law of Logarithmic Returns. ↩
Linch Zhang. Comment on “Seeking feedback on new EA-aligned economics paper”.

Quoted in full:

The toy model I usually run with (note this is a mental model, I neither study academic finance nor do I spend basically any amount of time on modeling my own investments) is assuming that my altruistic investment is aiming to optimize for E(log(EA wealth)). Notably this means having an approximately linear preferences for the altruistic proportions of my own money, but suggests much more (relative) conservatism on investment for Good Ventures and FTX, or other would-be decabillionaires in EA. In addition, as previously noted, it would be good if I invested in things that aren’t heavily correlated with FB stock or crypto, assuming that I don’t have strong EMH-beating beliefs.

↩
An informal literature review by Gordon Irlam, Estimating the Coefficient of Relative Risk Aversion for Consumption[^39], found that estimates of the coefficient of relative risk aversion tended to vary from 1 to 4, with lower estimates coming from observations of behavior and higher estimates coming from direct surveys.

I trust observed behavior more than self-reported preferences, so I prefer to use a lower number for relative risk aversion. More importantly, individuals are risk averse in ways that philanthropists are not. For AI safety work specifically, we’d expect the rate of diminishing marginal utility to look similar to any other research field, and utility of spending on research tends to be logarithmic (corresponding to a risk aversion coefficient of 1).

We could instead derive the coefficient of relative risk aversion from first principles, such as by treating the utility function as following the “drawing ideas from an urn” model (which implies a coefficient between 1 and 2), or by using the approach developed by Owen Cotton-Barratt in Theory Behind Logarithmic Returns. ↩
Takeoff would still have to be “slow” in the Christiano/Yudkowsky sense. fast. But it could be fast in the sense that AI starts meaningfully accelerating economic growth sooner than expected. ↩
If $\mu$ is the geometric mean and $\sigma$ is the standard deviation, then the arithmetic mean approximately equals $\mu + \frac{1}{2} \sigma^2$. ↩
You can certainly still prioritize AI safety even if you don’t expect transformative AI to emerge for a long time. But it’s common for people in the AI safety community to have short timelines. ↩
Facebook specifically has a historical volatility of around 30%. ↩
I could have used AQR’s simulated time series momentum benchmark, which subtracts out estimated fees and costs. But the AQR benchmark performed much better, so I’m suspicious that their methodology overestimates true returns.

One might reasonably be concerned about survivorship bias in selecting a relatively old fund. To check for this, I compared the Chesapeake fund return to the Barclay CTA Index, and they have roughly comparable performance.

(You might ask, why didn’t I just use the Barclay CTA Index? Well, dear reader, that’s because Chesapeake provides monthly returns, but the Barclay CTA Index only publicly lists annual performance.)

That being said, the returns aren’t actually that important because we’re looking at correlations, not returns. ↩
On my computer, I have a folder filled with dozens of charts like this one. You probably don’t care about most of them. Honestly you probably don’t care about this one either, but it’s sufficiently relevant that I decided to include it. ↩
Technically, if yields go up, you should only expect to earn a higher total return if your time horizon is longer than the bond duration. ↩
Jaime Sevilla, Lennart Heim, Anson Ho, Tamay Besiroglu, Marius Hobbhahn & Pablo Villalobos (2022). Compute Trends Across Three Eras of Machine Learning.

Read status: I did not read this paper at all, but I did read the associated Alignment Forum post. ↩
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu & Dario Amodei (2020). Scaling Laws for Neural Language Models.

Read status: I skimmed to find the relevant figures. ↩
Technically, they used negative exponents because they were looking at how loss scales. I’m measuring performance in terms of inverse loss, so I’m using e.g. 0.05 instead of –0.05. ↩
Joel Hestness, Sharan Narang, Newsha Ardalani, Gregory Diamos, Heewoo Jun, Hassan Kianinejad, Md. Mostofa Ali Patwary, Yang Yang & Yanqi Zhou (2017). Deep Learning Is Predictable, Empirically.

Read status: I skimmed to find the relevant figures. ↩
And ML benchmark performance is only a proxy for AI progress, so model size is a proxy for a proxy. ↩
A theoretical model that assumes log-normal price distributions and continuous rebalancing overestimates optimal leverage, but only by a small margin. See Do Theoretical Models Accurately Predict Optimal Leverage? ↩
We can see this by calculating the gradient of expected utility at an allocation of (200% MVO asset, 0% hedge asset). The gradient is largest in the direction of the MVO asset.

The intuitive explanation for this result is that the model wants to use more than 200% leverage, so if we cap leverage at 200%, it wants to squeeze out as much expected return as it can, and mission hedging reduces expected return. ↩
I realize that 2013 was 9 years ago, but this type of data set usually has a lag time of about a year, so we’d only have full data through 2020. ↩

The Review Service Vortex of Death

Mon, 20 Jun 2022 00:00:00 -0700

Consumers want to know which products are good without having to buy them first. One way to do that is by reading reviews.

There are third-party services that provide product reviews. Unfortunately, almost all of them are useless because they inevitably fall into the Review Service Vortex of Death.

Review services have a fundamental incentives problem stemming from two facts:

Businesses don’t want to get bad reviews and they will pay a lot of money to have fake good reviews written, or to have bad reviews removed.
Consumers can’t tell when a review service is removing bad reviews.

Therefore, any review service, even if consumers pay to use it, is incentivized to accept businesses’ money to remove bad reviews, thus making the service useless for consumers. And it can get away with this behavior for a long time.

The biggest reviewers—including Better Business Bureau, Yelp, and Trustpilot—have all fallen into the Review Service Vortex of Death, and should not be trusted by consumers, but they continue to be used because it’s not common knowledge that they delete bad reviews in exchange for money. (And indeed, it’s hard to even prove that they do.)

What can consumers do about this? I don’t know. Businesses like Amazon, that make their money from retail sales, are less likely to fall into the Vortex, but they’re still vulnerable to businesses giving themselves fake reviews.

(I did write a review article one time—a review of donor-advised fund providers—and a couple of providers have subsequently emailed me to ask me to include them in my article. But sadly¹, they didn’t offer me any bribes.)

Notes

This is a joke. If a company offered me a bribe to include them, then I’d have to exclude them as a matter of principle. ↩

Index Funds That Vote Are Active, Not Passive

Wed, 20 Apr 2022 00:00:00 -0700

As an investor, you might invest in an index fund because you want to get unbiased, diversified exposure to the market. You don’t have to figure out which stocks will beat the market if you simply buy every stock.

But when you invest in an index fund, usually that means the fund can now vote on your behalf. The more stock an index fund owns, the more voting power it has. Generally speaking, the big index fund providers (including Vanguard and BlackRock) will vote in ways that align with their own corporate values—their top (stated) priorities are to increase climate change mitigation and workforce gender/racial diversity.

Regardless of whether you want this voting behavior, it means these index funds are not passive. By putting your money in an index fund that votes, you are implicitly claiming that it will make better voting decisions than the market.

(For that matter, any time you hold something other than the global market portfolio, you’re making an active bet. Sadly (and surprisingly), there aren’t any single index funds that offer the global market portfolio. But I digress.)

A Preliminary Model of Mission-Correlated Investing

Mon, 04 Apr 2022 00:00:00 -0700

Summary

TLDR: According to my preliminary model, the altruistic investing portfolio should ultimately allocate 5–20% on a risk-adjusted basis to mission-correlated investing. But for the current EA portfolio, it’s better on the margin to increase its risk-adjusted return than to introduce mission-correlated investments.

Last updated 2022-04-06.

The purpose of mission-correlated investing is to earn more money in worlds where your money matters more. For instance, if you’re working to prevent climate change, you could buy stock in oil companies. In worlds where oil companies are more successful and climate change gets worse, you make more money.

Previous work by Roth Tran (2019)¹ proved that, under certain weak assumptions, philanthropists should invest more in so-called “evil” companies than they would from a pure profit-making standpoint. This result follows from the assumption that a philanthropist’s actions become more cost-effective when the world gets worse along some dimension.

That’s an interesting result. But all it says is altruists should invest more than zero in mission hedging. How much more? Am I supposed to allocate 1% of my wealth to mission-correlated assets? 5%? 100%?

To answer this question, I extended the standard portfolio choice problem to allow for mission-correlated investing. This model makes the same assumptions as the standard problem—asset prices follow lognormal distributions, people experience constant relative risk aversion, etc.—plus the assumption that utility of money increases linearly with the quantity of the mission target, e.g., because the more CO2 there is in the atmosphere, the cheaper it is to extract.

I used this model to find some preliminary results. Future work should further explore the model setup and the relevant empirical questions, which I discuss further in the future work section.

Here are the answers the model gives, with my all-things-considered confidence in each:

Given no constraints, philanthropists should allocate somewhere between 2% and 40% to mission hedging on a risk-adjusted basis,² depending on what assumptions we make. Confidence: Somewhat likely. [More]
Given no constraints, and using my best-guess input parameters:
- Under this model, a philanthropist who wants to hedge a predictable outcome, such as CO2 emissions, should allocate ~5% (risk-adjusted) to mission hedging.
- Under this model, a philanthropist who wants to hedge a more volatile outcome, for example AI progress, should allocate ~20% to mission hedging on a risk-adjusted basis.
If you can’t use leverage, then you shouldn’t mission hedge unless mission hedging looks especially compelling. Confidence: Likely. [More]
If you currently invest most of your money in a legacy investment that you’d like to reduce your exposure to, then it’s more important on the margin to seek high expected return than to mission hedge. Confidence: Likely. [More]
The optimal allocation to mission hedging is proportional to: (Confidence: Likely)
1. the correlation between the hedge and the mission target being hedged;
2. the standard deviation of the mission target;
3. your degree of risk tolerance;
4. the inverse of the standard deviation of the hedge.

Cross-posted to the Effective Altruism Forum.

Summary
Contents
Setup
Results
How accurate is this model?
Future work
Acknowledgments
Appendices
Notes

Setup

The goal

We have some utility function that says how much we value the state of the world. Our utility function cares about two things: (1) our wealth and (2) some mission target. The mission target is a measure of an outcome we care about. For example, if we want to hedge climate change, the mission target is the level of CO2 in the atmosphere. If we’re hedging factory farming, the mission target could be the number of animals on factory farms.

Suppose there are two things we can invest in: an MVO (mean-variance optimal) asset and a hedge asset. (Note: usually MVO stands for “mean-variance optimization”, but I’m using it to mean “mean-variance optimal”.) Let the MVO asset be the investment that maximizes expected utility of wealth without regard to the mission target³. Let the hedge asset be an investment that’s correlated with the mission target and uncorrelated with the MVO asset—for example, a climate change hedge could consist of buying oil stocks + shorting the S&P 500. We choose some proportions of our wealth to invest in each of the MVO asset and the hedge asset. This total could add up to more than 100% (by using leverage) or it could be negative (by short selling).

Then we want to find the asset proportions that maximize expected utility.

As an example, say we can invest in two securities: XOM (Exxon Mobil) and SPY (S&P 500 index fund). The MVO asset might be {90% SPY, 10% XOM}, and the hedge asset might be {100% XOM, –100% SPY}. That means, by assumption, {90% SPY, 10% XOM} maximizes expected utility when holding the mission target constant.

If we invest 90% in the MVO asset and 10% in the hedge asset, then our allocation becomes

        0.9 (90% SPY + 10% XOM) + 0.1 (100% XOM – 100% SPY)
        = 71% SPY + 19% XOM

That is, 71% long SPY and 19% long XOM, with the remaining 10% in cash.

The hedge asset is a combination of positions that (by definition) has no correlation to the MVO asset. But we might never invest directly in the hedge asset. In the example above, even though the hedge includes both long and short positions, our final allocation doesn’t require short-selling. And in practice, you probably wouldn’t think about how to allocate between (90% SPY, 10% XOM) and (100% XOM, -100% SPY). You’d just think about how to allocate between SPY and XOM. But for the sake of analyzing the problem, it’s easier to think in terms of a mean-variance optimal asset vs. an uncorrelated hedge asset.

Importantly, mission hedging climate change doesn’t just mean we allocate more than 0% to oil stocks. It means we allocate more to oil stocks than a typical investor would. In this example, a normal investor allocates 10% to XOM, while a mission hedger allocates 19%.

Note A: Mission hedging is a special case of mission-correlated investing. It’s the most intuitive case, so I will focus on it in this essay, but the model I present could be adapted to apply to other types of mission-correlated investing. See Appendix A for how this could be done.

Note B: This model of mission hedging uses an extension of mean-variance optimization, but it’s not obvious that this is a good approach. See Appendix B for more on why I used this method.

The utility function

Following Roth Tran (2019), we want our utility function to meet the following criteria. Plain descriptions on the left, mathematical definitions on the right, with U = utility, W = wealth, b = mission target (which Roth Tran called a “bad activity”, although in my model, it’s not necessarily bad):

More wealth is better	$\displaystyle\frac{dU}{dW} > 0$
More of the mission target is worse	$\displaystyle\frac{dU}{db} < 0$
More of the mission target makes wealth more valuable	$\displaystyle\frac{d^2U}{dW db} > 0$

This describes a broad class of utility functions—so broad that it can’t give much practical guidance. Let’s pin down some more requirements:

Wealth has diminishing marginal utility	$\displaystyle\frac{d^2 U}{dW^2} < 0$
Sufficient wealth can eliminate almost all of the mission target, but never quite all of it	$\displaystyle\lim\limits_{W \rightarrow \infty} \displaystyle\frac{dU}{db} = 0$
We have constant relative risk aversion with respect to wealth	$-W \displaystyle\frac{d^2 U}{dW^2} / \frac{dU}{dW} = \gamma$

A constant relative risk aversion (CRRA) utility function looks like

\begin{align} U(W) = \displaystyle\frac{W^{1 - \gamma} - 1}{1 - \gamma} \end{align}

where $\gamma$ (that’s the Greek letter gamma) is a constant that’s greater than 1. For our utility function to account for the mission target, we need to ensure (a) utility decreases with the mission target and (b) wealth mitigates the mission target, up to a maximum of 100% mitigation.

So we can define

\begin{align} U(W, b) = f(b) \displaystyle\frac{W^{1 - \gamma} - 1}{1 - \gamma} - \frac{f(b)}{\gamma - 1} \end{align}

given some function $f(b)$ that defines how utility scales with the mission target.⁴

The $\frac{W^{1 - \gamma} - 1}{1 - \gamma}$ term says that the utility of wealth scales with $f(b)$. The $\frac{f(b)}{\gamma - 1}$ term says that total utility decreases with $f(b)$.

What is the shape of the badness function? As an example, consider climate change. If the atmosphere contains twice as much CO2, you can remove it approximately twice as quickly.⁵ In general, interventions to prevent a bad activity are plausibly twice as effective when there’s twice as much of the bad thing. So we can say $f(b) = b$.

Our utility function simplifies to

\begin{align} U(W, b) = b \displaystyle\frac{W^{1 - \gamma}}{1 - \gamma} \end{align}

For example, $\gamma = 1.5$ gives

\begin{align} U(W, b) = \displaystyle\frac{-2 b}{\sqrt{W}} \end{align}

If we wanted to, we could generalize this to $U(W, b) = \displaystyle\frac{b^\lambda W^{1 - \gamma}}{1 - \gamma}$ for some constant $\lambda$. That would allow utility to change non-linearly with $b$, which more accurately describes many real-world situations. For this essay, to keep things simple, I will stick with $\lambda = 1$.

These assumptions imply that we should mission hedge rather than mission leverage. But there are related assumptions that suggest we should mission leverage. For example, if we change the third condition to $\displaystyle\frac{d^2U}{dW db} < 0$, that means wealth becomes more valuable as the mission target increases, so we should mission leverage.

Why this utility function?

Most of the listed criteria make perfect sense—e.g., of course more wealth is better and more of the mission target is worse. It’s not always true that when the mission target gets worse, our spending becomes more cost-effective. But it’s true in some cases, and those are the cases where we’d want to consider mission hedging.

The weirdest thing about this utility function: it has an upper bound. Why is that? Couldn’t we produce arbitrarily high value for the world if we had sufficient wealth?

A bounded utility function makes sense if we think of ourselves as committed to minimizing the harm of a particular mission target. For instance, there’s an upper bound to how much climate change we can prevent, namely, all of it. If we mitigate 100% of climate change, that provides an upper bound on how much good we can do.

Note: This utility function implies that impact investing doesn’t work—the way you invest doesn’t directly change the world. That might be false, but I’ll treat it as true for now.

Some more assumptions

The philanthropist doesn’t care about anyone else’s investment portfolio.

In reality, philanthropists should value the investments of other like-minded people. Instead of maximizing the expected utility of their personal portfolio, they should consider the aggregate portfolio across all value-aligned investors. They should invest in whatever way best improves the aggregate portfolio on the margin.

However, it makes more sense to start by modeling what philanthropists should do in the aggregate, and then later use that to figure out what to do on the margin. This post focuses on that first step.

The risk-free rate is 0%.

A more sophisticated model should include a term for the risk-free rate, but it’s simpler to assume that the rate is 0%, and this simplification doesn’t change much.

All random variables are lognormally distributed.

It’s common to assume that asset prices follow lognormal distributions. I also assume that the quantity of mission target is lognormally distributed, which makes sense if it’s something that tends to grow exponentially (CO2 emissions, AI progress, and numbers of farmed animals probably all behave this way).

The MVO asset has an arithmetic mean return of 8% and a standard deviation of 18%.

I could spend all day talking about market expectations, but in short, I believe these numbers appropriately represent an aggressive but appropriately-diversified portfolio (e.g., a global equity index fund).

The optimal degree of mission hedging isn’t determined by the expected return or standard deviation alone, but by the ratio between the two.

(I wrote more about market expectations in a previous essay.)

The hedge asset has an arithmetic mean return of 0% and a standard deviation of 18%.

By assumption, the MVO asset is the optimal selfish portfolio, and the hedge asset has zero correlation with the MVO asset. That means the optimal selfish portfolio contains no allocation to the hedge, and therefore the hedge asset cannot have an expected return greater than 0%.

If expected return equals 0%, then you can add leverage or mix in cash to get whatever standard deviation you want. I set the standard deviation at 18% so it’s the same as the MVO asset.

The MVO asset has zero correlation to the mission target.

This might not actually be true, but it keeps things simple. To the extent that the MVO asset is positively correlated with the mission target, that makes investing in the MVO asset look more compelling. It does not reduce the optimal allocation to the hedge, but it does reduce the optimal relative allocation. For example, increasing the correlation might change the optimal allocation of (MVO, hedge) from (200%, 30%) to (210%, 30%).

Table of variables

The model uses the following variables. m, h, b follow lognormal distributions (and thus their logarithms follow normal distributions), and x can refer to any one of m, h, b.

m	MVO asset price
h	hedge asset price
b	mission target
W	wealth
$\alpha_x$	log(mean of x)
$\mu_x$	mean of log(x) = log(geometric mean of x)
$\sigma_x$	standard deviation of log(x)
$\sigma_{xy}$	covariance between log(x) and log(y)
$\rho$	correlation between log(h) and log(b)
$\gamma$	coefficient of risk aversion
$\omega_x$	proportion of wealth allocated to asset x

Results

We now have a model of the expected utility of mission hedging. What does this model say a philanthropist should do?

That depends on the inputs. This model has four free variables:

$\rho$: correlation between the hedge asset and the mission target
$\gamma$: risk aversion
$\alpha_b$: arithmetic mean growth rate of the mission target
$\sigma_b$: standard deviation of the growth rate of the mission target

The model outputs are $\omega_m$ and $\omega_h$: the optimal allocations to the MVO asset and the hedge asset, respectively.

Unconstrained results

In this section, I provide a formula for the solution, then I offer some concrete answers for particular numeric inputs.

General solution

The MVO asset’s optimal portfolio allocation $\omega_m$ is given by

\begin{align} \omega_m = \displaystyle\frac{\alpha_m}{\sigma_m^2 \gamma} \end{align}

where $\alpha_m$ and $\sigma_m$ are the arithmetic mean and standard deviation of the MVO asset, respectively.

(This is identical to the solution to Merton’s portfolio problem for an ordinary investor.)

And optimal hedge allocation $\omega_h$ equals

\begin{align} \omega_h = \displaystyle\frac{\sigma_{bh}}{\sigma_h^2 \gamma} \end{align}

where $\sigma_{bh}$ is the covariance of the mission target and the hedge, and $\sigma_h$ is the standard deviation of the hedge.

We could also write this as

\begin{align} \omega_h = \displaystyle\frac{\rho \sigma_b}{\sigma_h \gamma} \end{align}

where $\sigma_b$ is the standard deviation of the mission target.

That means the optimal allocation to the hedge asset is proportional to its correlation with the mission target, proportional to the mission target’s standard deviation, and inversely proportional to the standard deviation of the hedge and to the philanthropist’s degree of risk aversion.

(See Appendix C for proof.)

Some qualitative observations:

$\omega_m$ and $\omega_h$ are independent, except that they both depend on risk aversion $\gamma$.
$\omega_h$ increases with $\rho$ and $\sigma_b$, decreases with $\sigma_h$ and $\gamma$, and does not change with $\alpha_b$. This makes intuitive sense:
- We want to mission hedge more when our hedge is more effective (higher covariance with the mission target), including when the mission target is more volatile.
- We want to mission hedge less when the hedge is more volatile. We get more “bang for our buck” with each dollar we put in the hedge, so we don’t need as much.
- We want to mission hedge less when we’re more risk-averse, because we prefer a safer (less volatile) portfolio.
It’s perhaps not immediately obvious why $\omega_h$ varies with $\sigma_b$ but not $\alpha_b$. The explanation is that we’re trying to hedge against future worlds where the mission target is unexpectedly prevalent, which is more likely to happen when $\sigma_b$ is large.⁶ $\alpha_b$ doesn’t affect how the hedge covaries with the mission target.
The optimal relative allocation $\displaystyle\frac{\omega_h}{\omega_m + \omega_h}$ does not depend on risk aversion $\gamma$. In other words, risk aversion doesn’t change our relative preference for hedging vs. traditional investing.

Specific numeric results

When provided reasonable inputs, what output does this model give?

Within a reasonable range of inputs, the optimal relative allocation to the hedge asset $\displaystyle\frac{\omega_h}{\omega_m + \omega_h}$ falls between 1.7% and 29%. So, at least given the model assumptions described previously, we know that 50% is too much and 0.1% is not enough.

Specifically, I tested input values within these ranges:

$\rho$ (correlation) from 0.25 (weak hedge) to 0.9 (very strong hedge)
$\gamma$ (risk aversion) from 1.1 (approximately logarithmic) to 2 (substantially risk-averse)
$\sigma_b$ (mission target volatility) from 3% (highly stable thing, e.g., CO2 emissions⁷) to 20% (volatile or hard-to-predict thing, e.g., AI progress⁸)

Mission hedging looks most favorable when $\rho = 0.9, \sigma_b = 20\%$. With those parameters, the optimal portfolio allocates 71% to the MVO asset and 29% to the hedge in relative terms. The absolute allocations depend on $\gamma$. With $\gamma = 1.1$, the optimal allocation is 227% MVO, 91% hedge (giving 3.18:1 leverage). $\gamma = 2$ gives 124% MVO, 50% hedge.

(I do think it’s plausible that $\sigma_b > 20\%$ for some causes, so the optimal relative allocation to mission hedging could perhaps be higher than 29%. $\sigma_b = 30\%$ gives a relative allocation of up to 38%.)

Mission hedging looks least favorable when $\rho = 0.25, \sigma_b = 3\%$. In that case, the optimal relative allocation is 98.3% MVO, 1.7% hedge.

Mission hedging boosts expected utility by 0.01% on the low end and 0.3% on the high end.

It’s still an open question as to whether we should invest closer to 1.7% or closer to 29% in mission hedging. That depends on (a) the correlation between the hedge and the mission target and (b) the volatility of the mission target, which I will leave as questions for future research.

For bigger tables of results, see Appendix D.

Results with a leverage constraint

All the optimal portfolios so far have required us to use leverage—sometimes a lot of leverage. But many investors can’t, at least not easily. What happens if we constrain the solution to disallow leverage?

This changes the results in one key way: Unless we’re particularly risk-averse, we can’t take on as much risk as we want to. If we want to mission hedge, we have to give up some expected return. That doesn’t mean we should never mission hedge, but it does change the tradeoffs.

Specifically, it means we should allocate 0% to mission hedging unless (a) mission hedging looks particularly compelling (high volatility of the mission target, high correlation) or (b) we’re very risk-averse, to the point that we wouldn’t want to use leverage anyway (given the return/risk expectations I used, this happens when $\gamma > 3$). At $\sigma_b = 20\%, \gamma = 1.5$, we only want to mission hedge if $r \ge 0.88$, which is probably unattainable.

Results with a legacy investment

One common scenario: You have a lot of money in some position. It’s not well-diversified, but you still hold it for historical reasons. Maybe you don’t want to sell all of your legacy asset right away, but you can sell a little bit and invest the proceeds in something else. Should you prioritize mission hedging, or is it better to invest in the mean-variance optimal asset?

Under this model, you should prefer the MVO asset, even given parameters that heavily lean toward mission hedging.⁹ Even if we already invest half our portfolio in the MVO asset (and the other half in the legacy asset), it’s still (slightly) better on the margin to move money into MVO.¹⁰

This is a much stronger result than what we got from the unconstrained model—the input parameters don’t change the outcome except at the extreme high end.

How accurate is this model?

Getting results out of a model always requires making a lot of assumptions. And every time you make an assumption, that’s another opportunity for the model to diverge from reality.

My mission hedging framework includes a lot of parameters, which are easy to modify. It also builds in some less-easily-fixable properties, including:

This model treats investment returns as lognormal. In reality, investments are more likely to show big declines than a lognormal distribution would suggest. A more accurate model would invest more conservatively, but it’s not clear how this would affect mission hedging. Intuitively, I’d expect an improved model to relatively favor whichever asset has a smaller left skew.
This model uses a particular class of utility functions. These utility functions have certain nice features, for example, the optimal asset allocation doesn’t depend on your time horizon or on how much money you start with. A philanthropist’s real-life utility function probably doesn’t look like the one I used, but real-life utility functions are notoriously difficult to ascertain. (We could perhaps improve the utility function by further investigating how our ability to do good varies with the target we’re trying to hedge.)
This model treats leverage, short positions, and trading as free. But even for investors who can use leverage and shorts, they have to pay some extra cost for those.
This model ignores the direct effects of investing. Buying stock in a company might cause the company to become more successful. And philanthropists might want to prioritize impact investing over mission hedging or profit maximization.

The most robust output of this model is the result with a legacy investment: if you currently hold most of your money in a legacy investment, then it’s better on the margin to shift toward the mean-variance optimal portfolio than to shift toward mission hedging. This result held true regardless of how I changed the model parameters (within reason). However, this result still depends on the choice of utility function, and it’s possible to construct a utility function where it’s better on the margin to put funds toward mission hedging.

For an unconstrained portfolio, it’s unclear how much to mission hedge. This model can produce a wide range of numbers depending on input parameters. The choice of risk aversion has a fairly small impact, choice of correlation has a moderate impact, and the standard deviation of the mission target matters a lot.¹¹ This suggests that, if we want to know more about what to do, we should prioritize figuring out the mission target’s volatility.

Future work

Six ideas for projects that could help philanthropists decide how to mission hedge:

1. What changes would make mission hedging worthwhile on the margin?

The strongest conclusion of my model is that, on the margin, mission hedging is worse than improving the risk-adjusted return of an investment portfolio. This result held across a range of input parameters. What assumptions could break this result? Are those changes reasonable?

If no such changes exist, we can conclude that mission hedging isn’t worth doing in practice.

2. Mission hedging vs. impact investing.

Many philanthropists want to directly do good with their investments. But impact investing often has the exact opposite prescriptions as mission hedging. Does divestment do enough good to damper—or even negate—the benefits of mission hedging? Should we over-weight investments in bad things, or under-weight them? How do we decide?

Jonathan Harris, founder of the Total Portfolio Project, has done some relevant work¹² on this.

3. Find the specific assets that provide the strongest mission correlation.

What can we invest in to mission hedge various causes? Which ones will work best? Or, in cases where mission leverage makes more sense than mission hedging, which assets provide the most effective mission leverage? And how can we tell?

For some causes, like climate change (and maybe farm animal welfare), plenty of data exists, and it’s just a matter of putting it together. For others, there’s not much existing data, so we’d have to get more creative.

4. Optimization over real-world historical data.

Pick some mission target, then get historical data on the growth rate/volatility of that target and find an investment asset that hedged it well. What would the historically optimal portfolio allocation have been?

Using real-world data alleviates certain concerns with using a model. The model assumes asset prices follow lognormal distributions and maintain consistent correlations over time, but those things aren’t true. Optimizing over historical data would show us how asset prices really behave (or, at least, how they used to behave).

Getting this data shouldn’t be too hard to do for some causes. For climate change, we could look at historical annual CO2 emissions, market returns, and returns for some plausibly good hedges like oil company stocks or oil futures.

5. Shape of the utility function.

Does the marginal utility of wealth really increase linearly with the mission target? Maybe it increases with some power function of the mission target, or maybe the relationship is more complicated. What if the mission target is a good thing (see here for examples)?

My current thinking on this subject is pretty primitive, so I expect there’s room to come up with something a lot better.

6. Alternative approaches for evaluating mission-correlated investing.

Could we look at the value of mission-correlated investing in an entirely different way? What other methods might we use to decide how much of our portfolio (if any) to dedicate to it? Could we decide on a percentage allocation using a qualitative, rather than quantitative, approach?

Acknowledgments

Thank you to Jonathan Harris for providing feedback on drafts of this essay.

Appendices

Appendix A: Mission leveraging

Mission leveraging is the opposite of mission hedging. With mission hedging, we invest in a way that gives us more money when the world gets worse. With mission leveraging, we get more money when the world gets better, doubling down on our mission instead of hedging it.

I won’t discuss when to hedge vs. leverage in full generality. (See here for a discussion plus some real-world examples.) But when should we hedge vs. leverage when we’re working with the class of utility functions discussed in this essay?

With any utility function that matches my stated assumptions, we always prefer to mission hedge. But there are similar utility functions under which we’d prefer to leverage.

We can take a utility function with the same form as before:

\begin{align} U(W, b) = b \displaystyle\frac{W^{1 - \gamma}}{1 - \gamma} \end{align}

But instead of $\gamma > 1$, let $\gamma < 1$. This changes two of the six stated assumptions:

Utility is now bounded below by 0, and has no upper bound.
$b$ is now a good thing: utility is positive instead of negative, which means a larger $b$ increases utility rather than decreasing it.

Because $b$ is now a good thing, by investing in an asset that’s correlated with $b$, we are leveraging rather than hedging.

If we want $b$ to be a bad thing, we can write the utility function as $\displaystyle\frac{1}{b} \frac{W^{1 - \gamma}}{1 - \gamma}$. The solution with this new utility function becomes:

\begin{align} \omega_h = -\displaystyle\frac{\sigma_{bh}}{\sigma_h^2 \gamma} \end{align}

So now, rather than buying the hedge asset, we want to short it. And the higher its covariance with $b$, the larger our short position should be. In other words, we want to mission leverage.

In broad terms, when utility of wealth grows quickly ($\gamma < 1$), we should leverage. And when utility of wealth grows slowly ($\gamma > 1$), we should hedge.

The exact conditions are:

When $\frac{dU}{db}$ is negative, $b$ is a bad thing. When it’s positive, $b$ is a good thing.
When $\frac{d^2U}{db dW}$ has the same sign as $\frac{dU}{db}$, we want to leverage. When it has an opposite sign, we want to hedge.

Appendix B: Why use this approach?

The model I use in this post is an extension of Harry Markowitz’s mean-variance optimization model (see Markowitz (1952)¹³). Markowitz won a Nobel Prize for this model, and it has been explored extensively in academic finance. At the same time, few investing practitioners use it.

Mean-variance optimization finds the portfolio with the best risk-adjusted return (a.k.a. Sharpe ratio) when given a set of assets with known means, standard deviations, and correlations with each other. This approach has some serious downsides:

If you feed the model historical data for return/volatility/correlation, the model only tells you what portfolio would have been optimal historically. As William Bernstein put it: “If you can predict the optimizer inputs well enough to come close to the future [optimal portfolio], then you don’t need an optimizer in the first place.”¹⁴
The Sharpe-ratio-maximizing portfolio is only optimal if standard deviation fully captures what people mean by “risk” (spoiler: it doesn’t).
Often, the resulting portfolio requires using so much leverage that it’s impossible to invest in in practice.

Most practitioners prefer other portfolio construction approaches. A simple equal-weight strategy—where you divide your money evenly across every asset class—works pretty well.¹⁵¹⁶ However, a few firms (such as RHS Financial) explicitly use portfolio optimization, but instead of naively inputting historical data, they take a sophisticated approach to generate more reasonable model results.

So mean-variance optimization doesn’t produce good results (at least if applied naively). Why, then, am I extending this model to mission-correlated investing?

One reason is that it’s a good starting point—this model is well-understood, and so a simple extension of it is easy to reason about.

A second reason is that, just as recent research has found more practical variations on the mean-variance approach, so too might we find a modification of my model that fixes the biggest issues. (It might be as simple as taking a published extension to the Markowitz model and incorporating mission-correlated investing.)

But most importantly, it’s because I don’t see how a non-quantitative approach could work. With ordinary investing, you can simply invest in an equal-weighted portfolio and you won’t go too wrong. Or you can use popular rules of thumb like 60% stocks, 40% bonds. But there’s no fallback approach for allocating to mission-correlated investing. Equal-weighting would tell us to put 50% in mission hedging/leveraging, which seems excessive. And we don’t have any convenient rules of thumb, either. Without any strong guiding heuristics, I believe the best approach is to make a bunch of reasonable(ish) assumptions and use those to explicitly calculate the portfolio that maximizes expected utility.

Appendix C: Proof of analytic solution

Thanks to Gordon Irlam’s Lifetime Portfolio Selection: A Simple Derivation, which provides a simple proof of the asset allocation result from Merton (1969)¹⁷, and which I found indispensable in proving the result in this section.¹⁸

Recall that we defined

\begin{align} U(W, b) = b \displaystyle\frac{W^{1 - \gamma}}{1 - \gamma} \end{align}

For this proof, let’s generalize this to

\begin{align} U(W, b) = b^\lambda \displaystyle\frac{W^{1 - \gamma}}{1 - \gamma} \end{align}

where $\lambda$ determines the rate at which it becomes easier to affect the mission target as the mission target grows.

Wealth is determined by the proportional allocation to the MVO asset and to the hedge asset and the returns of each. Each variable follows a lognormal distribution.

A single-variable lognormal distribution is given by $e^{\mu + \sigma N(0,1)}$ where $N(0,1)$ is a standard normal distribution (with mean 0 and standard deviation 1).

$b$ is a lognormal distribution given by $\exp(\mu_b + \sigma_b N(0,1))$.

Our objective is to maximize expected utility with respect to allocations $\omega_m$ and $\omega_h$:

\begin{align} \arg \max\limits_{\omega_m, \omega_h} E\left[b^\lambda \displaystyle\frac{W^{1 - \gamma}}{1 - \gamma}\right] \end{align}

Expanding $b$ and $W$, we get

\begin{align} \arg \max\limits_{\omega_m, \omega_h} E\left[\exp(\lambda \mu_b + \lambda \sigma_b N_b(0,1)) (\exp(\omega_m \mu_m + \omega_m \sigma_m N_m(0,1) + \omega_h \mu_h + \omega_h \sigma_h N_h(0,1)))^{1 - \gamma}\right] \end{align}

where $N_m(0,1), N_h(0,1), N_b(0,1)$ are all standard normal distributions.

(ignoring the $1 - \gamma$ in the denominator, because scaling by a constant does not change the argmax)

When taking an expected value, we can separate out any independent variables—$E[X Y] = E[X] E[Y]$. By assumption, $m$ is independent of both $b$ and $h$. So we can separate this into two maximization problems:

\begin{align} \arg \max\limits_{\omega_m} E\left[\exp(\omega_m \mu_m + \omega_m \sigma_m N_m(0,1)) \right] \end{align}

\begin{align} \arg \max\limits_{\omega_h} E\left[\exp(\lambda \mu_b + \omega_h \mu_h (1 - \gamma) + \lambda \sigma_b N_b(0,1) + \omega_h \sigma_h (1 - \gamma) N_h(0,1)) \right] \end{align}

The first problem is simply the traditional portfolio optimization problem, which has the solution

\begin{align} \omega_m = \displaystyle\frac{\mu_m + \frac{1}{2} \sigma_m^2}{\sigma_m^2 \gamma} \end{align}

If we let $\alpha = \mu_m + \frac{1}{2} \sigma_m^2$, we can write this as $\displaystyle\frac{\alpha_m}{\sigma_m^2 \gamma}$. This is convenient in some cases because $e^\alpha$ is the arithmetic mean of a lognormal distribution.

The second maximization problem does not have a pre-existing solution (to my knowledge), so let’s solve it.

If $N_b(0,1)$ and $N_h(0,1)$ have correlation $\rho$, then the part inside the exponent is a sum of dependent normally distributed random variables. The mean of the sum is simply the sum of the means. The standard deviation of the sum equals

\begin{align} \sigma_{b + h} = \sqrt{\sigma_b^2 + (\omega_h \sigma_h)^2 + 2 \rho \sigma_b (\omega_h \sigma_h)} \end{align}

Note that $\rho \sigma_b \sigma_h$ is the covariance between $b$ and $h$, so we can replace this with $\sigma_{bh}$.

The expected value of a lognormal distribution parameterized by $\mu$ and $\sigma$ is $e^{\mu + \sigma^2/2}$. Maximizing this quantity is equivalent to maximizing its logarithm, so we want to find

\begin{align} \arg \max\limits_{\omega_h} \left[ \lambda \mu_b + \omega_h \mu_h (1 - \gamma) + \frac{1}{2} \lambda^2 \sigma_b^2 + \frac{1}{2} \omega_h^2 \sigma_h^2 (1 - \gamma)^2 + \omega_h \sigma_{bh} (1 - \gamma) \right] \end{align}

At this point, it is convenient to replace $\mu$ with $\alpha - \sigma^2/2$ (recall that $\alpha$ is the log of the arithmetic mean). In an efficient market, an uncorrelated asset such as a mission hedge has $\alpha = 0$, which allows us to ignore the $\alpha_h$ term if we want to.

Setting the derivative to 0,

\begin{align} \displaystyle\frac{dE(U)}{d\omega_h} = \alpha_h (1 - \gamma) - \omega_h \sigma_h^2(1 - \gamma) + \omega_h \sigma_h^2 (1 - \gamma)^2 + \lambda \sigma_{bh} (1 - \gamma) = 0 \end{align}

Solving for $\omega_h$ and simplifying gives

\begin{align} \omega_h = \displaystyle\frac{\lambda \sigma_{bh} + \alpha_h}{\sigma_h^2 \gamma} \end{align}

And naturally, for $\lambda = 1, \alpha_h = 0$, this further simplifies to

\begin{align} \omega_h = \displaystyle\frac{\sigma_{bh}}{\sigma_h^2 \gamma} \end{align}

Jonathan Harris wrote an alternative proof here, starting from the model he developed in A Framework for Investing in Altruism—which is more general than my model—and plugging in some numbers. His model provides an approximation, not an exact general solution, but the solution happens to be exact in this case.

Appendix D: Tables of optimization results

Results for $\sigma_b = 3\%$

$\rho$	$\gamma$	market	hedge	ratio
0.25	1.1	2.245	0.038	1.7%
	1.5	1.646	0.028	1.7%
	2.0	1.235	0.021	1.7%
0.5	1.1	2.245	0.076	3.3%
	1.5	1.646	0.056	3.3%
	2.0	1.235	0.042	3.3%
0.9	1.1	2.245	0.136	5.7%
	1.5	1.646	0.100	5.7%
	2.0	1.235	0.075	5.7%

Results for $\sigma_b = 20\%, \gamma = 1.5$ (note: optimal absolute market allocation for a given $\gamma is the same as for$\sigma_b = 3\%$$)

$\rho$	market	hedge	ratio
0.25	1.646	0.185	10.1%
0.5	1.646	0.370	18.4%
0.9	1.646	0.667	28.8%

Results for $\sigma_b = 30\%, \gamma = 1.5$

$\rho$	market	hedge	ratio
0.25	1.646	0.278	14.4%
0.5	1.646	0.556	25.2%
0.9	1.646	1.000	37.8%

Results for $\sigma_b = 20\%$, no leverage allowed ($\omega_m + \omega_h = 1$), computed by numeric approximation:

$\rho$	$\gamma$	market	hedge
0.25	1.1	1.497	-0.497
	1.5	1.230	-0.230
	2	1.048	-0.048
0.5	1.1	1.370	-0.370
	1.5	1.138	-0.138
	2	0.978	0.022
0.9	1.1	1.177	-0.177
	1.5	0.992	0.008
	2	0.869	0.131

Appendix E: Numeric approximation of the optimal solution

It is possible to numerically approximate the optimal allocation for any utility function. In this section, I describe how I did this.

Epistemic status: Before I started working on this problem, I didn’t understand most of the required math, and I picked it up as I went along. So there’s a reasonable chance that I’m making some mistakes.

We want to find the utility-maximizing asset allocation, where:

Utility is a two-dimensional function of wealth and the mission target.
There are three lognormally-distributed random variables: price of the MVO asset $m$, price of the hedge asset $h$, and quantity of the mission target $b$.

The three variables are parameterized by a length-3 mean vector $\mu$ and a covariance matrix $\Sigma$, which together describe a three-dimensional multivariate normal distribution. The random variables we care about—$m, h, b$—are defined as the exponentials of three normally-distributed random variables $x_1, x_2, x_3$, which together form a vector $x$. That is, $m = e^{x_1}, h = e^{x_2}, b = e^{x_3}$.

The probability density function for a three-variable multivariate normal distribution is

\begin{align} f(x) = \displaystyle\frac{\exp(-\frac{1}{2}(x - \mu)^T \Sigma^{-1} (x - \mu))}{\sqrt{(2\pi)^3 \det(\Sigma)}} \end{align}

where $\Sigma^{-1}$ is the inverse of the covariance matrix $\Sigma$ and $\det(\Sigma)$ is its determinant.

Transforming this normal density function into lognormal space according to the multivariate change of variables formula, letting $y = [m, h, b] = \exp(x)$, we have

\begin{align} f(y) = \displaystyle\frac{\exp(-\frac{1}{2}(\log(y) - \mu)^T \Sigma^{-1} (\log(y) - \mu))}{m h b \sqrt{(2 \pi)^3 \det{\Sigma}}} \end{align}

where $\frac{1}{m h b}$ is the determinant of the Jacobian matrix of $y$. (This Stack Exchange post gives an intuitive explanation of why the change-of-variables formula works this way.)

Let $\omega_m, \omega_h$ be the asset proportions in the MVO asset and the hedge asset, respectively (recall that these can sum to greater or less than 1, and they can be negative). Wealth is then calculated as $w = \exp(\omega_m m + \omega_h h)$. The mission target $b$ is simply the third value of the random vector $y$.

Expected utility is given by

\begin{align} E[U(W, b)] = \int_{-\infty}^\infty U(W, b) f(y) dy \end{align}

Our goal is to maximize this function.

To solve this problem, I wrote a program to compute expected utility using numerical integration and then perform gradient descent to find the optimal asset proportions.

My numerical integration uses Richardson’s extrapolation formula¹⁹ with 12 and 24 trapezoids per dimension (up to 24³ = 13,824 four-dimensional trapezoids per integral). According to my tests, this method is accurate to within 0.02% (e.g., if the true optimal allocation is 10%, this method will give an answer between 9.998% and 10.002%).

I define the trapezoid bounds by first generating evenly-spaced 3D bases for a three-variable i.i.d. standard normal distribution. Then I transform the bounds of these squares according to $g(x) = e^{\mu + \sigma x}$, where $\sigma$ is the vector of standard deviations for the three variables in $x$. Then I use these new bounds given by $g(x)$ as the base of each trapezoid. This transformation function assumes the three variables are independent, which they’re not, but I found that this method was accurate enough.

My source code is available here.

Notes

Brigitte Roth Tran (2019). Divest, Disregard, or Double Down? Philanthropic Endowment in Objectionable Firms. ↩
An allocation of (say) 20% on a risk-adjusted basis means that 20% of the risk of my portfolio comes from mission hedging investments, which is not the same as a 20% allocation. For example, if my normal investments have a 15% standard deviation and my mission hedging investments have a 30% standard deviation, then I should allocate 11% to mission hedging, because 11% * 30% / (11% * 30% + 89% * 15%) = 20%. ↩
Technically, “mean-variance optimal” means that if you apply leverage or mix in cash, this portfolio maximizes the arithmetic mean return for any given variance, or (equivalently) minimizes variance for any given mean return. In realistic conditions, you might not want to invest in the MVO portfolio (e.g., because you don’t want to use leverage), so I’m generalizing the term “MVO” to refer to the optimal portfolio from a self-interested standpoint, even if it’s not technically mean-variance optimal. ↩
If $f(b) = 1$, this is equivalent to a standard CRRA utility function (plus a constant). ↩
I spoke to an expert in carbon capture, who suggested that the cost of removing CO2 from the atmosphere scales roughly with the square root of atmospheric concentration. As shown by the formula in Appendix C, using a square root function instead of a linear function would halve the optimal allocation to mission hedging. But this expert also suggested that higher atmospheric CO2 would drive innovation on carbon removal, bringing the cost down, possibly resulting in a super-linear cost-effectiveness function. For simplicity, I will stick with a linear function for this essay. ↩
$U(W, b)$ is defined as a CRRA utility function of $W$ scaled linearly by $b$, and CRRA utility functions are invariant with scale. Changing $\alpha_b$ changes the scaling, but that doesn’t change the optimal allocation. ↩
Using World Bank data, I estimated that CO2 emissions growth has an annual volatility of 3%. ↩
I estimated the volatility of AI progress using the Electronic Frontier Foundation’s AI Progress Measurement database, which aggregates ML model scores across a variety of benchmarks. I found a standard deviation of growth rate of about 15%. Let’s bump this up to 20% to increase the width of the parameter range. ↩
$\sigma_b=20\%, r=0.9, \gamma=2$ gives
\[\nabla E[U(\text{MVO}, \text{legacy}, \text{hedge})]_{[0, 1, 0]} = [5.43, -19.9, 3.60]\]
The gradient is largest in the direction of MVO, which means we maximize expected utility by moving in that direction. ↩
$\sigma_b=20\%, r=0.9, \gamma=2$ gives
\[\nabla E[U(\text{MVO}, \text{legacy}, \text{hedge})]_{0.5, 0.5, 0} = [3.35, -6.67, 3.33]\]
↩
Technically, optimal allocation varies linearly with both correlation and standard deviation of the mission target. But I have greater uncertainty about the standard deviation, and it can cover a wider range of plausible values. ↩
Jonathan Harris (2021). Investing for Impact in General Equilibrium. Working paper. ↩
Harry Markowitz (1952). Portfolio Selection. ↩
William Bernstein (1998). The Intelligent Asset Allocator. Kindle location 1082. ↩
Victor DeMiguel, Lorenzo Garlappi & Raman Uppal (2007). Optimal Versus Naive Diversification: How Inefficient Is the 1/N Portfolio Strategy? ↩
Georg Ch. Pflug, Alois Pichler & David Wozabal (2011). The 1/N investment strategy is optimal under high model ambiguity. ↩
Robert Merton (1969). Lifetime Portfolio Selection Under Uncertainty: The Continuous-Time Case. ↩
I didn’t actually find the analytic solution by proving it. I originally wrote a program to numerically compute the optimal allocation. After messing with this program for a while, I discovered that the outputs appeared to change in predictable ways with the inputs, and I wrote down what I believed to be an analytic solution. Then, already having some confidence in the answer, I attempted to prove it. This provided some useful guidance—my original proof contained some mistakes, which I noticed because the result of my proof did not match the formula that I knew empirically to be correct. ↩
This site has a good explanation of how it works. In short, if $F_n(x)$ is the numeric integral of a function $f(x)$ using $n$ trapezoids per dimension, then the Richardson estimate is $F_n(x) + \frac{1}{3} (F_{2n}(x) - F_n(x))$. ↩

How I Estimate Future Investment Returns

Fri, 01 Apr 2022 00:00:00 -0700

To make informed investing decisions, I want to estimate the future expected return of my portfolio. Markets are unpredictable, and future returns will likely significantly deviate from estimates—AQR believes there’s a 50% chance that 10-year realized equity returns will differ from their predictions by more than 3% per year. Still, it’s helpful to come up with a median guess.

In this post, I explain the projections that I use for my own financial planning.

Last updated 2022-06-16.

Market returns

For market returns, I look at three institutional forecasts:

They each use somewhat different methodology:

AQR uses the dividend discount model, the most standard method of estimating future returns.
RAFI uses the discounted dividend model, but also assumes that valuations tend to revert to the mean (like the Bogle expected return formula).
Vanguard uses a complicated model that incorporates lots of economic factors.

(If I’d had to guess in advance which of these three firms uses the most complicated model, I definitely wouldn’t have guessed Vanguard.)

Historical evidence suggests that valuations tend to mean revert, but not reliably. My best-guess estimate would incorporate partial but not complete mean reversion, so I believe it makes sense to take an average of AQR’s and RAFI’s return estimates. I don’t really understand how Vanguard came up with its estimates, but it tends to give numbers in between AQR’s and RAFI’s.

A table of estimates for the 10-year real geometric returns of various asset classes:

Asset Class	AQR	RAFI	Vanguard
US equities	3.6%	-0.7%	1.3%
developed ex-US equities	4.3%	4.4%	4.2%
emerging equities	5.3%	7.4%	3.2%
US 10-year Treasuries	-0.8%	0.9%	-0.3%
commodities	-1.5%	1.4%	N/A

Estimated standard deviations:

Asset Class	RAFI	Vanguard
US equities	15.2%	16.7%
developed ex-US equities	17.2%	18.4%
emerging equities	20.9%	26.8%
US 10-year Treasuries	3.3%	4.7%
commodities	16.3%	N/A

Taking an approximate average of these, I assume a 2% real return for US equities, 4% for developed ex-US, and 5% for emerging markets.

I would also like to know the expected return of the global market portfolio.

Vanguard does not provide any such estimate.
RAFI forecasts the global market portfolio to have a 1.5% real return with a 9.2% standard deviation.
AQR forecasts global 60/40 to earn a 2.0% real return (no standard deviation given), and believes that this return can be increased to 3.0% at the same level of risk by adding a little leverage, increasing the weight to bonds and low-volatility equities, and mixing in commodities.

We could also look at historical performance. Meb Faber’s Global Asset Allocation found that from 1973 to 2013, the global market portfolio earned a real return of 5.4% with an 8.8% standard deviation.

I approximated the global market portfolio using data from The Rate of Return on Everything, 1870–2015¹. I found that from 1950 to 2015 (the time range over which every country has annual data), global 60/40² had a nominal return of 10.3% with a standard deviation of 14.5%.

Putting all these together, my best-guess forecast is a 3% real return with a 9% standard deviation.

Factor premia

I make investments in certain factors that historically predicted asset returns, including value, momentum, and trendfollowing.

AQR and RAFI provide projected returns for some factors. (See also RAFI methodology.)

AQR projects a 0.5% excess return over the benchmark for a market portfolio with a single factor tilt, and 1% excess return for a portfolio with multiple factor tilts.
AQR projects a Sharpe ratio of 0.7–0.8 for a long/short multi-factor portfolio.
RAFI projects a 4–5% excess return for a long-only value strategy and a 1.5–2% excess return for a long-only momentum strategy.
Vanguard projects a 1% excess return for a long-only value strategy, but doesn’t offer any other factor projections.

AQR numbers are net of fees and transaction costs. RAFI estimates are net of transaction costs, but don’t account for fees.

None of them provide as much detail as I’d like on how they came up with these numbers. I do think RAFI overestimates value and underestimates momentum because they believe value looks unusually undervalued right now and momentum looks unusually overvalued. But predicting factor performance based on valuation probably doesn’t work as well as they think it does. (For more on factor timing, see Contrarian Factor Timing Is Deceptively Difficult³ from AQR.)

And anyway, the way I invest doesn’t match how AQR and RAFI came up with their estimates. I invest in concentrated, equal-weighted factor portfolios, which should earn higher returns than the sorts of portfolios AQR and RAFI looked at. Historically, concentrated strategies had 3–4x larger premia than weakly-tilted portfolios. On the other hand, I believe their estimates (especially RAFI’s) are too optimistic. AQR estimates future factor premia by dividing historical premia in half, which I believe is appropriately conservative, but I’m concerned that they underestimate the costs and tail risk of a leveraged long/short portfolio.

I came up with my own projections for concentrated factor returns by following these steps:

Run a backtest to find the historical factor premium for a comparable portfolio to the one I invest in.
Subtract fees and expected transaction costs.
Divide the result by two, on the assumption that factors will only work half as well in the future.

Why divide historical factor returns in half?

We have good reason to expect factors to continue to work.
But they might work less well in the future, simply because investment strategies tend to get worse over time.
To keep it simple, just cut expected returns in half.

This ends up giving factor return estimates that are a little more conservative than AQR or RAFI.

I get factor exposure through the Alpha Architect ETFs, which I believe are the best on the market for investors like me. They provide backtests of their methodology to 1973. I did my own backtests to 1926 using the Ken French Data Library, approximating the methodology as closely as I could, and got similar but slightly worse results (which I believe is explained by a weaker methodology, see footnote⁴). According to these backtests, concentrated value and momentum indexes each had an 8% premium before costs. With a 0.5% management fee, and conservatively assuming 1.5% transaction costs,⁵ that gives a 6% premium. Then divide this in half to get a 3% expected future premium.

I use equity trendfollowing to reduce market exposure during downtrends, like what the VMOT ETF does. Historical evidence suggests that this does not affect expected return, but it reduces equity volatility from ~16% to ~13%. (This understates the value of trendfollowing because trendfollowing tends to change an investment’s skewness from negative to positive.) See Faber (2013)⁶ for a review of trendfollowing across asset classes. I also performed my own backtests over 80+ years of equity, bond, and commodity data from various sources and got similar results.

In addition, I invest in long/short trendfollowing over commodities and fixed income, similar to what KMLM does. (I don’t invest in KMLM; I have an SMA with Alpha Architect where they directly run managed futures, which is more tax- and leverage-efficient.⁷ But if I weren’t doing that, I’d invest in KMLM.) The best data on long/short trendfollowing performance comes from Hurst et al. (2014)⁸, which found a 100-year historical return of 11.2% net of 2-and-20 fees with a 9.7% standard deviation. I’m reluctant to take even half this return as a future expectation because it just seems implausibly high—I expect trendfollowing to work, but not that well.⁹

The Barclay CTA Index, an index of commodity trading advisors who mostly use trendfollowing strategies, had an 8.8% return with 16.5% standard deviation from 1980 to 2021.

In my projections, I assume an aggressive long/short trendfollowing strategy will earn 4% real with a 15% standard deviation, which is about half the historical performance of the Barclay CTA Index.¹⁰

(As an aside, trendfollowing strategies perform much better than equities on risk measures that account for the severity of drawdowns, such as the Ulcer Index. So a trendfollowing fund with a 15% standard deviation feels much more pleasant to invest in than an equity fund with the same volatility.)

For volatility projections, there’s no strong reason to expect standard deviation to change over time—it might go up or down, but neither direction looks more likely than the other. So I’ll simply assume that historical volatility continues.

In summary, I expect VMOT to earn a 6% real return with a 13% standard deviation, for a Sharpe ratio of 0.6.¹¹ I expect VMOT + bond/commodity trendfollowing to do somewhat better than this—maybe 5% real with 11% standard deviation.¹² Remember that, even if this projection is exactly correct ex ante (which it isn’t), the true number will probably be significantly higher or lower.

Changelog

2022-05-31: Change projected trendfollowing return to incorporate Barclay CTA Index.

2022-06-16: Minor wording improvement.

Notes

Òscar Jordà, Katharina Knoll, Dmitry Kuvshinov, Moritz Schularick & Alan M Taylor (2019). The Rate of Return on Everything, 1870–2015 ↩
60/40 is supposed to be market cap weighted, but I weighted by GDP instead because the data set doesn’t include market cap. ↩
Cliff Asness, Swati Chandra, Antti Ilmanen & Ronen Israel (2017). Contrarian Factor Timing Is Deceptively Difficult. ↩
The Alpha Architect value and momentum ETFs mainly focus on the value and momentum factors (as the names suggest), but they also tilt toward the quality and low-volatility factors, which also show robust predictive power, although not as much as value or momentum. Those additional tilts should increase the excess risk-adjusted return. But the Ken French Data Library does not have the data I’d need to test those tilts. ↩
This is comparable to AQR’s estimated transaction costs based on their own live training data,¹³ but (1) AQR (during the sample period) managed about 100x more money than Alpha Architect does and (2) AQR’s strategies rebalanced monthly, and the Alpha Architect funds only rebalance every 3–6 months.

AQR found a realized cost of 0.20% per individual trade. At a typical turnover of 50% per 6 months for value or 50% per 3 months for momentum, that implies an annual trading cost of 0.40% for value and 0.80% for momentum. A significantly smaller firm could probably achieve lower trading costs. ↩
Meb Faber (2013). A Quantitative Approach to Tactical Asset Allocation. ↩
You can get something like 10:1 leverage for cheap by buying futures collateralized by your equity holdings (although I don’t have anywhere close to 10:1 leverage). To leverage an ETF, you have to use margin, which is a bit more expensive, and also wasteful because managed futures ETFs hold a lot of cash on their balance sheets, so you’re mostly just paying to get leverage on cash. ↩
Brian Hurst, Yao Hua Ooi & Lasse H. Pedersen (2014). A Century of Evidence on Trend-Following Investing. ↩
I compared AQR’s trendfollowing index to the performance of one of AQR’s actual trendfollowing funds, AQMIX. The index performed better by about 3 percentage points per year from 2010 to 2019. I’m not sure why—the performance difference varies a lot from year to year, which suggests that it’s not (entirely) due to transaction costs (or else we’d see consistent underperformance by a ~fixed amount). ↩
I started using the 4% return / 15% volatility projection before I looked up the historical returns of the Barclay CTA Index, so it’s somewhat coincidental that that’s about half the historical return of the index. I originally came up with 4% / 15% by taking the AQR index and applying some heavy discounts. ↩
The Sharpe ratio is the return in excess of cash divided by standard deviation. Right now, the risk-free rate is lower than inflation, so the excess return is greater than the real return by 1 to 2 percentage points. ↩
A backtest of this strategy, using Ken French factor data and AQR trendfollowing data, found a real return of 10% with an 11% standard deviation net of estimated fees and trading costs. ↩
Andrea Frazzini, Ronen Israel & Tobias J. Moskowitz (2014). Trading Costs of Asset Pricing Anomalies. ↩

Can Good Writing Be Taught?

Tue, 22 Mar 2022 00:00:00 -0700

Epistemic status: Highly speculative; unburdened by any meaningful supporting evidence.

I’ve written something like 200 essays for school. Writing those essays did not teach me how to write. Writing for fun taught me how to write.

When I was in high school, I used to complain that the essays I was required to write were both boring and unhelpful, and I’d learn more by writing essays about whatever I wanted. But if my teachers had let students write whatever they wanted, I don’t think most of them would have gotten very far. I don’t think I would have gotten very far, either. There’s a big difference between

me: I have an idea! I will write about it!

versus

teacher: Please have an idea and write about it.

me: What should I write about? I dunno, I guess I could write about X, I can probably force myself to come up with something to say about it.

Instead of writing something detached from ordinary life, like literary analysis, should high schoolers be taught to write something relevant, like emails?

In fact, I was taught how to write emails in high school (although that was only a small % of what we did), and the teaching was counterproductive. The way my teachers taught me to write emails was significantly wrong, and probably would have hindered my career if I had listened. (As a basic example, they said to always start an email with “Dear [name]”. Nobody starts emails that way in real life.) All the people with jobs who write emails somehow managed to un-learn the anti-lessons that they were taught.

But even if my teachers had taught me how to write emails correctly, it wouldn’t have mattered. If I have to slog through a purposeless assignment that I don’t care about, anything I learn from it doesn’t stick. I only learn from doing things if I’m doing them for a reason.

In conclusion, it’s impossible to force someone to learn good writing. They have to want to write.

Existential Risk Reduction Is Naive (And That's a Good Thing)

Fri, 18 Mar 2022 00:00:00 -0700

I see many people criticize existential risk reduction as naive or arrogant. “What, you think you can save the world?”

I’m not going to dispute this. Yes, it’s naive and arrogant, and that’s a good thing.

There are countless movies about saving the world. Lots of people fantasize about saving the world (or, at least, my friends and I did when we were kids, and I still do). Ask any five-year-old child, and they can tell you that saving the world is awesome. But it takes a particularly subtle and clever mind to understand that actually, trying to save the world is a silly waste of time.

But actually, the five-year old was correct all along. Saving the world is, in fact, awesome! We should do it!

The mature, adult response is that you can’t save the world, and you should be content with contributing to society in your own small way. I could make some clever argument about scope sensitivity or universalist morality or something, but I don’t need to. You already know that saving the world is awesome. Everybody knows it, they’ve just forgotten.

Climate change is the only mainstream cause that at least has a plausible case for saving the world. And indeed some climate change activists think in those terms. Even though I believe it’s unlikely that mitigating climate change can save the world, it’s still admirable to try. I would like to see more people try. Ask yourself: What could destroy the world, and how do we stop that from happening?

Altruistic Investors Care About Other Altruists' Portfolios

Fri, 18 Mar 2022 00:00:00 -0700

Confidence: Highly likely.

In some sense, altruists and traditional investors have the same investing goals—they want to own the portfolio with the best balance of return and risk. But self-interested people only care about their own portfolios. If you’re a philanthropist, you also care about other (value-aligned) philanthropists’ portfolios.

When the market goes up, you have more money, and you can donate more to charity. But other altruists also have more money, and they can donate more to charity, so your money isn’t as valuable. Conversely, when markets go down, you have less money to donate at the exact time when charities need funding the most.

That means you should not (necessarily) invest your money in the best overall portfolio. Instead, you should use your investments to move the pool of altruistic money in the direction of optimal.

An illustration:

Alice and Bob both donate to the Against Malaria Foundation (AMF). (For simplicity, let’s say they’re the only two donors.) AMF has diminishing marginal utility of money—once it distributes malaria nets in all the best places, the next round of nets won’t save quite as many lives. So Alice and Bob prefer to invest in a way that will earn good returns but without too much risk. Ideally, they’d both hold something like the total world stock market.

Bob lives in the United States, and he invests all his money in US stocks. Alice could simply buy the global stock portfolio, which is roughly 50% US stocks and 50% international stocks. But that would put their aggregate portfolio at 75% US stocks, 25% international stocks (assuming Alice and Bob have the same amount of money). So AMF is being funded by an investment portfolio that’s overweighted toward US stocks, which adds risk without any reward to compensate.

Alice can fix this by investing her entire portfolio in non-US stocks. Now the aggregate portfolio of AMF donors is 50% US stocks, 50% international stocks, just as it should be. It wouldn’t make sense for someone to hold 0% US stocks in their personal retirement portfolio, but this strategy works for Alice because she’s a philanthropist.

(Of course, Alice could also talk to Bob and persuade him to diversify his investments, which might be an even better idea!)

Now, I’m not trying to say that the global stock market is the best investment, or that Alice did the exact right thing in this scenario. This is just an illustration of a broader point: for an individual altruistic donor, the best investment portfolio on the margin might not be the same thing as the best overall portfolio. And altruists should pick the portfolio that’s best on the margin.

(I wrote this post to provide an easy reference for this concept. The concept is not original to me—I originally heard it from Paul Christiano’s Risk aversion and investment (for altruists).)

Should Earners-to-Give Work at Startups Instead of Big Companies?

Fri, 12 Nov 2021 00:00:00 -0800

Summary

Confidence: Likely.

Cross-posted to the Effective Altruism Forum.

Effective altruist earners-to-give might be able to donate more money if, instead of working at big companies for high salaries, they work at startups and get paid in equity. Startups are riskier than big companies, but EAs care less about risk than most people.

Working at a startup is easier than starting one. It doesn’t pay as well, but based on my research, it looks like EA startup employees can earn more than big company employees in expectation.

Does the optimal EA investment portfolio include a significant allocation to startups? To answer that question, I estimated the expected return and risk of startups by adding up the following considerations:

Find a baseline of startup performance by looking at historical data on VC firm returns.
VC performance is somewhat persistent. EAs can beat the average by working at startups that the top VC firms invest in.
Startup employees get worse equity terms than VCs, but they also don’t have to pay management fees, and they get meta-options. Overall, employees come out looking better than VCs.
Current market conditions suggest that future performance will be worse than past performance.
Startups are much riskier than publicly-traded stocks, and the startup market is moderately correlated with stocks (r=0.7).

All things considered, my best guess is that more earners-to-give should consider working at startups.

Summary
Contents
Framing the problem
The right question
Modeling the solution
What factors determine the expected return of startups?
- Returns for VC firms
- Returns for employees
- Forecasting future returns
- Giving now vs. later
Putting together the expected return
Risk and correlation of startups
Leverage
Startups vs. public equities
Startups vs. an optimized public investment portfolio
Alternative: Predictionless approach
Alternative: Models are bad. What if we don’t use a model?
Practical details
Conclusion
Areas for further research
Acknowledgements
Changelog
Appendix A: Startups for founders and investors
Appendix B: Some important tangents
Notes

Framing the problem

Suppose you’re an effective altruist and you want to donate as much money as possible. Perhaps you’ve heard the arguments that EAs should start startups. Starting a startup is a lot of work and requires special skills, so you’d rather not. Maybe you’d like to invest in venture capital, but the really good VC firms won’t accept your money. However, you wouldn’t mind working at a startup. You could also work at a big company that pays a high salary. Which should you choose?

This is how I would think about the problem:

Say a startup offers you an equity package that’s worth $X per year at the current valuation. At the same time, a big company offers you a salary that’s $X higher than your salary would be at the startup. Both compensation packages have the same face value.

If you work at the startup, you get $X per year of equity. Some number of years later, the startup might go public or get acquired, at which point you can sell your equity for $Y. Over that time, your equity earned a return of Y/X.

If you work at the big company, you could invest your extra $X salary in the stock market. Will that investment earn a higher or lower return than the startup equity? Whichever you expect to earn a higher return is the one you should pick. (Well, that’s not really true. Read the next section to find out why not.)

The right question

Which do we expect to earn a higher return, startups or the public stock market?

We don’t have good data on the historical performance of startup employees. We do have data on VC returns, so we can use that.

“Have VCs historically outperformed the public market?” is the wrong question, because some VCs consistently outperform the average.

“Have top VCs historically outperformed the public market?” is the wrong question, because future expected performance probably isn’t the same as past performance.

“Can we expect top VCs to outperform the public market?” is the wrong question, because startup employees don’t earn the same returns as VCs. Employee equity has a worse liquidation preference than VC equity, but investors in VC firms have to pay fund fees, which employees don’t. And employees have the meta-option to keep vesting when their company does well, or to quit when it does poorly. We can use these considerations to estimate the value of employee equity compared to VC equity.

“Can we expect startup employees to outperform the public market?” is the wrong question, because we need to consider leverage. Investors in public markets can use leverage to increase their risk and expected return, but startup employees can’t.

“Can we expect startup employees to outperform the leveraged public market?” is the wrong question, because an effective altruist’s goal isn’t to maximize their own portfolio return, it’s to maximize the expected utility of the overall EA portfolio. If no other EAs work at the startup where you choose to work, then you’re adding better diversification than if you invest in the public market.

“Do startup employees contribute more expected utility to the EA portfolio than if they invested in the leveraged public market?” is the wrong question, because if they did work at a big company and invest their salary, they might be able to invest in something better than the broad market. For example, I have previously discussed investing in concentrated value/momentum/trend portfolios, and I made a rough attempt to calculate the expected utility of doing so. For us to prefer to become startup employees, startups would have to look better than the best possible public investment (whether that’s value/momentum/trend or something else).

“Do startup employees contribute more expected utility to the EA portfolio than if they invested in the optimal set of public investments?” is more or less the right question.

Modeling the solution

Do startup employees contribute more expected utility to the EA portfolio than if they invested in the optimal set of public investments?

The answer to this depends on two things:

How do we model the answer?
What values should we use for the model inputs?

#2 is hard. #1 is sort of hard, but luckily, it’s already a solved problem. I described an applicable model in Asset Allocation and Leverage for Altruists with Constraints. In short, we set up a mean-variance optimization problem where we assume 99% of the capital is controlled by other people, and we can decide how to allocate the remaining 1%. Suppose we can allocate between three investment choices:

A typical investment portfolio, such as a stock market index fund
The optimal (ex ante) public investment portfolio
Startups

The simplest method is to assume we should put all our money into just one choice. What is the overall expected utility if we put our money into choice 1, choice 2, or choice 3?

If we tell our model the expected returns, standard deviations, correlations between these three portfolios, and a utility function, then the model will spit out the expected utility of each choice.

Let’s say the average EA investment portfolio equals the global equity market, which is at least sort of correct. What’s the expected return and standard deviation of global equities?

We have no idea how equities will perform in the short run. But in the long run, the market’s return is somewhat predictable. And over long time horizons, volatility stays pretty consistent, so we can simply assume the future standard deviation of global equities equals the historical standard deviation.

Similarly, we can approximate the future standard deviation of startups, and their correlation with global equities, by taking the historical standard deviation and correlation and assuming they will stay the same.

The most difficult input variable is the expected return of startups.

What factors determine the expected return of startups?

How to estimate the expected return of startups for employees:

Start with some index of VC returns, such as the Cambridge Associates Venture Capital Index.
These indexes usually provide returns net of fees. VC investors have to pay fees, but startup employees don’t. Add 2-and-20 fees back in to get the gross return.
Unlike with public market investors, VC firms that beat the market in the past tend to continue to beat the market. Employees can choose to work at startups with funding from top VCs. Add a premium to the expected return to account for this.
Maybe EAs can pick startups better than top VC firms. Possibly add a premium.
Employees get worse equity terms than VCs, so subtract some discount to account for this.
Startup employees get meta-options, which VCs don’t get. Add an appropriate premium.
Use the current market environment to forecast how future returns will look compared to past returns. (This is the sketchiest step, so we might skip this and just assume future returns equal historical returns. But I think we don’t want to skip this because we don’t even really know what historical returns were—more on that later.)
Giving now might be better than giving later. If so, that means we shouldn’t compare startups to public investments because public investments aren’t the best thing to do with money. Instead, we should compare startup equity to money donated now.

In the next four subsections, let’s break down each of these steps.

Returns for VC firms covers steps 1–3;
Returns for EAs covers steps 4–6;
Forecasting future returns covers step 7; and
Giving now vs. later covers step 8.

Returns for VC firms

I used Cambridge Associates’ Venture Capital Index 2020 report to find the aggregate historical return of VC firms from 1995 to 2018¹. According to this data set, VCs had a geometric mean return of 13.1% with a standard deviation of 18.9%. That’s a good starting point.

As the saying goes, a person with one clock always knows what time it is. Someone with two clocks is never quite sure. Cambridge Associates’ 2017 report has data from 1988 to 2016, which gives a geometric mean return of 14.9% and a standard deviation of 17.2%.²

According to these data sets, VC experienced several regimes:

Strong performance 1988–2000, and especially 1998–2000 at the peak of the tech bubble.
Very bad returns 2001–2003.
Mediocre returns 2004–2013, with a mix of good years and bad years.
Strong returns 2014–2020.

The “historical performance of VC” substantially changes depending on which time period you look at. And we don’t know what sort of regime will come next.

Another problem: Other VC return databases give entirely different numbers. For example, I could have used the VentureXpert database, which some (e.g., Koch (2014)³) claim is more accurate. (I used Cambridge Associates purely out of convenience.) Cambridge Associates tends to give higher VC returns than other databases (by a couple percentage points).⁴

I will use the 2020 Cambridge Associates report as a starting point. Just be aware that the startup market will likely behave very differently in the future.

Now let’s convert net returns to gross returns. If we assume VCs usually charge 2-and-20 fees, this step is pretty easy. Using the Cambridge Associates 1995–2018 data, we find a gross historical return of 18.1% with a 23.5% standard deviation.

Some people (especially VCs) like to talk about how the “top quartile” of VC firms persistently beat the market. This is true, but potentially misleading. VC firms who beat the market in year N are more likely than chance to beat the market in year N+1. But top-quartile firms are by no means guaranteed to stay in the top quartile.

How strongly do top-VC returns persist? Data from Harris et al. (2020)⁵ (1984–2014) is presented in the table below⁶.

The table uses these terms:

IRR = internal rate of return, or the annual return achieved by investors.
PME = public market equivalent, or the total return of VCs relative to the S&P 500 (see Kaplan & Schoar (2005)⁷). For example, if a VC fund earns a total return of 60% and the S&P earns 50% in the same period, then the PME is 60% / 50% = 1.2.

	IRR	PME
Average VC	14.8%	1.22
Top-quartile VC, backward-looking	45.3%	2.60
Top-quartile VC, forward-looking	26.3%	1.70

Results for VC firms 2001–2014:

	IRR	PME
Average VC	10.4%	(+)
Top-quartile VC, backward-looking	30.0%	2.11
Top-quartile VC, forward-looking	14.7%	1.20

(+) This figure is not provided by Harris et al.

In the full sample (1984–2014), top-quartile VCs retained about half of their outperformance out of sample, although they lost most of their relative outperformance (compared to the S&P 500). In the post-2001 sample, they lost most of their outperformance both in relative and absolute terms, but still showed nonzero persistence.

Harris et al. also did a regression analysis, and found that across all VC firms, one third of PME outperformance persisted.

(Note: Harris et al. used data from Burgiss, yet another source for VC returns.)

Returns for employees

Almost all startups give preferred shares to VCs and common shares to employees. Normally, preferred shares get a 1x liquidation preference. That means if the company exits, VCs are guaranteed to get back at least the money they put in before employees get anything. This makes employee equity worth less than it appears.

Example:

A startup raises funding at a $100 million valuation. VCs have $20 million of preferred shares; founders and employees have $80 million of common shares.
Later, the startup is acquired for $50 million.
VCs get back their $20 million. That leaves just $30 million to split among the common shareholders. The valuation went down by 50%, but employees lost 62.5% of their equity value.

This is basically standard practice. Some startups also give special advantages to VCs. There are lots of ways they can do this, such as:

a liquidation preference that’s higher than 1x (e.g., a 2x preference guarantees that VCs get to double their money before employees get anything)
a ratchet, which gives VCs protection against dilution at the expense of employees

These sorts of conditions are really bad for startup employees. You might just want to avoid any startup that offers terms like these. (If you work at a startup with no sketchy terms, and they raise a new round of funding that introduces sketchy terms, that alone might be enough reason to start looking for a new job.) For more on what to watch out for, read Ben Kuhn’s How bad are fundraising terms?

Even without any bad terms, employee stock options introduce some other problems:

If you don’t exercise your options, they could expire before the company exits.
If you don’t exercise your options and they go up in value, you might have to pay income tax or alternative minimum tax instead of capital gains tax.

You can avoid these problems by exercising your options as soon as they vest, or even early exercising if you can. But even if you do exercise, you might end up paying higher taxes when the startup exits because you’ll get pushed into a higher tax bracket. You can (at least partially⁸) mitigate this by donating the stock instead of selling it.

(A lot of people can’t afford to exercise their employee stock options. Perhaps an EA org could make grants or loans to help EAs exercise their options. That would be difficult to set up and they’d have to carefully vet grant recipients, but maybe it could work.)

According to my rough estimate, a 1x liquidation preference reduces the expected value of common shares by about 10%⁹. That equates to around 1–2% per year, depending on how long the company takes to exit. Let’s assume a 3% annual discount due to liquidation preference plus tax disadvantage.

Many startups pay below-market compensation by claiming that their equity is underpriced. Don’t buy it. The whole point of working at a startup is that your equity will earn (in expectation) above-market returns. If your employer adjusts for this by giving you less equity, that ruins the (monetary) advantage of working at a startup.

This essay is not about employee equity terms, but it’s an important topic for anyone considering working at a startup. For an in-depth guide, see The Open Guide to Equity Compensation by Joshua Levy et al. For something shorter, I recommend Ben Kuhn’s checklist for stock option offers.

Startup options are better than they look because employees get “meta-options”: your compensation package gives you the option to vest stock options at the current price for the next four years. If the company does well, you can “exercise” your meta-options by continuing to work there. If it doesn’t do well, you can quit. Neither startup founders nor startup investors can do this.

I piggybacked on Ben Kuhn’s meta-option model (my code here) and found that meta-options are worth an extra 16 percentage points of return (!!). I just did a quick calculation and didn’t perform a sensitivity analysis or anything, so this number could be way off, but let’s go with it for now. If correct, this number is so large that working at a startup looks more profitable than starting a startup, unless you believe you’d make for an unusually good entrepreneur.

It’s worth mentioning that you could work at a big company that offers equity compensation, which also behaves like a meta-option—albeit a much less valuable one, because big company stock is not as volatile. Using similar methodology, I found that meta-options at a big company are worth 5 percentage points. That means startup meta-options provide an extra 11 percentage points of value (16% – 5%).

As far as I know, Ben Kuhn invented the concept of meta-options, and no one has ever rigorously analyzed them. My own modification of his program could contain bugs or logical flaws. The value of meta-options could be large enough to dominate every other factor, or they could be worth nothing. This subject strongly warrants a deeper investigation.

–––––

If we can get around the practical concerns, EAs can easily match the returns of top VC firms by getting jobs at their portfolio companies. Can EAs do even better? Can we outperform VCs at picking winning startups?

Let me say up front that I don’t believe EAs in general can outperform top-quartile VC firms. But when I say I don’t believe it, what I really mean is I assign it less than a 50% probability. So it might still be worth trying.

(To be more precise, I would give an 80% probability that at least a handful of EAs could pick winning startups better than top VCs, but I don’t know how to identify those people in advance.¹⁰ I’d estimate a 30% chance that, if a group of EAs decide to go work at startups and make a conscious effort to pick winning startups, then they will do better than top-quartile VCs.)

EAs as a group are really smart. But professional investors are also really smart, and the overwhelming majority of them still fail to beat the market. Maybe EAs are smarter? Maybe EAs are more rational or clear-thinking in some way that most professional investors aren’t? I don’t know.

It’s possible that EAs could do a better job than VCs of identifying the best startups. On the other hand, EAs might also do a better job of identifying the best public investments. On the other other hand, startups are hard to invest in, so it might be easier to find underappreciated opportunities. If EAs have an edge in public markets, they probably have an even bigger edge in startups.

I don’t have strong evidence on this either way, so I’m leaning on my prior that almost nobody can beat the market. We do have at least some evidence, but I’m not sure how to interpret it.

The evidence we do have:

Over the past few years, EA investors have beaten the market. This is mostly driven by a single company (FTX), so I don’t know how much we can infer from this.
One person reviewed a few months’ worth of EAs’ proposed investing ideas and found that they had beaten the market over those few months. (I don’t want to go into specifics because this review was not shared publicly, but that’s the gist of it.)

In the rest of this essay, I will assume EAs can’t beat top-quartile VCs—not because I am confident that this is true, but because I don’t know how to evaluate the evidence. It could be a good idea to look into this in more depth.

Forecasting future returns

As discussed above, startup returns tend to vary a lot over time, so past performance does a poor job of predicting future performance. But we can’t choose between two investments (in this case, public investments vs. startups) unless we believe something about how they will perform. So what should we believe?

Public US equity and bond markets have unusually low return expectations right now, thanks to high valuations/low yields. When stocks and bonds look bad, money flows into alternatives, including VC. Therefore, it’s reasonable to expect VC to have worse future returns as well.
For the past few years, VCs have been investing much more money than the historical average (Statista, 2021). Crowdedness suggests low future returns. The only other time we saw similar crowdedness was in the year 2000, which was the beginning of a major losing streak for VC.
Private equity (that is, leveraged buyouts, not VC) has gotten more expensive over the last decade or so (Chingono & Rasmussen, 2015¹¹), which predicts muted future performance. And Harris et al.⁵ found that, while top VC firms’ persistence persisted in their whole sample, top private equity firms’ returns stopped persisting after 2001. It’s not clear why private equity’s persistence didn’t persist, but whatever the reason, the same thing might happen to VCs.

The outlook for VCs looks worse than usual. The question is, how much worse? Should we expect future returns to be 1 percentage point lower per year? Or 20 percentage points?

Well, how much worse to US equities and bonds look? We can reliably predict bonds’ long-term returns using the yield. 5-year bonds currently yield around 1%, compared to a 1984–2014 average nominal return of 8% (Damodaran, 2021).

Stock returns are harder to predict. In the short term, they’re almost impossible to predict, but we can estimate their return over 10+ year periods with reasonable accuracy. Under a more EMH-y model that assumes no change in market valuation (e.g., AQR, 2021), forward-looking US equity return expectations look around 5 percentage points worse than they did from 1984 to 2014 (4% vs. 9% after inflation). According to a model that assumes valuations will revert to their historical average (such as Research Affiliates, 2021), returns look 10 percentage points worse (-1% vs. 9%). The truth is probably somewhere in the middle.

In theory, all asset classes should have the same risk-adjusted return. Startups are riskier than stocks or bonds. So if return expectations for stocks/bonds go down by some amount, expectations for startups should go down by more than that. But we don’t know if this holds in practice, and we don’t even know exactly how risky startups are. If bonds look 7% worse, and stocks look between 5% and 10% worse, then maybe we could assume VC will perform 7% to 10% worse in expectation.

As for top-quartile VCs: according to Harris et al., over the full historical sample, they outperformed average VCs by a full 11 percentage points. In the post-2000 era, they only outperformed by 4 percentage points, and had a public-market equivalent performance of 1.2 (which means they only weakly outperformed the S&P 500). It seems fair to assume that the future for VCs will look more like 2001–2014 than like 1984–2000, as the pre-2000 VC market was probably less efficient. We could simply assume top VCs will perform 4 percentage points better than average VCs going forward. But it’s also possible that the gap between average and top VCs will continue to narrow.

Giving now vs. later

Working at a startup is comparable to working for a salary and investing it. But if giving now beats giving later, then you wouldn’t want to invest your salary. Instead, you’d want to donate it right away. This makes working at a startup look worse because you can’t donate your equity until it becomes liquid.

This possibility makes the comparison more difficult, so I will mostly ignore it. It’s not as simple as applying a fixed discount rate to the value of your startup equity. Just be aware that my methodology for comparing startups vs. big companies only works if giving later is at least as good as giving now, at least for the next few years.

Putting together the expected return

If we combine all the numbers I came up with in the previous section, we get:

13% after-fees historical return to VC firms, or 10% after inflation.
15% real historical return before fees.
Add 4% for the persistence of top-quartile VCs, giving 19%.
Add 0% for EAs’ extra outperformance. Still 19%.
Subtract 3% for employee equity terms, giving 16%.
Add 11% for meta-options, giving 27%.
Subtract 9% for the relatively poor market outlook, giving 18%.
Ignore giving now vs. later. Still 18%.

Thus, I predict a 18% real return for startup employees who try to maximize their earnings (by working at startups with funding from top VCs, getting good equity terms, and exercising their meta-options when necessary).

How big are the error bars on each of these numbers? In order:

Historical return depends a lot on what time period you look at. Wide error bars.
Calculating before-fees return just requires knowing fees, which are usually 2-and-20. Narrow error bars.
It wouldn’t be too surprising it top-quartile VCs had as much as 11% extra return or as little as 0%. Wide error bars.
Even if we have good reason to expect EAs to do better at picking startups than top-quartile VCs, it seems unlikely that they could perform much better. Narrow error bars.
Liquidation preference matters relatively little. Narrow error bars.
The concept of meta-options is complicated and has received hardly any attention. My best-guess estimate for their value is very large, but I could be way off. Extremely wide error bars.
Future performance is really hard to predict, even over long time horizons. Wide error bars.

By my estimate, startup employees’ expected returns could optimistically be as high as 52% (!); or they could be as low as -2%.¹² (Remember, these are expected returns. Realized returns could fluctuate by much more than this. Startups in aggregate could easily realize a 100% return next year, and I wouldn’t find that surprising, but I would be crazy to expect it to happen.)

Risk and correlation of startups

Expected return alone isn’t what we care about. We really want to know risk-adjusted return.

And we don’t just care about the risk-adjusted return of startups in isolation. We want to know how they fit into a broader EA investment portfolio.

There are two equivalent ways of looking at this:

Find the risk of startups, and their correlation to the aggregate EA investment portfolio. Then we can calculate whether EAs on the margin should work at startups instead of big companies.
Find the alpha of startups relative to the EA portfolio. If alpha > 0, that means at least some EAs should work at startups.

I will focus on the first because I find it more intuitive. I also calculated the second and got similar results (not presented in this essay).

I’m looking at the risk and correlation of the startup industry, rather than the average risk/correlation of a single startup. We can think of it as a collective decision by many EAs to work at a diversified group of startups, rather than the decision of a single person.

As with expected return, we have no way to know the future risk of startups, or their correlation to the EA portfolio. But with risk and correlation, we get to make some simplifying assumptions.

We can’t learn much about the future return of an asset class by looking at its past return. Markets are reasonably efficient, so if an asset class performs well, more money floods in and performance reverts to the mean. But the efficient market hypothesis doesn’t say risk mean-reverts. Studies show that, at least for public equities, historical volatility is a pretty good predictor of future volatility (e.g., Dreyer & Hubrich, 2017¹³).

Three different data sets all give similar(ish) numbers for startups’ standard deviations:

Data Set (Gross)	Standard Deviation
Cambridge Associates, 1988–2016	21.4%
Cambridge Associates, 1995–2018	23.5%
Harris et al., 1984–2014	28.5%

Note that, according to Cambridge Associates, top-quartile VCs have higher standard deviations than average VCs (25% or 28%, depending on which time horizon you use). So if we only work at top startups, we should bump these numbers up by a few percentage points. Also, startups don’t have public prices the same way stocks do, and VCs have some leeway to value their portfolios however you want. I expect that they tilt their portfolios toward low volatility to make themselves look better, so the “true” volatility is probably higher.

Woodward (2009)¹⁴ argues that, because startup valuations tend to lag the market, a naive regression doesn’t show the true relationship between startups and public equities. The paper finds that startups have a stock market beta of a little over 2, which corresponds to a standard deviation of about 35%.

For correlation, as with standard deviation, we can assume the future looks the same as the past. Historical correlation between startups and public equities was around r=0.7. (My own analysis found a correlation of 0.6, and Woodward (2009)¹⁴ found a correlation of around 0.7–0.8 using more limited data but better methodology. Woodward’s analysis suggests that the naive approach underestimates the true correlation. So let’s use 0.7.)

Leverage

Many EA investors probably want to use leverage. But startup employees can’t leverage their equity: they get however much they get based on their employment contract, and there’s no way to borrow money to get more equity.¹⁵ Instead of comparing startup equity to a public investment portfolio, we should compare startup equity to an optimally leveraged public investment portfolio (taking into account that leverage typically costs more than theoretical models assume).

Startups vs. public equities

Now that we’ve discussed the main considerations, we can return to the original question: is it better for earners-to-give to work at high-paying companies and invest their salaries in the market, or to work at startups and “invest” in startup equity?

Some additional assumptions:

Our goal is to maximize the geometric return of the overall EA investment portfolio. (This is consistent with logarithmic utility of money.)
We can only invest in two things: public equities or startups.
We control 1% of the EA portfolio. We can’t affect the other 99%.
EAs currently invest all their money in public equities, and none in startups. (The latter is obviously false, but it’s also sort of true: on the margin, earners-to-give can consider working for startups that don’t already have any EAs working for them. That set of startups has 0% EA investment.)
If we buy public investments, we can use up to 2:1 leverage.¹⁶
Public equities earn an expected real return of 3% with a standard deviation of 16%.¹⁷

Recall from above that we’re giving startup equity an 18% expected real return, a 35% standard deviation, and a 0.7 correlation to public equities.

Our two choices:

Work at a big company. Invest our salary in public equities with 2:1 leverage.
Work at a startup.

Given all the stated assumptions, working at a startup is more than four times better than working at a big company (37 expected utility vs. 200 expected utility, according to a scaled logarithmic utility function¹⁸).

Suppose we hold everything else constant but reduce the expected real return of startups. The return needs to be as low as 1% before the startup looks like the worse choice. (Notice that that’s lower than the 3% expected return of the public stock market, even before accounting for leverage. Startups with a 2% expected return are still (barely) preferable to public equities with a 3% return because we’re assuming startups have a lower correlation to the EA portfolio.) So even under a much more pessimistic projection for startup returns, they still look preferable to big companies.

Startups vs. an optimized public investment portfolio

Buying an index of public equities might not be the best way to invest one’s big-company salary. I personally prefer to invest in concentrated value, momentum, and trend strategies. Some EAs believe cryptocurrency or AI stocks will beat the market. The specifics don’t matter too much. What matters is that if you believe some other investment has substantially better expected performance than a broad index fund, then you should use that other investment as your benchmark instead. And startups need to look better than that benchmark.

My best guess is that a concentrated value/momentum/trend portfolio will earn an expected real return of 6% with a standard deviation of 11%. (Of course, as with my estimates for startup returns, these numbers are not remotely robust.) If we also use 2:1 leverage, then value/momentum trend still looks somewhat worse than startups, although not by a as big of a margin (126 expected utility points vs. 200). If startups returned 10% instead of 18%, then value/momentum/trend would be the better choice.

Alternative: Predictionless approach

Alternatively, take the same basic model as above, but don’t try to predict the future. Instead, assume asset classes will perform exactly as well in the future as they performed in the past. As I discussed above, this approach has issues—performance fluctuates a lot over time, so past performance doesn’t tell us what will happen in the future. But there’s also something appealing about this method. Trying to predict future performance leaves lots of room for you to bias the outcome toward what you (perhaps subconsciously) want. It’s harder to introduce bias if you just use past performance.¹⁹

For the predictionless approach, I estimated the expected return to employee equity as:

13% after-fees historical return to VC firms, or 10% after inflation
15% real historical return before fees
Subtract 2% for employee liquidation preference, giving 13%
Add 11% for meta-options, giving 24%
Add 4% for the persistence of top-quartile VCs, giving 28%

For public equities and for my value/momentum/trend portfolio, instead of making projections, I used the (estimated) historical return after inflation from 1995 to 2018:²⁰

	Return	Std Dev
Public Equities	9%	19%
Val/Mom/Trend	15%	12%

The three choices have the following expected utilities:

Public equities: 142 utility
Val/Mom/Trend: 288 utility
Startups: 279 utility

Startups look preferable to public equities, but slightly worse than value/momentum/trend.

Alternative: Models are bad. What if we don’t use a model?

I love using quantitative models like the one in this essay. I think more people should use them. But most models are bad, including mine. They depend on lots of assumptions, and you can change the model output by making small changes to the assumptions.

How could we reason through this decision without using an explicit model? Let’s review some arguments, both pro- and anti-startup.

Argument from risk preferences: Most startup employees don’t want to donate all their equity. That makes them much more risk-averse than EAs who work at startups. If they’re acting rationally, we should expect them to demand higher equity to compensate for the risk. Therefore, startup equity should look particularly compelling to EAs.

Argument from inefficiency: The market for startups is illiquid and has high barriers to entry. We might reasonably expect it to be less efficient than public markets, which means we have a better chance of identifying startups that will outperform.

Argument from investability: The most reputable VC firms usually don’t accept new investors. Even if they can beat the market, you can’t invest with them, so it doesn’t matter. But there’s nothing stopping you from getting jobs at top VCs’ portfolio companies.

Argument from overpopularity of startups:

A lot of people want to work at startups because startups are cool, and they’re willing to accept below-market compensation.
Total VC investment dollars have increased a lot over the past few years, even though the number of startups hasn’t changed much. So the startup market might be overinflated.

Argument from underappreciated risk: In my experience, almost nobody understands how risky individual startups are. Even medium-sized companies are about 3x as risky as the S&P 500. I don’t have sufficiently granular data on startups, but startup-sized public companies are about 5–6x as volatile as the S&P 500, and my guess is startups are even worse. When I see people discussing the value of startup equity, they almost never properly account for this.

Argument from diversification: If you get a job at a startup where no other EAs work, you’re adding an entirely new investment to the EA portfolio. That could be a good thing even if that particular investment has a worse expected value than the market. On the other hand, there are other ways of diversifying that might be better.

These qualitative arguments don’t obviously lean one way or the other. My intuition from my time working at startups and knowing lots of startup employees is that most people overvalue startups and underestimate risk, which means they probably push down the market rate for equity compensation. But even if most startup employees don’t behave consistently with their personal risk appetite, they still might behave more risk-aversely than EAs ought to.

Practical details

If more EAs want to work at startups, there are some ways that people or organizations could support this effort, such as:

Maintain a list of startups with funding from top VCs, or startups that look particularly promising for whatever reason.
Coordinate to identify companies that don’t already have EAs working at them, or that might provide the most diversification benefits to the EA portfolio.
Help EAs review employment contracts from prospective employers.
Make loans or grants to EAs to help them exercise stock options as soon as they vest.
Career support/recruiting services for EAs who want to work at startups.
Support for EAs whose startups fail. Maybe even offer some kind of insurance to reduce risk, e.g., if you go work for a startup and it fails, we will pay you to compensate for the earnings you could have had.
Help people negotiate for better equity terms.

Some of these ideas are logistically difficult, maybe even impossible. I’m not sure the best way to provide support for earners-to-give who choose to work at startups, but it’s something to consider. I believe it would be valuable if an organization existed that helped EAs with these practical details.

Conclusion

Assuming my model is approximately correct, what type of person might want to work at a startup?

Someone who wants to earn to give.
Someone who doesn’t have the right skills or temperament to start a startup, but still might want to work at one.
Someone with special insight into a field who thinks they can identify the most promising companies.
Perhaps someone who can’t invest with leverage, or who doesn’t want to use leverage, but who is comfortable with the risk of startup equity.

Who might not want to work at a startup?

Someone who’s not comfortable with the risk (equity risk or career risk or both).
Someone who believes they can see particularly good investment opportunities /outside/ of startups, and wants to earn a high salary so they can invest in those other opportunities.
Someone who thinks donating now is significantly better than donating a few years from now, and therefore doesn’t want to wait for startup equity to vest.

My analysis suggests that working at a startup has good expected value under ideal conditions. If you get a job offer from a startup, remember to pay attention to the specifics of the offer:

Is your total compensation competitive with what you’d get at a big company? (Taking startup equity at face value)
Does your equity contract include any sketchy terms?
etc. (Too many specifics to list all of them)

Areas for further research

Many subjects warrant a deeper investigation:

Historical VC returns
Historical returns earned by startup employees
How employee equity terms affect the value of equity
Value of meta-options
How most startup employees decide where to work—most importantly, how sensitive are they to the value of equity?
Why EAs might or might not expect to beat the market
How Woodward (2009)¹⁴’s analysis looks if you update it to include more recent data
Relevance of giving now vs. later
Career capital from working at startups. Does working at a startup train you to start a startup or a nonprofit?
Other considerations worth including

Acknowledgements

Thanks to Linchuan Zhang for commissioning this research project and providing support. Thanks to Charles Dillon for feedback.

Changelog

2024-04-25: Upgraded confidence from “somewhat likely” to “likely” based on the fact that readers did not spot any major flaws in my reasoning, and based on calculating the value of meta-options using Black-Scholes (see here for explanation) which roughly agreed with Ben Kuhn’s script that I’d used previously.

Appendix A: Startups for founders and investors

This essay has looked at startups from the perspective of employees. How do startups look for other types of people?

Founders: Similar to employees in many ways. The upside is you get a lot more equity. (Hall & Woodward (2012)²¹ found that VC-backed startup founders on average made much more money than salaried employees.) The downside is you have to actually start a startup, which is much harder and may require an entirely different skillset.

Other people have written about whether EAs should start startups:

Applied Divinity Studies, Life Advice: Become a Billionaire
Mathieu Putz, An update in favor of trying to make tens of billions of dollars
Carl Shulman, Salary or startup? How do-gooders can gain more from risky careers
Brian Tomasik, Calculator for Expected Utility of Founding a Startup

VC limited partners: If you give your money to a VC firm to invest, this is probably worse than being a startup employee (although it does have the advantage that you don’t need to get a new job). You have to pay VC fees and you don’t get meta-options. You do get a better liquidation preference, but that’s usually not worth as much. For a more detailed discussion on investing in VC, see Mulcahy et al. (2012)²².

Angel investors: If you become an angel investor, you don’t have to pay VC fund fees, but you do have to evaluate startups on your own.

Appendix B: Some important tangents

The points below are all important, but they distract from the thesis of this essay, so I’m not commenting on them in detail.

Within the context of my model, some big companies’ compensation packages behave more like startups’. Companies such as Facebook and Google offer equity to employees. Unlike with startups, you can sell the equity as soon as you get it. But you usually have to wait a year, and a big company’s equity grant still behaves like a meta-option.
There are many non-monetary pros and cons to working at a startup. For instance, see Big companies vs. startups and Early career EA’s should consider joining fast-growing startups in emerging technologies.
An unimportant point that I nonetheless want to address: Startup employees’ equity will get diluted by future fundraising rounds. This doesn’t matter because VCs will get diluted by the same amount, so it doesn’t make employee equity look worse relative to VC equity (unless the employment contract contains sketchy terms around who gets diluted, in which case maybe you shouldn’t work there). Normally, VCs have the option to invest more money in future rounds to negate their dilution, but this also doesn’t matter because it doesn’t change the return on their initial equity purchase.

Notes

The report includes VC returns up to 2020, but it only includes detailed data up to 2018. So when I did my analysis, I used the 1995–2018 data. ↩
Somewhat concerningly, these two data sets show different numbers even for years where they overlap. For example, the 1988–2016 data set quotes a 60.09% return for the year 1996, whereas the 1995–2018 data set claims a 63.46% return for the same year. This discrepancy is at least partially because the 1995–2018 series includes more VC firms, but I haven’t read the Cambridge Associates reports in enough detail to say if that’s the only reason. ↩
Koch (2014). The risk and return of venture capital. ↩
Woodward (2009). Measuring risk for venture capital and private equity portfolios. ↩
Harris, Jenkinson, Kaplan & Stucke (2020). Has Persistence Persisted in Private Equity? Evidence from Buyout and Venture Capital Funds ↩ ↩²
Figures are copied or inferred from Harris et al. (2020), Table 1, Table 2, and Table 4. ↩
Kaplan & Schoar (2005). Private Equity and Performance: Returns, Persistence, and Capital Flows. ↩
At least in the United States, if you donate stock to charity, you can only deduct up to 30% of your income. If you get lucky and make a bunch of money when your startup exits, your startup equity could account for something like 90% of your income. You can only deduct 30%, so you’re stuck paying taxes on the other 60%. ↩
Very roughly, a startup has a 70% chance to be worth 0x, 20% chance of 0–1x, and 10% chance of >1x. Liquidation preference only matters in the 0–1x case, where common shares are worth about half as much as their face value.

If you don’t filter out sketchy terms, the appropriate discount is more like 36%. ↩
Actually, I know a few people who I believe could do a good job of identifying top startups if they took the time to conduct lots of interviews and due diligence. But they’re not going to do that because they’re busy doing other important EA-related activities. ↩
Chingono & Rasmussen (2015). Leveraged Small Value Equities. ↩
For the optimistic estimate, I assumed: top-quartile VCs’ average out-of-sample return of 26% (or 23% real) fully persists; EAs perform 5% better than top VCs; and meta-options are worth 20% (which follows from optimistic assumptions about how meta-options behave).

For the pessimistic estimate, I assumed: public equity valuations fully mean revert, and startups perform even worse due to higher risk; top quartile VCs’ returns do not persist at all; and meta-options are worthless (probably because there’s some flaw with them that I haven’t thought of). ↩
Dreyer & Hubrich (2017). Tail Risk Mitigation with Managed Volatility Strategies. ↩
Woodward (2009). Measuring Risk for Venture Capital and Private Equity Portfolios. ↩ ↩² ↩³
Technically, if you get employee stock options, your equity is leveraged. But the amount of leverage approaches zero as the stock price increases. And most stock options are only a little bit leveraged to begin with. For example, if your company stock was last valued at $4 and you get options with a $1 strike price, that’s only 1.33:1 leverage. If the stock price doubles to $8, now you only have 1.14:1 leverage. ↩
In keeping with my previous work on leverage, I assume that borrowers must pay 1% plus the risk-free rate. ↩
This is the average of the projections by AQR and Research Affiliates as of October 2021. ↩
The utility function takes the geometric mean return as the utility and multiplies by 100,000 to make the numbers more readable. As a baseline, it calculates the expected utility of the EA portfolio without your investment, and then subtracts that from the total expected utility of the EA portfolio including your investment. ↩
Still not impossible, because you could pick the data set or the time series that most closely matches the outcome you want. ↩
For public equities, I used the historical return of the total US stock market, assuming zero fees or trading costs. To find the historical return of Val/Mom/Trend, I created a hypothetical portfolio that invested 80% in Alpha Architect’s Value Momentum Trend Index and 20% in AQR’s Managed Futures Index, which roughly reflects how I actually invest my money. Both indexes subtract estimated fees and trading costs. The historical returns are hypothetical, not actual. I didn’t have data for 2018, so I calculated summary statistics over 1995–2017 instead. ↩
Hall & Woodward (2012). The Burden of the Nondiversifiable Risk of Entrepreneurship. ↩
Mulcahy, Weeks & Bradley (2012). We Have Met the Enemy…and He Is Us: Lessons from Twenty Years of the Kauffman Foundation’s Investments in Venture Capital Funds and the Triumph of Hope over Experience. ↩

Obvious Investing Facts

Fri, 29 Oct 2021 00:00:00 -0700

Last updated 2025-05-01.

Many investors, even professionals, are ignorant about obvious facts that they really should know. When I say a fact is “obvious”, what I mean is that you can easily observe it by looking at widely-available data using simple statistical tools.

A list of obvious but underappreciated facts:

Fact 1. A single large-cap stock is about 2x as volatile as the total stock market. A small-cap stock is about 4x as volatile. It’s common knowledge that individual stocks are risky, but most people don’t know how to quantify the risk, and I believe they tend to underestimate it. I wrote a whole essay about this because I think it’s the most important underappreciated investing fact.

Fact 2. A basket of 50 randomly-chosen stocks isn’t much more volatile than the total stock market. This is kind of the opposite of fact 1. According to my backtest,¹ the total US stock market had an annual standard deviation of 17.3%, whereas a randomly-chosen selection of 50 stocks had a standard deviation of 20.2%.² That’s higher, but not by much.

Baskets of 30 stocks had standard deviations of 21.7%; 10 stocks had 25.5%; 5 stocks had 31.9%.

(For more on this, see Elton & Gruber (1977), Risk Reduction and Portfolio Size: An Analytic Solution.)

My anecdote about this: When I was considering opening a donor-advised fund (DAF), I had phone calls with a few investment advisors who I thought might do a good job of managing it. I remember one advisor in particular who I liked at first because she said she was a fan of value and momentum investing. I asked her if she could invest my DAF in a concentrated portfolio of about 60 value stocks, like I describe in this essay. She declined, saying that a 60 stock portfolio would be far riskier than the broad stock market, and without any extra expected return to compensate. This is obviously false if you simply look at the data. A randomly-chosen collection of 60 stocks is a little riskier than the broad market, but not much riskier. And it’s true that there’s no reason to take on risk if there’s no benefit, but if you’re specifically investing in value and momentum strategies, then historically, concentration risk also brought higher return, as I showed previously.

Fact 3. Stocks can underperform bonds for 30+ years at a time. Investment manager Meb Faber wrote an article about this, inspired by two amusing Twitter polls:

What stretch of underperformance by stocks vs. bonds would you be willing to tolerate before selling your stock allocation?
— Meb Faber (@MebFaber) February 6, 2020

Would you be willing to invest in an asset that historically outperforms bonds by a few percentage points per year, but has, multiple times, generated zero outperformance for stretches lasting over 30 years?
— Meb Faber (@MebFaber) February 6, 2020

53% of investors said they would not tolerate 10 years of stocks underperforming bonds. What did they do from 2000 to 2010, when US stocks performed worse than US bonds? (Bonds have beaten stocks many times, but 2000 to 2010 is such a recent decade that you’d think people would remember it.) Fully 76% of respondents said they wouldn’t tolerate 30 years of underperformance. So either they don’t invest in stocks at all (unlikely), or they don’t know that stocks have in fact underperformed bonds for 30 years on more than one occasion.

Longtermtrends.net has a nice chart of US stock vs. bond performance going back 150 years, using data from McQuarrie (2021),³ FRED,⁴ and NASDAQ.⁵ (Be sure to look at the “Total Return Stock Index” chart. This chart only shows a maximum of 22 years of underperformance since 1900, but it’s using a corporate bond index and I assume Meb was talking about Treasury bonds.)

Fact 4. Bonds are pretty risky after adjusting for inflation. US short-term T-bills—supposedly the safest investment around—have experienced a historical drawdown of –49% after inflation. On another fun Twitter poll by Meb Faber, only 36% of respondents guessed that the drawdown was worse than –45%. (Meb Faber’s Twitter is a great source of underappreciated investing facts.)

Fact 5. The last 3–5 years of performance (of a fund or asset class) tell you almost nothing about future performance. For instance, see Hoffstein (2016), Are 3-year track records meaningful?

And yet, 89% of managers would replace an investment strategy that has underperformed for 3 years. And when investment managers switch strategies after a period of poor performance, the new strategy tends to perform worse than the old one would have⁶.

Fact 6. Buybacks are financially equivalent to dividends. Many investors like dividends but dislike buybacks. But these are just two different ways of returning capital to shareholders. In an efficient market, if a company buys back 2% of its shares and then you sell 2% of your holdings, that’s exactly the same as if the company paid a 2% dividend. Conversely, if the company pays you a 2% dividend and you use that money to buy more shares, that’s exactly the same as if the company did a 2% buyback. These scenarios are slightly different for tax purposes, but in financial terms, they’re identical.

Sort-of-obvious but controversial facts

These facts in this section are less obviously true, either because they come from theoretical models that don’t necessarily hold in practice, or they rely on mixed empirical data. But I believe they’re still underappreciated, and it’s somewhat surprising just how underappreciated they are among investment professionals.

Fact 7. Actively-managed mutual funds are just as risky as index funds. I don’t know if this is a common misconception among ordinary investors, but I frequently hear investment professionals claim that actively-managed funds are less risky than index funds, or that they perform better in market drawdowns. The evidence on this is mixed, but generally doesn’t look promising for the actively-managed funds (e.g., see Vanguard (2018), specifically Myth #5).⁷

I could be wrong about this one, because the evidence isn’t great either way. But the mutual fund managers who confidently claim that they can reduce risk are definitely not standing on solid ground.

“It is difficult to get a man to understand something, when his salary depends on his not understanding it.” Most people know by now that active mutual funds don’t outperform index funds. So in an attempt to justify their existence, investment managers claim that they provide value by reducing risk. This is (probably) false, but it’s not as well-known that it’s false, so they think they can get away with it.

Fact 8. If the efficient market hypothesis and CAPM are strictly true, then the mean-variance-efficient portfolio is the global market portfolio.⁸ Not the S&P 500. Not 60/40. Not global equities.

Maybe this fact isn’t exactly obvious—figuring it out required inventing new math. But it was proven over half a century ago, and the man who proved it went on to win a Nobel Prize in economics, in large part thanks to precisely this result. You’d think it would be more well-known by now.

A weaker version of this fact: if you want international exposure, buy a world stock market fund. Don’t buy a US market fund and then claim you’re getting international exposure because some US companies sell internationally. I’ve seen many investing professionals recommend the latter (including Jack Bogle). History shows that an international stock index is not perfectly correlated to a US index, so you can trivially get better diversification by buying internationally.

Fact 9. In theory, the best way to increase expected return is by using leverage, not by increasing the ratio of stocks to bonds. This isn’t an empirical fact, but it easily follows from CAPM, the standard theory of investment risk. CAPM makes some assumptions that don’t hold in practice. But under certain real-life conditions, it does indeed make sense to use leverage than rather than overweight risky assets. A few sophisticated investors, such as Bridgewater Associates, abide by this principle, but the vast majority of investors don’t.

(Bridgewater pioneered risk parity investing, where you allocate to each asset class such that each asset class exposes you to equal risk, and then add leverage. Risk parity is theoretically optimal when every asset class has the same risk-adjusted return.)

Fact 10. Gold is not a great inflation hedge.⁹ Many people claim that gold can hedge inflation, and there are reasonable theoretical reasons to expect it to. But gold is very volatile, and it frequently performs poorly even in times of high inflation. It might still make a good addition to a portfolio, but it doesn’t provide the straightforward inflation protection that a lot of people believe it does.

Fact 11. Art has not outperformed the stock market. Art sellers like to claim that art is a great investment (e.g., 1, 2), and they cite studies showing the returns to classic art. These results are due to selection bias¹⁰¹¹. You can only measure an artwork’s change in value if it gets sold once and then sold again later. And art only tends to get re-sold when its value goes up, so your sample excludes most of the artwork that depreciated over time.

Obvious facts that are becoming more well-known

20 or 30 years ago, most people didn’t know these facts. but fortunately, they’ve become more widely understood in recent years.

Fact 12. High fees are really bad.

Fact 13. Actively-managed funds generally can’t outperform index funds.

Many people know these facts, and the market for low-fee index funds is growing rapidly. Even so, about 2/3 of investment dollars still reside in expensive actively-managed mutual funds, so we have a long way to go.

Some more facts

AQR has written three papers about investing facts and fictions. Most of AQR’s facts are obvious in the sense that you can easily verify them using empirical data. They’re somewhat less obvious than the facts above in that they’re about more specific types of investments, rather than the general behavior of markets.

Their three papers (PDF warning):

Fact, Fiction and Momentum Investing (2014)
Fact, Fiction and Value Investing (2015)
Fact, Fiction and the Size Effect (2018)

Why do so many investment professionals not know these obvious facts?

I’m not sure. I can see two common themes:

Investment professionals believe things because they can make more money by believing them, not because they’re true.
Investment professionals don’t look at empirical data, so regardless of how obvious an empirical fact might be, they will never notice it.

One possible explanation is that when people think they know something, they don’t feel the need to look into it. For example, if you “know” that bonds are safe after adjusting for inflation, why would you look up their historical performance? Even so, that would mean financial professionals don’t make a habit of looking at simple data that’s relevant to their work, which seems bad. Or, like, you’d think this information would propagate. Somebody tweets about an obvious but little-known fact; their followers realize how wrong they were, and tell their colleagues about it; those colleagues tell their other colleagues; and pretty soon, everybody knows. (I must admit that I didn’t know bonds had experienced a –49% drawdown until I read Meb Faber’s tweet, so I’m also guilty of not looking at obvious data sometimes.)

(I do think professional credentials provide a positive signal, at least. In my experience, CFAs are more likely to know these obvious facts.)

Maybe experts in most fields don’t know obvious facts. I don’t know much about most fields, so I can’t say. The two areas that I know a lot about are computer science and investing (and I know more about computer science because I studied it in college). In my experience, there aren’t a lot of obvious computer science misconceptions among programmers/computer scientists (unless I also believe all the common misconceptions).

Maybe investment advisors aren’t the right reference class for “experts”. Maybe the experts are the people publishing academic papers on investing.¹² I’d guess that academics in finance know obvious facts more often than investment advisors do.

Sometimes, people don’t know important facts, but they behave correctly anyway. I frequently hear people touting advice such as, “pick an investment allocation and stick with it no matter what.” You might come up with this advice if you knew that stocks can underperform bonds for 30 years at a time, and that 3–5 years of historical performance tells you almost nothing about future performance. And many people follow this advice even without knowing those facts. If they end up doing the right thing, it doesn’t really matter if they know why they’re doing it. On the other hand, almost everyone behaves incorrectly in other ways—overweighting their home country stock, or holding on to concentrated investments for too long.

(This reminds me of Statistical Literacy Among Doctors Now Lower Than Chance: doctors perform depressingly poorly on the Bayes mammogram problem, but when they get a positive test result in real life, they still do the correct thing—rather than jumping to conclusions, they order more tests.)

Changelog

2022-11-18: Add a data source for the fact “Stocks can underperform bonds for 30+ years at a time.”
2025-05-01: I realized the facts were numbered incorrectly so I updated the numbers.

Notes

My analysis took about two hours, which is long enough that someone wouldn’t do it on a whim, but short enough that I still think it’s fair to describe the result as “obvious”. ↩
My methodology:
1. Each year from 1973 to 2013, choose 50 stocks uniformly at random.
2. Simulate a portfolio that buys the chosen stocks, weighted by market cap.
3. Calculate the standard deviation over the portfolio’s annual total return.
4. Repeat 100 times and take the arithmetic mean of all the standard deviations.
The standard error was 0.17 percentage points. ↩
McQuarrie, Edward (2021). Stocks for the Long Run? Sometimes Yes. Sometimes No. ↩
Federal Reserve Bank of St. Louis (2022). ICE BofA US Corporate Index Total Return Index Value. ↩
NASDAQ (2022). S&P Real Price by Month. ↩
Stewart, Neumann, Knittel & Heisler (2009). Absence of Value: An Analysis of Investment Allocation Decisions by Institutional Plan Sponsors. ↩
Ptak (2018), Will Active Stock Funds Save Your Bacon in a Downturn? found that actively-managed funds did tend to outperform in drawdowns, but not by enough (in the author’s opinion) to make up for their overall bad performance.

What I’d really like to see is how much alpha mutual funds have in up or down markets, but I haven’t found any research on that.

See also financial advisor Josh Kruk’s take on the matter. He did his own analysis and found that active funds did not outperform in down markets. ↩
Sharpe (1964). Capital Asset Prices: A Theory of Market Equilibrium Under Conditions of Risk. ↩
Erb, Harvey & Viskanta (2020). Gold, the Golden Constant, COVID-19, ‘Massive Passives’ and Déjà Vu. ↩
Korteweg, Kräussl & Verwijmeren (2013). Does It Pay to Invest in Art? A Selection-Corrected Returns Perspective. ↩
Disclosure: The linked article did not convince me that classic art returns are due to selection bias. Rather, I was confident a priori that the apparent high returns came from selection bias, and this was the first empirical result I found that supported my prior hypothesis. ↩
Successful day traders and quants are experts in some sense, but their work doesn’t actually have much in common with the way most people invest. ↩

Future Funding/Talent/Capacity Constraints Matter, Too

Mon, 18 Oct 2021 00:00:00 -0700

Last updated 2021-10-20.

People who talk about talent/funding/capacity constraints mostly talk about what’s the biggest constraint right now. But it also matters what the constraints will be later.

Right now, the EA community holds a lot of wealth—more wealth than it can productively spend in the next few years, at least on smaller cause areas such as AI safety, cause prioritization research, and wild animal welfare. Those newer fields need time to scale up so they can absorb more funding.

That doesn’t mean EAs should stop earning to give. Maybe most EAs could do more good this year with their direct efforts than with their donations. But perhaps 10 years from now, the smaller causes will have scaled up a lot, and they’ll be able to deploy much more money. Earners-to-give can invest their money for a while, and then deploy it once top causes develop enough spending capacity.

Even if top cause areas don’t currently have the capacity to spend more money, earning-to-give still looks valuable if three conditions all hold:

The capacity:funding ratio will greatly increase in the future.
The money spent in high-capacity organizations will still produce a lot of utility per dollar.
The discount rate isn’t too much higher than the investment rate of return.

In general, I believe it should be easier to grow capacity than to grow funding. By comparison, it’s not uncommon for startups to double the number of employees year over year. Charitable organizations usually can’t grow like startups, but they can at least get within the same ballpark.

Over the past few years, funding has grown faster than capacity, not the other way around. So I could be wrong.

Once capacity grows, will orgs still be able to do a lot of good per dollar? They will use up the best giving opportunities first, so by the time capacity expands, the remaining opportunities won’t be as good. How much difference does that make? It depends on how much it costs to solve the world’s most important problems. If it cost a lot, we will still need lots of money. If it doesn’t cost much (relatively speaking), then future dollars will be worth much less than dollars today.

Is the discount rate high enough that we should care substantially less about future funding? I don’t think so. (What we actually care about is the discount rate minus the investment rate, because the investment rate determines how much money we will have in the future if we invest it now.) The altruistic discount rate minus the investment rate might be around 5% on the high end (it’s probably less than that, and it could be negative). I would be surprised if we couldn’t grow capacity much more quickly than 5% per year. So this factor is unlikely to be decisive.

My personal predictions on the three conditions:

The capacity:funding ratio will indeed greatly increase in the future. Confidence: Likely.
I don’t know how quickly the value of money will decrease over time. My best guess is that all current EA funds combined are not enough to solve the world’s biggest problems, so we will need as much additional funding as we can get. Confidence: Possible.
The discount rate is low enough that it doesn’t affect the outcome of this question. Confidence: Highly likely.

Ultimately, this argument doesn’t tell us much about whether any particular person should go into direct work or earning to give. And even if earning to give looks valuable, it might still be true that, in general, more people should be doing direct work.

The first argument:

Altruists have a low discount rate.

[some other premises]

Therefore, they should give later rather than now.

Therefore, earning to give is good.

(Phil Trammell presents an argument like this here)

This is a related argument in the sense that it considers the value of marginal dollars now vs. later. But the argument I presented does not require altruists to have a low discount rate.

The second argument:

There is an optimal percentage of resources for the community to spend vs. invest in a given year.

The community may end up spending above this level in the future.

If that happens, having people switch to earning to save may be one of the best ways to deal with it.

(source: Ben Todd, An argument for keeping open the option of earning to save)

Like my argument, this argument also considers present vs. future constraints. But it’s talking about the tradeoff between spending and investing for people who are already earning to give, not about the choice to earn money vs. do direct work.

Low-Hanging (Monetary) Fruit for Wealthy EAs

Sat, 16 Oct 2021 00:00:00 -0700

Confidence: Likely.

Cross-posted to the Effective Altruism Forum.

Ordinary wealthy people don’t care as much about getting more money because they already have a lot of it. So we should expect to be able to find overlooked methods for rich people to get richer.¹ Wealthy effective altruists might value their billionth dollar nearly as much as their first dollar, so they should seek out these overlooked methods.

If someone got rich doing X (where X = starting a startup, excelling at a high-paying profession, etc.), their best way of making money on the margin might not be to do more X. It might be to do something entirely different.

Some examples:

(Edit 2024-03-18: This paragraph did not age well…although the point about retaining equity is still valid.)

Sam Bankman-Fried increased his net worth by $10,000,000,000 in four years by founding FTX. He earned most of those zeroes by doing the hard work of starting a company, and there’s no shortcut around that. But, importantly, he managed to retain most of his original stake in FTX. For most founders, by the time their company is worth $10 billion or more, they only own maybe 10% of it. If Sam had given away a normal amount of equity to VCs, he might have only gotten $2 billion from FTX instead of $10 billion. In some sense, 80% of the money he earned from FTX came purely from retaining equity.²

(Bill Gates might be another person who got richer than usual by retaining ownership.³)

Warren Buffett is an investing genius. But if he had only been an investing genius, he would have made a lot less money. In addition to earning high investment returns, he borrowed money at below-market interest rates using insurance float. That let him compound his returns much faster than he otherwise would have.

Long-Term Capital Management was an extremely successful hedge fund (at least until it blew up, but let’s not talk about that part). Much of its success didn’t come from its trading strategies, but from the fact that it negotiated unusually favorable deals with its banking partners.

A quasi-example: Instead of trying to make more money, try to (legally) pay less tax. This idea is not exactly overlooked, so it’s not a central example of what I’m talking about. But I have noticed that most “normal-level” rich people (with between $1 million and $10 million) don’t pay nearly enough attention to taxes. I know a lot of people who could save $10,000 to $100,000 per year with 20 hours of work or less.

In these examples, it’s not necessarily that the person (or company) came up with a trick that nobody else had thought of. Obviously every startup founder would love to retain as much of their stock as they can. But they usually have to choose between giving up equity and running out of funding, and they rightly choose to keep their company alive. My point isn’t that it’s easy. My point is that, on the margin, EA founders should probably pay more attention to retaining equity. Maybe a founder could do some extra work and find a way to keep 11% of their equity instead of 10%, without meaningfully reducing their startup’s chances of success. Getting that extra 1 percentage point of equity might be a lot easier than raising the company’s ultimate valuation by 10%.

Wealthy EAs care more about marginal dollars than ordinary wealthy people, and they can take advantage of that fact. They can look for (relatively) easy ways of getting money. I only spent about five minutes coming up with the examples I gave, so I’m sure there are plenty of other ideas.

Notes

For people with about average wealth, the best way to get more money is usually the obvious thing: get a higher salary. But there are still some neglected strategies. One that comes to mind is salary negotiation. ↩
The mathy way to say this is that, on a logarithmic scale, most of his wealth came from doing the hard work. But on a linear scale, it mostly came from retaining equity. If most people have logarithmic utility of money but you have linear utility, that changes how you should behave. ↩
Elon Musk did something sort of similar, where he got the Tesla board of directors to grant him highly leveraged call options. When Tesla stock went up, Elon Musk’s net worth went up by even more. ↩

Do I Read My Own Citations?

Fri, 08 Oct 2021 00:00:00 -0700

It has often been said that scholars don’t read their own citations. Out of curiosity, I decided to go through one of my longer essays to see how many of my citations I read.

(I actually did this exercise a while ago, around the time I published the original essay. Today I was going through my personal journal and found my notes on the exercise, and I thought it might be worth sharing publicly.)

The results:

My essay cites a total of 37 academic papers.
For 9 citations, I read the entire thing top to bottom and took notes to help me remember.
For another 6 citations, I skimmed them but didn’t read carefully or take notes.
For the remaining 22, I only read the abstracts.

I also replicated the results of 7 of the 37 citations. I didn’t do full replications, but I did reproduce at least one result from each paper. 6 out of those 7 were what you might call “trusting” replications: I got data from the study authors and used that to replicate their findings, but I’m still trusting that they constructed their data set correctly. For one of the 7 replications, I used an independent data set.

(There’s another level of reading that’s less rigorous than a replication but more rigorous than a full read-through: a “close reading” with the intentional goal of identifying mathematical or logical errors. I did not do that for any of my citations. Replicating a paper is (arguably) harder than doing a close reading, but I preferred to do outright replications because I find it more fun.)

Why did I approach the citations this way? Here’s why:

A minority of papers were directly relevant to my thesis, so I read those in full. Whether I read or skimmed a paper largely depended on how well it was written. I can barely force myself to read papers with bad prose, so I just skim those.
For most citations, I just wanted to know some specific fact, and I searched the literature until I found one that included the fact I wanted. For example: I wondered, “do mutual funds have any net exposure to the value factor?” I found a couple of papers that purportedly answered this question—Blitz (2017) and Lettau et al. (2018)—skimmed them to find the relevant tables, and then cited them, trusting that they were methodologically sound.
Another common cause of me not reading citations is that I’d read an interesting fact in an article or book, decide to put the fact in my essay, and then cite the same paper that that article or book cited. (Pretty much the classic “don’t cite Wikipedia, just read Wikipedia and then copy its citations” technique.)

As a heuristic, if multiple studies find similar results, that gives me more confidence in their conclusions without having to read them. It’s not perfect, but it’s a reasonable way to save time. For example, I haven’t read Fama & French (1992) beyond the abstract, but I know it’s been replicated lots of times.

Sure, in an ideal world, I should carefully analyze every paper I cite to check for mistakes. But given time constraints, it makes sense for me not to read most of the papers I cite. Still, I can see why bad papers often get widely cited—nobody reads them! I wish I had more time and energy to read more stuff. (I also wish most academic papers weren’t written so poorly. If they were better written, reading them might actually be fun.)

Summaries Are Important

Mon, 04 Oct 2021 00:00:00 -0700

Every informative essay or research paper should include a summary at the beginning. Write your summary with the expectation that most readers will ONLY read the summary. The summary should tell most readers everything they need to know. The body of the article only exists to provide context and supporting evidence.

(That was my summary. You can probably stop reading now.)

The summary of your article should describe your central conclusions, which is ultimately the point of your article. A summary is high-context/high-trust, while the body of an article is low-context/low-trust. A summary is high-context because you write it for readers who have enough context to understand what you’re talking about. It’s high-trust because it’s for readers who trust that you know what you’re talking about, and who you trust not to deliberately misinterpret you. Then you write the body of the article to give additional context, and to justify your conclusions to skeptical or inquisitive readers.

There are basically two types of articles that should have a summary:

In a research paper, the goal is to test some hypothesis. (That hypothesis doesn’t have to be empirical, e.g., your paper could prove a mathematical theorem.) Most readers only care about what the hypothesis is, and whether it’s true or false.¹ So put that in your summary.
In a persuasive essay, the central argument is usually pretty simple. Put the central argument in the summary. This might be be sufficient to persuade open-minded readers who have enough background to understand the argument. Then, in the body of the essay, explain the argument and provide supporting evidence.

If most readers only read the summary, that means you should spend much more time on the summary to make sure it conveys exactly the message you want. When I write an essay, I usually spend 5–10 times longer per word writing the summary than writing any other part.

Academic papers almost always include abstracts. An abstract is like a summary. Unfortunately, many (most?) papers don’t have good abstracts. Some common mistakes:

The abstract uses overly convoluted language, making it hard to understand. (An abstract/summary is high-context, but that doesn’t mean it has to be hard to understand.)
The abstract describes what the paper does, but doesn’t describe what conclusions it reaches.

Examples of bad and good summaries

I don’t want to pick on anyone, so I’ll write my own bad summary.

Luke Muehlhauser’s How Feasible Is Long-Range Forecasting? has an excellent summary, which I’ll talk about in a minute. But first, here’s my own (bad) summary of this article, written in the style of a typical academic paper’s abstract:

Long-term projects or endeavors depend on predictions of distant future events under low information and high uncertainty. This paper investigates historical evidence on long-term track records. There are significant concerns regarding historical forecasts with respect to quantification of degrees of confidence, null hypotheses, and forecaster behavior. Some difficulties are discussed for assessing the accuracy and determining the value of long-term forecasts. In conclusion, this organization’s current attitude toward long-range forecasting is provided.

This summary is bad for several reasons:

The sentences are long and hard to read.
It doesn’t actually explain the findings of the paper. “Some difficulties are discussed”? What difficulties, specifically? Don’t make me read the entire article to learn the article’s results.
It tries too hard to sound objective, which contorts the language. “[T]his organization’s current attitude toward long-range forecasting is provided”? Why not, “We provide our current attitude toward long-range forecasting”? Or, even better, instead of saying that you’ll provide your attitude, simply provide your attitude: something like, “We don’t know how to make good long-term forecasts, but we plan to continue making them so we can get better at it.”

Now let’s read Muehlhauser’s actual summary:

Long-range forecasts are often stated too imprecisely to be judged for accuracy. [More]

Even if a forecast is stated precisely, it might be difficult to find the information needed to check the forecast for accuracy. [More]

Degrees of confidence for long-range forecasts are rarely quantified. [More]

In most cases, no comparison to a “baseline method” or “null model” is possible, which makes it difficult to assess how easy or difficult the original forecasts were. [More]

Incentives for forecaster accuracy are usually unclear or weak. [More]

Very few studies have been designed so as to allow confident inference about which factors contributed to forecasting accuracy. [More]

It’s difficult to know how comparable past forecasting exercises are to the forecasting we do for grantmaking purposes, e.g. because the forecasts we make are of a different type, and because the forecasting training and methods we use are different. [More]

Take the first sentence of this summary:

Long-range forecasts are often stated too imprecisely to be judged for accuracy.

This sentence forms a complete thought and tells me something that I might not have known. I don’t have any way of knowing whether this statement is true, so the article needs to justify it. But if I trust that the author knows what he’s doing, I could stop reading at this point without losing much.

For most readers, those seven conclusions are all they need to know. Every point on this list is easy to understand and tells you something interesting. There’s no need to read the rest of the article. But if you want to know the details, you can follow the links or continue reading to see how the author came to those conclusions.

How to write a good summary

I don’t really know how to write a good summary. I know one when I see one, but writing is harder than reading.

Some strategies I use:²

Write a summary that matches the overall structure of the article. If your article includes three parts, write one sentence for each part.
If your summary covers several independent points, write them as a bulleted list, not a big paragraph. Bulleted lists are easier to read (in my experience).
Make each sentence as short as possible without losing any critical information.
If you must choose between precision and clarity, choose clarity. People often end up writing overly convoluted sentences because they’re trying to make their statements impossible to misinterpret. That’s okay sometimes, but don’t overdo it, especially not in a summary. Keep the summary clear, even if that means readers have to read between the lines a little.
Rewrite each sentence multiple times. Pick whichever version is clearest.

A lot of the advice for writing a good summary is the same as the advice for writing well in general. But even if you don’t meticulously comb through your entire article to make it as clear as possible, you should spend a lot of time time on your summary.

Notes

Obviously, a paper on an empirical subject cannot definitively prove it true or false. Rather, the paper provide evidence pointing to “true” or to “false”, and the reader wants to know which way the evidence points. ↩
I did not take all of my own advice in this essay. This essay is short, and these strategies are more relevant to long articles. ↩

Argumentative Tactics I Would Like to Stop Seeing

Mon, 04 Oct 2021 00:00:00 -0700

I realize I’m fighting a losing battle on this one and most people don’t care if they behave uncharitably toward their opponents. People are always gonna mock people they disagree with. But I don’t like it.

Argument from my opponent made a silly face

News articles about a particular person, especially when the writer wants to make that person look bad, will often include a cover photo of that person making a silly face.

Have you ever paused a movie during a dialogue scene and noticed that the speaker appeared to be making a silly face? It’s easy to take a video of a person speaking and pick out still frames where they look goofy. News articles love doing this because they can make someone look bad while still claiming to be objective.

Here are some examples of silly faces that I found in news articles with various political leanings:

Argument from my opponent sounds silly

People sometimes quote their opponents as mispronouncing certain popular phrases, and that’s their entire argument.

Examples:

“freeze peach”
“muh X” for various values of X (“muh jobs”, “muh freedom”)
“well, aktually”
“dead gaem” (I see this one a lot on gaming forums)
“they tuk er jerbs!” (I had no problem with South Park using this phrase because it was a joke. I do have a problem with people repeating it as if it’s an actual argument)
aNy TiMe SoMeOnE tYpEs LiKe ThIs

Argument from my opponent has unstylish facial hair

It is widely recognized that calling someone ugly is not an acceptable argumentative tactic. But for some reason, people make an exception for the term “neckbeard”.

My Experience Trying to Force Myself to Do Deep Work

Mon, 23 Aug 2021 00:00:00 -0700

Inspired by Applied Divinity Studies’ Unemployment Part 2

Many people, such as Cal Newport, say that you can only do about four hours of deep work per day. I am a lot worse at deep work than that.

When I worked full-time as a software developer, I tried pretty hard to avoid distractions and stay focused on work. At the end of each day, I made a quick estimate of how much I got done that day. I rated myself on a 5-point productivity scale. A fully productive day, where I spent the bulk of the day doing meaningful work, earned the full 5 points. My estimates were by no means objective, but according to my own perception, I scored 5 points on a total of 91 out of 602 work days (that’s 15%). A 5-point day usually meant I spent around four hours doing deep work, and most of the rest of the day doing important shallow work.

When I failed to do sufficient deep work, it usually wasn’t because I spent too much time on meetings and emails (although that did happen occasionally). Most of the time, I was simply too dumb to get enough useful work done.

Most of my work required having insights. If I’m trying to debug a problem, figure out how to extend some bit of code, or plan out a new feature, I need to have a series of small ideas about how to make progress. For example, if I’m looking for the source of a bug, I might have the idea, “The problem might be in this particular function. I’ll step through the function and look for any anomalies.” On my less productive days, I would simply fail to generate any such ideas, preventing me from making any progress.

On my worst days (which happened about 15% of the time), I couldn’t manage even simple tasks. This was more about focus than about insight. It would usually go like this:

Me: Here’s the task that we need to do. As you can see, I’ve opened the relevant file and I’m currently looking at it. We need to do X to this file. Brain: I don’t want to do that. Me: Come on, you know exactly what to do, you should just do it. Brain: No. Can we do something fun instead? Me: We have to do this. Brain: Well I’m not gonna.

Since I refused to get distracted and my brain refused to do the actual work, I’d end up staring at the screen doing nothing for about an hour. Eventually I’d give up on that and go read Hacker News or think about investing—things that feel work-adjacent but don’t qualify as genuine work.¹ After doing that for about 15 to 30 minutes, I’d feel guilty enough to go back to staring at my code while doing nothing.

What predicts productivity?

I tracked a bunch of variables to see if anything could predict whether I’d have a good or bad day. I looked at sleep, how much I ate, the glycemic index of my lunch², whether I went to the gym, and whether I had melatonin the previous night; none showed a statistically significant effect. The only thing that clearly mattered was whether I had caffeine. On a five-point scale, caffeine boosted by average productivity by about 0.7 points.

(I’m highly confident that I perform worse when I don’t get enough sleep. But my sleep schedule was consistent enough that I didn’t have a sufficiently large sample of sleep-deprived work days to show a visible effect.)

(I didn’t blind myself, so any positive results could be explained by a placebo effect. But I only got one positive result, and I wasn’t expecting to get it before I analyzed the data—subjectively, I felt about the same on caffeine days and non-caffeine days.)

About a year ago, when I started working full-time on independent research, the frequency of fully-productive days went up from 15% to 24% (53 out of 221). But median productivity declined, probably because I stopped trying to force myself to work when I didn’t want to.

My productivity tended to come in waves. I’d think of some interesting and digestible sub-problem, make a bunch of progress on it for a few days or weeks (depending on the size of the problem), figure out all the obvious stuff, and then lose steam and stop being productive for a while.

During waves of low productivity, the issue wasn’t that I didn’t know what to work on. I had plenty of ideas about important research areas. The problem was that I didn’t feel motivated to work on them. Usually, I’d only feel motivated to work on a problem shortly after I came up with it. Once the problem became stale, I’d lose interest. But sometimes I’d get a sudden second wind on an old stale problem. It would be nice if I knew what caused those second winds, or if I could find a way to reliably trigger them. But so far I haven’t figured out how to do that.

Notes

Some of the time spent on investing genuinely improved my life, and maybe even helped improve other people’s lives. But I also spent a lot of time doing obviously-useless things like checking the current value of my portfolio, or changing my hypothetical allocation to a particular strategy from 5% to 10% and then changing it back later.³ ↩
I didn’t have any precise way of measuring this. I just kept a list of common foods and their glycemic indexes, and then estimated the average glycemic index of my meal based on how much of each food I ate. ↩
I say hypothetical because I have a rule that I only change my actual allocation if I consistently believe I should change it for a while—maybe a month or longer (depending on the scope of the change). I want to avoid what you might call “tinkering bias”—the desire to tinker with my allocation because tinkering is fun, not because I’m actually improving it. So I rarely change my actual investments. ↩

Mission Hedgers Want to Hedge Quantity, Not Price

Wed, 18 Aug 2021 00:00:00 -0700

Summary: Mission hedgers want to hedge the quantity of a good, but can only directly hedge the price. As a motivating example, can we mission hedge climate change using oil futures or oil company stock? Based on a cursory empirical analysis, it appears that we can, and that oil stock makes for the better hedge. But this answer relies on some questionable data (either that, or my methodology is bad).

Introduction

The purpose of mission hedging is to earn more money in worlds where your money matters more. For instance, if you’re working to prevent climate change, you could buy stock in oil companies. In worlds where oil companies are more successful and climate change is worse, you make more money—at least in theory. But it might be hard in practice to find a good way to hedge climate change.

Let’s start with a simple example. Suppose you’re considering hedging climate change by buying oil futures. Does that work?

If people burn more oil, that directly contributes to climate change. You’d like to make money as the quantity of oil goes up. If you buy oil futures, you will make money as the price of oil goes up. The problem is, the quantity and price of oil aren’t necessarily related.

Before we talk about the economics of oil prices, let’s clarify some terminology.

Supply and demand

The basic drivers of supply and demand:

Consumers are willing to buy more of some product if the price is lower.
Producers are willing to sell more of the product if the price is higher.

In aggregate, consumers’ willingness to buy the product creates a downward-sloping demand curve, which indicates the quantity consumers will buy at any given price—or, conversely, the price they will pay for any given quantity. Similarly, producers’ willingness to sell gives us an upward-sloping supply curve. These two curves cross over at some point, and that point tells us the actual market price and quantity of a product.

A poorly-drawn illustration of what the supply and demand curves look like:¹

Demand for oil might increase if, say, people start owning more cars. If demand increases, both the price and quantity of the good go up:

Supply increases when produces are willing to sell more. For example, supply of oil could increase if oil companies discover a new source of oil. If supply increases, quantity goes up and price goes down:

And, naturally, supply and demand can move in the other direction:

If demand decreases, both quantity and price go down.
If supply decreases, quantity goes down while price goes up.

Back to mission hedging

So, can we mission hedge by buying oil futures?

If we own oil futures, we make money when the price goes up. But climate change gets worse when the quantity goes up. And there’s no obvious relationship between price and quantity. If demand changes, price and quantity move together. But if supply changes, price and quantity move oppositely.

If oil prices are mostly driven by changes in demand, we can mission hedge by buying oil futures. If prices are mostly driven by supply, we can hedge by short-selling oil. If supply and demand matter about equally, price and quantity will be uncorrelated and we can’t use oil futures as a hedge.

What’s the relationship between oil price and quantity in practice? I did about 10 minutes of research and found some data from the U.S. Energy Information Administration (EIA)’s Short-Term Energy Outlook. Based on this data, it appears that price and quantity are moderately correlated (r=0.67, p<1e-16)². If this is correct, we can at least moderately hedge climate change by buying oil futures.

(I’m not at all confident that this 0.67 figure is accurate. I found a different, seemingly-reasonable data set that shows a negative correlation—see Methodological concerns.)

Hedging with oil stocks

Can we do better by buying/short-selling stock in oil companies instead?

We want to make money when the quantity of oil goes up. It takes four steps to get from quantity to stock return:

The revenue of an oil company roughly equals [quantity of oil sold] x [price of oil].
Profit equals revenue multiplied by profit margin.
Companies can return value to shareholders by paying out dividends and buybacks.
The price of a stock (theoretically) equals the long-run value of all profit that will be returned to shareholders. Stock return equals dividends plus buybacks plus price change due to changes in future expectations.

Each step in this chain [quantity -> revenue -> profit -> dividends -> stock return] leaves some wiggle room. It’s not obvious that any step is correlated with the next step, much less that the first step correlates with the last.

In a perfectly competitive market, increases in revenue should incentivize competition, driving down profit margins such that profits don’t change at all. So we should see no correlation between oil quantity and stock return. But markets are not perfectly competitive, so that argument might not apply in practice.

We could answer this empirically by looking at the correlation between oil stock returns and the quantity of oil consumed.

I took the Energy Information Administration data from before and combined it with historical returns for the Vanguard Energy Fund (VGENX). They had a correlation of 0.85 (p<1e-36)—much stronger than the relationship between the oil quantity and oil futures prices. (A correlation of 0.85 means changes in the quantity of oil consumption explain 72% of the variation in VGENX’s price (0.85^2 = 0.72). So we might say we can hedge 72% of climate change by buying oil stock.)

How much of this comes from a correlation between oil consumption and stocks in general? It happens that oil consumption correlates with the S&P 500³ at 0.83 (p<1e-33). Oil stocks barely do better, with r=0.85. However, a combined portfolio of 50% S&P 500, 50% VGENX correlates with oil consumption at r=0.96 (p<1e-68). So according to this, VGENX does provide some hedging power beyond what the S&P 500 gives.

But as with the data on oil futures, we have some reason to question this empirical result.

Methodological concerns

In an earlier draft of this essay, I didn’t realize EIA had data on world consumption, so I used the closest proxy I could find—non-OPEC oil production—combined with their data on WTI real prices (GDP deflated). I found a correlation of –0.21 (p < 0.07). Later, I found EIA’s data on world oil consumption and used that instead.

I would expect these two pairs of datasets to produce slightly different correlations, but I’m surprised by just how different they look. Why does one pair show a meaningful positive correlation, and the other shows a weak negative correlation? The first suggests mission hedging works pretty well, and the second suggests it barely works at all. Which one is correct?

Where does the difference come from? EIA-reported real oil prices correlate with investing.com-reported oil futures prices at r=0.95 (p<1e-42). World oil consumption correlates with non-OPEC oil production at r=0.15 (p<0.17). That’s weirdly low—according to EIA, non-OPEC countries produce 60% of the world’s oil, so their production should strongly relate to world consumption. This seems to suggest that non-OPEC oil production is not a good metric to look at. But also, even though EIA oil prices look very similar to oil futures prices, and oil futures prices moderately correlate with world oil consumption (r=0.67, p<1e-16), EIA oil prices only weakly correlate with oil consumption (r=0.11, p<0.33). Part of this happens due to a difference in time horizons: the VGENX, world consumption, and futures data go back to 1990, but EIA oil prices and non-OPEC production only goes to 2001. If I only include data starting in 2001, futures prices only correlate with world consumption at r=0.26 (p<0.02). This seems weird both in that it’s still a lot stronger than the EIA price <> world consumption correlation, and it’s much worse than the full time series (I’d expect shortening the time series to increase the p-value, but not to change the correlation much).

World oil consumption seems more relevant than non-OPEC production. But slide 7 on an EIA slideshow claims “price increases have coincided with lower consumption”, which suggests the correlation should be negative, not positive. That makes me suspicious of the data on world oil consumption, or suspicious of my methodology.

It’s possible that there’s something wrong with my methodology. On the one hand, I don’t have much experience with data analysis, so I could have made a mistake. On the other hand, all I’m doing is correlating two lists of numbers, how hard could that be?

In conclusion, I’m confused.

Questions for future research

Why do these different data sources produce seemingly contradictory results?
What is the portfolio with the strongest correlation to oil consumption?
We actually want to hedge global temperature, which isn’t perfectly related to oil consumption. Is there a better hedge we could use?
How can we apply this same approach to other areas worth hedging, such as factory farming or AI progress?

Notes

In high school AP Economics, we had to draw graphs like this about three thousand times. I’m leaving out a few details to make the graph simpler—if I drew this on the AP test, I’d probably fail the test. ↩
I arrived at this number by downloading EIA’s quarterly data on total world consumption of oil and other liquid fuels and investing.com’s data on WTI oil futures prices, and then calculating the correlation. ↩
Instead of looking at the S&P 500, which only includes the largest US stocks, I’d rather compare oil consumption to the global stock market. But I couldn’t quickly find quarterly global stock returns going back to the beginning of 1990. ↩

How Do AI Timelines Affect Giving Now vs. Later?

Mon, 02 Aug 2021 00:00:00 -0700

Cross-posted to the Effective Altruism Forum.

How do AI timelines affect the urgency of working on AI safety?

It seems plausible that, if artificial general intelligence (AGI) will arrive soon, then we need to spend quickly on AI safety research. And if AGI is still a way off, we can spend more slowly. Are these positions justified? If we have a bunch of capital and we’re deciding how quickly to spend it, do we care about AI timelines? Intuitively, it seems like the answer is yes. But is it possible to support this intuition with a mathematical model?

TLDR: Yes. Under plausible model assumptions, there is a direct relationship between AI timelines and how quickly we should spend on AI safety research.

Let’s start with some simplifying assumptions. We want to distill reality into a simple framework while still capturing the key features of the world that we care about. We could do this in lots of different ways, but here’s an attempt:

We start with some fixed amount of capital that we can choose how to spend. At some point in the future, an artificial general intelligence (AGI) will be developed. This AGI will either be friendly or unfriendly. If it’s unfriendly, everyone dies. We don’t know exactly when AI will be developed, but we have at least an expected timeline.

To ensure the AGI is friendly, we will need to do some amount of AI safety research, but we don’t know exactly how much. Once per decade, we decide how much to spend on safety research. Any money we don’t spend can be invested in the market. Then, after AGI emerges, if it’s friendly, we can spend any leftover capital on whatever amazing things will presumably exist at that point.

(That’s just a high-level explanation; I’m skipping over the mathy bits. Appendix A contains the full details.)

If we do enough research by the time AGI is developed, everything works out okay. If we don’t do enough research, we go extinct. The objective is to choose the spending schedule that maximizes the welfare we get out of our remaining capital after AGI emerges. (If we go extinct, we get no welfare. To meet our objective, we need to spend money on preventing unfriendly AI.)

Philanthropists face this basic tradeoff:

If we spend more now, we’re more likely to get enough research done in time if AGI arrives soon.
If we spend more later, we earn more return on our investments. That way, (a) we can do a greater total amount of research, and (b) we will have more money left over at the end to spend on good things.

If we run the math on this model, this is what it says to do:

If AGI is very unlikely to emerge this decade, don’t spend any money on research yet. Invest all our capital.
Once we get close to the median estimated date of AGI (to within a few decades), start spending around 30% of our capital per decade / 3% per year.
In the decades after the median date of AGI (assuming AGI hasn’t emerged yet), reduce the spending rate.

The model’s optimal spending rate varies based on the median date of AGI:

AI Impacts’ review of AI timeline surveys found that survey respondents estimated a 50% chance of AGI by around 2050. Given that timeline, the model recommends a peak spending rate of 3% per year.
For a much longer median timeline of 100 years, the model suggests spending nothing for the first 50 years, then spending around 1% per year after that.
If we assume a very short timeline of only one decade, the model says to spend 5% per year for the first decade, and 1–2% per year after that if AGI still hasn’t appeared.

Obviously this is a toy model that makes lots of unrealistic simplifications. For instance, you can’t instantly cause more research to happen by throwing more money at it. But the model corroborates the intuitive notion that if AI timelines are shorter, then we should spend more quickly.

I have a hard time trusting this intuition on its own. The question of how much to spend now vs. later is really complicated: it’s affected by the exponential growth of investments, the decay in expected value of future worlds where extinction is a possibility, and the complex relationship between research spending and productivity. Humans don’t have good intuitions around that sort of thing. A lot of times, when you do the math, you realize that your seemingly-reasonable intuition was totally off base. So even though this model has many limitations, it confirms that the intuition is not a mistake arising from a failure to comprehend exponential growth. The intuition could still be wrong, but if so, it’s not because of a basic math error¹.

It’s also noteworthy that under this model, even with an aggressive AGI timeline, the optimal spending rate doesn’t exceed 5% per year.

So, do short timelines mean we should spend more quickly? Yes. Maybe. If this model is correct. Which it’s not. But even if it’s wrong, it might still be correct in the ways that matter.

Python source code is available here. It’s hard to properly describe the model’s output in words, so you might find it more illustrative to download the source code and play with it.

Appendix A: Some properties of this model

This model embeds all of the assumptions listed below, any of which could easily be wrong. This list does not cover every assumption, just the explicit ones plus all the implicit ones I could think of in five minutes.

We represent all donors supporting AI safety research. We can collectively decide on the optimal spending rate.
We decide how much to spend once per decade. (This makes the problem tractable. If we could spend on a yearly basis, the model would have too many independent variables for Python’s optimization library to handle.)
We only care about spending decisions for the next two centuries. Ignore anything that happens after that. (Again, this is to make the problem computationally tractable.)
Prior to the emergence of AGI, we don’t want to spend money on anything other than AI safety research.
After AGI is developed, we get an amount of utility equal to the logarithm of our remaining capital.
It’s possible to instantly convert money into research at any scale.
The date of AGI follows a log-normal distribution. A log-normal distribution has some relevant properties:
1. It’s fat-tailed, which means the longer we go without developing AGI, the more additional time we expect it to take.
2. Unlike, say, an exponential distribution, a log-normal distribution allows for a non-trivial probability that our median estimate is off by an order of magnitude. If our median timeline is 30 years, then we might still think it’s plausible that AGI could take 300 years. (Exactly how plausible depends on what standard deviation we use.)
3. On the other hand, unlike, say, a Pareto distribution, our probability quickly diminishes as we move out by more orders of magnitude. For example, if we estimate there’s a 50% chance of AGI within 30 years and a 95% chance within 300 years, that implies an extremely confident 99.995% chance of AGI by the year 5000.
Research spending required to avert catastrophe follows a log-normal distribution, so it also has the properties listed above.

This model has six input variables:

how much capital we start with
the investment rate of return
median research spending required to make AGI safe²
standard deviation of research spending required
median date of AGI
standard deviation of date of AGI

Appendix B: Alternative models

Tom Sittler’s “The expected value of the long-term future”³ presents some models that treat x-risk as an ongoing concern that can be reduced by deliberate effort (I will refer to these as “periodic models”) rather than a single one-off event that occurs at an unknown time. I find his models more realistic, plus they’re more similar to the Ramsey model that comes up a lot in economics literature.

About a year prior to writing this essay, I spent quite some time working with periodic models like the ones Sittler gives. The problem with them is that they’re much harder to solve. I couldn’t find optimal solutions for any of them, and I couldn’t even find approximate solutions with a convex optimization program.⁴

As a way around this, I tried restricting the decision to only two points in time: now or one century from now. This allowed me to preserve the basic structure these models while making it possible to find an optimal solution. But this restriction is highly limiting, which means the models’ optimal solutions tell us little about what we should do in reality.

The new model I presented above has some nice properties. To my knowledge, no previous model achieved all of these:

Civilization has a nonzero probability of going extinct in any particular year, but the probability of survival does not quickly approach zero.
We must decide how much of our remaining budget to spend in each period. We cannot reduce our decision to a binary “fund x-risk or don’t”.
The optimal spending schedule is feasible to find.

Of the four models that Sittler presented:

“Trivial model” (section 3.1.2) has property 3, but not 1 or 2;
“Constant risk, temporary effects” (section 3.1) has property 2, but not 1 or 3;
“Variable risk, temporary effects” (section 3.2) and “Constant risk, lasting effects” (section 3.3) have properties 1 and 2, but not 3.

The models in my previous essay are similar to Sittler’s. They gain solvability (property 3) at the expense of periodic decisionmaking (property 2).

However, the model in this essay does make some fairly specific assumptions, as discussed in Appendix A. Perhaps the most important assumptions are (a) there is only a single potential extinction event and (b) the long-term value of the future is bounded.

In an earlier draft of this essay, my model did not assign value to any capital left over after AGI emerges. It simply tried to minimize the probability of extinction. This older model came to the same basic conclusion—namely, shorter timelines mean we should spend faster. (The difference was that it spent a much larger percentage of the budget each decade, and under some conditions it would spend 100% of the budget at a certain point.⁵) But I was concerned that the older model trivialized the question by assuming we could not spend our money on anything but AI safety research—obviously if that’s the only thing we can spend money on, then we should spend lots of money on it. The new model allows for spending money on other things but still reaches the same qualitative conclusion, which is a stronger result.

None of these models is perfect, but some are more realistic than others. Which one is more realistic largely depends on an important question: what happens after we develop AGI? For example:

Will the AGI behave like a better version of a human, allowing us to do all the stuff we would have done anyway, but at a faster rate? Or will it be so radically better as to make the world unrecognizable?
Will the AGI be able to prevent all future x-risks, or will we still need to worry about the possibility of extinction?
Does it matter how much capital we have? If we invest more now, that might give the AGI a useful headstart. But the AGI might so radically change the economy that the state of the economy prior to AGI won’t matter, or altruistic capital might become (relatively) useless.

The answers to these questions could meaningfully change how much we should be spending on AI safety (or on other forms of x-risk mitigation).

It’s at least plausible that the world economy allocates far too little to x-risk, therefore thoughtful altruists should spend their entire budgets on x-risk reduction. But the same could be argued for other effective and neglected causes such as farm animal welfare, so you have to decide how to prioritize between neglected causes. And that doesn’t get around the problem of determining the optimal spending schedule: even if you should spend your entire budget on x-risk, it doesn’t follow that you should spend the whole budget now.

Notes

Unless, of course, my model contains the same math error. Which is entirely possible. ↩
We could parameterize the distribution using the mean rather than the median, but I find medians a little more intuitive when working with log-normal distributions. ↩
Published 2018-01-02. Accessed 2021-08-02. ↩
Last year I spent a good 200 hours trying to figure out how to model this problem. Then, after not working on it for a year, I suddenly get an idea and write up a working program in an hour. Funny how that works. ↩
On the old model, the only downside to spending now rather than later was that you lose out on investment returns, so you can spend less total money. When investments could earn a relatively low return and timelines were short, the model would propose spending a little each decade and then spending the entire remaining budget at a specific point, usually on or shortly before the decade of peak risk. When investments could earn a high return or timelines were long, the model would never spend the whole budget at once, preferring to always save some for later. ↩

Metaculus Questions Suggest Money Will Do More Good in the Future

Wed, 21 Jul 2021 00:00:00 -0700

Cross-posted to the Effective Altruism Forum.

Update 2021-10-06: I believe I was overconfident in my original interpretations of these Metaculus questions. Some EA Forum commenters pointed out alternative interpretations of people’s answers that could allow us to draw orthogonal or opposite conclusions. For example, on question 1, Metaculus users might predict GiveWell’s top charities to drop off the list by 2031 not because better charities are discovered, but because current charities run out of room for more funding.

In the giving now vs. later debate, a conventional argument in favor of giving now is that people become better off over time, so money spent later will do less good. But some have argued the opposite: as time passes, we learn more about how to do good, and therefore we should give later. (Or, alternatively, we should use our money now to try to accelerate the learning rate.)

Metaculus provides some evidence that the second argument is the correct one: money spent later will do more good than money spent now.

This evidence comes from two Metaculus questions:

A brief explanation for each of these and why they matter:

On question 1: As of July 2021, Metaculus gives a 30% probability that one of GiveWell’s 2019 top charities will be ranked as the most cost-effective charity in 2031. That means a 70% chance that the 2031 charity will *not** be one of the 2019 recommendations. This could happen for two reasons: either the 2019 recommended charities run out of room for more funding, or GiveWell finds a charity that’s better than any of the 2019 recommendations. This at least weakly suggests that Metaculus users expect GiveWell to improve its recommendations over time.

On question 2: Metaculus estimates that GiveWell’s top charity in 2031 will need to spend $430 per life saved equivalent (according to GiveWell’s own analysis). For comparison, in 2019, GiveWell estimated that its most cost-effective charity spends $592 per life saved equivalent. (These figures are adjusted for inflation.)

As with question 1, this does not unambiguously show that GiveWell top charities are expected to improve over time. Perhaps instead Metaculus expects GiveWell’s estimate is currently too pessimistic, and it will converge on the true answer by 2031. But the cost reduction could also happen because GiveWell top charities truly get more effective over time.

Some caveats:

These Metaculus answers only represent the opinions of forecasters, not any formal analysis. (Some forecasters may have incorporated formal analyses into their predictions.)
Neither question directly asks whether money spent in 2031 will do more good than money spent now. (I don’t know how to operationalize a direct question like that. Please tell me if you have any ideas.)
These questions only ask about GiveWell top charities. Even if GiveWell recommendations become more effective over time, the same might not be true for other cause areas.

Reverse-Engineering the Philanthropic Discount Rate

Fri, 09 Jul 2021 00:00:00 -0700

Summary

How much of our resources should we spend now, and how much should we invest for the future? The correct balance is largely determined by how much we discount the future. A higher discount rate means we should spend more now; a lower discount rate tells us to spend more later.
In a previous essay, I directly estimated the philanthropic discount rate. Alternatively, We can reverse-engineer the philanthropic discount rate from typical investors’ discount rates if we know the difference between the two. [More]
In theory, people invest differently depending on what discount rate they use. We can estimate the typical discount rate by looking at historical investment performance. But the results vary depending on what data we look at. [More]
We can also look at surveys of experts’ beliefs on the discount rate, but it’s not clear how to interpret their answers. [More]
Then we need to know the difference between the typical and philanthropic discount rates. But it’s difficult to say to what extent philanthropists and typical investors disagree. [More]
Some additional details raise more concerns about the reliability of this methodology. [More]
Ultimately, it looks like we cannot effectively reverse-engineer the philanthropic discount rate, even if we spend substantially more effort on the problem. But under some conditions, we prefer to give later as long as we discount at a lower rate than non-philanthropists, which means we don’t need to make precise estimates. [More]

Cross-posted to the Effective Altruism Forum.

Summary
Contents
Introduction
Estimating the typical discount rate $\delta$
Getting from the typical discount rate to the philanthropic discount rate
Additional considerations
- Rate of risk aversion ($\eta$)
- Why is the risk-free rate so low?
Conclusion
Appendix
- Appendix A: Optimal consumption under uncertainty
- Appendix B: Details on Harvard endowment annual report
Notes

Introduction

Last time, I attempted to estimate the philanthropic discount rate. We should discount according to the probability that our resources become worthless in the future. I identified three main reasons why this might happen: existential catastrophe, expropriation, and value drift. Then I estimated the probabilities of each of these individually.

As an alternate approach, what if we reverse-engineer the philanthropic discount rate (call it $\delta_P$) from the typical discount rate (call it $\delta$)?

We can find $\delta_P$ as a combination of:

the typical discount rate $\delta$
the difference between $\delta$ and $\delta_P$

For example, if we know that $\delta = 2\%%$ and $\delta_P$ is 0.5 percentage points less than $\delta$, then $\delta_P = 1.5\%$.

This approach has some advantages estimating $\delta_P$ directly. We can only speculate as to the risk of existential catastrophe, expropriation, or value drift. But we can empirically observe $\delta$ by looking at financial markets. Then all we need to know is how philanthropists and ordinary investors differ.

Estimating the typical discount rate $\delta$

We can estimate most investors’ $\delta$ by looking at investment returns. According to standard economic theory, we can determine $\delta$ if we know four things:

The market rate of return
The volatility of the market
People’s appetites for risk
How much people consume vs. invest

In this section, I will avoid the mathematical details, but I provide the exact formula in Appendix A.

Let’s see if we can use this formula to estimate $\delta$ using historical data.

(Short answer: We can’t—it’s too hard to pin down the values of the four inputs.)

Deriving $\delta$ from GDP and market investment return

At a national scale, the consumption rate $g$ equals the GDP growth rate, and the investment rate represents the aggregate investment return achieved by capital markets. We can use these to estimate $\delta$.

The Rate of Return on Everything, 1870-2015 (henceforth “RORE”) provides equity, bond, and housing returns, as well as GDPs, for 16 countries back as far as 1870.

Table 1 includes:

Real GDP growth $g$
Mean $\mu$ and standard deviation $\sigma$ of log real returns for the weighted combination of all investment assets in the country (weighting determined by RORE)
The derived value of $\delta$ for various values of $\eta$, where $\eta$ is the rate at which marginal utility diminishes with consumption (also known as the rate of relative risk aversion)

Table 1: RORE 1870–2015 by country

	$g$	$\mu$	$\sigma$	$\delta$
$\eta$				$\eta$=0.5	$\eta$=1	$\eta$=1.5	$\eta$=2	$\eta$=3
AUS	3.37	5.47	7.23	3.9	2.1	0.5	-1	-3.6
BEL	2.26	5.90	9.11	4.9	3.6	2.6	1.8	0.8
CHE	2.45	5.29	6.82	4.1	2.8	1.7	0.6	-1.1
DEU	3.83	6.14	6.32	4.3	2.3	0.4	-1.3	-4.6
DNK	2.74	6.93	6.43	5.6	4.2	2.9	1.7	-0.5
ESP	3.02	4.10	8.62	2.7	1.1	-0.3	-1.6	-3.5
FIN	3.91	8.03	15.52	6.4	4.1	2.5	1.4	1.1
FRA	2.38	4.48	8.75	3.4	2.1	1	0.1	-1.1
GBR	2.32	4.28	8.02	3.2	2	0.9	0	-1.4
ITA	3.12	4.62	6.72	3.1	1.5	0	-1.4	-3.8
JPN	4.29	5.04	7.45	3	0.8	-1.3	-3.3	-6.7
NLD	3.48	5.29	6.92	3.6	1.8	0.1	-1.4	-4.2
NOR	3.28	6.64	7.26	5.1	3.4	1.8	0.3	-2.1
PRT	2.94	5.04	8.88	3.7	2.1	0.7	-0.4	-2.2
SWE	3.06	6.77	8.62	5.3	3.7	2.3	1	-0.9
USA	3.12	5.51	8.39	4	2.4	0.9	-0.4	-2.4

Table 2: RORE 1870–2015, summary statistics

	$g$	$\mu$	$\sigma$	$\delta$
$\eta$				$\eta$=0.5	$\eta$=1	$\eta$=1.5	$\eta$=2	$\eta$=3
median	3.09	5.38	7.74	3.95	2.2	0.9	-0.2	-2.15
mean	3.1	5.6	8.19	4.14	2.5	1.04	-0.24	-2.26

(These $\delta$s are the median/mean values from Table 1, rather than calculated from the mean/median $g, \mu, \sigma$.)

How should we measure consumption?

In the above tables, I used GDP as the measure of consumption because it’s the most well-known metric (and the only consumption metric that comes with the RORE data set). But it’s not obvious that we should use GDP. Arguably, we should use net national income, which excludes depreciation of manufactured capital. Or perhaps we should use GDP minus investment.

Subtracting investment from GDP, leaving only the “consumption-y” part, reduces $g$ by about 20% (World Bank, 2020). This increases $\delta$, and the magnitude of the increase depends on $\eta$.

Table 3: RORE 1870-2015, subtracting 20% from GDP

	$g$	$\mu$	$\sigma$	$\delta$
$\eta$				$\eta$=0.5	$\eta$=1	$\eta$=1.5	$\eta$=2	$\eta$=3
median	2.47	5.38	7.74	4.25	2.9	1.65	0.9	-0.3
mean	2.48	5.6	8.19	4.45	3.11	1.98	1.01	-0.42

Correlation between r and g

According to this model, $r_t$ is proportional to $g_t$ for any period $t$, so they should be perfectly correlated. That is to say, people’s spending in a particular period depends on how their investments performed. They’re willing to spend more when their investments perform better, and vice versa.

In real life, $r_t$ and $g_t$ are only weakly correlated. In the RORE data set, they had a correlation of only 0.14 (Online Appendix Q, Table A.26). So aggregate consumption has little to do with investment return.

Our model predicts perfect correlation between $r_t$ and $g_t$, but this prediction is not even close to true. That casts doubt on the reliability of this model.

Some other caveats

These figures for market return represent gross return, not net. If we used net returns instead, we would get a higher value for $\delta$ (assuming $\eta > 1$).
Most investors cannot practically invest in a representative sample of the entire market, because doing so requires buying diversified real estate holdings. A less-diversified portfolio would probably not have lower expected return, but it would have greater volatility, resulting in a lower value of $\delta$ (assuming $\eta > 1$).

Deriving $\delta$ from endowment portfolios

We can estimate $\delta$ by looking at how universities manage their endowments. This approach has a few advantages:

Universities are legally required to publish their financial data.
Universities often intend for their endowments to last for centuries.
Large universities typically hire teams to manage their endowments, and these teams make intentional, strategic choices about how to spend endowment money over time. (Some of them might even explicitly use the same model as I use to inform their decision-making.)

Let’s use the Harvard endowment as an example. Harvard provides annual financial reports going back to fiscal year 2003.

(To get a more complete perspective, we could combine data from multiple universities, but for now I will only look at the Harvard endowment.)

Over 2003-2019, Harvard had an average annual consumption rate $\lambda$ of 4.2%. If we define $\lambda_G$ as the consumption rate when excluding gifts, we get an average $\lambda_G$ of 3.2%. (For both $\lambda$ and $\lambda_G$, the median equaled the mean to within 0.1 percentage points). As explained in Appendix A, we can use the consumption rate to derive the discount rate.

See Appendix B for the full table of yearly consumption rates.

I measure both $\lambda$ and $\lambda_G$ because it is unclear how we should treat gifts. If we treat gifts simply as part of the portfolio, we get $\lambda$. But arguably, if the endowment receives outside income in the form of gifts, that should not count as part of the portfolio, in which case we can use $\lambda_G$ instead. Ideally, we would treat the gifts as coming from an external asset, determine the value of that asset, and count it as part of the endowment; but we don’t have enough data to properly do this.

I calculate the consumption rate $\lambda$ (or $\lambda_G$) using the expenditures for the current year divided by the value of the endowment at the end of the current year. Arguably, we should use the value of the endowment at the beginning of the year, because the university executives make budget decisions based on the endowment’s value at the beginning of the year, not the end. This method would give a median/mean $\lambda$ of 4.4% and a median/mean $\lambda_G$ of 3.3%/3.2%, respectively.

The following table gives implied $\delta$ for various values of $\eta$, using $\mu = 6\%, \sigma = 7\%$ (which approximately equal the long-run market return and standard deviation according to RORE).

Table 4: Implied δ from Harvard endowment statements

$\lambda$	0.5	1	1.5	2	3
3.2	4.7	3.3	1.9	0.7	-1.3
4.4	5.3	4.5	3.8	3.2	2.5

Universities have sources of revenue other than the endowment (most notably including tuition and research funding). A proper analysis would consider all source of revenue and all expenditures. However, as with gifts, we cannot easily model these other revenue sources as capital assets, which makes it harder to deduce $\delta$ using this model. The above analysis treats the endowment as an isolated system that doesn’t know anything about the rest of the university’s financial activities.

Deriving $\delta$ from public companies’ shareholder yield

Let’s look at how publicly-traded companies make consumption choices. To simplify, when companies earn income, they can do two things with it: invest it back into the company (via R&D, acquisitions, etc.), or allow shareholders to “consume” it by paying out dividends or buybacks.

In some sense, dividends don’t count as consumption, because shareholders might invest the money back into other assets. But it can still be considered consumption from the perspective of the company because the company can no longer invest with that money, and shareholders now have the money to use however they like. We could say that money feeds into shareholders’ utility functions, and the company is agnostic as to how shareholders use the money, even if they invest it.

According to Yardeni (2020), “S&P 500 Buybacks & Dividends”, the S&P 500 shareholder yield (that is, dividend yield plus buyback yield) from 1998 to 2020 has varied from about 2% to 7%, with an average around 4.5%. (Yardeni does not report exact figures; I’m estimating these based on the graph on page 9.)

Shareholder yield equals dividends + buybacks divided by market capitalization. Is that really the right denominator to use? Market cap represents the discounted present value of the market, not the capital of the underlying companies per se. Arguably, we should use total assets on the denominator instead (calculating shareholder yield as (dividends + buybacks) / assets), or perhaps balance sheet capital (which can be defined in several ways; one such definition is property, plant, and equipment + current assets - debt in current liabilities - cash and short-term investments). Using any of these would give a higher consumption rate.

I don’t believe any denominator based on a company’s balance sheet properly reflects the value of the capital a company possesses. Companies hold a great deal of value in non-balance-sheet assets such as human capital. In some sectors (such as technology), the majority of a company’s assets do not appear on the balance sheet. The market cap of a company represents something like the net value of all its capital, whether physical, human, or otherwise. I’m not entirely convinced that we should use market cap as the definition of “capital”, but it seems better than using anything else.

If we assume a 4.5% shareholder yield, along with $\mu = 5\%, \sigma = 17\%$ for the global equities market¹², we can construct this table of possible values of $\delta$ depending on $\eta$:

Table 5: Implied δ from public companies' shareholder yield

$\lambda$	0.5	1	1.5	2	3
4.5	5.2	4.6	4.8	5.7	9.6

The substantially higher volatility of equities (compared to the global market portfolio, which also includes bonds and real estate) results in a higher estimate for $\delta$.

Deriving $\delta$ from GDP growth

Liu (2012), Inferring the rate of pure time preference under uncertainty, uses similar methodology to estimate $\delta$ as a function of GDP growth, volatility in GDP growth, and the risk-free rate of return, calculated with US data 1889 to 1978 (taken from Mehra & Prescott (1985), The Equity Premium: A Puzzle). They find that, for $\eta \le 1$, $\delta$ “lies within ±1% from zero”; and for $\eta > 1$, $\delta$ “tends to be negative.”³

Experts’ beliefs about $\delta$

Drupp et al. (2017), Discounting Disentangled, presents a survey of experts asking their beliefs about the value of the discount rate.

Before proceeding, we should clarify two terms: (1) the utility discount rate $\delta$ and (2) the rate of pure time preference. A pure time preference represents the extent to which we discount future beings merely because they live in the future. Many philosophers argue (and I would agree) that we should not admit any pure time preference. The utility discount rate $\delta$ includes the rate of pure time preference, but it can also account for other reasons to discount the future, such as the possibility of extinction or expropriation.

Unfortunately, Drupp et al. conflate between these two terms. In their survey, they explicitly ask respondents to estimate the “[r]ate of societal pure time preference (or utility discount rate)”, as if they mean the same thing. Based on the descriptions in the paper, the authors appear to assume that a pure time preference is the only reason to discount future utility. But it is unclear how survey respondents might interpret the prompt, and they probably interpret it in different ways, which makes it difficult to say how we can use the responses. The modal discount rate was 0%, which at least suggests that many participants interpreted the question as asking for the rate of pure time preference.

Surveyed experts reported a mean discount rate of 1.1% and a median of 0.5%. It’s not clear whether we should interpret these as the utility discount rate or as the rate of pure time preference (or perhaps as a weighted average of the two, if some survey respondents used one interpretation and some used the other).

We could also attempt to derive the utility discount rate from experts’ beliefs about the social discount rate $r$ using this formula: $\delta = r - \eta g + 1/2 \eta (1 + \eta) \sigma^2$ (taken from Gollier et al. (2016), Declining discount rates: Economic justifications and implications for long-run policy, section 2.1). Using the median $\sigma$ from RORE and experts’ median estimates gives a $\delta$ of 1.0%. But this move seems questionable because we don’t know if surveyed experts would agree with this estimate for $\sigma$.

Getting from the typical discount rate to the philanthropic discount rate

In the previous part, we attempted to estimate $\delta$ using several methods. These methods did not produce consistent answers. But if we did manage to determine the value of $\delta$, how to we then derive the philanthropic discount rate $\delta_P$?

Most people behave as though they have a positive pure time preference. If we know the average rate of pure time preference (call it $\rho$), we can calculate $\delta_P$ as $\delta - \rho$. This gives us the “patient” discount rate, where we only discount due to empirical factors such as the probability of extinction, not due to a pure time preference.

But well-informed philanthropists might substantially disagree with most people about things like the probability of extinction. In that case, we might estimate a different discount rate.

If we assume factual agreement + pure time preference

We can break down the discount rate into the patient discount rate plus the rate of pure time preference. The patient discount rate tells us the extent to which we should discount based on empirical factors such as the probability of extinction or the probability that we lose access to our funds. If we assume everyone agrees on the empirical discount rate, then we can determine the value of the philanthropic discount rate (which equals the patient discount rate) by taking the typical discount rate and subtracting the rate of pure time preference.

What is the rate of pure time preference?

Unfortunately, research on the rate of pure time preference has produced mixed results.

Discounting Disentangled, discussed previously, gave expert opinion on the discount rate, but it did not clearly distinguish between the utility discount rate and the rate of pure time preference.

Frederick (2003), Measuring Intergenerational Time Preference: Are Future Lives Valued Less?, reviewed prior surveys that attempted to measure pure time preference. Then it conducted a series of new surveys, and found that people’s reported pure time preference varied depending on the exact question asked. Overall, average answers varied from 0% to 6% depending on survey wording. So it’s not clear whether most people exhibit a pure time preference at all, much less what rate they do exhibit. If most people do exhibit a 0% rate, that means the philanthropic discount rate equals the typical discount rate. But the two rates might differ by as much as six percentage points.

For more on the problems with observing pure time preference, see Frederick et al. (2002), Time Discounting and Time Preference: A Critical Review.

Non-factual disagreements other than pure time preference

Philanthropists and typical investors might discount at different rates for reasons other than pure time preference, even if they agree about all empirical facts. Philanthropists might naturally differ from typical investors for two main reasons:

Typical investors don’t care about value drift.
Typical investors do care about dying.

Value drift occurs when an altruist becomes less altruistic over time. If I stop spending money to do good, then from an altruistic perspective, my money is being wasted. So I should discount my future spending based on the probability that this will happen. But self-interested investors shouldn’t behave this way. Any money that I freely spend must be in accordance with my values at the time (assuming I’m behaving rationally), so I still consider it valuable from a self-interested perspective.

But unlike altruistic investors, typical investors don’t care as much about what happens to their money after they die. Many investors do want to ensure their children or grandchildren can lead good lives, but it often makes sense for them to discount the future based on the probability that they die before then. Whereas for an altruist, death does not much diminish the value of money. So this gives a reason why typical investors might discount the future when altruists don’t.

If we don’t assume factual agreement

When I directly estimated the philanthropic discount rate, I identified three key reasons to discount: extinction/global catastrophe, expropriation, and value drift. As discussed in the previous section, typical investors don’t care about value drift in the same way. In addition, many people in the effective altruism community have fairly unusual beliefs that might affect how they discount the future: namely, they pay more attention to existential risks. Many effective altruists assign much higher probabilities to existential catastrophes than most people do. However:

It’s possible that most sophisticated investors do generally agree with relatively pessimistic assessments of existential risk.
Perhaps most effective altruists aren’t particularly pessimistic about x-risk, and only differentially prioritize it because they care more about the long-run future.

So effective altruists might discount the future more heavily based on their beliefs about x-risk, but this is not obviously the case.

Disagreements about x-risk probably don’t substantially affect the overall discount rate. Even a fairly pessimistic 1% annual probability of existential catastrophe does not matter much in comparison to a 10% value drift rate, or a 2% probability of dying.⁴

Additional considerations

Rate of risk aversion ($\eta$)

As discussed previously, if we want to infer the discount rate from the market rate of return, we need to know how risk-averse most investors are. I presented estimates of the discount rate given various assumptions about risk aversion. But exactly how risk-averse are most investors?

Unfortunately, as with all the other variables we tried to estimate in this essay, we cannot reliably pin down the rate of risk aversion $\eta$. Different methodologies give different results.

In Discounting Disentangled, surveyed experts were asked to estimate the value of $\eta$. They gave a mean of 1.35 and a median of 1.0 (a value of 1.0 corresponds to a logarithmic utility function), but their answers showed high variance.

Studies of the relationship between income and happiness tend to find a roughly logarithmic relationship; for example, Stevenson & Wolfers (2013), Subjective Well-Being and Income: Is There Any Evidence of Satiation? (DOI: 10.1257/aer.103.3.598).⁵

Gordon Irlam’s Estimating the Coefficient of Relative Risk Aversion for Consumption summarizes a range of attempts to determine the value of $\eta$. Estimates vary greatly—Irlam claims that you could justifiably use a value anywhere between 1 and 4.

Why is the risk-free rate so low?

As of July 2021, real Treasury yields are negative: -1.60% for 5-year bonds and -0.21% for 30-year bonds. How do we square this with a positive time preference? Why would anyone with a positive discount rate ever invest in bonds?

Weil (1989), The equity premium puzzle and the risk-free rate puzzle, describes this issue as the risk-free rate puzzle. More recent publications have attempted to resolve the puzzle, for example Maki & Sonoda (2010), A solution to the equity premium and riskfree rate puzzles: an empirical investigation using Japanese data, which explains low risk-free rates as a product of trading costs. The existence of this puzzle somewhat invalidates the approach I used to derive the discount rate based on historical investment performance.

Conclusion

To summarize what we’ve learned so far:

In theory, we can reverse-engineer the philanthropic discount rate by estimating the typical discount rate and then subtracting the pure time preference.

We can determine the typical discount rate from market data in various ways: by looking at broad market returns, behavior of endowment portfolios, public companies’ shareholder yield, or GDP growth. Or we can simply survey experts and ask them what they think.

But when we attempt to reverse-engineer the philanthropic discount rate, we run into some problems:

Different methods to estimate the typical discount rate disagree with each other and have wide margins of error.
All of these methods require that we can accurately estimate how risk-averse people are, which we can’t. Attempts to estimate people’s risk aversion have produced inconsistent results.
We don’t know what rate of pure time preference most people use.
Philanthropists and typical investors might disagree about the discount rate for reasons other than pure time preference, and it’s hard to say how much they disagree.
Even if we can accurately estimate all of these parameters, this whole exercise relies on a theoretical model that doesn’t match reality—for example, it can’t explain why the risk-free rate is as low as it is.

Could we fix all of these problems? Perhaps. But some of these are open research areas that have already received substantial attention from academics, and still remain mysterious. It’s hard to say how much additional effort we would need to get a more realistic model with more accurate estimates.

In conclusion, it doesn’t look like we can do a good job of reverse-engineering the philanthropic discount rate.

Under some circumstances, we don’t care about the philanthropic discount rate; all we care about is the relationship between the philanthropic discount rate $\delta_P$ and the typical discount rate $\delta$. If patient philanthropists want to fund a particular cause that’s also funded by other actors, and if $\delta_P < \delta$, then the cause may be over-funded according to the philanthropists’ discount rate. In that case, the philanthropists should invest all their money to give later. Sometimes we might have good reason to believe $\delta_P < \delta$, even if we don’t know the exact difference. This greatly simplifies our decision, and we don’t need to precisely determine the philanthropic discount rate. For more on this, see Philip Trammell’s working paper Dynamic Public Good Provision under Time Preference Heterogeneity⁶ (especially section 3, “Interactions between patient and impatient funders”), and his 80,000 Hours interview.

Appendix

Appendix A: Optimal consumption under uncertainty

The Ramsey model defines the relationship between consumption and investment. Traditionally, this model assumes a fixed investment rate. In real life, nearly everyone invests at least some of their money in risky assets like stocks, and even “risk-free” assets like short-term Treasury bills still carry some degree of volatility. If we want to estimate the discount rate using real financial data, our model needs to incorporate uncertainty.

Levhari and Srinivasan (1969)⁷ provide a solution. First, some definitions⁸:

Let $c_t$ be consumption at time $t$.
Let $\delta$ be the discount rate.
Let $\eta$ be the rate at which marginal utility diminishes with consumption.
Let $r_t$ be the investment return at $t$. $\log(1 + r_t)$ follows a normal distribution parameterized by mean $\mu$ and standard deviation $\sigma$ (which is to say $r_t$ follows a lognormal distribution).
Let $k_t$ be the value of capital at $t$. Capital grows according to $k_{t+1} = (k_t - c_t)(1 + r_t)$.
Let $u(c)$ be a utility function of consumption. Suppose we have an isoelastic utility function, that is, \begin{align} \displaystyle \begin{cases}{\frac {c^{1-\eta}-1}{1-\eta}} & \eta \neq 1 \\ \ln(c) & \eta =1\end{cases} \end{align}
Our goal is to maximize total expected discounted utility \begin{align} E \left[\displaystyle\sum\limits_{t=0}^\infty (1 - \delta)^t u(c_t) \right] \end{align} (We discount by $(1 - \delta)^t$ instead of $e^{-\delta t}$ because this model follows discrete time, not continuous. In the limiting case as the length of a time step approaches zero, these two discount factors are equal.)

Under these conditions, the optimal consumption rate is a constant (call it $\lambda$). $\lambda$ is given by

\begin{align} \lambda = 1 - \exp[\frac{-\delta}{\eta} + \frac{1-\eta}{\eta}(\mu + \sigma^2/2) - (1 - \eta)\sigma^2/2] \end{align}

\begin{align} \lambda = 1 - \exp[\frac{-\delta}{\eta} + \frac{1-\eta}{\eta}\mu + \frac{1}{\eta}(1 - \eta)^2 \sigma^2/2] \end{align}

(Levhari and Srinivasan do not provide this exact result, but it easily follows from what they do give.)

Using this equation, we can derive the value of $\delta$ as a function of $\lambda, \eta, \mu$, and $\sigma$:

\begin{align} \delta = (1-\eta)\mu + (1 - \eta)^2\sigma^2/2 - \eta \log(1 - \lambda) \end{align}

If we do not know the value of $\lambda$, we can derive it from the investment rate and consumption growth rate for any particular period.

The consumption growth rate $g(c_t)$ must equal the capital growth rate because consumption is a fixed proportion of capital. By definition, the capital growth rate equals $\frac{k_{t+1} - k_t}{k_t} = (1 - \lambda)(1 + r_t) - 1$. If we can empirically observe $g_t$ and $r_t$, we can use these to derive $\lambda$:

\begin{align} \lambda = 1 - \frac{1 + g_t}{1 + r_t} \end{align}

In theory, this equation must hold for all times, because the actor will decide how much to grow their consumption ($g_t$) based on how much their portfolio earned during that period ($r_t$). In practice, it doesn’t hold, but we’ll get to that later.

Now combine our equations for $\lambda$ and $\delta$. As a simplifying assumption, suppose $g_t = \log(1 + g_t)$ and $r_t = \log(1 + r_t)$. Strictly speaking this is false, but it’s true in the limit as time steps become arbitrarily small.

This gives us a new formula for $\delta$:

\begin{align} \delta = \eta (r_t - g_t) + (1 - \eta)(\mu + \sigma^2/2) - \eta(1 - \eta)\sigma^2/2 \end{align}

We can simplify this a bit further. Suppose we identify the period with median investment return $r$ and median consumption growth $g$ (these must occur in the same period because $g$ is monotonic with $r$). That means $r = \mu$ (strictly speaking, $r = e^\mu - 1$, but again, this gives $r = \mu$ when time steps are infinitesimally small). Now we can simplify the formula to

\begin{align} \delta = r + (1 - \eta)^2\sigma^2/2 - \eta g \end{align}

These equations allow us to derive $\delta$ if we can observe the consumption rate $\lambda$ or, failing that, we can derive the optimal consumption rate from the interest rate $r_t$ and consumption growth rate $g_t$ at a particular time $t$.

Appendix B: Details on Harvard endowment annual report

The columns in the table below are defined as follows. All numbers are in millions, except for $\lambda$ and $\lambda_G$, which represent percentages.

“Endowment”: The total value of the endowment
“Gifts”: Value of gifts made to the endowment
“Return”: The investment return of the endowment portfolio
“Expenditures”: Money withdrawn from the endowment to fund the university’s operations
“Savings”: Any portfolio return that’s not used for expenditures (= Return - Expenditures)
”$\lambda$”: Consumption rate (= Expenditures / Endowment)
”$\lambda_G$”: Consumption rate excluding gifts (= (Expenditures - Gifts) / (Endowment - Gifts))

	Endowment	Gifts	Return	Expenditures	Savings	$\lambda$	$\lambda_G$
2003	19294	262	2059	770	1289	4	2.7
2004	22587	257	3800	807	2993	3.6	2.5
2005	25853	285	4044	854	3190	3.3	2.2
2006	29219	273	4113	933	3180	3.2	2.3
2007	34912	277	6499	1043	5456	3	2.2
2008	36926	336	2880	1201	1679	3.3	2.4
2009	26138	194	-9591	1416	-11007	5.4	4.7
2010	27557	240	2630	1320	1310	4.8	4
2011	32012	212	5500	1321	4179	4.1	3.5
2012	30745	226	0	1422	-1422	4.6	3.9
2013	32689	222	3267	1499	1768	4.6	3.9
2014	36429	512	4688	1539	3149	4.2	2.9
2015	37615	338	1956	1594	362	4.2	3.4
2016	35665	491	-625	1706	-2331	4.8	3.5
2017	37096	550	2651	1787	864	4.8	3.4
2018	39233	646	3332	1821	1511	4.6	3
2019	40929	613	2326	1908	418	4.7	3.2

Notes

Dimson, Marsh, and Staunton (2020). Summary Edition Credit Suisse Global Investment Returns Yearbook 2020. ↩
I don’t have data on the global shareholder yield, so I am naively assuming that it equals the yield of the S&P 500. ↩
The author distinguishes between risk aversion and intertemporal substitutability of consumption, which I do not, so they present the results a little differently. ↩
Technically, the probability of dying varies depending on age. 30-year olds only have about a 0.1% annual probability of death, increasing to 2% by age 70 and 5% by age 80. ↩
The paper does not explicitly find a logarithmic relationship; it is simply attempting to argue that income continues to increase happiness on the margin, even at relatively high income levels. But the paper’s data is public available. I fitted the data to a hyperbolic curve and found that it best fit at $\eta = 0.86$, which is a little less risk-averse than a logarithmic utility function. ↩
Accessed 2021-07-09. ↩
Levhari and Srinivasan (1960). Optimal Savings Under Uncertainty. ↩
My definitions differ from Levhari and Srinivasan in a few ways:
1. They define $r_t$ as one plus the interest rate, and I define it as the interest rate.
2. They use a discount factor $\beta$, and I use a discount rate $\delta$ (such that $\beta = 1 - \delta$).
3. They refer to the rate of diminishing marginal utility as $\alpha$, and I refer to it as $\eta$.
↩

How Can We Increase the Frequency of Rare Insights?

Mon, 19 Apr 2021 00:00:00 -0700

In many contexts, progress largely comes not from incremental progress, but from sudden and unpredictable insights. This is true at many different levels of scope—from one person’s current project, to one person’s life’s work, to the aggregate output of an entire field. But we know almost nothing about what causes these insights or how to increase their frequency.

Cross-posted to LessWrong.

Incremental progress vs. sudden insights

To simplify, progress can come in one of two ways:

Incremental improvements through spending a long time doing hard work.
Long periods of no progress, interspersed with sudden flashes of insight.

Realistically, the truth falls somewhere between these two extremes. Some activities, like theorem-proving, look more like the second case; other activities, like transcribing paper records onto a computer, look more like the first. When Andrew Wiles proved Fermat’s Last Theorem, he had to go through the grind of writing a 200-page proof, but he also had to have sparks of insight to figure out how to bridge the missing gaps in the proof.

The axis of incremental improvements vs. rare insights is mostly independent of the axis of easy vs. hard. A task can be sudden and easy, or incremental and hard. For example:¹

	incremental work	sudden insights
easy	algebra homework	geometry homework
hard	building machine learning models	proving novel theorems

Insofar as progress comes from “doing the work”, we know how to make progress. But insofar as it comes from rare insights, we don’t know.

Some meditations on the nature of insights

Why did it take so long to invent X?

Feynman on finding the right psychological conditions

Physicist Richard Feynman talks about this in Take the World from Another Point of View:

I worked out the theory of helium, once, and suddenly saw everything. I’d been struggling, struggling for two years, and suddenly saw everything at one time. […] And then you wonder, what’s the psychological condition? Well I know at that particular time, I simply looked up and I said wait a minute, it can’t be quite that difficult. It must be very easy. I’ll stand back, I’ll treat it very lightly, I’ll just tap it, and there it was! So how many times since then, I’m walking on the beach and I say, now look, it can’t be that complicated. And I’ll tap it, tap it, nothing happens.

Feynman tried to figure out what conditions lead to insights, but he “never found any correlations with anything.”

P vs. NP

A pessimistic take would be that there’s basically no way to increase the probability of insights. Recognizing insights as obvious in retrospect is easy, but coming up with them is hard, and this is a fundamental mathematical fact about reality because P != NP (probably). As Scott Aaronson writes:

If P=NP, then the world would be a profoundly different place than we usually assume it to be. There would be no special value in “creative leaps,” no fundamental gap between solving a problem and recognizing the solution once it’s found. Everyone who could appreciate a symphony would be Mozart; everyone who could follow a step-by-step argument would be Gauss; everyone who could recognize a good investment strategy would be Warren Buffett. It’s possible to put the point in Darwinian terms: if this is the sort of universe we inhabited, why wouldn’t we already have evolved to take advantage of it?

I’m not quite so pessimistic. I agree with Scott Aaronson’s basic argument that solving problems is much harder than recognizing good solutions, but there might still be ways we could make it easier to solve problems.

johnswentworth on problems we don’t understand

The concept of sudden-insight problems relates to johnswentworth’s concept of problems we don’t understand. Problems we don’t understand almost always require sudden insights, but problems that require sudden insights might be problems we understand (for example, proving theorems). johnswentworth proposes some types of learning that could help:

Learn the gears of a system, so you can later tackle problems involving the system which are unlike any you’ve seen before. Ex.: physiology classes for doctors.

Learn how to think about a system at a high level, e.g. enough to do Fermi estimates or identify key bottlenecks relevant to some design problem. Ex.: intro-level fluid mechanics.

Uncover unknown unknowns, like pitfalls which you wouldn’t have thought to check for, tools you wouldn’t have known existed, or problems you didn’t know were tractable/intractable. Ex.: intro-level statistics, or any course covering NP-completeness.

I would expect these types of learning to increase the rate of insights.

Learning how to increase the frequency of insights

Insights happen less frequently under bad conditions: when you’re sleep-deprived, or malnourished, or stressed out, or distracted by other problems. Some actions can increase the probability of insights—for example, by studying the field and getting a good understanding of similar problems. But even under ideal conditions, insights are rare.

Interestingly, most of the things that increase the frequency of insights, such as sleep and caffeine, also increase the speed at which you can do incremental work. It’s possible that these things speed up thinking, but don’t increase the probability that any particular thought is the “right” one.

I can come up with one exception: you can (probably?) increase the frequency of insights on a problem if you understand a wide variety of problems and concepts. I don’t believe this does much to speed up incremental work, but it does make sudden insights more likely. Perhaps this happens because sudden insights often come from connecting two seemingly-unrelated ideas. I’ve heard some people recommend studying two disparate fields because you can use your knowledge of one field to bring a unique perspective to the other one.

Overall, though, it seems to me that we as a society basically have no idea how to increase insights’ frequency beyond a basic low level.

Instead of directly asking how to produce insights, we can ask how to learn how to produce insights. If we wanted to learn more about what conditions produce insights, how might we do that? Could we formally study the conditions under which geniuses come up with genius ideas?

If someone gave me a pile of money and asked me to figure out what conditions best promote insights, what would I do? I might start by recruiting a bunch of mathematicians and scientists to regularly report on their conditions along a bunch of axes: how long they slept, their stress level, etc. (I’d probably want to figure out some axes worth studying that we don’t already know much about, since we know that conditions (like sleep quality) do affect cognitive capacity.) Also have them report whenever they make some sort of breakthrough. If we collect enough high-quality data, we should be able to figure out what conditions work best, and disambiguate between factors that help provide insights and factors that “merely” increase cognitive capacity.

I’m mostly just speculating here—I’m not sure the best way to study how to have insights. But it does seem like an important thing to know, and right now we understand very little about it.

Notes

Some more specific examples from things I’ve worked on:
- Easy, incremental work: A Comparison of Donor-Advised Fund Providers just required me to read the websites of a bunch of providers and talk to them about their offerings.
- Easy, sudden insight: The Risk of Concentrating Wealth in a Single Asset comes from the simple but oft-overlooked observation that you can compare different investments using their certainty-equivalent returns. I probably spent dozens of hours thinking about how to evaluate investments before this occurred to me. (This still qualifies as easy in the grand scheme of things.)
- Hard, incremental work: Asset Allocation and Leverage for Altruists with Constraints required me to set up an optimization problem, write a program to solve it, and run through various scenarios that could change the results.
- Hard, sudden insight: GiveWell’s Charity Recommendations Require Taking a Controversial Stance on Population Ethics presents a fairly simple argument. But it was hard to come up with—my impression is that I had a low before-the-fact probability of thinking of it.
↩

Investment Strategies for Donor-Advised Funds

Tue, 06 Apr 2021 00:00:00 -0700

A donor-advised fund (DAF) is an investment account that allows you to take a tax deduction now and give the money to charity later. When you give money to a DAF, you can deduct that money just as you would deduct a charitable contribution. The DAF invests the money tax-free until you are ready to donate it to charity. But DAFs only allow limited investment options. How can we best make use of a DAF to optimize expected investment performance?

Contents
Caveats
What investment options do DAFs provide?
Investment strategies
(Even more) advanced ideas
- Create a new DAF
- Launch a new leveraged fund
Comparing the choices
Appendix: More ideas
- Hire an advisor and let them decide how to invest
- Find an advisor who can generate alpha
Notes

Caveats

Before we decide how to invest in a DAF, first we must ask: Should we use a DAF at all? Or should we keep our money in a taxable account? Or should we simply donate it all right away? The answer depends on the circumstances. I review some pros and cons of DAFs in Donor-Advised Funds vs. Taxable Accounts for Patient Donors. In this essay, I will assume we’ve already decided to use a DAF.

I only cover DAFs in the United States because that’s what I’m familiar with. Many other countries probably have similar investment vehicles.

I am not a lawyer or an investment advisor and none of this should be taken as legal or investment advice.

What investment options do DAFs provide?

The three most prominent donor-advised fund providers are Vanguard Charitable, Schwab Charitable, and Fidelity Charitable. A fourth provider, Greater Horizons, is worth including for reasons we will discuss below. All four DAF providers charge similar fees. Fees matter, but these four providers have similar enough fees that we can ignore them for now.

The providers offer index funds covering the following markets:

	Fidelity	Schwab	Vanguard	Greater Horizons
US stocks
US small-cap
international stocks
Europe stocks
Pacific stocks
emerging stocks
US bonds
international bonds
TIPS
money market

If your account has over $250,000, Fidelity and Schwab (but not Vanguard) allow you to appoint a financial advisor to manage the account. This advisor can invest in things other than the pre-selected funds, as long as they abide by certain restrictions (which are documented here for Schwab and here for Fidelity). Greater Horizons allows you to open an advisor-managed account with no minimum account size, and imposes fewer restrictions (see here).

I believe Greater Horizons is the most likely to allow advisors to follow risky or unusual strategies¹. Possibly due to legal restrictions, none of the DAFs allow investors to use leverage via margin, futures, options, or swaps² (but they do allow leveraged ETFs).

For more what different DAF providers offer, see A Comparison of Donor-Advised Fund Providers.

Investment strategies

Let’s review some possible investment strategies. This list includes all the ideas that I personally have considered, but I’m sure there are some good ideas that I don’t know about.

Baseline: global market portfolio

If the efficient market hypothesis is strictly true, then we should invest in the global market portfolio. The investment selections available in a Fidelity, Schwab, or Vanguard DAF don’t include every asset class in the world, but they do include most of the biggest ones. A DAF can get pretty close to the global market portfolio with something like this:

US stocks	30%
international stocks	30%
US bonds	40%

If you use Vanguard, you can include international bonds, so you can get even closer to the global market portfolio:

US stocks	30%
international stocks	30%
US bonds	20%
international bonds	20%

Many altruistic investors can tolerate more risk than this. We can increase risk by removing bonds and only investing in stocks—something like 50% in a US stock fund and 50% in an international stock fund.

Next, let’s look at a few strategies that might improve on this baseline.

Caveat: We can never know for certain how well an investment will perform in the future. For the rest of the strategies in this essay, we have at least some reason to expect them to outperform the global market portfolio in the long run, but nothing is guaranteed.

Overweighting emerging markets

In theory, the best way to increase expected return is to use leverage: borrow money to invest more than you have. But most DAFs don’t allow leverage. The only way to increase expected return is to tilt toward high-risk, high-return asset classes. We can do this by investing in all stocks and no bonds. Can we go further?

To increase expected return, we can concentrate in the region of the world where equities have the highest expected return. To do so, we need to make an educated guess as to which region that is.

We don’t have any perfect way to determine future expected returns, and we could take many different approaches. Several institutions provide return projections using various methodologies, and they generally agree that emerging market equities currently have higher expected returns than other asset classes (e.g., AQR, Research Affiliates, Vanguard). This higher expected return comes with higher risk. So we can simulate leverage by buying emerging market stocks.

(Of the major DAF providers, only Vanguard offers an emerging markets index fund as a pre-selected option. Investors who want to hold emerging markets, but don’t want a professionally managed account, would need to use Vanguard.)

How much higher of a return should we expect from emerging markets than from global equities? That depends on what methodology we use to estimate future returns. According to AQR, emerging markets have an expected excess return of about one percentage point, while Research Affiliates expects a three percentage point excess return (as of March 2021).

The main reason for this difference is that Research Affiliates adjusts for stock market valuations, while AQR does not. Emerging market stocks look cheaper than stocks in developed markets. We know that cheap countries tend to outperform, sometimes by a lot (Faber, 2012³; Radha, 2020⁴). More generally, cheap assets tend to outperform expensive assets both within and across markets (Asness et al., 2013⁵). There’s mixed evidence on what causes this, but it could happen due to a combination of home country bias (Swedroe, 2019; Saphier et al., 2019), common behavioral biases (Lakonishok et al., 1993⁶), and limitations on sophisticated investors that prevent them from correcting this mispricing (Lewellen, 2011⁷; Shleifer & Vishny, 1997⁸). That said, Research Affiliates’ estimates might over-correct for the valuations of different markets, so my own estimate for the expected return of emerging markets would fall somewhere between AQR’s and Research Affiliates’ estimates.

Momentum across asset classes

As discussed in the previous section, cheap assets tend to outperform expensive assets. Value isn’t the only investment factor that systematically predicts performance. There’s also the momentum factor: assets that have outperformed over the past 6-12 months tend to continue to outperform for a few months (Jedageesh & Titman, 2011⁹; Fama & French, 2008¹⁰; Berger et al., 2009¹¹). Similarly, assets that have performed well on an absolute basis tend to continue to perform well (D’Souza et al., 2016¹²; Lempérière et al., 2014¹³; Babu et al., 2019¹⁴; Faber, 2013¹⁵))—this is known as absolute momentum or trendfollowing.

A DAF can’t invest in individual high-momentum stocks, but it can invest in whichever of the available funds show the strongest momentum. Gary Antonacci describes a version of this concept in his book, Dual Momentum, and in a paper¹⁶; a similar approach was studied in D’Souza et al. (2016)¹². (The strategy is called Dual Momentum because it uses both relative and absolute momentum.) Dual Momentum works like this:

Start with three index funds: a US stock fund, an international stock fund, and a bond fund.
If both stock funds had negative total return over the prior 12 months, buy the bond fund.
Otherwise, buy whichever stock fund had higher return over the prior 12 months.
Rebalance monthly.

In a backtest from 1950 to 2018, Dual Momentum performed much better than a diversified buy-and-hold strategy:

	Dual Momentum	Buy and Hold
Return	15.8%	10.0%
Stdev	11.5%	9.8%
Sharpe Ratio	0.96	0.57
Max Drawdown	-18%	-41%

We don’t have to use US stocks + international stocks + bonds as our asset classes. A similar strategy would have performed well on many different combinations of asset classes. For example:

Faber (2010)¹⁷ finds strong historical performance for a momentum strategy that rotates among US stocks + international stocks + REITs + commodities + bonds¹⁷.
Moskowitz & Grinblatt (1999)¹⁸ identify a momentum premium across industries.

(I was able to replicate these results in and out of sample using data from the Ken French Data Library, the US Federal Reserve, and AQR’s commodity index.)

The momentum premium shows up everywhere⁵—it is the most consistent and robust market anomaly. For more detailed evidence that momentum-based (as well as valuation-based) asset allocation strategies work, including a discussion on expected future performance, see Blitz & van Vliet (2008)¹⁹.

We could run an asset class momentum strategy in a DAF through Vanguard Charitable:

Consider five funds: US stock, Europe stock, Pacific stock, emerging stock, and US bonds²⁰.
Out of these five funds, hold whichever two have the highest prior 12-month return.
But, if fewer than two funds had positive 12-month return, put the remaining money into a money market fund.
Rebalance monthly.

Or we could run Dual Momentum in Fidelity or Schwab, using three asset classes (US stocks, international stocks, and bonds) instead of five.

(There are lots of variations of asset class momentum: using a different set of funds; using a time horizon other than 12 months, such as 6 or 9 months; rebalancing quarterly instead of monthly; etc. Every variation has pros and cons. The important thing is to capture the momentum premium, and any reasonable implementation will do that.)

What kind of future returns might we expect from asset class momentum?

As a basic approach, let’s look at how well asset class momentum has performed in the past compared to buy-and-hold. This tells us the historical size of the “asset class momentum premium”. But the premium will likely shrink in the future²¹. If we roughly assume that asset class momentum will work half as well in the future as it has in the past and that the volatility won’t change, then we can expect asset class momentum to earn about a 5-6% net real return with 12% standard deviation²².

Note: Asset class momentum strategies often move in and out of positions many times in a year. DAFs generally don’t want investors to trade excessively. If they believe you are trading too frequently, they might decide to limit your trades. I expect they wouldn’t do that, but it’s a risk worth considering.

Leveraged ETFs (with a managed account)

By default, DAF providers only allow you to buy funds from their pre-selected list. But most DAF providers allow you to appoint a professional investment manager. (Advisor-managed accounts typically have a $250,000 minimum, although Greater Horizons has no minimum.) That manager can invest however they want as long as they follow certain guidelines. They can’t get leverage via margin, futures, or options, but they’re allowed to buy leveraged ETFs.

Leveraged ETFs tend to cost more than other types of leveraged investments. In a previous essay, I estimated that leveraged ETFs on average cost about 2% plus the risk-free rate (although the exact number varies a lot over time). Investment managers typically charge around 1% for small(ish) investors and 0.5% for large investors, so holding leveraged ETFs in a managed account will cost 2.5 to 3 percentage points more than investing in a DAF’s pre-selected funds. Fidelity Charitable allows you to self-manage your DAF if you have over $5 million, so you could cut out the advisor fee.

For truly risk-neutral investors, this is probably worth it. Under most plausible assumptions, leveraged ETFs have a high enough expected return to make up for the cost. For investors with logarithmic utility (which is probably still too aggressive for individual investors, but might be appropriate for altruists), the higher cost of leveraged ETFs probably cancels out their higher expected utility. (For details, see The True Cost of Leveraged ETFs.)

Factor funds (with a managed account)

The value and momentum factors have historically predicted stock performance. (There are some other factors as well, such as size, low-volatility, and quality. These aren’t as well-supported as value and momentum, but they might be worth incorporating.) Some ETFs and mutual funds provide exposure to these factors. With a managed account, your investment advisor could use factor funds to try to outperform the market.

Factor funds (at least the good ones) are much cheaper than leveraged ETFs. To be worth buying, factor funds need to outperform the market after both fund expenses and the advisor’s fee. That’s probably achievable in expectation.

You can run asset class momentum yourself (and skip the advisor’s fee), so it seems less likely that a factor fund portfolio can outperform asset class momentum after costs. But using an advisor means you don’t have to do any work, and the time savings might justify the extra cost.

Concentrated stock picking (with a managed account)

If we’re willing to take heavier bets on value and momentum, we can concentrate our investments in a small number of value/momentum stocks (follow the link for a more detailed explanation). Backtests suggest that the top 10% of small-cap value and momentum stocks earned about double the return of the market before fees (with more than 1x the volatility of the market, but less than 2x). A portfolio of individual stocks would have to pay trading costs plus a management fee. This would probably cost less than a leveraged ETF, but definitely more than an un-managed DAF portfolio.

How good is a concentrated value/momentum strategy? That depends on how well we expect the value and momentum factors to perform in the future. It seems likely that they will continue to work at least to some extent, and the value factor in particular looks attractive relative to history (as of early 2021). Still, it’s difficult to forecast future performance. But even if a concentrated value/momentum strategy has zero risk-adjusted premium over the market, it should still outperform a leveraged ETF by having lower costs.

A related approach: Instead of buying individual value/momentum stocks or a set of diversified factor funds, we could buy a concentrated mutual fund or ETF that looks particularly attractive. For example, as of this writing, the Russian stock market looks cheaper than almost any other market, so we could buy the iShares MSCI Russia ETF (ERUS). Buying ERUS is the same basic strategy as buying an emerging market index fund as discussed previously, but it’s higher risk and has the potential to deliver a higher return. We can’t get as much concentration with an ETF as with a basket of stocks, but this approach has the advantage that we only need to buy one thing.

(Even more) advanced ideas

Create a new DAF

Legally, any registered charity can operate a DAF. (At least that’s my understanding from reading IRS guidelines.) The charity just needs to accept money from donors, invest that money, and allow donors to recommend grants to other charities. But it’s a lot of work to set up all the necessary infrastructure. A sufficiently motivated organization can do it if they believe they’ll get enough assets under management to justify the effort.

Charities have to abide by certain legal restrictions about how they can invest, but the restrictions are not well-specified and I don’t understand them very well. A new DAF might be able to provide better investment options than any of the ones I’ve discussed so far, or it might not, depending on the legal situation.

Launch a new leveraged fund

The leveraged ETFs on the market today have some problems that make them less than ideal for long-term investing. But a new leveraged ETF or mutual fund could fix these problems. In particular:

Existing leveraged ETFs charge high fees. Most people who buy leveraged ETFs use them for short-term trading, so they don’t care much about the fees. But these ETFs have high profit margins, so a few fund could undercut them while still making a profit.
Existing leveraged ETFs rebalance daily. A long-term risk-friendly investor doesn’t need their account to perfectly track the leveraged index on a daily basis. A new leveraged ETF could rebalance less frequently (perhaps monthly), which would reduce trading costs.
Existing leveraged ETFs focus on a single market or sector. This makes each individual fund riskier. It also means an investor who buys multiple leveraged ETFs will need to rebalance their holdings manually. But a new leveraged ETF could invest in the global market portfolio (or something like it). That way, long-term investors could buy a single fund and would never need to rebalance.

Investors with a minimum of around $25 to $50 million could sponsor the creation of a new leveraged fund. Such a fund might be a better way to invest than any of the other options we’ve discussed.

Comparing the choices

Confidence: Possible.

So far we’ve looked at seven investment strategies that might perform better than the global market portfolio:

Buy and hold an emerging markets index fund.
Run asset class momentum on the pre-selected asset classes.
With an investment manager, buy and hold leveraged ETFs.
With an investment manager, buy and hold some factor funds.
With an investment manager, invest in a concentrated basket of value/momentum stocks or a concentrated ETF.
Create a new DAF that offers more investment options.
Sponsor the creation of a new leveraged ETF or mutual fund.

I don’t know what legal restrictions would apply to a new DAF, so I can’t say much about that idea. Creating a new leveraged fund looks like the best option, but isn’t possible for most investors. Existing leveraged ETFs seem not worth buying. That leaves four choices: buying an emerging markets fund, running asset class momentum, buying factor funds, or buying a concentrated basket of stocks. Investors who do not meet the minimum requirement for a managed account (usually $250,000) must pick between the first two choices.

When possible, I would prefer to invest in a concentrated basket of stocks. Historically, they offered substantially higher returns than the market, and I expect this to continue (although probably to a lesser extent). Buying and holding factor funds is less risky, but also has worse expected return. You’ll have a much easier time finding an investment advisor who’s willing to buy factor funds, so it might be a more realistic option. (Buying factor funds is a pretty normal thing that lots of advisors do. Buying a few dozen deep value/momentum stocks is a weird thing that almost nobody does.)

Between the two strategies that do not require a professionally managed account—asset class momentum and an emerging markets index fund—I lean toward asset class momentum, for three reasons:

Asset class momentum diversifies across multiple asset classes. All else equal, diversification is good.
According to backtests I ran, if we went back in time to 1990 and bought the asset class that performed best over the next 30 years, we still would have underperformed asset class momentum. Even with perfect foresight, asset class momentum still beat buy-and-hold.²³
According to my (very unreliable) future return projections, asset class momentum has a higher expected return than emerging markets.

I don’t put much credence in the last two points, but they provide at least weak evidence in favor of asset class momentum.

On the other hand, it seems hard to deny that emerging markets currently have a higher expected return than other parts of the world. Asset class momentum seems likely to continue working, but I’m not quite as confident about that. Holding an emerging markets funds has the added advantage that you never have to rebalance, so if you want to do as little work as possible, it might be a better choice.

Appendix: More ideas

Hire an advisor and let them decide how to invest

In general, this is a bad idea. Most investment advisors cannot outperform the market, and they’ll charge you a fee for essentially no added value. You’d be better off using the DAF’s pre-selected funds to build the global market portfolio (or get as close to it as you can).

If your advisor is familiar with the research on value and momentum investing, they’re more likely to invest your money well.

Find an advisor who can generate alpha

It’s not too hard to find an investment advisor who can build you a portfolio of value/momentum factor funds that’s better than the global market portfolio. But it’s much more difficult to find someone who can generate true alpha on top of those factors. A few such people probably exist, but they’re nearly impossible to identify before the fact. In general, I don’t think it’s worth trying, although there might be exceptions.

Notes

I spoke to representatives at each donor-advised fund and to a few financial advisors who manage DAFs. Based on those conversations, it appears that Greater Horizons offers the most flexibility. ↩
Unfortunately, the legal requirements are not clear. A DAF provider has a fiduciary duty to make prudent investments, and it’s largely up to courts’ interpretation as to what this means. DAF providers disallow investments that they think might violate fiduciary duty. For more, see the FDIC Trust Examination Manual: Investment Principles, Policies and Products. ↩
Faber (2012). Global Value: Building Trading Models with the 10 Year CAPE. For more detail, see Faber’s book on the same subject, Global Value: How to Spot Bubbles, Avoid Market Crashes, and Earn Big Returns in the Stock Market. ↩
Radha (2020). Using CAPE to Forecast Country Returns for Designing an International Country Rotation Portfolio. ↩
Asness, Moskowitz & Pedersen (2013). Value and Momentum Everywhere. ↩ ↩²
Lakonishok, Shleifer & Vishny (1993). Contrarian Investment, Extrapolation, and Risk. ↩
Lewellen (2011). Institutional Investors and the Limits of Arbitrage. ↩
Shleifer & Vishny (1997). The Limits of Arbitrage. ↩
Jedageesh & Titman (2011). Momentum. ↩
Fama & French (2008). Dissecting Anomalies. ↩
Berger, Israel & Moskowitz (2009). The Case for Momentum Investing. ↩
D’Souza,, Srichanachaichok, Wang & Yao (2016). The Enduring Effect of Time-Series Momentum on Stock Returns Over Nearly 100-Years. ↩ ↩²
Lempérière, Deremble, Seager, Potters & Bouchaud (2014). Two Centuries of Trend Following. ↩
Babu, Levine, Ooi, Pedersen & Stamelos (2019). Trends Everywhere. ↩
Faber (2013). A Quantitative Approach to Tactical Asset Allocation. ↩
Antonacci (2016). Risk Premia Harvesting Through Dual Momentum. ↩
Faber (2010). Relative Strength Strategies for Investing. ↩ ↩²
Moskowitz & Grinblatt (1999). Do Industries Explain Momentum? ↩
Blitz & van Vliet (2008). Global Tactical Cross-Asset Allocation: Applying Value and Momentum Across Asset Classes. ↩
This list does not include the international stock fund because that fund is simply the sum of the Europe, Pacific, and emerging stock funds. Unlike Antonacci’s Dual Momentum, I treated bonds as a risky asset. I believe Antonacci’s backtest makes bonds look better than they are because bond yields consistently decreased for most of the tested period. It makes more sense to use a money market fund as the risk-free asset. ↩
We should expect the asset class momentum premium to shrink for two main reasons:
1. Bond yields are lower than they were in the backtest period. That reduces how much return an asset class momentum strategy can earn when it’s not holding stocks.
2. Momentum strategies have become more popular, which could cause them to perform worse going forward.
The first point seems hard to dispute. Regarding the second point, as a simple rule of thumb, we could assume the premium will decline by half²⁴. Relevant research:
- McLean & Pontiff (2016)²⁵ found that published anomalies historically shrunk by 58% after publication, of which 26% could be attributed to data mining, and 32% to markets becoming more efficient.
- Jensen, Kelly & Pedersen (2021)²⁶ expanded McLean & Pontiff’s research and found about a 1/3 decline post-publication, all of which could be attributed to data mining.
- Blitz (2017)²⁷ found that ETFs do not harvest the momentum premium²⁸.
Asset class momentum appears more likely than stock momentum to survive increasing popularity because (1) it trades larger, more liquid positions, and (2) most momentum investors use stock momentum, not asset class momentum. So arguably we should make a more optimistic assumption. ↩
I came to this expected return figure as follows:
1. As a baseline, assume Dual Momentum will perform as well as the average of the US and international stock markets. If we take the median of AQR’s and Research Affiliates’ projections, we get an expected return of 3.4%.
2. Based on backtests, the historical Dual Momentum premium was about 6%. Halve this to get 3% and add to the baseline performance for a total expected return of 6.4%.
3. Subtract 1% for transaction costs to get 5.4%.
↩
I backtested two different pairs of asset classes: (US stocks, global ex-US stocks) and (US stocks, Japan stocks). I specifically chose Japan because Japan had the weakest performance of any major stock market over the sample period. If asset class momentum over (US, Japan) outperformed US stocks alone, that would give the strongest possible evidence in favor of asset class momentum that my data set could produce. And indeed, it turned out that asset class momentum did (slightly) outperform. I don’t consider this strong evidence in an absolute sense, but it’s the best I can do given the data I have access to (I used the Ken French Data Library). ↩
In How Can a Strategy Everyone Knows About Still Work? , Cliff Asness suggests making this “conservative” assumption. ↩
McLean & Pontiff (2016). Does Academic Research Destroy Stock Return Predictability? ↩
Jensen, Kelly & Pedersen (2021). Is There a Replication Crisis in Finance? ↩
Blitz (2017). Are Exchange-Traded Funds Harvesting Factor Premiums? ↩
We might be interested in how hedge funds behave rather than ETFs, because hedge funds probably represent more ‘savvy’ investors. I am not aware of any research on whether hedge funds in aggregate exploit the momentum premium. But another 2017 study by Blitz, Are Hedge Funds on the Other Side of the Low-Volatility Trade?, found that hedge funds systematically reverse load on the low volatility premium (that is, they take the losing side of the trade), which weakly suggests they may do the same for momentum. ↩

A Comparison of Donor-Advised Fund Providers

Mon, 05 Apr 2021 00:00:00 -0700

Last updated 2025-05-09.

A donor-advised fund (DAF) is an investment account that lets you take a tax deduction now and give the money to charity later. When you give money to a DAF, you can deduct that money just as you would deduct a charitable donation. The DAF invests the money tax-free. At any time, you can write a grant from your DAF to a charity of your choice.

You can open a DAF through a donor-advised fund provider. A provider charges an administrative fee to invest your DAF and make donations when you recommend them.

For donors in the United States,¹ which DAF provider is best?

The short answer

All the big DAF providers offer similar features. For most people, it doesn’t matter much which one you choose.

If you already have a DAF, you might as well keep using it.
If you have a brokerage account at Fidelity, Schwab, or Vanguard, then the easiest thing to do is to open a DAF with your brokerage account. That way, you can manage all your investments in one place.

Otherwise, this handy flowchart can help you choose a DAF provider that fits your preferences.

The long answer

That flowchart might not cover everything you care about, and it doesn’t offer nuance. In the rest of this post, let’s look in detail at how DAF providers compare.

Cross-posted to the Effective Altruism Forum.

Contents
My process
Comparison
The best DAF provider(s)
- More on Fidelity, AEF, NPTrust, and T. Rowe Price
Appendix: Table of administrative fees
Disclaimers
Changelog
Notes

My process

I made a list of every United States nationwide DAF provider I could find. I excluded regional DAF providers (example: Silicon Valley Community Foundation), providers that only support certain causes (example: National Christian Foundation), and providers that don’t work with individual donors (example: American Online Giving Foundation). Your local community foundation might offer a better DAF than any of the national providers of my list, but there are too many community foundations for me to look at them all.

I ended up with nine DAF providers (in alphabetical order):

American Endowment Foundation (AEF)
Charityvest (see disclaimer)
Daffy (see disclaimer)
Fidelity Charitable
Greater Horizons
National Philanthropic Trust (NPTrust)
Schwab Charitable
T. Rowe Price Charitable
Vanguard Charitable

This list is probably not comprehensive, but it’s all the DAF providers I could find that meet my criteria.

I spoke to representatives at these providers to fill in some gaps in my knowledge. I also spoke to a few DAF users and financial advisors who manage DAFs at different providers.

I then eliminated four DAF providers:

T. Rowe Price Charitable funds have excessively high fees (0.6% or higher).
AEF and NPTrust both charge higher administrative fees than Vanguard/Schwab, offer a worse user experience, and don’t have any special features to compensate.
Fidelity is nearly identical to Schwab, but it’s slightly worse in a few ways (for more, see here).

Five DAF providers remain: Charityvest, Daffy, Greater Horizons, Schwab, and Vanguard.

Caveat 1: I don’t have good firsthand knowledge of any of these DAF providers except for Fidelity Charitable (which I used to use) and Greater Horizons (which I currently use). I created accounts at Charityvest, Daffy, Schwab, and Vanguard to get a sense of how they work, but I haven’t tried to do anything fancy like set up an advisor-managed account. I had to make subjective judgments on things like UI, so don’t take my claims as definitive.

Caveat 2: This article is not about whether you should open a DAF in the first place. You might want to keep your money in a taxable account, or maybe donate it all right away. But if you’ve already decided you want a DAF, then I hope this article will help you choose a provider.

Comparison

Let’s compare Vanguard, Schwab, Greater Horizons, Daffy, and Charityvest on these questions:

What fees do they charge? [More]
Do they have contribution/grant minimums? [More]
What investment choices do they offer? [More]
Do they have reasonable default investment options? [More]
How flexible are their advisor-managed accounts? [More]
Can you contribute complex assets such as cryptocurrency or real estate? [More]
How good is the user experience? [More]
How risky are the newer DAF providers (Daffy and Charityvest)? [More]

Fees

Minimum fees:

Charityvest has no minimum fee for cash accounts, and a minimum annual fee of $100 for accounts that hold stocks and bonds.
Daffy has three tiers of accounts, with minimums of $36, $60, and $240 per year.
Greater Horizons has a minimum annual fee of $500.
Schwab has no minimum annual fee.
Vanguard has a minimum annual fee of $250.

Charityvest, Greater Horizons², Schwab, and Vanguard offer tiered fee structures based on the account value. Daffy charges a monthly fee but no percentage-based fee. See Appendix for full details on the administrative fees for each provider.

Daffy has a unique pricing structure. It offers three plans: a basic account for $3/month, a family account for $5/month, and a premium account for $20/month, and has no percentage-based fee.

Daffy is the least expensive DAF provider if you have a basic account with at least $6,000 or a premium account with at least $40,000. But beware that Daffy does not allow you to contribute more than a lifetime total of $25,000 in non-cash assets to the basic plan, or $50,000 to the family plan. If you make $75,000 and you donate 10% of your income, you’ll hit the basic plan’s limit in 4 years and the family plan’s limit in 7 years.

Daffy’s $20/month plan has no limit. You can keep $10 million in your DAF and still only pay $20/month.

If you only use a DAF as a convenient way to donate appreciated stock (or other assets), and you don’t plan on keeping money in the DAF long-term, you want a DAF with no minimum fee. Your best options are Schwab and Charityvest:

Schwab has no minimum fee.
Charityvest has a basic plan with no minimum fee, but with the basic plan you can’t hold any investments in your account, only cash. (You can still donate non-cash assets.)

In brief:

Daffy’s basic account is the cheapest of all the providers if you have at least $6,000. Its premium account is the cheapest if you have at least $40,000.³
Schwab is the cheapest for accounts with less than $6,000 because it’s the only provider with no minimum fee. (Unless you have a Charityvest cash-only account.)
For accounts with over $16,667, the cheapest (other than Daffy basic) is a three-way tie between Charityvest, Fidelity, and Schwab up to account values of $1 million.⁴
Above $1 million, the cheapest might be Charityvest, Schwab, or Vanguard, depending on exactly how much money you have, but the differences between the three are small.
If you have enough money (probably starting in the tens of millions), you can negotiate lower fees.

Daffy and Charityvest are startups, which makes them riskier. If we exclude the two startup-y DAF providers:

If you have less than $25,000, Schwab is cheapest.
If you have between $25,000 and $1 million, the three providers (Greater Horizons, Schwab, Vanguard) charge the same fees.
For larger accounts, Vanguard is cheapest.

Minimums

Donor-advised funds have three types of minimums:

Minimum contribution to create an account
Minimum additional contribution
Minimum grant size
Minimum size for an advisor-managed account

	Charityvest	Daffy	Greater Horizons	Schwab	Vanguard
Account Min	$0	$0	$0	$0	$25,000⁵
Contribution Min	$15	$0	$0	$0	$5,000
Grant Min	$15	$18	$0	$50	$500
Advisor Min	$1 million	N/A	$0	$0	N/A

Investment choices

If you don’t know much about investing or you want to invest in whatever default your DAF provider uses, skip to the next section.

Let’s compare the investment options for Charityvest, Greater Horizons, Schwab, and Vanguard. Daffy provides many more investment options than the other providers, so I discuss Daffy in its own section.

The DAF providers offer index funds covering the following asset classes.

	Charityvest	Greater Horizons	Schwab	Vanguard
US stocks
US small-cap
international stocks
emerging market stocks
US bonds
international bonds
TIPS
money market

Vanguard offers funds covering many other market segments, including US growth/value, US REITs, European stocks, Pacific stocks, corporate bonds, dividend growth stocks, and commodities. Vanguard absolutely has the best variety in terms of index fund offerings.

The expense ratios on the funds themselves are low enough not to matter as long as you stick with the passively-managed funds. (Vanguard and Schwab offer a few actively-managed funds with higher fees, which you should avoid.) You can use any of these DAF providers to construct a globally diversified portfolio for an average expense ratio of about 0.05%.

Vanguard and Charityvest investors can approximately replicate the global market portfolio with 30% US stocks, 30% international stocks, 20% US bonds, and 20% international bonds. Investors with Schwab or Greater Horizons don’t have access to international bonds, so the closest they can get is something like 30% US stocks, 30% international stocks, 40% US bonds.

Vanguard, Daffy, Charityvest, and Greater Horizons offer a few ESG funds with low expense ratios. Schwab also has ESG funds, but they’re unreasonably expensive.

If you want to invest in cryptocurrency, you can either do that through an advisor-managed account⁶, or you can invest in a cryptocurrency fund through Daffy. Daffy is the only provider that offers crypto funds.

In summary:

If you want the global market portfolio, use Vanguard or Charityvest because they offer an international bond index.
If you want ESG funds, use any except Schwab.
If you want cryptocurrency (and you don’t want an advisor-managed account), use Daffy.
If you want a wide diversity of options, use Vanguard.
Otherwise, any provider is a good choice.

Default investments

Each of the five DAF providers offers a few default investment allocations. The defaults cater to investors with different levels of risk tolerance (conservative/balanced/aggressive).

Schwab’s default investments are not good because they charge unconscionably high fees.⁷ Schwab does offer some low-fee options (listed in the previous section), so if you use Schwab, make sure you choose their low-fee funds instead of the defaults.

The other four providers—Charityvest, Daffy, Vanguard, and Greater Horizons—charge low fees on their default investments. So if you use one of those providers, you don’t need to worry about fees.

All five providers offer ESG portfolios. Again, though, Schwab’s ESG funds are unreasonably expensive.

All four providers’ default funds allocate too much to US stocks relative to the global market portfolio. I personally wouldn’t use a default allocation because I’d want to invest more into international stocks. Every provider lets you choose your own allocation if you want to.

Daffy custom portfolios

Daffy is the only DAF provider that lets you build a custom portfolio from a long list of eligible ETFs. Not every ETF is eligible, but there are around 500 choices.

Custom portfolios are only available for the $20/month tier, so this is not a cost-effective offering for smaller DAFs.

There is no full list of the available ETFs, but I used the search function⁸ to find that Daffy allows all of these ETFs:

All the big Vanguard index funds (VT, VTI, VXUS, VUG, VTV, BND, BNDX, VB, etc.)
Alternative assets: emerging market stocks (VWO), high-yield bonds (USHY), commodities (COMT), gold (AAAU), bitcoin (FBTC), ethereum (FETH), REITs (VNQ/VNQI)
Sector ETFs (technology, healthcare, consumer staples, etc.)
Factor funds from Vanguard, iShares, and Schwab; including value funds (VFVA, IVLU), momentum funds (VFMO, IMTM), and multi-factor funds (VFMF, ISCF, EMGF)

(The ETFs I listed are just examples. Daffy has multiple options for most categories.)

Unfortunately, Daffy does not include some of my favorite ETF providers: AlphaArchitect, Avantis, Cambria, and Return Stacked. It also doesn’t include any managed futures ETFs.

Advisor-managed accounts

Schwab, Charityvest, and Greater Horizons allow donors to appoint an investment advisor who can invest in things other than the pre-selected funds. Their program guidelines (links: Greater Horizons, Schwab (p. 11–14)) dictate what investments are allowed. Charityvest’s advisor-managed accounts must hold a minimum of $1 million.

Vanguard and Daffy allow advisor-managed accounts, but advisors don’t get to invest in anything beyond what individuals get.

Some examples of restrictions that apply to advisor-managed accounts at Schwab/Charityvest/Greater Horizons:

“Risky” investments are not allowed, including margin, short sales, options (except covered calls/puts), futures, and swaps, unless held within a mutual fund or ETF structure. (Return stacked funds should be allowed, and I confirmed that at least Greater Horizons definitely allows them.)
The advisor may not charge higher fees to the DAF than to the donor’s personal accounts.
The donor’s family members may not serve as investment advisors.

Schwab has some additional restrictions that Greater Horizons does not:

The account must trade using a Schwab brokerage account.
The advisor must establish a benchmark and then track that benchmark reasonably closely. If the account deviates substantially from the benchmark, the advisor is accountable to Schwab’s investment committee.
The account must meet certain asset allocation requirements. For example, it cannot allocate more than 25% to any one security, more than 25% to emerging market equities, or more than 50% “to publicly traded funds that pursue alternative or non-diversified investment strategies, e.g., commodities or cryptocurrencies”.
Advisors may not invest in funds that they own or manage.

(Charityvest allegedly also has some restrictions, but they’re not publicly listed.)

Greater Horizons provides the most flexibility to investment advisors. If I wanted to do anything unconventional, I would use Greater Horizons. (In fact, I do use Greater Horizons for exactly that reason.)

Contributing complex assets

Four of the five providers—Greater Horizons, Schwab, Charityvest, and Vanguard—can accept donations of complex assets, including cryptocurrency, private equity, real estate, and more.⁹ Daffy can accept cash, stocks/ETFs/mutual funds, and cryptocurrency, but not other types of assets.

These four providers claim to accept pretty much any type of asset that can legally be donated. But if you want to donate a large position in a complex asset, you should contact your DAF provider of choice to confirm they can receive it.

These providers all manage donations of complex assets through the same third party (Charitable Solutions LLC).¹⁰

(Amusingly, AEF, NPTrust, and Schwab claim they’re “uniquely flexible” when handling complex assets. If they’re all uniquely flexible, I guess that means none of them is?)

User experience

Most donors probably only care about the basic features of a DAF—they don’t need fancy investment options or the ability to donate complex assets. So the main deciding factors for most people are (1) fees and (2) user experience.

Of the five providers I’m focusing on, Greater Horizons easily has the worst sign-up process. You have to request that a customer service representative reach out to you. Then they send you a PDF form to fill out and the representative manually creates the account for you. In contrast, the other providers all let you sign up online and you can fill out your personal information on the website.

In the process of writing this article, I created accounts at Schwab, Vanguard, Daffy, and Charityvest and went through a few common use cases. I found them all easy to use. Vanguard was slightly more complicated than the others.

Another important aspect of user experience is the quality of customer service. Good customer service matters, but it’s difficult to assess—if you have a good/bad customer service experience, that might have more to do with the specific person you talked to than the quality of service in general. That said, I communicated through email and over the phone with all of the providers to get answers to my questions, and had decent experiences with all of them. I don’t know if my experience with Charityvest would generalize because the CEO specifically reached out to me to ask me to review it (see disclaimer). Daffy also reached out to me, but then didn’t do a great job of answering my questions. I have heard anecdotes from a personal friend¹¹ and from an independent financial advisor; they report that Daffy customer service was helpful and responsive when dealing with issues.

I’ve heard that Schwab has particularly good customer service. I’ve heard Vanguard the brokerage has worse customer service, but that Vanguard Charitable has a separate customer service department and it’s better than Vanguard the brokerage.

A side note: You might think it’s a good idea to evaluate user experience by looking at reviews on a site like Trustpilot or Better Business Bureau. I would avoid looking at those sites because they are protection rackets¹²—they offer to remove bad reviews in exchange for money. So the companies with good reviews are just the ones who pay to get the bad ones deleted.

I can’t give objective metrics for user experience, but my ranking based on my personal experience is Charityvest > Schwab > Daffy > Vanguard > Greater Horizons.

How risky are new DAF providers?

Daffy and Charityvest only started a few years ago. If they can’t earn enough money to sustain themselves, they might have to dissolve customers’ accounts and use their money to pay their bills (which they have a legal right to do, as established by National Heritage Foundation v. Highbourne Foundation). You might not want to use Daffy or Charityvest if you’re concerned about the risk.

I believe that:

There is a non-trivial probability that Daffy/Charityvest will close down and you’ll have to transfer your money to another DAF provider. (I have no reason to believe they’re riskier than the average startup, but most startups go out of business.)
If they do close down, they will almost certainly let you transfer your money to another DAF provider—it’s unlikely that you will lose your money permanently.

Here’s why I don’t believe you are likely to lose your money:

Charityvest and Daffy both use a financial structure that is (as far as I know) novel for DAF providers. They were founded concurrently with sister corporations (Vennfi, Inc. for Charityvest and Aside, Inc. for Daffy). Vennfi/Aside receive funding from VC firms and Charityvest/Daffy pay the corporations to develop their technology. This lets the corporations run at a loss using VC funding until the DAF providers grow large enough to make the corporations profitable.

This structure works in donors’ favor. If the for-profit corporation declares bankruptcy, VC funders can only make bankruptcy claims against the corporation, not against the DAF provider, so donors’ money is safe. And according to their 2022 public filings (viewable through the Florida Check-A-Charity website), Charityvest/Daffy have very little debt (or at least they did as of 2022).¹³

The best DAF provider(s)

This flowchart shows the best DAF provider depending on your circumstances.

A text description of this flowchart:

If you want an advisor-managed account, use Greater Horizons.
If you want to set your own investment allocation from a list of index funds:
- If you feel comfortable giving your money to a startup with an unproven track record, use Charityvest.
- Otherwise, use Vanguard (if your DAF will hold at least $25,000) or Schwab (otherwise).
If you are happy to use the DAF provider’s default investment allocation:
- If you feel comfortable giving your money to a startup with an unproven track record, use Daffy (if your DAF will hold at least $40,000) or Charityvest (otherwise).
- Otherwise, use Vanguard (if your DAF will hold at least $25,000) or Schwab (otherwise).

Some other factors you might care about, that I didn’t put on the flowchart because it would be too big:

If you only plan on keeping money in your DAF for a short time, use Schwab or Charityvest because they let you keep an empty account without charging you (the others charge a flat fee).
If you want to invest in cryptocurrency (without using an advisor-managed account), use Daffy.
If you want to donate complex assets (such as real estate or artwork), don’t use Daffy, but any of the other providers should be able to accommodate you.

This table illustrates my evaluation of the providers along five dimensions, ranked as bronze, silver, or gold.

	Fees	Investment Variety	User Experience	Advanced Features	Riskiness
Charityvest
Daffy
Greater Horizons
Schwab
Vanguard

So far, I have only looked at Charityvest, Daffy, Greater Horizons, Schwab, and Vanguard. That’s not because I believe these are better across the board than the other providers I didn’t discuss. For example, I think Fidelity has a better UI than Vanguard; and NPTrust offers more flexibility than Schwab for advisor-managed accounts. Rather, I chose these five DAF providers because each one is the best at something, while the other providers are not the best at anything (even though they are good at some things).

More on Fidelity, AEF, NPTrust, and T. Rowe Price

Even though I didn’t compare them in detail, here are my impressions of the other four DAF providers, ranked from best to worst.

Fidelity Charitable is as good as Schwab for most purposes. I like it better than T. Rowe Price, AEF, or NPTrust. I like Fidelity a little bit less than Schwab because it provides fewer investment options and the UI is slightly harder to use¹⁴, but Fidelity still seems like a fine choice as long as you’re careful to avoid their high-fee investment options.
- Bonus fact: If you have at least $5 million, Fidelity Charitable lets you manage your own investments (without appointing an advisor). I don’t believe any of the other DAF providers let you do that.
NPTrust and AEF are maybe passable but they have some serious issues:
- They’re more opaque than the other DAF providers, and I had a bad experience talking to them.¹⁵
- They make it more difficult to set up an account.
- For advisor-managed accounts, they impose more restrictions on the investment advisor than Greater Horizons does.
- They charge somewhat higher administrative fees than Fidelity/Schwab/Vanguard.
- NPTrust’s pre-selected investment funds are too expensive. NPTrust should only be used with an advisor-managed account. AEF does not have pre-selected funds—you must appoint an investment advisor.
- AEF’s website is so buggy that it’s unusable on Firefox. Even ignoring the bugs, AEF has a much worse UI than Fidelity/Schwab/Vanguard.
- A former AEF user reached out to me to give some more information about AEF:
  - The donor portal has limited functionality, for example there is no way to get a statement of transactions.
  - AEF charges a 1% fee if you withdraw or grant >80% of your funds within six months of opening the DAF.
T. Rowe Price Charitable is too expensive to be worth using. If you have a DAF at T. Rowe Price, you should consider switching to a new provider.

Appendix: Table of administrative fees

Fees can be found at these links: AEF¹⁶, Charityvest, Fidelity, NPTrust, Schwab, T. Rowe Price, Vanguard.

Greater Horizons does not publish its fees online. I learned what it charges by speaking to a representative. I don’t like it when companies don’t publish their fees, but I will respect Greater Horizons’ preferences by not disclosing them. If you want to know the specifics, contact them directly.

Minimum fees:

	AEF	Charityvest	Daffy	Fidelity	NPTrust	Schwab	T. Rowe Price	Vanguard
Minimum Fee	$500	$48	$36	$100	None	None	None	$250

Percentage-based fees:

Account Value	AEF	Charityvest	Fidelity	NPTrust	Schwab	T. Rowe Price	Vanguard
First $250K	0.70%	0.60%	0.60%	0.85%	0.60%	0.50%	0.60%
Next $250K	0.70%	0.60%	0.60%	0.70%	0.60%	0.50%	0.60%
Next $500K	0.35%	0.30%	0.30%	0.60%	0.30%	0.39%	0.30%
Next $1.5M	0.25%	0.20%	0.20%	0.45%	0.20%	0.18%	0.13%
Next $2.5M	0.15%	0.15%	0.15%	0.25%	0.15%	0.12%	0.13%
Next $5M	0.15%	0.10%		0.15%	0.13%	0.10%	0.13%
Next $5M	0.15%			0.10%	0.12%	0.10%	0.13%
Next $15M	0.10%				0.10%	0.09%	0.10%
Over $30M							0.05%

Caveats:

Charityvest has no minimum fee on cash-only accounts.
Daffy charges $36 per year, $60 per year, or $240 per year, depending on your account type. It does not charge any percentage-based fees.
Fidelity uses a flat fee schedule on accounts with over $5 million.

Some observations:

Daffy is the cheapest if you have more than $40,000.
Other than Daffy, Charityvest is cheapest or tied for cheapest above $6,000, and Schwab is cheapest below $6,000.
T. Rowe Price has the 3rd lowest administrative fees, but its investment options charge high expense ratios. It only offers two reasonably-priced funds (an S&P 500 index fund and a “balanced” index fund). T. Rowe Price only has the 3rd lowest all-in fee if you invest in the S&P 500 and nothing else.
Even though Schwab has lower fees than Vanguard at one tier, there is no account value at which Vanguard has a higher total fee.
Even though NPTrust has lower fees than Schwab at one tier, there is no account value at which Schwab has a higher total fee.
Fidelity’s flat fee structure above $5 million means that near the bottom of a tier (e.g., $11 million), it’s cheaper than Schwab or Vanguard, and near the top of a tier (e.g., $19 million), it’s more expensive.¹⁷

Disclaimers

The original version of this article did not include Charityvest or Daffy. I updated the article when representatives from Charityvest and Daffy reached out to ask me to include them. Everything I wrote about them is my own opinion.

I learned about Charityvest in May 2021, when the CEO, Stephen Kump, emailed me to ask me to review it prior to its public launch. He gave me access to the beta (which I used to check out Charityvest) and added $20 to my account (which I did not use, to avoid any conflict of interest). I did not receive any other compensation for writing about Charityvest.

In early 2022, a representative from Daffy reached out to me, and I proceeded to forget about their message for a year. I rediscovered it in late 2023, when a couple of other people asked me for my opinion on Daffy and I decided to update this article to include it. I did not receive any incentive from Daffy.

Changelog

2022-03-10: Add a Charityvest review and add it to the recommendation flowchart.
2023-01-06: Update fee minimums.
2024-04-24: Significant revisions:
- Add changelog.
- Add a Daffy review.
- Add a new section on default investment options.
- Correct an error: I previously wrote that Vanguard’s account minimum is $0, but it is $25,000.
- Update information on T. Rowe Price: I originally wrote that it only offered one reasonably-priced fund, but now it offers two.
- Substantially change recommendations based on a number of factors:
  - Add consideration to a provider’s default investment options. Previously, I hadn’t considered them at all, but I expect many (most?) DAF users will use the defaults, so that was an oversight.
  - Downgrade my recommendation of Schwab. Previously, I had Schwab as my top recommendation for most people, but their default funds are too expensive.
  - Add Daffy as a possible recommendation and rearrange the flowchart accordingly.
  - Remove the “best DAF provider for most people” recommendation (previously I recommended Schwab). Every DAF provider has some hiccup that a significant number of people won’t like, so I don’t think I can reasonably give a general recommendation. (Schwab’s default funds are too expensive; Vanguard is too expensive for small donors; Charityvest/Daffy are new orgs that might go out of business.)
  - Generally rewrite to increase clarity.
2025-01-31: Add some more information about AEF.
2025-02-05: Update to reflect Charityvest’s recently raised fees. Previously Charityvest was cheaper than Fidelity/Schwab at most account values, and now it’s a three-way tie up to $5 million.
2025-05-09:
- Daffy now allows you to choose your own ETFs. Update to explain this and incorporate it into the flowchart.
- On the previous update, I forgot to account for Charityvest’s raised fee when determining which DAF provider is cheapest. Update to fix this.

Notes

I don’t know how DAFs work outside the United States. If you do have DAFs in your country, you could use this post as a guide for what sorts of things to look for when choosing a DAF provider. ↩
I can’t provide a link for Greater Horizons because its fees are not publicly available. But I will say that they’re broadly similar to Vanguard and Schwab. ↩
At $40,000, Charityvest/Schwab/Vanguard’s 0.60% fee costs $240 per year, which exactly equals Daffy’s fee.

At $6,000, Charityvest is (marginally) more expensive than Daffy’s basic account. But Schwab’s 0.6% fee costs $36/year, the same as Daffy. ↩
At $16,667, Schwab’s 0.6% fee equals Charityvest’s $100/year minimum. ↩
Vanguard requires an initial contribution of $25,000, but you do not need to maintain a balance of $25,000. ↩
An advisor-managed account will probably still have limits on how much cryptocurrency you can hold, something like 50% of your portfolio. Daffy lets you invest up to 100% of your portfolio in crypto. ↩
Schwab actually got sued for its high fees (see Philip Pinkert v. Schwab Charitable Fund), but the court ruled in Schwab’s favor on the basis that Schwab legally owns donated funds so they have no fiduciary duty toward donors. ↩
Note: You have to be logged in to view this link. You can create an account without paying a membership fee. ↩
Vanguard’s website does not state that they can accept cryptocurrency, but I confirmed with a representative that they take donations of cryptocurrency if the value of the contribution is at least $50,000. ↩
AEF, Fidelity, and NPTrust also use Charitable Solutions LLC. It seems that Charitable Solutions LLC has something of a monopoly on managing donations of complex assets. ↩
h/t Barak Gila for talking to me about his experiences with Daffy. ↩
I have strong private evidence for this claim. The public evidence for this claim is weak because review site protection rackets, much like the mafia, know how to avoid leaving any incriminating evidence. But you can find many anecdotes, such as here, here, and here (see also the comment section on LinkedIn) from business-owners who claim Trustpilot extorted them, and here from an alleged former employee of Trustpilot. I didn’t look for evidence on Better Business Bureau, but in my experience it’s more widely known that BBB is a protection racket, so evidence shouldn’t be hard to find. ↩
They both had less than $20,000 of debt against tens of millions of dollars of assets. The debt was entirely accounts payable, with no other sources of debt. I had to look up what “accounts payable” means but if I understand correctly, that means they were charged a fee (presumably by the partner corporation) and they haven’t paid the fee yet. ↩
Fidelity’s and Schwab’s UIs are nearly identical. The only real difference I noticed is that Schwab does a better job of conglomerating your accounts. If you have both a brokerage account and a DAF at Schwab, you can easily switch between them from the landing page. If you have both types of accounts at Fidelity, you can navigate to your DAF from the landing page, but it sends you to a separate URL and sometimes requires you to log in again. ↩
My incredible customer service experience with NPTrust:
- I send them an email to ask what types of investments they allow.
- A representative replies and says we need to set up a phone call.
- On the phone call, I repeat my question. They then email me a PDF that contains their investment requirements.
I’m tempted to send them one of those “this meeting could have been an email” coffee mugs. Or, better yet, don’t make me email you, just list your offerings on your website. ↩
AEF does not provide a fee schedule, only a fee calculator. I used the calculator to reverse-engineer the fee schedule. ↩
For the mathematically inclined, this function converts Fidelity’s flat fee into an equivalent tiered fee on amounts over $5 million, given two variables AUM and flat_fee:
```
tiered_fee = ((AUM * flat_fee) - $11,250)
             / (AUM - $5,000,000)
```
↩

More Evidence on Concentrated Stock Selection

Thu, 01 Apr 2021 00:00:00 -0700

In Can Investors Simulate Leverage via Concentrated Stock Selection?, I used the Ken French Data Library to show that concentrated value or momentum factor portfolios historically had both higher returns and higher risk than diversified value or momentum portfolios. Thanks to a newly released dataset, we can replicate this finding across 137 factors.

Jensen, Kelly & Pedersen (2021), Is There a Replication Crisis in Finance?, attempted to replicate 153 factors that supposedly predict stock performance. More importantly for our purposes, the authors publicly released their source code and data.

Previously, using the Ken French Data Library, I found that equal-weighted (EW) factor portfolios performed better than value-weighted (VW) portfolios. This performance could not be explained by factor exposure—equal-weighted portfolios had “alpha” on top of the value-weighted factor. (I use scare quotes because it’s not true alpha. It’s still a form of factor exposure, but it’s factor exposure that a value-weighted factor construction fails to capture.) Can we replicate this finding using the Jensen, Kelly & Pedersen dataset?

Note: This essay assumes the reader has some familiarity with CAPM and factor investing. My previous essay provides a gentler introduction to the relevant concepts.

I filtered the dataset down to just those factors that had data for US stocks starting in 1960. The full dataset goes back to 1926, but many factors are impossible to calculate that far back because the necessary data doesn’t exist. I started in 1960 because we have enough data by that point to calculate the returns of almost all the factors (137 out of 153).

The hypothesis is that EW factors provide a stronger signal. If a particular VW factor has a positive return, then its EW equivalent should have a higher return. And if a VW factor has a negative return, then the EW version should have a lower return.

I tested this hypothesis in three ways:

For each of the 137 factors, does EW have a larger absolute value of return than VW, and with the correct sign?
For each of the 137 factors, does EW have a larger absolute value of risk-adjusted return?
Does EW provide a stronger signal if we restrict to just those factors where VW has a risk-adjusted return of at least 0.3?

My reasoning for the third test is that factors with small risk-adjusted returns might not have any predictive power—they’re more likely to be noise. If we restrict our dataset to just the value-weighted factors with good performance (taking 0.3 as an arbitrary threshold for “good”¹), then we should expect their equal-weighted equivalents to have even better performance.

I calculated the returns (or risk-adjusted returns) for every factor and took the difference between EW and VW. Then I calculated the following summary statistics:

number of eligible factors
number of factors for which the EW formulation provided a stronger signal than the VW version
mean difference between EW and VW factors
standard error of differences
t-statistic for the null hypothesis that the mean equals zero

	Absolute Return	Risk-Adjusted Return	Risk-Adjusted (> 0.3)
n	137	137	56
n(EW>VW)	110	112	54
mean	1.67%	0.245	0.493
stderr	0.21%	0.032	0.041
t-stat	7.6	8.0	12.2

In all three cases, EW provided a stronger signal for a large majority of factors, and all three differences had extremely high t-statistics. The t-statistic was highest when we restricted the sample to only factors with a risk-adjusted return of 0.3 or higher.

This supports my previous finding that, before fees and transaction costs, EW factor portfolios outperform VW portfolios.

Two important caveats:

The Jensen, Kelly & Pedersen dataset only includes long/short factors. It’s possible that most of the outperformance of equal-weighted factors happens on the short side. This was not the case on the Ken French data, so it stands to reason that it’s not the case for most of these new factors, either. But we can’t say for sure without getting more data.
This analysis does not account for fees and transaction costs. Equal-weighted portfolios incur higher costs, and they’re prohibitively expensive for sufficiently large investors (with perhaps $1 billion or more). Small investors can invest in equal-weighted portfolios without paying much in transaction costs. I discussed this in more detail previously.

Disclaimer: This should not be taken an investment advice. Any given portfolio results are hypothetical and do not represent returns achieved by an actual investor.

Notes

I chose 0.3 as the threshold because it’s a bit less than the risk-adjusted return of the US stock market minus T-bills. My reasoning is that if a factor performed close to as well as the stock market itself, then it’s a pretty good factor (at least ex post). ↩

The True Cost of Leveraged ETFs

Thu, 04 Mar 2021 00:00:00 -0800

Updated 2025-01-18 to fix some flaws in the calculations and add new data for 2021–2024. Tables have been updated to include the new results. For more, see 2025 Update: How have things changed? and Appendix A: Changes to calculation methodology.

Under some circumstances, altruists might prefer to leverage their investments. The easiest way to get leverage is to buy leveraged ETFs. But leveraged ETFs charge high fees and incur other hidden costs. These costs vary substantially across different funds and across time, but on average, leveraged ETFs have historically had annual excess costs of about 1.5% per 100% leverage, or around 1% on top of the expense ratio.

Given reasonable expectations for future returns, leveraged ETFs most likely have substantially higher arithmetic mean returns than their un-leveraged benchmarks. They also appear to have higher geometric mean returns than their benchmarks, but only by a small margin. More pessimistic estimates would find that adding leverage decreases geometric return.

Note: Many investors can get leverage more cheaply via other methods, such as margin loans or futures. Even if leveraged ETFs appear better than un-leveraged investments, other forms of leverage might be better still.

Disclaimer: This should not be taken as investment advice.

Contents
Measuring the cost of leveraged ETFs
Are leveraged ETFs worth it?
2025 Update: How have things changed?
Source code
Appendix A: Changes to calculation methodology
Appendix B: Original tables, for posterity
Notes

Measuring the cost of leveraged ETFs

To calculate costs, I created a list of leveraged ETFs and looked up their historical returns from the start of 2016 to the end of 2020. I paired each leveraged ETF with an equivalent un-leveraged benchmark ETF. I calculated what return you would have earned if you had taken that ETF and applied the appropriate amount of leverage (either 2x or 3x, depending on how much the corresponding leveraged ETF uses), paying only the risk-free rate.¹ Then I subtracted the actual return of the leveraged ETF to find the excess cost.

My analysis included these ETFs:

ETF	Leverage	Index	Index ETF
SPXL	3	S&P 500	SPY
UPRO	3	S&P 500	SPY
SSO	2	S&P 500	SPY
UMDD	3	S&P 400 (mid cap)	IJH
URTY	3	Russell 2000 (small cap)	IWM
EFO	2	EAFE	EFA
EURL	3	Europe	VGK
EZJ	2	Japan	EWJ
EET	2	emerging markets	EEM
EDC	3	emerging markets	EEM
TQQQ	3	NASDAQ	QQQ
TMF	3	long-term US Treasury bonds	TLT

The following table shows the total excess cost and after-fee cost for various leveraged ETFs (2016–2024). Excess cost is shown per 100% leverage (so the excess on a 3x ETF is divided by two). After-fee cost tells us the excess cost minus the expense ratio.² r gives the correlation between the leveraged ETF and the simulated leveraged benchmark ETF.

ETF	Excess Cost	After Fee	r
SPXL	1.14%	0.82%	0.999
UPRO	1.17%	0.85%	0.999
SSO	1.51%	0.80%	0.999
UMDD	0.94%	0.54%	0.991
URTY	1.22%	1.03%	0.999
EFO	2.31%	1.48%	0.947
EURL	1.37%	0.98%	0.995
EZJ	2.32%	2.20%	0.936
EET	1.83%	1.04%	0.987
EDC	1.17%	0.72%	0.998
TQQQ	1.09%	0.97%	0.999
TMF	0.77%	0.48%	0.997
Average	1.40%	0.99%

The leveraged ETFs generally had an excess cost of 1.43%, or 1.01% in excess of the ETF expense ratios. Almost all fund pairings had high correlations, which tells us that the leveraged ETFs do a good job of tracking their benchmarks (with the possible exceptions of EFO and EZJ).

Excess costs were not stable—they varied substantially from year to year. The next table shows excess cost for each individual year:

ETF	2016	2017	2018	2019	2020
SPXL	1.39	1.63	0.85	1.77	2.74
UPRO	0.99	1.52	0.86	2.22	2.48
SSO	1.44	1.74	1.39	2.57	2.63
UMDD	1.40	1.94	0.56	1.43	0.13
URTY	1.14	1.00	0.46	1.50	3.71
EFO	0.84	2.91	2.05	4.24	1.52
EURL	1.43	2.61	0.72	3.33	1.44
EZJ	2.88	1.63	1.84	3.49	1.78
EET	2.23	1.96	1.37	1.85	1.35
EDC	1.41	1.95	0.49	1.91	1.12
TQQQ	1.05	1.75	0.64	2.30	3.51
TMF	0.71	0.56	0.12	0.94	1.71
Average	1.41	1.77	0.95	2.30	2.01

New data through 2024:

ETF	2021	2022	2023	2024
SPXL	1.94	0.29	1.61	2.13
UPRO	2.06	0.44	1.97	2.14
SSO	2.37	0.86	2.09	2.46
UMDD	1.66	0.37	1.68	1.94
URTY	0.70	0.40	1.71	1.24
EFO	2.71	2.28	3.83	2.28
EURL	2.01	0.32	1.61	1.20
EZJ	2.49	2.08	2.87	2.75
EET	1.08	1.63	3.31	2.10
EDC	1.15	0.44	1.32	1.63
TQQQ	0.66	0.16	0.70	0.99
Average	1.74	0.80	2.12	1.92

For individual years, excess costs ranged from as low as 0.13% to as high as 4.24%.

Some factors that might contribute to this year-to-year variance:

The ETFs’ counterparties charge different rates based on perceived risk.
Market liquidity varies over time.
The ETFs might experience some tracking error.

Excess cost does not include the risk-free rate, so the year-to-year variance has nothing to do with changes in the risk-free rate.

Some people observe that 3x leveraged funds usually provide less than a 3x return due to volatility drag. To be clear, the excess cost I found is not related to volatility drag. Volatility drag equals the difference between the arithmetic mean and geometric mean of an investment. If two investments A and B have the same arithmetic return but B has higher volatility, then B will have lower geometric return.

Specifically, the geometric mean of an investment equals $\alpha - \sigma^2/2$, where $\alpha$ is the arithmetic mean and $\sigma$ is the standard deviation.

The geometric mean of a 3x leveraged ETF is less than 3x the arithmetic mean. For example, if the benchmark has a geometric mean of 5% and an a standard deviation of 16%, then we calculate the geometric mean of the leveraged ETF as:

Arithmetic mean of the benchmark is $5\% + 16\%^2/2 = 6.28\%$
Arithmetic mean of the leveraged ETF is $3 \times 6.28\% = 18.84\%$
Standard deviation of the leveraged ETF is $3 \times 16\% = 48\%$
Geometric mean of the leveraged ETF is $18.84\% - 48\%^2/2 = 7.32\%$

In this example, adding 3x leverage only increases the expected geometric return from 5% to 7.32%. If we assume a 3x leveraged ETF has a 2.5% excess cost, that reduces its expected geometric return to 4.82%, which is lower than the return of an un-leveraged ETF.

Are leveraged ETFs worth it?

Mathematically speaking, an agent with a logarithmic utility function wants to maximize geometric mean, while a risk-neutral agent wants to maximize arithmetic mean. A logarithmic utility function is probably still too aggressive for self-interested investors, but it might be appropriate for altruists. Under some circumstances, altruists might even be close to risk-neutral.

For risk-neutral investors, leveraged ETFs pretty clearly have higher expected utility than normal ETFs. (Although they might prefer to get leverage some other way if they can, since leveraged ETFs are one of the most expensive forms of leverage.)

Are leveraged ETFs worth it for investors with logarithmic utility? That is, can we expect leveraged ETFs to have a higher geometric return than ordinary ETFs?

That depends on three things: the expected return of the benchmark, volatility, and excess cost.

Let’s assume an excess cost of 2% and a risk-free rate of 0%. Long-run market volatility generally varies from 15% (for more stable markets like the S&P 500) to 22% (for more volatile markets like US small-caps or emerging markets). Then we can ask, what expected geometric return does an index need to have for a leveraged ETF to be worth it? Let’s assume we can use the optimal amount of leverage, not just 2x or 3x. (We can combine a 3x ETF with an un-leveraged ETF to get an intermediate amount of leverage.)

At a 15% standard deviation, an investment needs an expected (geometric) return higher than 3.125% for leverage to be worth it. At a standard deviation of 22%, the investment must return more than 4.42%.³

If we assume a total cost of 2.5% (e.g., an excess cost of 2% plus a risk-free rate of 0.5%, or an excess cost of 2.5% when the risk-free rate is zero), then an investment with 15% standard deviation must return 3.625%, and one with 22% standard deviation must return 4.92%.

What returns can we expect from various market indexes? We can’t know with high confidence, but we can try to estimate it. The following table gives return estimates for various equity regions according to Research Affiliates (RA) and AQR, and standard deviation estimates from Research Affiliates.⁴

Index	Return (RA)	Return (AQR)	Stdev (RA)
US large	2.0%	5.9%	15.4%
US small	4.3%	N/A	20.7%
EAFE	6.3%	6.5%	17.3%
Europe	6.4%	6.3%	19.2%
Japan	5.8%	6.3%	17.6%
emerging	7.6%	7.0%	21.7%

The next table gives the expected (geometric) return of an optimally-leveraged portfolio⁵ minus the expected return of the index, constructed using a leveraged ETF with a 1.5% cost.⁶

Index	Improvement (RA)	Improvement (AQR)
US large	0.1%	2.2%
US small	0.1%	1.5%
EAFE	1.8%	2.1%
Europe	1.3%	1.2%
Japan	1.2%	1.7%
emerging	1.5%	1.1%

Applying leverage at least marginally increases expected geometric return in every sample case. However, leverage provides only a modest improvement, with the best case being 2.2 percentage points, and the projected improvements in some cases being as low as 0.1 percentage points.

Vanguard’s return expectations produce similar numbers (not shown).

Of course, these results heavily depend on what inputs we use for expected return, volatility, and costs, and the true numbers might differ significantly. If costs turn out to be closer to 3%, or if true (ex-ante) expected returns are one percentage point lower, that would eliminate the benefit of using leverage.

Or suppose you are trying to decide between two ways of investing:

Buy leveraged ETFs in a donor-advised fund (DAF).
Buy un-leveraged ETFs in a taxable account, and use margin to get leverage.

Say your DAF has to pay a 0.3% fund fee plus a 1% fee for an investment manager, and margin costs 0.5% over the risk-free rate. That means the DAF has to pay an extra 2.3% cost above what the taxable account pays (DAF fees plus leveraged ETF cost minus margin cost), whereas the taxable account must pay taxes. 2.3% is a lot. It’s not obvious whether it outweighs the tax benefits of a DAF, but I personally would lean toward keeping my money in a taxable account.

2025 Update: How have things changed?

I ran the numbers again to cover the four years since this post was first published. In short:

My previous methodology had some flaws, and overestimated the true average cost by about 0.5 percentage points. (See Appendix A for a full explanation.)
Since 2021, excess costs have continued to vary a lot from year to year.
Excess costs were just as high after 2020 as they were before 2020.

2016–2020 had an average annual cost of 1.77% (taking the arithmetic mean across all ETF-year pairs) and 2021–2024 had an average of 1.74%. The difference was highly insignificant (p = 0.87).

Source code

Source code for the 2025 revision of this article is available on GitHub.

Appendix A: Changes to calculation methodology

In the 2025 update to this post, I made several changes to my original methodology that changed the numbers.

Based on the script I originally used, the average excess cost of ETFs (2016–2020) was 2.09%.
I discovered that I had been calculating the risk-free rate incorrectly. I converted annualized T-bill yields into daily interest by raising to the power of 1/365. That’s not correct because there are not 365 trading days per year—there are approximately 252. Raising to the power of 1/252 instead reduced the average excess cost to 1.64%.
I completely re-wrote the calculations because I lost my original script (I later re-discovered it). The new script uses a more rigorous method for calculating the cost of leverage. Instead of assuming the interest paid is uniform across 252 trading days, it pays interest on every calendar day, using the last known rate on days when the market is closed. The new script also correctly handles days where the stock market is open but the Treasury is not (which happens occasionally). The new script produces an excess cost of 1.61%.

There is one additional issue that has no simple resolution: there are multiple ways to define “average excess cost”, and it’s not clear which one is best.

We can calculate excess cost in at least three ways:

Take the total return of the simulated leveraged ETF; annualize it by raising to the power of the number of years in the sample; do the same for the actual leveraged ETF. Let the excess cost be the difference between those two numbers. (This is the method my original script used.)
For each individual year, calculate the difference in annual return between the simulated and actual ETFs. Let the excess cost be the simple average of each year’s costs. (That’s the method I used above when I wrote that 2016–2020 had an average annual cost of 1.69%.)
Let the excess cost be the number X such that, if you took the simulated ETF and deducted X from the return every year, the resulting total return (across the full sample) would equal the total return of the actual ETF.

The third method seems most reasonable to me, so it’s what I used in my new script. I like the third method because if an ETF charged a management fee, the third method is how you would calculate the impact of the fee.

While the first method found an average cost of 1.65%, the third method found a cost of 1.35%.

The table above reported an average of 1.43%, not 1.35%, because (1) the 2025 version includes a longer date range and (2) it includes two new leveraged ETFs.

For the 2025 edition, I added these leveraged ETFs and their corresponding index ETFs:

Leveraged ETF	Leverage	Index	Index ETF
SSO	2	S&P 500	SPY
TQQQ	3	NASDAQ	QQQ

Updated again 2025-01-22. I realized there was a data entry error where for 2016–2017 I incorrectly used the 4-month Treasury yield as the risk-free rate rather than the 3-month Treasury yield. This only slightly changed the results.

Appendix B: Original tables, for posterity

These are the tables I originally included in the post in 2021. I made some mistakes in calculating these so the figures are not correct.

ETF	Excess Cost	After Fee	r
SPXL	2.23%	1.72%	0.995
UPRO	2.12%	1.64%	0.995
UMDD	1.31%	0.84%	0.995
URTY	2.08%	1.61%	0.993
EFO	2.47%	1.14%	0.978
EURL	1.97%	1.44%	0.984
EZJ	2.79%	1.84%	0.981
EET	2.18%	1.25%	0.994
EDC	1.67%	1.20%	0.994

ETF	2016	2017	2018	2019	2020	Min	Max
SPXL	1.39	2.02	1.22	3.25	3.01	1.22	3.25
EET	2.73	2.63	1.55	2.71	1.58	1.55	2.73
UPRO	1.03	1.90	1.27	3.51	2.76	1.03	3.51
UMDD	1.58	0.85	1.04	2.03	0.06	0.06	2.03
URTY	1.05	1.59	0.87	2.48	3.88	0.87	3.88
EURL	1.51	3.08	1.15	4.21	1.34	1.15	4.21
EZJ	2.05	2.07	2.11	4.71	1.87	1.87	4.71
EDC	1.45	2.55	0.73	2.89	1.30	0.73	2.89
EFO	0.83	2.73	2.51	4.02	1.32	0.83	4.02
Min	0.83	0.85	0.73	2.03	0.06	0.06	2.03
Average	1.51	2.16	1.38	3.31	1.90	1.03	3.47
Max	2.73	3.08	2.51	4.71	3.88	1.87	4.71

Index	Improvement (RA)	Improvement (AQR)
US large	<0%	1.6%
US small	0.0%	N/A
EAFE	1.3%	1.5%
Europe	0.9%	0.8%
Japan	0.8%	1.2%
emerging	1.1%	0.7%

Notes

I used the 3-month T-bill yield as the risk-free rate. ↩
I looked at the ETFs’ current expense ratios (as of 2025-01-18) and assumed that they stayed the same over the whole sample period, which might not be true.

I did not use the actual expense ratios for EFA and EEM. Instead I used the expense ratios for VEA and VWO respectively. EFA/VEA and EEM/VWO cover the same markets but use different benchmarks so they are not perfectly correlated. Realistically, if you wanted to invest in EAFA or emerging markets, you’d buy VEA or VWO respectively, not EFA/EEM. But I used EFA/EEM as the benchmarks for EFO/EET/EDC because they’re more strongly correlated, which means the estimated excess costs have less variance. ↩
In general, a leveraged ETF has the same geometric return $\mu$ as the index when $\mu = \sigma^2 / 2 + R + c$ for standard deviation $\sigma$, risk-free rate $R$, and excess cost $c$.

Another way to put it: when $\mu = \sigma^2 / 2 + R + c$, optimal leverage is exactly 1x. For any higher value of $\mu$, adding leverage increases geometric return. ↩
Research Affiliates estimates are as of 2021-01-31; AQR estimates are as of January 2021.

I did not update these numbers for the 2025 revision because the exact numbers don’t matter too much, they’re more meant as an illustration of how different return assumptions affect the viability of leveraged ETFs. ↩
If geometric mean maximizing leverage is less than 3x (which it usually is), then we can construct an optimally-leveraged portfolio by holding both a 3x leveraged ETF and an un-leveraged ETF and rebalancing their weights daily. ↩

Improvements can be calculated using the following Python code:

def return_improvement(mu, sigma, cost):
    leverage = (mu + sigma**2/2 - cost) / (sigma**2)
    return (
        leverage * (mu + sigma**2/2) - (leverage * sigma)**2/2
        - (leverage - 1) * cost
        - mu
    )

↩

Investors Can Simulate Leverage via Concentrated Stock Selection

Mon, 08 Feb 2021 00:00:00 -0800

Summary

Last updated 2022-04-15; see Errata.

Confidence: Highly likely.

Some altruists are much less risk-averse than ordinary investors, and may want to use leverage. But foundations and donor-advised funds legally cannot access most forms of leverage. As an alternative approach, leverage-constrained investors could buy concentrated positions in small-cap value and momentum stocks. For example, instead of buying an ETF that holds the best half of the market as ranked by value or momentum, they could buy the top 10%.

According to backtests, when portfolio concentration increases, both return and risk increase, and return increases more than risk (so that concentrated portfolios have higher risk-adjusted returns).

Large investors cannot hold concentrated portfolios without moving the market, so they probably prefer to use leverage if they can. Small investors probably prefer to buy concentrated investments because they offer higher risk-adjusted returns than leveraged broad portfolios.

Disclaimer: This should not be taken as investment advice. Any given portfolio results are hypothetical and do not represent returns achieved by an actual investor.

Summary
Contents
Motivation
Known approaches
Increasing expected return via concentration
Fees, transaction costs, and taxes
How does concentrated investing compare to using leverage?
On future expectations
Finding concentrated ETFs
Errata
Appendix A: Significance tests
Appendix B: Replication on international equities
Appendix C: Factor regression on selected portfolios
Notes

Motivation

Some investors cannot use leverage. For example, foundations, donor-advised funds, and IRAs are prohibited from using certain types of investments, including margin loans. Other investors have access to leverage, but have to pay high fees to get it. But sometimes, investors want to increase their return and risk. Can we do that without explicitly using leverage?

Known approaches

Some stocks are more volatile than others. One way to simulate leverage might be to buy stocks with high volatility. Unfortunately, this doesn’t work: high-vol stocks do not provide enough return to justify the higher risk.¹² This may happen because leverage-constrained investors already overpay for stocks with high volatility.

As a different approach, we could tilt toward small-cap, value, and momentum stocks. These tilts all tend to increase both risk and return.³⁴⁵⁶ (Gordon Irlam has proposed that altruists use small-cap value rather than leverage as a (sometimes) better method of increasing return.) Most funds that implement these tilts hold around half to a third of the total market, and end up with a little bit higher volatility and expected return than the market, but not much higher.

Increasing expected return via concentration

Small-cap stocks, value stocks, and especially small-cap value stocks tend to earn higher return (and higher risk) than the broad market. Small-cap and value index funds typically hold about half the market. Can we enhance our return⁷ by concentrating in a narrower corner of the market, such as 10% instead of 50%?

Let’s use the Ken French Data Library to look at portfolios formed on size, value, and momentum at various levels of concentration.

Ignoring size for now, did more concentrated value and momentum portfolios historically perform better?

The chart below shows historical returns for US stocks using three valuation metrics and one momentum metric:

value: book to market (B/M)
value: earnings to price (E/P)
value: cash flow to price (CF/P)
momentum: 12-month return, excluding the most recent month (Mom)

It includes both equal-weighted (EW) returns (where each stock receives the same weight) and value-weighted (VW) returns (where stocks are weighted in proportion to market cap).⁸

For each of the four metrics, return monotonically increased with concentration (with the sole exception of B/M value-weighted moving from top 20% to top 10%).

Let’s narrow in on value stocks as measured by B/M. The next chart shows B/M returns over the total market as well as over small caps (defined as the 10% smallest stocks):⁹

We see a similar pattern, where more concentration means higher return. Within small caps, this pattern has a steeper slope, and small-cap value stocks consistently outperformed all-cap value stocks except at the weakest level of concentration. Relatedly, equal-weighting outperformed value-weighting—equal-weighting gives relatively more weight to small caps, so this is a different form of small cap tilt.

Most off-the-shelf value and momentum funds hold about half the market, value-weighted. If instead of buying one of those funds, we held only the top 10% and used equal weighting, historically we could have increased returns by a lot:

	50% VW	10% EW	Improvement
B/M	11.0%	18.7%	7.7%
E/P	13.8%	17.8%	4.0%
CF/P	12.6%	18.1%	5.5%
Mom	12.6%	19.2%	6.6%

As return increased with concentration, volatility increased at a similar pace. Equal-weighted portfolios had higher risk-adjusted returns across the board than their value-weighted counterparts; but equal-weighted portfolios had similar risk-adjusted returns at all levels of concentration (including 50%, 40%, 30%, 20%, and 10%).

These differences in returns across concentration, market cap, and weighting are highly statistically significant, as are the differences in volatility (see Appendix A).

The following table shows the risk-adjusted returns (Sharpe ratios) for value and momentum portfolios:

	50% VW	10% EW	Improvement
B/M	0.45	0.57	0.12
E/P	0.71	0.74	0.03
CF/P	0.63	0.71	0.08
Mom	0.57	0.73	0.16

We can get higher returns (accompanied by higher risk) by going even deeper into small-cap value and momentum. According to some backtests I ran on a different (non-public) data set, a portfolio of the top 30 value stocks (equal-weighted) would have historically returned about 30% per year gross of costs, with a standard deviation of about 45%. This is similar to the historical performance of a US total market index with 3:1 leverage. However, this portfolio would mainly have consisted of very small stocks (market cap $200 million or less), so trading costs could destroy a lot of the value of this strategy in practice.¹⁰

Fees, transaction costs, and taxes

So far, I have focused on gross returns. But real-world strategies have to pay various costs, including management fees, trading costs, and taxes. How much do these matter?

I did not find much published research on real-world transaction costs and I only have limited personal experience trading individual stocks, so take the following with a grain of salt.

Investors with less than $1 million probably shouldn’t worry about transaction costs. Even micro-cap stocks (with market caps between $50 million and $200 million) are liquid enough that small investors will probably end up paying 0.25% or less per trade. If you buy value stocks and rebalance once a year, transaction costs will probably be low enough not to matter. Momentum requires more frequent rebalancing, so even small investors might prefer to restrict themselves to some minimum market cap threshold (somewhere in the $200 to $500 million range) or take other measures to reduce costs.

Larger investors must take care not to move the market when they buy and sell stocks. Investors with substantially more than $1 million might prefer not to trade micro-cap stocks. Investors with much more money (perhaps around $100 million) can’t use equal weighting because they will move the market too much if they try to put substantial positions into small-caps.¹¹

If you hire an investment manager to run a strategy like this, they will charge a fee—probably around 1% per year. This dampens the value of investing in a concentrated portfolio rather than simply buying an off-the-shelf mutual fund or ETF. Or you could manage your own portfolio, avoiding the fee but giving yourself more work to do. I used to manage my own concentrated portfolio of value stocks, and it took me two hours of work per year. A momentum strategy would probably require somewhere around 10–20 hours per year. But if you manage the portfolio yourself, you need to be confident that you won’t make trading mistakes.

Value and momentum portfolios both have some turnover, and turnover means you pay taxes on gains. The more concentrated a strategy, the more turnover it has. This may be prohibitive for taxable investors, who will pay substantial taxes if they have to sell positions regularly.¹² Taxable investors could avoid turnover by buying an ETF that uses an appropriately concentrated strategy. ETFs charge fees, and concentrated funds tend to have substantially higher fees than broad ETFs. But these fees are probably still lower than the cost of hiring an investment manager, and lower than the tax drag from trading individual stocks.

How does concentrated investing compare to using leverage?

Adding leverage to a portfolio increases both return and volatility while keeping risk-adjusted return the same (before costs). In the backtests we looked at, more concentrated portfolios had both higher returns and higher risk-adjusted returns.

Two other important considerations:

Small investors have no difficulty buying an equal-weighted portfolio of small-cap stocks. Large investors cannot buy small-caps without moving the market.
Small investors face relatively high costs of leverage. Large investors can take advantage of their size to get leverage more cheaply.

These considerations suggest that small investors should invest in concentrated portfolios while large investors should use leverage.

Under some conditions, small investors may still prefer to hold leveraged broad portfolios if they can, mainly for two reasons:

Manually managing a basket of stocks is both less tax-efficient and more time-consuming than buying and holding an ETF (or group of ETFs) and applying leverage.
The analysis in this essay only considered stock portfolios. Adding bonds, real assets, or other factors might improve an investor’s overall risk-adjusted return.

On future expectations

So far, we have looked at simulated historical performance of different strategies. That doesn’t tell us how these strategies will perform in the future.

We care about two questions:

Will value/momentum investing continue to outperform the market?
Will concentrated value/momentum investing continue to beat the market, and continue to beat diversified value/momentum investing?

The first question has been addressed elsewhere in detail. For a deep dive into arguments on why value investing might not work anymore and why they’re probably wrong, see Israel, Laursen & Richardson (2020), Is (Systematic) Value Investing Dead? and Cliff Asness’ (less rigorous, easier to read) article of the same name. The future of momentum investing is much less clear. Israel & Moskowitz (2013)’s The Role of Shorting, Firm Size, and Time on Market Anomalies provides at least some reason to expect momentum’s outperformance to persist, but no strong evidence.¹³

For the second question, if we expect value and maybe momentum to persist, then we should probably expect concentrated portfolios’ even better performance to persist for the same reasons. In fact, the argument that concentrated value/momentum will persist appears even stronger. If strategies like value and momentum become more popular among large sophisticated investors, then they will tend to perform worse. But large investors cannot buy equal-weighted small cap portfolios. And small investors who do use value/momentum typically invest via broad mutual funds or ETFs, not concentrated baskets of stocks.

As a counterpoint, just because a strategy has higher risk doesn’t mean you’re compensated for that risk. For example, a basket of 30 randomly-chosen stocks is riskier than an index fund, but doesn’t have higher expected return.¹⁴ Investing in concentrated value/momentum portfolios only makes sense if they’re able to outperform randomly-chosen concentrated portfolios.

Finding concentrated ETFs

The etf.com screener can be used to identify concentrated ETFs. For example, the following table lists every ETF filtered by “Strategy: Value” and “Weighting Scheme: Equal”. This list does not include every concentrated value ETF because (a) some funds are concentrated but not equal-weighted, and (b) some funds don’t have sufficiently detailed metadata on etf.com to show up in these screens. I attempted to estimate concentration by comparing the number of stocks in each fund to the underlying index, but the numbers might not be fully accurate.

ETF	Region	Market Cap	Concentration
EEMD	emerging markets	mid+large	10%
IVAL	developed ex-US	mid+large	5%
QVAL	US	mid+large	5%
SPDV	US	mid+large	10%
SVAL	US	small	12.5%

Disclosure: I invest in QVAL and IVAL.

Errata

2022-04-15:

The original version of this post claimed that concentrated and broad factor portfolios have similar risk-adjusted returns. That is incorrect: equal-weighted concentrated portfolios have higher risk-adjusted returns than value-weighted broad portfolios.

Originally, I calculated the Sharpe ratio using the geometric mean, that is, (geometric mean - risk-free rate) / standard deviation. I should have calculated it using the arithmetic mean: (arithmetic mean - risk-ree rate) / standard deviation. The geometric Sharpe ratio understates the difference in risk-adjusted return between (relatively) low-volatility and high-volatility portfolios.

This change suggests that small investors should favor concentrated portfolios over leveraged broad portfolios even if they have access to cheap leverage.

I added a new table to report Sharpe ratios for various portfolios. I also corrected the table in Appendix B and updated it to include historical data through 2022-02.

Appendix A: Significance tests

This table shows p-values for the excess return of top 10% equal-weighted over top 50% value-weighted for various metrics.¹⁵¹⁶

Tested using monthly data.
Calculated over log returns on the assumption that log returns follow a normal distribution.
Using a two-sided T-test on the null hypothesis that the excess return of the concentrated portfolio is zero.

metric	mean	stdev	n	p-val
B/M	0.66	5.401	1132	3.2e-5
E/P	0.32	3.294	832	6.8e-3
CF/P	0.43	3.536	832	6.4e-4
Mom	0.53	3.615	1119	9.6e-7

Interestingly, among the two null hypotheses

a concentrated factor portfolio has the same return as a broad factor portfolio, and
a concentrated factor portfolio has the same return as the broad market,

the first hypothesis is rejected much more strongly. This happens because the two factor portfolios are highly correlated, so their difference has a relatively low standard deviation.

The excess returns of (a) top 10% VW over top 50% VW, (b) top 10% EW over top 50% EW, (c) top 50% EW over top 50% VW, and (d) top 10% EW over top 10% VW were generally statistically significant, but to a lesser degree. The next table gives p-values for these four portfolio differences using each of the four factors:

		B/M	E/P	CF/P	Mom
10% VW	50% VW	0.28	0.08	0.03	2e-4
10% EW	50% EW	2e-3	2e-3	3e-3	2e-4
50% EW	50% VW	2e-5	0.06	3e-3	1e-4
10% EW	10% VW	2e-5	0.11	0.03	0.03

(9 out of 16 are significant at p=0.01, and 5 are significant at p=0.001.)

This suggests that both methods of increasing concentration (moving from 50% to 10%, and from value-weight to equal-weight) by themselves increased return.

The next table shows p-values from significance tests for standard deviation over B/M (equal-weighted). Differences in standard deviations between portfolios were highly significant.

Tested using monthly data, n=1132.
Differences in standard deviations were tested using a two-sided F-test.

		p-val
50% BM	10% BM, small 10%	1e-22
50% BM	10% BM	1e-11
30% BM	10% BM	1e-6
10% BM	10% BM, small 10%	1e-8
30% BM	30% BM, small 10%	1e-10

Caveat #1: These t-tests assume that stock returns follow a normal distribution. Although this is a common assumption, it’s not quite accurate, as stocks tend to experience tail events more frequently than a normal distribution would predict. That means the “true” p-values are higher than what’s reported above, and the lower the reported p-value, the more wrong it is. For example, at one point the concentrated B/M portfolio experienced an excess drawdown of 43% over a 10-month period, which “should” only happen once every 21,000 years.¹⁷ (Similarly, US equities “should” experience an 80% drawdown only once every 70,000 years, and yet it happened during the Great Depression.)

Caveat #2: I am not particularly well-versed in significance testing, so I could have made mistakes in these calculations.

Appendix B: Replication on international equities

The Ken French Data Library only includes international equity returns back to 1990 (instead of 1926), and only reports value/size quintiles rather than deciles. So this international replication is more limited, but it illustrates the same patterns.

This table gives summary statistics for a variety of equal-weighted and value-weighted portfolios on B/M and size at various levels of concentration. As with US equities, geometric return increased with higher concentration, smaller size, and equal weighting. These three relationships were all statistically significant at p<0.001, even when excluding the first portfolio in the table (which had a much higher return than the others).¹⁸¹⁹

	Sharpe ratio	Return	Standard Deviation
20% BM, small 20% EW	0.97	18.1%	16.3%
20% BM EW	0.50	10.2%	17.5%
40% BM, small 40% EW	0.62	11.6%	16.3%
40% BM EW	0.47	9.2%	17.0%
20% BM, small 20% VW	0.63	11.4%	15.4%
20% BM VW	0.40	8.1%	16.9%
40% BM, small 40% VW	0.48	9.0%	15.5%
40% BM VW	0.39	7.7%	16.3%

Appendix C: Factor regression on selected portfolios

This table provides a monthly time series factor regression on several portfolios, using factors as defined in the Ken French data library.

Beta: market beta factor.
SMB: size factor.
HML: value factor.
Alpha: excess return not explained by these three factors.
P-values are reported for the null hypothesis that alpha = 0.
I use 30% as the diversified portfolio rather than 50% because I cannot fully accurately construct a 50% value-weighted portfolio using the Ken French data. For the data on historical returns, I wasn’t concerned about this because it doesn’t make much difference; but for a factor regression, I want to be as accurate as possible.

	Beta	SMB	HML	Alpha	p-val
30% BM VW	1.07	0.22	0.79	-0.05	0.07
10% BM VW	1.18	0.55	1.08	-0.23	0.002
10% BM, small 10% VW	0.97	1.57	1.14	0.05	0.74
30% BM EW	1.01	1.06	0.85	0.24	2e-5
10% BM EW	1.03	1.33	1.11	0.31	0.004
10% BM, small 10% EW	0.97	1.71	1.21	0.59	0.0008

Based on this limited sample, it appears that more concentrated portfolios do not have higher market beta, but do tend to have greater exposure to the size and value factors, as well as higher alpha. Most notably, the equal-weighted portfolios have more alpha than the value-weighted ones.

If we add leverage to the 30% VW portfolio in an attempt to replicate a more concentrated portfolio like 10% EW, the former will have less alpha and more market beta. This may be undesirable in the context of a broader portfolio that includes other allocations to equities.

I found this result surprising. I would have predicted that concentrated portfolios’ higher return would primarily come from higher factor exposure, especially exposure to the value factor (HML). While my prediction is at least partially true, even the most concentrated portfolio only had 1.21x loading on HML, and a significant portion of the outperformance could not be explained by any factor.

Conventionally, HML is defined as the return of the top 30% of value stocks minus the bottom 30%, value-weighted. If we redefine HML as the top 30% minus the bottom 30% equal-weighted (keeping the other factor definitions the same), we get an interesting result:

	Beta	SMB	HML EW	Alpha	p-val
30% BM VW	1.13	0.03	0.53	-0.21	8e-5
10% BM VW	1.26	0.27	0.78	-0.47	9e-8
10% BM, small 10% VW	1.03	1.15	1.08	-0.37	0.004
30% BM EW	1.06	0.79	0.72	-0.02	0.7
10% BM EW	1.09	0.96	0.99	-0.06	0.6
10% BM, small 10% EW	1.02	1.24	1.23	0.09	0.6

We can see that higher factor loadings on SMB and HML EW fully explain the return of the more concentrated portfolios. Meanwhile, the value-weighted portfolios have statistically significant negative alpha.

One possible explanation is that switching from value-weighting to equal-weighting “unlocks” more of the value premium by allowing investors to assign relatively higher weight to the most undervalued stocks, and this component of the value premium is invisible to HML VW.

The other value metrics, E/P and CF/P, exhibit a similar effect: when we regress on a value-weighted HML factor using E/P or CF/P (instead of B/M), the equal-weighted portfolios have statistically significant alpha at p<0.001, with 10% EW having more alpha than 30% EW. On an equal-weighted HML factor, value-weighted portfolios have negative alphas, but none are statistically significant at p<0.001 (or even p<0.01).

The following table gives factor regressions for selected momentum portfolios. “Mom” is the momentum factor.

	Beta	SMB	Mom	Alpha	p-val
30% Mom VW	1.07	0.08	0.37	0.03	0.35
20% Mom, small 20% VW	1.09	1.32	0.23	0.44	1e-4
30% Mom EW	1.03	0.74	0.27	0.31	1e-11
10% Mom EW	1.11	0.94	0.45	0.28	1e-5
20% Mom, small 20% EW	1.05	1.38	0.17	0.65	1e-6

Here we see qualitatively similar results to the first table, where more concentrated portfolios tend to have stronger exposure to the size and momentum factors, as well as significantly positive alpha.

When we use an equal-weighted momentum factor instead of value-weighted, concentrated portfolios still have statistically significant alpha. However, when we replace the momentum factor with an equal-weighted long-only factor (that is, 30% Mom EW minus the risk-free rate), none of the momentum portfolios have alpha (either positive or negative).²⁰ Relatedly, while equal-weighted momentum portfolios did earn higher risk-adjusted returns than comparable value-weighted strategies, the difference was not as large as for B/M. These two observations suggest that switching from value-weighting to equal-weighting does not matter as much for momentum as it does for value. This makes intuitive sense: equal-weighting implicitly tilts toward small-cap value, so it unlocks more of the value premium, but there’s no reason to expect it to do the same for momentum.

Notes

Ang, Hodrick, Xing & Zhang (2006). The Cross-Section of Volatility and Expected Returns. ↩
Frazzini & Pedersen (2013). Betting Against Beta. ↩
Fama and French (1992). The Cross-Section of Expected Stock Returns. ↩
Jegadeesh and Titman (1993). Returns to Buying Winners and Selling Losers: Implications for Stock Market Efficiency. ↩
Asness, Moskowitz, and Pedersen (2013). Value and Momentum Everywhere. ↩
Baltussen, Swinkels, and van Vliet (2019). Global Factor Premiums. ↩
All returns reported in this essay are geometric returns, not arithmetic. ↩
B/M and Mom historical returns were calculated over the period 1927–2020. E/P and CF/P were calculated over 1951–2020. ↩
I can only show this chart for B/M because that’s the only metric for which the Ken French Data Library includes the necessary data. ↩
It makes sense that the highest returns would show up in stocks that where it’s difficult to actually take advantage of their high return. ↩
The paper Trading Costs by AQR Capital Management found that their own trading costs were significant enough that they probably could not run an equal-weighted small cap value strategy like I described above, but they can still implement value and momentum in mid to large caps. AQR invests billions to tens of billions of dollars in each of its funds, so this information doesn’t tell us much about what smaller investors can do, but it at least indicates that investors with billions of dollars probably can’t increase risk and return via concentration in the manner I describe in this essay. Such investors who want to increase return probably need to use leverage. ↩
In Joel Greenblatt’s book, The Little Book that Beats the Market, in which he proposes following a concentrated value investing strategy, he offers a trick to increase tax efficiency:

For individual stocks in which we are showing a loss from our initial purchase price, we will want to sell a few days before our one-year holding period is up. For those stocks with a gain, we will want to sell a day or two after the one-year period is up. In that way, all of our gains will receive the advantages of the lower tax rate afforded to long-term capital gains […], and all of our losses will receive short-term tax treatment […].

This works for US investors; I can’t comment on how taxes work in other countries. ↩
Perhaps the best argument that momentum will persist: There’s good reason to believe that the value premium will persist, and if value can persist, why not momentum? ↩
If 30 stocks are chosen uniformly at random, they would have outperformed historically, because equal weighting has outperformed market cap weighting. But if the 30 stocks were selected at random in proportion to market cap, then they’d have the same expected return as the market, but with higher volatility. ↩
All p-values are rounded up. For any p-values smaller than 1e-5, I rounded up to the nearest power of 10. ↩
We have discussed concentration along three dimensions: factor concentration, weighting, and market cap. This table compares the most concentrated to the least concentrated portfolios along the first two dimensions, but does not use market cap because not all factors had data on portfolios sorted by market cap. ↩
Technically, the probability (assuming a normal distribution) is higher than that, because I cherry-picked the worst rolling 10-month period rather than looking at non-overlapping periods. ↩
This surprised me—I thought some of the relationships would be statistically insignificant because the data sample only covers 30 years. For comparison, the long/short value factor (HML) was statistically insignificant over this sample (p=0.34), as was market beta (p=0.09). ↩
I did not establish a significance threshold in advance. p<0.001 was the smallest power of 10 at which all three tests were significant. ↩
All alphas fall between -0.05 and 0.07, with the lowest p-value being 0.28. ↩

Asset Allocation and Leverage for Altruists with Constraints

Mon, 14 Dec 2020 00:00:00 -0800

Summary

Altruistic investors differ from ordinary investors in that they don’t just care about their own investments, but about the investments of all altruists.

We can use our own investments to improve the overall altruistic portfolio in two key ways:

Increase expected return by investing in high-return assets or by using leverage.
Reduce risk by investing in assets with low correlation to the altruistic portfolio.

At the margin, we have to choose between either increasing expected return or reducing correlation. How do we make that decision?

We can extend the commonly-used technique of mean-variance optimization (MVO) to derive optimal asset allocations under various assumptions. We don’t know which assumptions apply to the real world, but we can draw some general lessons. The result suggest that we should try to both increase expected return and decrease correlation, but that we should prioritize increasing expected return.

Disclaimer: This should not be taken as investment advice. Any given portfolio results are hypothetical and do not represent returns achieved by an actual investor. Any asset allocation described as “optimal”, how an investor “should” invest, or similarly, is only considered such for the goal of maximizing geometric return under specific theoretical conditions, and may not be optimal for any actual investors.

Cross-posted to the Effective Altruism Forum.

Summary
Contents
Key takeaways
Introduction
Problem setup
Optimal asset allocation
Impact of assumptions
Conclusions
Appendix
- Appendix A: Derivation of mean/standard deviation/correlation estimates
- Appendix B: Sensitivity analysis of risk-constrained allocation
  - Variations on Market + Val/Mom + VMOT + ManFut
  - Variations on US + International + Commodities + Bonds
Notes

Key takeaways

Altruistic investors care not just about their own investment performance, but about the performance of all other value-aligned altruistic investors. This suggests that altruists should pay particular attention to assets that are uncorrelated to typical investments. [More]
Mean-variance optimization (MVO)¹ is the standard approach for determining the theoretically optimal asset allocation. But MVO only works for a fully self-contained portfolio: it cannot determine how altruistic investors can maximize the expected utility of the overall altruistic portfolio. We can transform this unsolvable problem into a solvable one if we model altruists as having logarithmic utility of money. In that case, small investors want to find the personal asset allocation that maximizes the geometric mean return of the altruistic portfolio. [More]
Under certain assumptions, this model produces the following optimal asset allocations given various leverage and risk constraints:
1. Investors who cannot use leverage should (probably) invest all their money into whichever asset has the highest geometric mean return. [More]
2. Investors who can use leverage but cannot short-sell should invest in the same way as un-leveraged investors, but with more leverage. [More]
3. Investors who can use both leverage and shorts, but who do not want their portfolios to exceed some maximum volatility, should invest in a more complex mix of assets. The optimal allocation probably includes assets with high expected return and with low correlation, but with more emphasis on high return. It also includes a moderate short market position to reduce correlation to other altruistic investors. [More]
  - Notably, the short position is much smaller than the long position. The optimal allocation is not totally uncorrelated to the typical altruistic portfolio, sacrificing some diversification in the interest of increasing expected return.
There is a tradeoff between increasing return and decreasing correlation. Within this model, each unit of correlation is worth about 2 to 3 units of return (e.g., decreasing correlation by 0.1 is as good as increasing return by 0.2 to 0.3 percentage points). [More]
For some assumptions, we can see how altering them changes the theoretically optimal portfolio: [More]
1. If we expect any factors to outperform the market, the optimal portfolio will include substantial allocations to these factors, even if we only expect them to outperform by a thin margin. But reducing expected factor performance reduces the magnitude of the short market position.
2. The optimal portfolios described above assumed that typical investors both overly concentrate in certain asset classes (e.g., US stocks) and neglect the highest-return opportunities. If most altruistic dollars are concentrated in the highest-return asset(s) and use appropriate leverage, then marginal investors should invest in diversifying assets while shorting the overweighted assets (even though those assets have the highest expected return).
3. Similarly, if most altruists diversify well but don’t take on enough risk, then marginal altruists should concentrate in high-return assets.
4. If we account for the costs of leverage and short positions, then the optimal allocation uses a little bit less leverage and shorting, but not much less. [More]
5. For large investors, maximizing the geometric mean of the total altruistic portfolio looks more similar to simply maximizing the geometric mean of the investments they control. [More]
For other assumptions, if we alter them, we can no longer derive the geometric mean maximizing portfolio via the same method. But we can consider how the change qualitatively affects the optimal asset allocation:
1. Most assets experience deep drawdowns more frequently than theoretical models predict. Therefore, in practice, the optimal asset allocation probably takes on less risk than the models suggest. [More]
2. Most assets’ correlations tend to increase during bad times. Therefore, diversification does not reduce risk by as much as theoretical models indicate. However, some assets do not tend to experience negative correlations during equity market drawdowns, so these still have strong diversification benefits. [More]
3. The output of an optimization model heavily depends on the provided return and risk estimates. Given that these estimates are uncertain, it makes sense to diversify more than theoretical models suggest. [More]
4. Maximizing geometric mean is only optimal for actors with logarithmic utility. Altruists might be more or less risk-averse than this, in which case they should diversify more or less (respectively). [More]
This approach of geometric mean maximization has substantial limitations and uncertainties, so we should not draw overly strong conclusions. That said, this analysis provides at least moderate evidence that:
- Altruistic investors who cannot use leverage should invest in whatever asset has highest expected (geometric) return, as long as they are comfortable with the risk.
- Altruistic investors who can use leverage and shorts should invest in a somewhat diversified portfolio of assets with high expected return, along with a modest short market position to reduce their correlation to other altruistic investors.
- Increasing return is moderately more important than decreasing correlation, but not overwhelmingly so.

Introduction

In some ways, altruists and traditional investors have the same incentives regarding how they should invest—they want to invest in the portfolio with the best balance of return and risk. But self-interested people only care about how much money they have, while altruists care about how much money all other (value-aligned) altruists have.

That means altruists should not invest their money in the optimal overall portfolio. Instead, they should attempt to push the pool of value-aligned philanthropic money in the direction of optimal.

I have discussed this fact in the past, especially regarding how it affects optimal leverage and the importance of uncorrelated investments. Namely: most altruistic investors probably don’t use enough leverage, and don’t diversify well enough. So at the margin, altruists should pay extra attention to leverage and diversification, more so than they would if they only cared about their own investments.

In theory, altruists should invest in assets with zero correlation to the typical altruistic portfolio, and then apply an absurdly high amount of leverage (like 100:1 or more). In practice, we can’t do this. If we want to decrease correlation by short-selling popular investments, we also reduce expected return, and we can’t fully make up for it by adding leverage. So we must make tradeoffs between increasing expected return and increasing diversification? How do we decide?

In this essay, I will attempt to precisely determine, under specific conditions, the optimal asset allocation for altruistic investors at the margin.

Problem setup

The optimization problem

At a basic level, the problem we are facing is not unique to altruists: we want to construct the best portfolio from among a set of available assets. Traditionally, we could (in theory) solve this problem using mean-variance optimization (MVO).¹ If we know the expected return, standard deviation, and correlation of two or more asset classes, we can use MVO to determine how much to allocate to each of them. (In practice, MVO doesn’t actually work very well, but more on that later.)

MVO can work in two ways:

Find the allocation that maximizes expected (arithmetic mean) return for a given level of volatility—or, equivalently, the allocation that minimizes volatility for a given expected return.
Find the allocation with the highest risk-adjusted return (or Sharpe ratio).

Unfortunately, neither of these techniques can help us optimize the altruistic portfolio. We want to find the asset allocation among our own assets that maximizes the expected utility of the total altruistic portfolio. The first MVO method—maximizing expected return for a given volatility—only works if we already know what level of total volatility maximizes expected utility.

The second method—maximize Sharpe ratio—works for independent investors, because (under certain assumptions) the maximum-Sharpe-ratio allocation also maximizes utility if you add cash or add leverage to adjust the level of risk. But this doesn’t work for altruistic investors because if we maximize the Sharpe ratio of our own holdings, that probably doesn’t maximize the Sharpe ratio of the overall altruistic allocation. And if we maximize the altruistic Sharpe ratio, we can’t mix in cash or leverage on the whole portfolio, only on our own money, so knowing the maximum-Sharpe portfolio doesn’t tell us how to maximize utility.

We can solve the problem by starting with a reasonable assumption: altruists have logarithmic utility of money. This is a common assumption, and it might not be exactly true, but it seems at least approximately correct. With logarithmic utility, we want to find the asset allocation that maximizes the geometric mean. (For interested readers, this is equivalent to the Kelly criterion, but over long-term investments rather than discrete bets.²)

We could also find the optimal asset allocation if we used a different utility function, but logarithmic utility is simple and intuitively plausible, so it’s a good place to start.

Even if we want to take on more risk with our altruistic investments than with our personal investments, we still can’t take on arbitrarily high risk. In practice, we face constraints—typically either a maximum allowed quantity of leverage, or a maximum acceptable volatility. Subject to constraints on our own investments, we want to maximize the geometric mean return of the total altruistic portfolio.

There is no good formula for the exact geometric return of a portfolio, but we can use a close approximation (Estrada, 2010³⁴):

\[\text{Geometric Mean} = \log(1 + \mu) - \displaystyle\frac{\sigma^2}{2(1 + \mu)^2}\]

where $\mu$ is the portfolio arithmetic mean and $\sigma$ is the portfolio standard deviation, both of which we can precisely calculate.

This is a convex optimization problem, which we can solve with an optimization tool such as Python’s scipy.optimize.

Assumptions

In addition to the setup given in the previous section, let’s make some more assumptions to clarify and simplify the problem:

We care about real return (that is, return after inflation), not nominal.
Asset returns follow a log-normal distribution.
The real risk-free rate is 0%. (This is approximately true, and it makes the math easier.)
We control only a small portion of the total altruistic portfolio. For the calculations in this essay, I assume we control 1%.
If we use leverage, we only have to pay the risk-free rate in interest. We can short-sell securities for free.

Later, I will scrutinize these to see how they might affect the outcome.

Sample assets

For this essay, we will mainly consider four investable assets. The first represents the broad market, and the next three represent what I believe to be some of the best investment opportunities that are accessible to most investors. In a previous essay, Uncorrelated Investments for Altruists, I explain why I like these three investments.

The four assets are:

Market: The global equities market. This serves as a proxy for the average altruist’s investment portfolio.
Val/Mom: A concentrated long-only value and momentum strategy, similar to the strategies used by various ETFs.⁵
VMOT: A value/momentum strategy with a trendfollowing component, similar to the strategies used by the ETFs VMOT⁶ and GMOM.⁷⁶ (VMOT stands for Value Momentum Trend.)
ManFut: A trendfollowing managed futures strategy, like the strategy studied in Moskowitz et al. (2012)⁸.

To find the optimal allocation, we need to make some assumptions about these assets’ (geometric) means, standard deviations, and correlations to each other. I use the following numbers:

	Market	Val/Mom	VMOT	ManFut
Geometric Mean	3%	6%	6%	3%
Standard Deviation	16%	16%	13%	15%

And the correlation matrix:

	Market	Val/Mom	VMOT
Val/Mom	0.8
VMOT	0.5	0.8
ManFut	0	0	0.2

See Appendix A for details on how I came up with these numbers.

In a later section, I will look at results using more traditional assets (stocks, bonds, and commodities), but my analysis will primarily focus on the set of four assets listed above. The specific choice of assets doesn’t actually matter much—we can draw general lessons that apply to any assets.

Optimal asset allocation

With no constraints, the optimization problem is simple:

Find the allocation that the altruistic portfolio as a whole should have.
Take on gigantic leveraged and short positions to the point where you singlehandedly push the altruistic portfolio to the optimum point.

But you can’t do this because it requires using unrealistically large amounts of leverage. In practice, investors—even altruistic investors with high risk tolerance—have constraints. Let’s look at two types of constraints: leverage and risk.

I will begin by looking at the problem of portfolio optimization with leverage constraints. Then I will derive optimal portfolios under risk constraints, including some variations on the problem. Risk constraints allow for more variations and deeper analysis, so I will spend more time on them.

With a leverage constraint

Most investors do not use leveraged or short positions, either because they cannot or because they prefer to stick with simpler investments. If we assume that we cannot use leverage or shorts, what is the optimal asset allocation? More generally, what is the optimal allocation where we can only use up to some maximum amount of leverage?

Let’s begin with the case of an investor who cannot use any leverage or shorts.⁹ Under the assumptions described previously, an investor maximizes the geometric mean of the total altruistic portfolio by investing 100% of their wealth in VMOT. Without the ability to use leverage, altruistic investors should (according to this model) simply invest in whichever asset has the highest return and not too much correlation to the market.

An investor who can use up to 2:1 leverage (e.g., anyone using Reg T margin) has essentially the same optimal asset allocation, but with more leverage. Namely, they should invest 200% in VMOT.

This differs substantially from the optimal self-interested portfolio. An investor who simply wishes to maximize their own geometric return should invest 51% in Val/Mom and 49% in VMOT. (Notice that this portfolio does not allocate any to the two lower-return assets, Market and ManFut. Maximizing geometric return tends to result in fairly aggressive portfolios. A more risk-averse investor would want to diversify more.)

Does the altruistic investor prefer VMOT because it’s the highest-return asset (tied with Val/Mom), and has lower risk than Val/Mom? Or is it because VMOT has a lower correlation to the market? We can test this by supposing that VMOT has a perfect correlation to the market. In that case, the optimal portfolio allocates 100% to Val/Mom instead. So we see that the optimizer cares more about reducing correlation than about reducing volatility. If we revise the expected return of VMOT to 6.5% (along with a perfect correlation to Market), the optimal allocation now puts half into Val/Mom and half into ManFut. Roughly speaking, that means increasing expected return by 0.5 percentage points is as good as reducing correlation by 0.2.

Leverage constraint, but with shorts allowed

If we maintain the leverage constraint but allow short selling, we get some unusual and perhaps surprising results:

Allocation
Market	–44360%
Val/Mom	45521%
VMOT	–1854%
ManFut	793%

Summary Statistics
Total Altruistic Return	9.64%
Personal Return	–221.69%
Personal Standard Deviation	4435.67%

Recall that our portfolio represents 1% of the overall altruistic portfolio, so a –9900% short market position would be enough to bring the altruistic market allocation down to 0%. This allocation goes far beyond that point, down to a –44360% short position.

This allocation has an extremely bad geometric mean return of –222%. This is done in the service of diversifying the total altruistic portfolio.

But this seems slightly unrealistic. Let’s posit that the short side cannot be larger than 100% (that is, your short position cannot exceed the net value of your account). An investor who can short can probably also use leverage, so let’s allow up to 2:1 leverage with a 50% short margin requirement (i.e., if you short $1, you can use up to $0.50 of that to buy more assets). These constraints produce the following allocation:

Allocation
Market	–100%
Val/Mom	0%
VMOT	250%
ManFut	0%

Summary Statistics
Total Altruistic Return	3.12%
Personal Return	8.96%
Personal Standard Deviation	28.15%

So when we allow modest short positions, the optimizer will short the market and then buy as much VMOT as possible.

With a risk constraint

Most investors, including purely altruistic investors, want to limit their exposure to risk. Possible reasons for this include:

Overly risky leveraged portfolios might get margin called. This will never happen to a continuously-rebalanced portfolio, but in real life it’s not possible to rebalance continuously. It’s typical to only rebalance once a month, so we want a portfolio that’s exceedingly unlikely to experience a 100% drawdown within a month.
Even if extremely high risk is optimal in theory, most investors aren’t comfortable with that much volatility.

We can formally limit risk by mandating that our portfolio does not exceed some maximum (predicted) standard deviation. Standard deviation does not perfectly capture what we mean by “risk” (more on that later), but it’s a reasonable proxy that’s easy to work with.

The US stock market historically has had about a 20% standard deviation,¹⁰ so let’s begin by taking this as our maximum acceptable volatility. Altruists might want to take on more risk than that, but we need to pick a number, and 20% seems like a good starting point.

Our goal: build the asset allocation that maximizes the geometric mean of the overall altruistic portfolio while keeping the volatility of our personal portfolio below 20%. We can use leverage and shorts as much as we want, as long as we do not exceed a 20% standard deviation. Within these constraints, the optimal portfolio is:

Allocation
Market	–76%
Val/Mom	92%
VMOT	77%
ManFut	57%

Summary Statistics
Total Altruistic Return	3.10%
Personal Return	8.87%
Personal Correlation to Market	0.23

(“Return” refers to geometric return.)

A negative allocation indicates that the asset is sold short. The allocation adds up to more than 100%, which means we must use leverage. Also notice that this allocation has low correlation to the market, but it’s not totally uncorrelated.

How does this allocation compare to the optimal self-interested asset allocation, where we don’t care about the performance of the altruistic portfolio?

Allocation
Market	–36%
Val/Mom	82%
VMOT	71%
ManFut	52%

Summary Statistics
Total Altruistic Return	N/A
Personal Return	9.22%
Personal Correlation to Market	0.47

We can see that, according to this model, altruists are willing to sacrifice some return (from 9.22% down to 8.87%) in order to diversify more. The altruistic asset allocation has a much larger short market position, and somewhat larger long positions for the other assets.

But the altruistic allocation does not take as large a short position as it could. Qualitatively speaking, the algorithm does not seem willing to sacrifice much expected return to get more diversification. If we optimized with the goal of maximizing the Sharpe ratio of the total altruistic portfolio instead of the geometric return, we would see a much larger short market position (–109%, to be precise).

If we increase our risk tolerance, the portfolio composition largely does not change, but simply scales up linearly. For example, if we allow up to 40% standard deviation instead of 20%, the allocation to each asset approximately doubles (including doubling the size of the short market position). This is true as long as our portfolio only represents a tiny fraction of the total altruistic portfolio. However, for particularly large investors, increasing the maximum standard deviation will substantially affect portfolio allocation, because the large investor’s allocation will meaningfully influence the overall altruistic allocation.

Sensitivity analysis

The optimal allocation depends on our expectations for asset returns, standard deviations, and correlations. In this section, I will provide some qualitative sensitivity analysis, with full numerical details provided in Appendix B.

Changing the expected future return of the value, momentum, and trend factors does not much change the optimal allocation. If we assume each of these factors will only beat the market by one percentage point, the resulting optimal allocation looks much the same as the one above.

Even reducing expected factor performance to 0%, the optimal allocation still substantially allocates to Val/Mom and ManFut. The main difference in this case is that, rather than shorting the market, the optimal allocation includes a small (20%) long market position.

II.

How much of the asset allocation is driven by attempting to improve return, and how much by trying to diversify? The answer is obscured by our model’s assumption that most altruistic investors concentrate in a single asset (global equities), and that that asset has the lowest risk-adjusted return. So moving away from that asset could make sense either as a way to diversify or to increase return.

We can answer this by assuming other altruists invest all their money in VMOT—the asset with the highest risk-adjusted return (according to the model assumptions)—and use optimal leverage. In this case, our optimal allocation shorts both the market and VMOT, while holding large long positions in Val/Mom and ManFut.¹¹ Apparently, if the altruistic portfolio uses optimal leverage but doesn’t diversify well, then marginal investors should prioritize increasing diversification. But if the typical altruistic portfolio is both under-levered and overly concentrated, then investors should prioritize increasing return.

III.

In the estimates I made for return/standard deviation/correlation, I believe most of the numbers are well-supported. But I’m much more uncertain about what to expect from managed futures. The century-long backtest in Hurst et al. (2014)¹² found an 11.2% nominal return after estimated fees and transaction costs with a 9.7% standard deviation. If we levered this up to 15% volatility, that would give a historical return of about 15%, or 11% after inflation. Performance this strong seems unlikely to continue into the future, for various reasons.¹³ In my return assumptions, I substantially discounted this backtested 13% real return down to 3%.

This might be overly conservative.¹⁴ What if we make more optimistic assumptions about managed futures performance? Naturally, if we expect better return from managed futures, then we should give it a larger allocation. But as we increase the expected return of managed futures, its allocation only changes slowly. We need to expect a return as high as 10% before the optimizer allocates more to ManFut than to Val/Mom and VMOT combined. So it seems likely that the “true” optimal allocation includes more value/momentum stocks than managed futures.

Using more traditional assets

What result do we get if we use a more traditional set of assets? Let’s include US equities, international developed-market equities, commodities, and intermediate-term Treasury bonds.¹⁵ Assume the overall altruistic portfolio allocates 60% to US equities, 20% to international equities, and 20% to bonds.¹⁶

	US	International	Commodities	Bonds
Geometric Mean	0%	5%	1%	–1%
Standard Deviation	16%	17%	16%	4%

Correlation matrix:

	US	International	Commodities
International	0.9
Commodities	0.3	0.4
Bonds	–0.3	–0.3	–0.1

The optimal allocation, subject to a 20% maximum standard deviation:

Allocation
US Market	–250%
International	276%
Commodities	–11%
Bonds	85%

Summary Statistics
Total Altruistic Return	1.23%
Personal Return	11.14%
Personal Correlation to Altruistic Portfolio	0.16

When optimizing personal holdings only, with no regard to the external altruistic portfolio:

Allocation
US Market	–228%
International	272%
Commodities	–11%
Bonds	68%

Summary Statistics
Total Altruistic Return	N/A
Personal Return	11.31%
Personal Correlation to Altruistic Portfolio	0.30

As before, when we account for the investments of altruists other than ourselves, we prefer a bigger short market position and a larger position in each of the long assets. But the two portfolios don’t look much different. This suggests that how much we care about other altruists’ investments depends on what assumptions we make about investment performance.

Compared to the set of assets we looked at before, this more traditional set includes a much larger short position. The optimizer wants to allocate most of the money to international equities. But those are pretty highly correlated to US equities, so the optimizer seeks greater uncorrelated return by using more leverage and a bigger short position.

The return/correlation tradeoff

We want to maximize return while minimizing correlation to the typical altruistic portfolio. We are willing to give up some return in order to decrease correlation, or accept higher correlation to earn greater return. How exactly do we trade off between these?

As an example, let’s say (as before) that global equities have a 3% expected (geometric) return and 16% standard deviation. Suppose we can invest in equities, or we can invest in some other asset with the same volatility but a different return. What return + correlation would make the other asset be as good an investment as global equities?

On the rightmost point of this line, we see that if an asset is perfectly correlated to global equities, then we require it to have the same return. (In effect, this point represents the equity market itself.) At the other extreme, if an asset is perfectly anti-correlated with equities, it can have a return as low as –1.75% and still be worth investing in. In the middle, an asset with zero correlation must earn an expected return of 0.62%.

This line¹⁷ has a slope of 2.38. That means if we can increase return by (say) 0.238% while increasing correlation by less than 0.1, then we should do it, and vice versa.

As another example, suppose we can invest in leveraged VMOT¹⁸, which has the same volatility as global equities, an expected return of 7%, and a correlation of 0.5. Holding volatility fixed, what return/correlation would another asset need to have to match the expected value of VMOT?

For an asset with perfect correlation to global equities, we would demand a return of 8.19%. For an uncorrelated asset, we would accept a return of 5.81%. And for an asset with a perfect negative correlation, the expected return can go as low as 3.44%.

Impact of assumptions

The analysis in this essay made a number of assumptions. How realistic are they? If we change them, how might the results change?

Leverage and shorts are free

So far, we have assumed that we can get leverage at the risk-free rate and shorts for free. But this is usually not true in practice. Let’s take the more realistic assumptions that leverage costs 1% on top of the risk-free rate, and short positions cost 0.25%.¹⁹

This gives the following asset allocation:

Allocation
Market	–38%
Val/Mom	39%
VMOT	115%
ManFut	50%

Summary Statistics
Total Altruistic Return	3.08%
Personal Return	7.92%
Personal Standard Deviation	20%
Personal Correlation to Market	0.32

Notice that the short market position is smaller than before: –38% instead of –76%. This portfolio also has a smaller long position (204% instead of 226%). Interestingly, it actually increases the allocation to VMOT. This happens because Market has a stronger correlation to Val/Mom than to VMOT, so when the optimizer reduces the size of the short position, it no longer cancels out the high correlation of Val/Mom, so it prefers to shift some money to an asset with lower correlation.

For comparison, the optimal self-interested asset allocation looks like this:

Allocation
Market	0%
Val/Mom	42%
VMOT	95%
ManFut	42%

Summary Statistics
Total Altruistic Return	N/A
Personal Return	8.33%
Personal Standard Deviation	20%
Personal Correlation to Market	0.58

This allocation eschews short positions entirely.

We control only a small portion of the total altruistic portfolio

So far we have assumed that, in total, other value-aligned altruists donate much more money than we do. (Specifically, my calculations assumed we control 1% of the overall portfolio, although the exact number doesn’t matter much.)

If we control a large fraction of the altruistic portfolio—e.g., if we manage a large foundation, or if we donate to a cause that receives almost no attention—then our portion of the altruistic portfolio should look much more similar to the optimal overall altruistic portfolio. Specifically, that means:

We should care less about reducing correlation to other altruistic investors.
We might not want to get as much leverage as possible, instead getting an amount that’s closer to the overall optimum level.

Assets follow log-normal distributions

I’m using standard deviation as the measure of risk. That only works if asset prices follow log-normal distributions. In practice, most assets have a left-skewed distribution, meaning they see very bad performance more often than a log-normal distribution would predict.

Therefore, standard deviation does not properly capture the risk of most assets. You should usually use less leverage than you’d think just from looking at the standard deviation. (In a previous essay, I compared theoretically optimal leverage (over log-normally distributed assets) to the (simulated) historical performance of leveraged portfolios. Over the asset classes I examined, theoretically optimal leverage somewhat overestimated truly optimal leverage, but not by a huge margin.)

Usually, people don’t actually care about the standard deviation of a portfolio: they care about drawdowns. I like the ulcer index as a way to measure the tendency of a portfolio to experience drawdowns. Unfortunately, we can’t easily run an ulcer index minimization algorithm, because that would require having full price data of each asset, not just the return/volatility/correlations. Even if we do have full price data, the results would be highly sensitive to which particular drawdowns occurred during the sample period.

We discussed optimal portfolios under leverage constraints and risk (i.e., standard deviation) constraints. But perhaps a more better approach is to use a drawdown constraint. Assume I can only accept, say, a 90% drawdown on my altruistic investments. What maximum drawdowns do I expect from each asset, and how likely are the different assets to experience big drawdowns at the same time? Then I can use this to derive an optimal portfolio. This is approximately how I decide on an asset allocation in my personal account (although I certainly have a much lower drawdown tolerance than 90% on my personal funds!). I’m not sure how to formalize this, so I just make ad-hoc estimations.

It should be noted that some assets do not skew left. Importantly, managed futures have historically been right-skewed. For example, Chesapeake Capital’s managed futures fund⁶ experienced a standard deviation of 19% from 1988–2019, as compared to the US stock market’s 14%. But the Chesapeake fund performed substantially better on other measures of risk:

	Chesapeake	US Equities
Skewness	0.86	–0.67
Max Drawdown	32%	50%
Ulcer Index	10.5	14.2

(Notably, Chesapeake experienced its maximum drawdown not in 2007–2009 like US equities, but in 2010–2013, while the market was going up.)

If we expect this to continue in the future, then standard deviation is actually too conservative a measure of risk for assets like this one.

Asset correlations are stable over time

In ordinary times, stocks, commodities, and real estate have fairly low correlations to each other. But in bad times, correlations tend to go to 1. During the recession of 2008, all three asset classes experienced large drawdowns at the same time. So even if we build a diversified portfolio, that diversification might fail us at exactly the wrong time.

Some assets and strategies, most notably trendfollowing (including managed futures), have actually shown positive return during equity drawdowns.²⁰ Including such strategies can help alleviate concerns about correlations increasing during market downturns.

Optimization techniques can predict optimal future asset allocations

This essay implicitly assumed that we can use techniques such as geometric mean maximization to derive optimal asset allocations. But substantial literature has shown that the related technique of mean-variance optimization (MVO) tends to over-fit to past results and does a bad job of estimating forward-looking optimal portfolios—in fact, it maximizes estimation error (Michaud, 1989²¹). Geometric mean maximization does a better job than MVO of predicting out-of-sample performance (Estrada, 2010³), but still isn’t entirely reliable.

In my analysis, I used future return projections rather than naively taking historical returns, which should fix the problem of over-fitting to historical performance. But this still requires my estimates to be (reasonably) accurate, which they might not be.

Some more complex methods, such as Bayesian methods and portfolio resampling, can incorporate uncertainty into the portfolio optimization process (Fuhrer and Hock, 2019²²). These methods generally favor more diversified portfolios: if you don’t know which assets will perform best in the future, it makes sense to hedge your bets.²³

Ultimately, the takeaway from this essay should not be that a particular allocation shown is close to optimal. We should focus on the general principles rather than specific numbers—for example, the principle that altruistic investors should probably short the market, but their short position should be smaller than their long positions.

Altruists have logarithmic utility

Investors with logarithmic utility functions want to maximize geometric mean. So by focusing on the geometric mean, I have implicitly assumed that altruists have logarithmic utility. This seems at least approximately true, but it might not be true exactly. If altruists experience more rapidly diminishing marginal utility, then they want to diversify more than this essay suggests, and vice versa.

(Note that self-interested individuals are probably more risk averse than logarithmic utility would suggest. It seems plausible that altruistic endeavors on the whole are less risk-averse than individuals.)

Conclusions

In brief:

If most altruists don’t use enough leverage, or under-allocate to the assets with highest expected return, then the top priority is to get a (leveraged) allocation to the high-return assets.
The optimal allocation probably includes a modest short position to reduce correlation to the typical altruistic portfolio.
If altruists do use enough leverage but don’t fully diversify, then altruists on the margin should invest in diversifying assets, perhaps exclusively.

In practice, it seems most likely that altruists do under-allocate to the value, momentum, and trend factors, as well as some high expected return markets like emerging-country equities. This suggests that, on the margin, altruists should hold whichever investments they believe have the highest expected return, while shorting over-subscribed markets such as US equities.

How has writing this essay changed my mind?

Previously, I was moderately convinced by the theoretical argument (given here) that altruistic investors should exclusively hold uncorrelated investments. I wasn’t sure this was true in practice, because I suspected that the overhead costs of maintaining market-neutral long/short positions would exceed the benefits.

The analysis in this essay somewhat justifies my practical concerns, although I was right for the wrong reason. Yes, the costs of leveraged and short positions do make such positions somewhat less appealing, but not much. More importantly, if you believe that most altruistic investors use insufficient leverage or under-weight the highest-return assets, then—at least according to the analysis in this essay—you should primarily invest in the assets with highest expected (geometric) return, even if they’re not uncorrelated with typical investments. In particular, investors who can’t use leverage may simply want to invest in whichever asset has the highest expected return, without regard to its volatility or correlation.

Calculations for this essay were done using mvo.py.

Appendix

Appendix A: Derivation of mean/standard deviation/correlation estimates

Mean, standard deviation, and correlation for global equities, US equities, international equities, commodities, and bonds are based on forward projections from Research Affiliates’ Asset Allocation Interactive. Accessed 2020-11-27.

Means for Val/Mom, VMOT, and ManFut were derived by taking historical return in excess of the market, adding expected market return, subtracting projected costs, and dividing the premium in half based on the assumption that value, momentum, and trendfollowing will work less well in the future than they have in the past.

Baseline historical returns for Val/Mom and VMOT are taken from a 20-year backtest by Alpha Architect and my own 90-year backtest using the Ken French Data Library. Baseline historical returns for ManFut are taken from Moskowitz et al. (2012)⁸ and Hurst et al. (2014)¹².

Standard deviations for Val/Mom and VMOT are taken from the same backtests with no adjustments, on the basis that we have no reason to expect future volatility to be higher or lower than it was in the past.

Standard deviation for ManFut comes from the fact that high-volatility managed futures mutual funds generally target about a 15% standard deviation.

Correlation matrix was calculated with a backtest using the Ken French Data Library and AQR’s Time Series Momentum: Factors, Monthly data set, on the assumption that future correlations will resemble historical correlations. I revised some of the correlations slightly downward because my backtest only included US stocks, but an actual portfolio would include globally diversified stocks. Managed futures correlations were validated using the actual historical performance of the Chesapeake Capital managed futures fund, 1988–2019.

Appendix B: Sensitivity analysis of risk-constrained allocation

Variations on Market + Val/Mom + VMOT + ManFut

Asset allocation with maximum 40% standard deviation, but return expectations unchanged

Allocation
Market	–145%
Val/Mom	171%
VMOT	164%
ManFut	114%

Summary Statistics
Total Altruistic Return	3.20%
Personal Return	14.65%
Personal Correlation to Market	0.23

Asset allocation with 1% excess return for each factor (Val/Mom return = 4%, VMOT return = 3%, ManFut return = 1%)

Allocation
Market	–62%
Val/Mom	141%
VMOT	11%
ManFut	68%

Summary Statistics
Total Altruistic Return	3.05%
Personal Return	4.70%
Personal Correlation to Market	0.44

Asset allocation with 0% excess return for each factor (Val/Mom return = 3%, VMOT return = 2%, ManFut return = 0%)

Allocation
Market	20%
Val/Mom	62%
VMOT	50%
ManFut	50%

Summary Statistics
Total Altruistic Return	3.03%
Personal Return	3.54%
Personal Correlation to Market	0.72

Asset allocation when other altruists invest 100% of their money into VMOT

Allocation
Market	–45%
Val/Mom	102%
VMOT	47%
ManFut	64%

Summary Statistics
Total Altruistic Return	5.92%
Personal Return	9.16%
Personal Correlation to VMOT	0.45

Asset allocation when other altruists invest all their money into VMOT, with optimal leverage (405%), and allowing us to use up to 52% standard deviation (so that we can match 405% leverage)

Allocation
Market	–189%
Val/Mom	430%
VMOT	–121%
ManFut	270%

Summary Statistics
Total Altruistic Return	16.02%
Personal Return	14.70%
Personal Standard Deviation	52%
Personal Correlation to VMOT	0.62

Same, but only allowing us a 20% standard deviation

Allocation
Market	–73%
Val/Mom	166%
VMOT	–47%
ManFut	104%

Summary Statistics
Total Altruistic Return	15.94%
Personal Return	7.91%
Personal Standard Deviation	20.00%
Personal Correlation to VMOT	0.62

Asset allocation with more optimistic expectations for managed futures (ManFut return = 5%)

Allocation
Market	–75%
Val/Mom	104%
VMOT	41%
ManFut	85%

Summary Statistics
Total Altruistic Return	3.11%
Personal Return	10.12%
Personal Correlation to Market	0.20

Asset allocation with highly optimistic expectations for managed futures (ManFut return = 10%)

Allocation
Market	–66%
Val/Mom	112%
VMOT	–16%
ManFut	119%

Summary Statistics
Total Altruistic Return	3.16%
Personal Return	14.65%
Personal Correlation to Market	0.14

Asset allocation using historical performance, net of estimated costs (market return = 5%, Val/Mom return = 12%, VMOT return = 12%, ManFut return = 9%, ManFut standard deviation = 10%)

Allocation
Market	–82%
Val/Mom	125%
VMOT	13%
ManFut	142%

Summary Statistics
Total Altruistic Return	5.17%
Personal Return	22.40%
Personal Correlation to Market	0.19

Variations on US + International + Commodities + Bonds

Asset allocation when US equities return 5%

Allocation
US Market	82%
International	37%
Commodities	11%
Bonds	–30%

Summary Statistics
Total Altruistic Return	4.07%
Personal Return	6.02%
Personal Correlation to Altruistic Portfolio	0.99

Asset allocation when bonds return –3%

Allocation
US Market	–225%
International	263%
Commodities	5%
Bonds	58%

Summary Statistics
Total Altruistic Return	0.82%
Personal Return	10.16%
Personal Correlation to Altruistic Portfolio	0.30

Asset allocation when bonds have 0.0 correlation to each other asset

Allocation
US Market	–247%
Global ex-US	269%
Commodities	–7%
Bonds	84%

Summary Statistics
Total Altruistic Return	1.19%
Personal Return	10.90%
Personal Correlation to Altruistic Portfolio	0.19

Notes

The effective altruism movement, and charity in general, are over-represented in the United States.
Most investors over-weight investments within their own country.
Some altruistic investors probably hold something like a “standard” 60/40 stocks/bonds portfolio, and others probably hold stocks only. So maybe it averages out to around 80/20.

Haugh (2016). Mean-Variance Optimization and the CAPM. Lecture notes from IEOR E4706: Foundations of Financial Engineering. ↩ ↩²
For more on this, see MacLean, Thorp, and Ziemba. The Kelly Capital Growth Investment Criterion: Theory and Practice. ↩
Estrada (2010). Geometric Mean Maximization: An Overlooked Portfolio Approach? ↩ ↩²
A similar approximation, with more discussion of accuracy, is given by Bernstein and Wilkinson (1997): Diversification, Rebalancing, and the Geometric Mean Frontier. ↩
There are lots of value ETFs and a fair number of momentum ETFs, but most of them aren’t concentrated—they just replicate a market index while only weakly tilting toward value or momentum. Some examples of concentrated ETFs:
- US value: QVAL, SYLD
- International value: IVAL, FYLD, EYLD, GVAL
- US momentum: QMOM
- International momentum: IMOM
- US combined value/momentum: VAMO
Disclaimer: I invest in some of these ETFs.

It’s also fairly easy to build your own value/momentum stock portfolio if you have access to a good stock screener, although you may end up owing more taxes this way. ↩
Disclaimer: I have money invested in this fund. ↩ ↩² ↩³
The backtests I did for this essay attempted to replicate the methodology of VMOT. GMOM uses different methodology and will probably perform substantially differently, but it’s spiritually similar. ↩
Moskowitz, Ooi, and Pedersen (2012). Time Series Momentum. Backtest data available here: https://www.aqr.com/Insights/Datasets/Time-Series-Momentum-Factors-Monthly ↩ ↩²
Managed futures include embedded leveraged and short positions, and VMOT includes shorts (although it’s possible to implement something like VMOT without shorting). But investors can still buy these assets even if they can’t use any leverage or shorts within their own portfolios. ↩
Dimson, Marsh, and Staunton (2020). Summary Edition Credit Suisse Global Investment Returns Yearbook 2020. ↩
In practice, it doesn’t make sense to short VMOT while buying Val/Mom, because they’re pretty similar in a way that’s not captured by their correlation. But the optimizer doesn’t know that. ↩
Hurst, Ooi, and Pedersen (2014). A Century of Evidence of Trend-Following Investing ↩ ↩²
These reasons include:
1. Trendfollowing strategies are more crowded than they were for most of the 20th century, although probably not by enough to reduce the expected return to zero. See AQR (2018), Trend Following in Focus. Babu et al. (2019), You Can’t Always Trend When You Want showed that the poor recent performance of trendfollowing is almost entirely explained by the low absolute Sharpe ratios of markets, and is consistent with history. The paper actually found no reduction in return due to over-saturation, although on priors I would expect at least a little bit of reduction, and this result might be due to luck.
2. The risk-free rate is lower than it used to be, which means trendfollowing strategies earn a much lower return on the cash they hold as collateral.
3. The paper’s methodology in calculating costs might not account for all trading costs. I compared the recent performance of AQR’s trendfollowing backtest against actual managed futures funds (including AQR’s own fund), and the funds performed a few percentage points worse.
4. It is reasonable to assume some degree of mean reversion in backtested results due to data mining, although the high statistical significance of trendfollowing suggests that we should not expect much mean reversion. See Harvey and Liu (2015), Backtesting.
5. I additionally discount results based on a strong prior belief that any particular strategy cannot produce positive excess return. My subjective expectation for the return of a strategy is a weighted average of the prior expected return of zero and the (unweighted) expected return supported by the evidence. (It would be better to incorporate this Bayesian reasoning directly into the optimization model, as discussed in Fuhrer and Hock (2019)²², but that’s more complicated.)
↩
Chesapeake Capital’s managed futures fund returned 11% nominal (net) from 1988–2019, with a standard deviation of 19%. If we discount this based on inflation, the risk-free rate, and the assumption that managed futures will work somewhat less well in the future than it has in the past, then we get roughly a 3–5% real return. ↩
Performance expectations are taken from the spreadsheet downloaded from Research Affiliates’ Asset Allocation Interactive. Accessed 2020-11-27. The names of the assets in my analysis correspond to Research Affiliates’ names as follows:

US = US Large International = EAFE Commodities = Commodities Bonds = US Treasury Intermediate ↩
This asset allocation is an educated guess based on: ↩
This is not actually a straight line, but it’s pretty close. We can see this by looking at the formula we use to approximate the geometric mean: the geometric mean over two assets is a linear function of their correlation, and an almost-but-not-quite-linear function of expected return of one of the assets. ↩
We add leverage such that VMOT’s standard deviation matches the market’s. ↩
Interactive Brokers charges 1% over the risk-free rate on margin loans between $100,000 and $1 million, and lower rates above $1 million. Shorting widely-traded index funds (such as SPY) generally costs 0.25%. Shorting more esoteric funds costs more, but the optimal portfolios in this model mostly just short the market, so we don’t need to worry about the costs of shorting the other assets. ↩
AQR (2018). It Was the Worst of Times: Diversification During a Century of Drawdowns. ↩
Michaud (1989). The Markowitz Optimization Enigma: Is ‘Optimized’ Optimal? ↩
Fuhrer and Hock (2019). Uncertainty in the Black-Litterman Model - A Practical Note. ↩ ↩²
I am not confident about this claim. I’ve read that’s true, e.g., in Fuhrer and Hock (2019) cited above. But when I tried to replicate this result, I found that portfolio resampling did not change the optimal allocation. I might have been doing something wrong in my replication. ↩

Uncorrelated Investments for Altruists

Mon, 23 Nov 2020 00:00:00 -0800

Summary

Altruists care a lot about finding investments with low correlation to other altruists’ portfolios. [More]
Altruists can probably decrease correlation by investing in under-utilized asset classes, most notably (1) commodities and (2) long/short equity indexes. [More]
But there’s a better way to decrease correlation. Some factors—such as value, momentum, and trend—have been shown to predict investment performance. [More]
We don’t know for sure if these factors will continue to work in the future, but evidence suggests that they will. [More]
There are publicly-available funds that provide concentrated factor exposure. [More]
Factor investing is psychologically painful for most people. We can do a few things to mitigate this, but ultimately it seems unavoidable, and this will prevent most people from investing in value/momentum/trend in practice. [More]

Disclaimer: This should not be taken as investment advice. This content is for informational purposes only. Any given portfolio results are hypothetical and do not represent returns achieved by an actual investor.

Cross-posted to the Effective Altruism Forum.

Last updated 2022-05-31.

Summary
Contents
Why uncorrelated investments?
- How valuable are uncorrelated investments?
Are there uncorrelated asset classes?
Factor investing
Uncorrelated investments in practice
- Funds I like
- Behavioral concerns
Appendix
Notes

Why uncorrelated investments?

As a philanthropist, if you have some investments that you plan to donate eventually, then you should be somewhat risk-averse with respect to those investments. When the stock and bond markets go up, you have more money, and you can donate more to charity. But other altruists also have money, and charities receive more funding, so your money isn’t as valuable. Conversely, when markets go down, you have less money to donate at the exact time when charities need funding the most.

You can avoid this by buying uncorrelated investments. If you can find a way to invest that has a low (or even negative) correlation to the overall altruistic portfolio, then there will be times when most altruists lose money, but you gain money, so you can make up for other altruists’ losses. And there will be times where you lose money and others gain money, but you’re not bothered by losing money because other altruists can now provide more funding. (And of course sometimes you’ll gain or lose money at the same time as other altruists, but not as often as if you held stocks and bonds.)

So if you can find an uncorrelated investment with positive expected return, that would be really valuable.

How valuable are uncorrelated investments?

We can use a simple model to get a sense of how much uncorrelated investments matter. Let’s start with the following assumptions:

Charitable causes have logarithmic utility of money.
My assets only represent a small percentage of the total altruistic portfolio.
Altruists all invest in a standard stock/bond portfolio, and use the optimal amount of leverage.
The unleveraged portfolio has a long-run expected return of 3%, and a standard deviation of 12%.
I have the ability to invest in an alternative asset with a modest 1% return and 12% standard deviation, but zero correlation to the standard stock/bond portfolio.
Asset returns follow a log-normal distribution.

The easiest way to think about the value of my portfolio is in terms of the certainty-equivalent interest rate: what guaranteed interest rate would I consider equally as good as this risky portfolio?

Under these assumptions, if I take my portfolio out of stocks/bonds and invest it all in the alternative asset, the new portfolio has a certainty-equivalent excess return of 0.7%.¹ If I have $1 million, that means switching to the alternative investment is as good as getting a guaranteed $7,000 per year—and the amount increases as my portfolio grows.

If the alternative asset earns a 3% expected return instead of 1%, switching my portfolio to the alternative asset provides a certainty-equivalent return of 5.9% (!).

And that’s assuming I keep using the same amount of leverage. If I lever up my investment to the point where the overall altruistic portfolio allocates the correct amount to this alternative asset, then I get a certainty-equivalent return of over 300%. (But this requires that I use on the order of 400:1 leverage, which isn’t even remotely realistic.)

This is just an illustrative example, and it’s probably not accurate. The key takeaway is that allocating to uncorrelated investments can potentially provide a lot of value to a portfolio.

Update 2020-12-14: Since originally writing this, I have somewhat changed my views on the significance of uncorrelated investing. I believe that for most altruistic investors in practice, earning higher expected return is more important than decreasing correlation. I explain why in Asset Allocation and Leverage for Altruists with Constraints.

I still believe that, if an altruistic investor can can decrease correlation without sacrificing expected return, then they should. And I still agree with what I wrote in this essay regarding the diversification benefits of different types of investments.

Are there uncorrelated asset classes?

Investors can choose from essentially four asset classes: stocks (a.k.a. equities), bonds, real estate, and commodities. Most people already hold stocks and bonds. Can we get uncorrelated return with real estate and commodities? And can we find under-utilized investments within equities?

(I wrote about most of these in a previous essay. Here, I will go into a little more detail.)

Real estate

For most individual investors, it doesn’t make sense to buy real estate as an investment because it exposes them to too much undiversified risk. But it might make sense for altruists to buy real estate as part of their altruistic investments—this could help diversify the overall altruistic portfolio. Still, there are a few problems with this:

Most people aren’t rich enough to buy a real estate property as an investment. Even fewer people are rich enough to buy real estate as part of their altruistic investments, i.e., after already investing enough personal money to ensure their financial security.
If altruists do buy real estate, they should make sure to diversify globally. People who own investment properties tend to buy them in the same city where they live, which makes the property much easier to manage. If effective altruists as a group do this, they will overly concentrate their investments in a few cities where EAs are over-represented, such as Oxford and Berkeley.

You can gain exposure to a basket of real estate investments by buying a REIT, which is a publicly-traded company that earns money primarily by owning real estate. But REITs behave more like stocks than like real estate,² so they don’t provide the same diversification benefits as direct investments in properties would.

Real estate makes sense as an investment for the extremely wealthy (for example, university endowments often own a lot of land). But it seems like altruists would have a hard time diversifying with real estate, even in the aggregate.

Commodities

Over the past century and a half, commodities have on average produced a positive return on top of the risk-free rate, with fairly low correlation to stocks (r=0.23) and essentially zero correlation to bonds (Levine et al., 2016).³⁴ Commodities aren’t perfectly uncorrelated with a conventional stock/bond portfolio, but they do provide some diversifying power.⁵

And unlike real estate, commodities are easy to buy. Various ETFs and mutual funds provide exposure to the broad commodities market (as of this writing, the most popular such ETF is PDBC). Commodity funds generally charge higher fees than equity or bond funds, but probably not enough to destroy the commodity return premium.

Startups

As suggested by Paul Christiano:

[I]f you start or invest in a small company, your payoff will depend on that company’s performance (which is typically quite risky but only weakly correlated with the market). […] This special case is only possible because the entrepreneur or investor is putting in their own effort, and moral hazard makes it hard to smooth out all of the risk across a larger pool (though VC funds will invest in many startups). You shouldn’t expect to find a similar situation in investments, except when you are providing insight which you trust but the rest of the market does not (thereby preventing you from insuring against your risk).

Individual companies still tend to move with the market at least somewhat, so for a truly uncorrelated approach, you could invest in the startup while shorting the market.

But note that, at least according to basic theory, you want the overall altruistic portfolio to hold stock in a startup in proportion to the global market portfolio. So if the startup represents 0.0000001% of global wealth, then value-aligned altruists should invest 0.0000001% in that company—which would probably be no more than about $10,000.

Can we find uncorrelated return within equities?

Thanks to home country bias, most investors put too much money in their home countries and not enough into international equities. Effective altruists are over-represented in a few countries like the United States, the UK, and Switzerland. So there might be an opportunity to diversify by over-weighting countries where not many effective altruists live. For example, you could buy an emerging markets index fund like VWO.

But foreign equities are still 80-90% correlated with the domestic market. You can make truly uncorrelated portfolio with a long/short portfolio: buy (go long) emerging markets while shorting domestic equities.

In general, this strategy has an expected return of zero: in the long run, all equity markets should in expectation behave the same on a risk-adjusted basis. But even a zero-expected-return investment might still be a good idea if it decreases the volatility of the overall altruistic portfolio.

But we might have reason to expect some regions of the world to perform better than others. We can fairly reliably predict returns over the span of a decade or so using John Bogle’s expected return formula:

Future Market Returns = Dividend Yield + Earnings Growth +/- Change in P/E Ratio

This formula has a strong theoretical justification, and it has worked well at predicting future returns over the past century (Bogle and Nolan, 2015⁶; Radha, 2020⁷). I won’t go into detail about why this formula makes sense, but for a good layperson’s explanation, see Colby Davis’ Does Reality Even Matter Anymore? Part 1 and Part 2.

Research Affiliates’ Asset Allocation Interactive uses similar methodology to project the 10-year expected return and volatility for a wide variety of asset classes. As of October 2020, it predicts the following expected real returns for various regions:

	US	Europe	Japan	Emerging Markets
Return	0.2%	5.2%	4.5%	6.9%
Standard Deviation	15%	19%	18%	21%

According to the Bogle return formula, we can get something like a 6% expected return by going long emerging markets and short US equities (probably closer to 4-5% after costs). And if altruistic investors in general under-weight emerging markets (which I expect they do), then we’re diversifying by doing so.⁸

We can do a backtest to get a sense of how this regional long/short strategy might look. Over the past 30 years, the US stock market has outperformed the rest of the world. What would have happened if we had bought US equities while shorting international equities?

This backtest doesn’t really tell us how a regional long/short strategy might perform in the future because (1) we’re cheating by picking the best region in hindsight, and (2) emerging market equities tend to be more volatile than developed markets like the US. But it at least tells us how this general type of strategy might perform. (These return figures are nominal, not real, and don’t include fees or transaction costs.)

	US	International	Long US, Short International
Return	10.2%	5.6%	3.5%
Standard Deviation	14.5%	16.4%	11.3%

If the Bogle return formula is to be believed, investors can earn a positive (in expectation) and uncorrelated return by buying emerging markets while shorting US stocks. Even if we believe the US and emerging markets have similar expected (risk-adjusted) return, most investors under-weight emerging markets, so holding a long emerging/short US equity portfolio could improve the overall altruistic portfolio. (But it could be unpleasant for the investor who knowingly holds a risky investment with zero expected return.)

Factor investing

Some of the asset class strategies identified in the previous section look at least somewhat promising. But we can probably do better.

The best way to get uncorrelated positive return might be to find a gifted investment manager who can beat the market, and then also short the market (or find a manager who gets positive performance with net zero market exposure). But among well-established managers, the best ones don’t want to take your money—they can only maintain their high returns if they keep their list of clients short. (The best investment fund in the world, Renaissance Medallion Fund, caps the fund at $10 billion and doesn’t accept any outside money.) And it’s hard to evaluate managers who don’t have an established track record. I don’t know how a manager without a long track record could produce compelling evidence that they have enough skill to beat the market.⁹ And if a manager uses a discretionary process, even if they’ve made money in the past, I have no way to know if they will continue to outperform.¹⁰

I prefer to focus on strategies that demonstrably work. And the current state of the evidence suggests that the most demonstrably effective way to get uncorrelated return is via factor investing.

What is factor investing?

First, some background. In the sixties, some very smart economists came up with the Capital Asset Pricing Model (CAPM). According to the original formulation of this theory, a stock can only earn a higher expected return if it has greater exposure to market movements. This exposure is called its market beta. When the market moves up or down, a high-beta stock should move up or down even more. And, importantly, there should be no way to predict a stock’s return other than by its exposure to market beta.

But there’s a problem with this theory. In a seminal 1992 paper, Eugene Fama and Kenneth French found¹¹ that they could predict stocks’ returns much more effectively using three factors: market beta, size, and book to market ratio (“value”). In their sample, small companies tended to perform better than CAPM predicted, and so did companies with high book to market ratios. The new Fama-French three-factor model had much greater explanatory power than the CAPM one-factor model.

Later, researchers gathered more evidence on the size and value factors and found that they replicated well out of sample. Additional factors were identified, such as momentum (Jegadeesh and Titman, 1993¹²) and profitability (Fama and French, 2004¹³). Soon, hundreds more papers were published in the quest to identify more factors. Many of these supposed factors failed to replicate, but a few stand out.

What are the factors?

Berkin and Swedroe’s book, Your Complete Guide to Factor-Based Investing, reviews the literature and presents the factors that the authors believe have the strongest supporting evidence and might be worth investing in. To earn its place in the book, a factor must meet five criteria:

Persistent — It holds across long periods of time and different economic regimes.

Pervasive — It holds across countries, regions, sectors, and even asset classes.

Robust — It holds for various definitions (for example, there is a value premium whether it is measured by price-to-book, earnings, cash flow, or sales).

Investable — It holds up not just on paper, but also after considering actual implementation issues, such as trading costs.

Intuitive — There are logical risk-based or behavioral-based explanations for its premium and why it should continue to exist.¹⁴

In a book review, Wesley Gray lists the factors from the book with his own assessment of the quality of evidence behind each, ranging from gold to bronze:¹⁵¹⁶

Market beta: Gold. Pure risk. Legit.

Term Factor: Gold. Pure risk. Legit.

Momentum Factor: Gold. Part risk, part systematic mispricing.

Value Factor: Gold. Part risk, part systematic mispricing.

Trend Following Factor: Gold. Part risk, part systematic mispricing.

Size Factor: Silver. Part risk, maybe part systematic mispricing.

Carry Factor: Silver. Mostly risk, potentially some systematic mispricing.

Profitability & Quality Factors: Bronze. Systematic mispricing? Risk? Unclear. Best used as supplement for size/value factors.

Low-volatility Factor: Bronze. Systematic mispricing? Best used as supplement for size/value/momentum factors.

Default (or credit) Factor: Bronze. Systematic mispricing? Doesn’t pay to own credit risk, historically.

(Gray’s review also includes a list of links for further reading under each factor.)

We already talked about market beta as part of CAPM. The term factor represents the fact that long-term bonds tend to outperform short-term bonds. Almost all investors already expose themselves to these first two factors simply by buying stocks and bonds. If we want uncorrelated investments, we should avoid the market beta and term factors.

Let’s look at the other three factors that Gray identifies as “gold”: momentum, value, and trend.

In brief, what do these factors mean?

Value: Cheap stocks (as measured by the price-to-book ratio, price-to-earnings, or some other similar metric) tend to outperform expensive stocks.
Momentum: Assets (such as stocks, bonds, or commodities) with good performance over the past 6-12 months relative to other assets tend to continue to outperform.
Trend: Assets with good absolute performance over the past 6-12 months tend to continue to perform well.

(To illustrate the difference between momentum and trend: suppose stock A is up 50% over the past 12 months, and stock B is up 10%. The momentum factor would buy stock A and short stock B, because stock A went up more. But the trend factor would buy both, because both went up in absolute terms. Momentum and trend sound similar, but they tend to operate differently enough that they’re worth counting as separate factors—see Factor correlations for a more quantitative justification.)

In practice, investors can use these factors to outperform the market by buying assets that have low valuations/high momentum/positive trend (and maybe also by short-selling expensive/low-momentum/negative-trend assets).

How do we know these factors are real, and not the result of data mining?

In the rest of this section, I will present a series of research papers on the value, momentum, and trend factors, and explain their significance. If you’re already familiar with the research or if you just want to take my word for it that factor investing works, feel free to skip ahead to Uncorrelated investments in practice.

Evidence on factor investing

In 2013, Asness, Moskowitz, and Pedersen published an impressive paper titled Value and Momentum Everywhere.¹⁷ According to this paper, value stocks and momentum stocks outperformed in four different equity markets and in country equity indexes, government bonds, currencies, and commodities. It also showed that the performance of value and momentum could not be explained by macroeconomic risk, liquidity risk, or transaction costs.

Geczy and Samonov (2016)¹⁸, Baltussen, Swinkels, and van Vliet (2019)¹⁹, and Samonov (2020) assembled much longer historical data sets to show that momentum and value have worked for two centuries.

Hou, Xue, and Zhang (2017)²⁰ attempted to replicate 447 supposed market anomalies and found that most did not replicate, but the value and momentum factors appeared robust. (The paper did not examine the trend factor.) In an interview, when asked which factors he expected to still work in 50 years’ time, Zhang said,

I can only speculate what would happen in 2067. I would bet on value, momentum, investment, and [return on equity] (as well as different combinations of these variables).²¹

Fact, Fiction and Momentum Investing²² lists some common arguments for why momentum supposedly doesn’t work in practice, and references research explaining why those arguments are wrong. For instance, it cites Trading Costs of Asset Pricing Anomalies²³, which used real-world data to show that value and momentum still outperform the market after transaction costs. (This study was later extended by Frazzini et al., Trading Costs²⁴, to look at $1.7 trillion of real-world trades, which found “actual trading costs to be an order of magnitude smaller than previous studies suggest”.)

What about the trend factor? Moskowitz, Ooi, and Pedersen’s paper Time Series Momentum²⁵ observed a highly significant (p<0.00001) trendfollowing factor “in equity index, currency, commodity, and bond futures for each of the 58 liquid instruments we consider”, even after controlling for the value and momentum factors. Later papers replicated this finding over a full century of data²⁶ and across 82 previously unstudied securities²⁷.

Why have these factors persisted?

If investors can increase their performance by investing in these factors, why doesn’t everyone invest in them? Why haven’t they been arbitraged away? It’s not entirely clear, but these factors likely persist due to a combination of risk and common behavioral biases (Lakonishok et al., 1993²⁸), and limitations on sophisticated investors that prevent them from correcting this mispricing (Lewellen, 2011²⁹; Shleifer and Vishny, 1997³⁰).

I won’t go into too much detail on the risk vs. behavioral bias question or on the details of the theories behind each. This slide deck summarizes more research on the question. Additionally, Berkin and Swedroe cover this subject in their book, with references to more academic literature.

Will these factors continue to work?

Obviously, we can’t see into the future, and we don’t know if the value, momentum, and trend factors will continue to predict performance. In particular, the value factor has come under scrutiny recently because it has underperformed since about 2007, and trendfollowing has performed poorly over the past decade (at least until the 2020 market crash, where it performed well). How concerned should we be that two out of these three factors haven’t worked well recently?

This recent poor performance is probably due to bad luck. Research on this question supports the idea that these factors will indeed continue to work:

Ilmanen et al. (2019)³¹ found “little evidence for arbitrage activity influencing returns”.³² That means sophisticated investors are not exploiting these factors to an extent that causes the factors to become less effective. Similarly, according to Blitz (2017)³³:

[F]or each factor there are not only funds which offer a large positive exposure, but also funds which offer a large negative exposure toward that factor. On aggregate, all factor exposures turn out to be close to zero, and plain market exposure is all that remains. This finding argues against the concern that factor premiums are rapidly being arbitraged away by investors in [exchange-traded funds].

Lettau et al. (2018)³⁴ found a similar result: in aggregate, mutual funds, ETFs, and hedge funds had negative exposure to the value factor, even among funds that claimed to be “value” funds.

Arnott et al. (2020)³⁵ looked at the value factor in particular, which has had the worst performance in recent years, and concluded that the recent underperformance does not suggest that value no longer works. They reviewed a number of popular explanations for why value investing might not work anymore, and found that while they had some explanatory power, none of them could fully explain value’s recent underperformance. Instead, according to the paper, value investing underperformed primarily because cheap companies became cheaper, and expensive companies became more expensive. Value investing outperforms when undervalued companies revert to reasonable valuations. But recently, undervalued companies have become even more undervalued. That means, in the words of Arnott et al., “the stage is set for potentially historic outperformance of [the value factor] over the coming decade.”

Dan Rasmussen’s 2020 letter An Apology for Small-Cap Value explained value’s recent poor performance in terms of increasing high-yield spreads and valuation spreads, and expressed a similar sentiment to Arnott et al.: “value is uniquely positioned to capitalize on perhaps a once- or twice-in-a-century convergence of opportunities.”

In short: If value investing had stopped working, we would expect to see a narrowing of the valuation spread between cheap and expensive stocks. Instead, the spread has widened, which suggests that, if anything, value will work better in the coming years than it did in the past (although changes in earnings growth could dampen the performance of value).

The 2019 paper You Can’t Always Trend When You Want³⁶ broke down the performance of the trend factor into two components: the existence of trends and the ability of trendfollowing strategies to profit off of them.³⁷ If trendfollowing doesn’t work anymore, we’d expect to see a decrease in trading profit per trend. But profits have been remarkably close to the long-run historical average. Instead, the recent poor performance of trendfollowing is explained by a lack of trends in most markets. The paper found that both the existence of trends and the ability to make money off of them vary over time, and the current period is not particularly unusual.

(Additionally, many investors benchmark their portfolios against the S&P 500, which has performed particularly well over the past decade. So all sorts of diversifying strategies, including not just factors but international equities, bonds, commodities, etc., look bad by comparison.)

Blitz (2011)³⁸ showed that “the theoretically optimal strategic allocation to these [factors] is sizable, even when using highly conservative assumptions regarding their future expected magnitudes.” The optimal portfolio should include tilts toward value, momentum, and trend, even if these factors will only provide slightly positive outperformance going forward. That means if the overall altruistic portfolio has net zero exposure to these factors, altruists on the margin might want to put their entire portfolios into value/momentum/trend, even if the factors have a low expected return (as long as the expected return is positive).

In a 2015 article, How Can a Strategy Everyone Knows About Still Work?,³⁹ Cliff Asness gives some reasons to expect factor investing to continue to work even when it’s well known, and how to think about factors and their expected performance going forward.

For more on this subject, see Berkin and Swedroe, Chapter 8: “Does Publication Reduce the Size of Premiums?”

So the value, momentum, and trend factors can predict asset performance. And although we can’t know for sure, it seems likely that these factors aren’t going away any time soon.

Berkin and Swedroe also identified size, carry, and profitability/quality as factors, and low-volatility and credit as potential factors. The evidence on these isn’t as strong, so let’s focus on the most robust factors: value, momentum, and trend. How can we use these factors to construct an uncorrelated investment portfolio?

Factor correlations

The value and momentum factors, as constructed in the previously-cited paper Value and Momentum Everywhere,¹⁷ had nearly zero correlation to the market and negative correlations to each other (r=–0.5). Hurst, Ooi, and Pedersen (2013)⁴⁰ showed that a simple trendfollowing strategy had close to zero correlation to stocks, bonds, or commodities. According to Time Series Momentum,²⁵ the trend factor was only weakly explained by the market beta, value, and momentum factors (r²=0.3).

As a simple replication with a longer but narrower data set, I used the Ken French data library to look at the correlation between the market beta, value, and momentum factors on US stocks from 1927 to 2019. According to the data, market beta and value had a correlation of r=0.23, market beta and momentum had r=–0.34, while value and momentum had r=–0.41.

Maybe correlation is too technical of a measurement. How does including value, momentum, and trend actually affect the performance of a portfolio? In particular, how do these factors tend to behave when the market performs poorly?

AQR (2018)⁴¹ investigated what investments historically provided the biggest benefit during market downturns. It looked at six different investment strategies over the past 90 years to see how they interacted with a stock portfolio both in general and in bad times (plus an additional five strategies over the most recent 30 years). Two strategies stood out as the best: trendfollowing and “styles”—their term for a long/short factor portfolio using value, momentum, and two other factors (carry and low-volatility).

My favorite chart from the AQR report:

If we don’t short individual assets

An academic factor portfolio buys stocks (or other types of assets) that look good according to a particular factor, and short sells assets that look bad. By shorting stocks, we can cancel out the market beta factor: when the market goes up, the long positions rise while the short positions fall (and vice versa), so the fund overall doesn’t move (much). This allows the fund to gain pure exposure to other factors with no exposure to the market.

But short-selling individual stocks can be expensive because you have to borrow the stocks you sell, and short lenders often charge high interest rates. Can we still build a value/momentum portfolio with low correlation to the stock market without shorting?

Let’s do a simple test with the Ken French data library. Instead of investing in the long/short value and momentum factors, what if we invest in the top 10% best value stocks (measured by book-to-market, and equal weighted), as well as the top 10% momentum stocks?

Using data from 1927–2019, the backtest gives the following results for the market, value stocks, and momentum stocks (ignoring fees and transaction costs):⁴²

	Market	Value	Momentum
Return	10.0%	19.4%	20.0%
Standard Deviation	18.5%	35.6%	25.3%
Sharpe Ratio	0.35	0.44	0.64
Correlation to Market	1	0.75	0.84

The value and momentum portfolios show impressive performance compared to the market, but unfortunately, they still have pretty high correlation. If we want uncorrelated returns, this doesn’t seem like it will work.

What if instead, as suggested by Alpha Architect, we short the index? We exploit the value or momentum factor on the long side, but on the short side we just hold the market. According to the linked article, we can short a market index for relatively cheap, and they’re more tax-efficient than stocks.

If we take the article’s assumption that the short proceeds equal the risk-free rate minus 0.25 percentage points, then according to the backtest, the long-factor/short-market portfolios performed as follows:⁴³

	Value	Momentum	Combined Val/Mom
Return	9.6%	9.5%	10.1%
Standard Deviation	24.8%	13.9%	16.6%
Sharpe Ratio	0.24	0.43	0.52
Correlation to Market	0.34	0.21	0.34

That’s more like it—these portfolios have much lower correlations to the market. We could even get the correlation down to zero by increasing the size of the short position, at the cost of worse return.

Optimized historical portfolios

Nobody knows how factor investing will perform in the future. For illustrative purposes, let’s construct theoretically optimal portfolios under different possible futures and see what they look like.

For simplicity, suppose we have three investment choices: the global equities market, the value factor, and the momentum factor. According to Asness et al. (2013)⁴⁴, these investments had historical volatilities of 16%, 11%, and 12%, respectively. Both factors had close to zero correlation to the market, and had –0.5 correlation to each other. Let’s assume all these numbers will continue into the future, but make different assumptions about future returns.

If we naively take the historical performance figures from Asness et al., we get about a 5% expected return for the market, 6% for value, and 7% for momentum. Under these assumptions, the optimal portfolio⁴⁵ allocates 9% to the market, 47% to value, and 44% to momentum.⁴⁶ Among these three investment options, the optimal historical portfolio contained almost no exposure to the market!

Let’s see what happens if we conservatively assume that the value and momentum factors will return a mere 1% each, while the market return stays at 5%. In this case, the theoretically optimal allocation is 39% to market, 32% to value, and 29% to momentum. Even in this conservative case, the (theoretically) best portfolio still allocates more than half to the value and momentum factors. (Blitz (2011)³⁸ finds a similar result, although it looks at long-only factor portfolios rather than long/short.)

We could try to tightly optimize our factor allocation by projecting future returns and then investing in the optimal portfolio according to our projections. But it’s hard to predict optimal allocations in advance, and equal-weighted portfolios tend to work just as well out of sample (DeMiguel, Garlappi, and Uppal, 2007⁴⁷). So in this example, we could simply allocate 1/3 to the market, 1/3 to value, and 1/3 to momentum.

These simulated examples suggest that the altruistic portfolio should allocate substantial sums to value, momentum, and maybe other factors. Almost no altruists do this. Therefore, it seems plausible that, on the margin, people should allocate 100% of their altruistic investments to long/short factors.

Uncorrelated investments in practice

If we want to move from the land of academia into the real world, we need to deal with some practical considerations:

How can we actually invest in the value, momentum, and trend factors?
How can we compensate for our emotions and behavioral biases that make factor investing difficult?

Funds I like

What are some actual investable funds that implement factor strategies like the ones I’ve discussed?

I haven’t spent much time surveying the landscape of funds, and I can’t claim to know which ones are best. If I were investing a lot of money, I’d want to spend much more time learning about what investment options are available and how they work. But it feels unfair of me to talk about factor investing and then not provide any practical suggestions. So as a starting point, I can talk about a few of the funds that I like.

Finding a good factor fund requires a little more than just searching for an ETF or mutual fund with “value” or “momentum” in its name. Many purported value/momentum funds are just index funds in disguise. We want to find funds that provide concentrated factor exposure, not just market beta with a weak tilt.

Astute readers will notice that I’ve cited a lot of papers published by researchers at AQR Capital Management. It sure would be nice if they would use all this great research to build some mutual funds that take advantage of the value, momentum, and trend factors!

Luckily for us, they do exactly that. I briefly reviewed all the mutual funds run by AQR, and based on what I read in the fund documents, I believe two funds give the best exposure to factors with low correlation to stocks and bonds:

AQR Style Premia Alternative Fund for exposure to value and momentum
AQR Managed Futures Strategy HV Fund⁴⁸ for exposure to trendfollowing

(The term “managed futures” can describe a range of investment strategies, but it usually refers to a trendfollowing strategy, as it does in the case of this fund). Buying these two funds together should provide exposure to the value, momentum, and trend factors. AQR Style Premia Alternative Fund also invests in the carry and “defensive”⁴⁹ factors. I haven’t talked about those factors in this essay because they’re not backed by as much evidence as value/momentum/trend, but allocating some money to them might not be a bad idea. (Notice that these are the same four factors used in the “styles” strategy from AQR (2018)⁴¹, cited previously in this essay.)

I found these funds independently of Berkin and Swedroe, but as a corroboration, their book recommends these two funds for combined factor exposure and for trendfollowing exposure, respectively.⁵⁰

Unluckily for some people, AQR funds have a $1 million minimum investment, so you can’t invest in both these funds unless you have $2 million. (There are three exceptions: (1) you can buy AQR funds in a retirement account with a lower minimum (but it probably doesn’t make sense to keep your altruistic funds in a retirement account); (2) if you live in Australia, you can buy AQR Australia funds with a minimum investment of $25,000; (3) some investment advisors can invest their clients’ money in AQR funds, even if they have less than $1 million.)

But good trendfollowing funds “all pretty much [do] the same thing”, according to Eric Crittenden, former manager of a trendfollowing fund-of-funds.⁵¹ I like the KFA Mount Lucas Index Strategy ETF (KMLM),⁵² which has a lower fee and no minimum investment.

If you want other ideas, this graphic shows the largest trendfollowing managed futures funds as of 2018.

Outside of AQR, it’s a little harder to find a good value or momentum fund with no correlation to the stock market. The investment firm Alpha Architect offers my favorite long-only value and momentum stock ETFs: QVAL, IVAL, QMOM, and IMOM. (Disclosure: I invest in these funds.) I like these funds because they provide much higher factor exposure than most. A more traditional fund such as Vanguard Value ETF provides large exposure to market beta, but only weakly tilts toward the value factor. The Alpha Architect funds are more expensive than Vanguard funds, but they’re just as cheap after adjusting for factor exposure.

The Alpha Architect founders⁵³ have published two books, Quantitative Value and Quantitative Momentum, describing their methodology in detail and explaining why they made the implementation choices that they did. They make some (well-justified) changes to the pure value and momentum factors, incorporating tilts toward quality and low volatility. So their funds may perform a bit better in expectation than a vanilla value or momentum fund (although the main benefit comes from concentrated exposure to the value and momentum factors, not the minor implementation changes). I won’t talk about how exactly these four funds work or why I expect them to outperform “pure” value/momentum, but the books go into lots of detail on this. Alpha Architect has also published two shorter articles summarizing how the funds work: The Quantitative Value Investing Philosophy and The Quantitative Momentum Investing Philosophy.

But these funds are long-only rather than long/short. As we saw previously, long-only equity funds likely have a high correlation to the broad market. According to Alpha Architect’s simulated historical data going back to 1992, an equal-weighted combination of the the four strategies (QVAL/IVAL/QMOM/IMOM) would have had a correlation to the stock market of 0.68.⁵⁴ But if we bought the Alpha Architect ETFs while shorting market index funds, as discussed previously, the resulting portfolio actually had a negative correlation to the market (r=–0.30). Alpha Architect’s Jack Vogel suggests a similar approach in his article, How to Create a Tax-Efficient Hedge Fund. He proposes buying value/momentum stocks while shorting the S&P 500. He claims that shorting indexes is “much cheaper on an after-cost, after-tax basis for a taxable investor” than shorting individual stocks.

(In practice, it probably doesn’t make sense to go fully market neutral. In theory, the larger your short position, the more leverage you can use, so your expected return doesn’t change; but after considering real-world costs and frictions, increasing the size of the short position reduces expected return. I believe the short side should be around 1/3 to 1/2 as big as the long side, based on an analysis I did here, in which I explain how I derived this estimate.)

I like Alpha Architect’s funds because they provide concentrated factor exposure, and the firm openly discusses the intricate details of their methodology. But unlike the AQR Style Premia Alternative Fund, the Alpha Architect funds only use equities. The AQR fund also uses bonds, commodities, and currencies, so it can probably provide better diversification (although it’s somewhat debatable whether a value/momentum factor fund should include asset classes other than equities). On the other hand, the Alpha Architect funds concentrate in a small number of stocks, which works better for small investors. And ETFs are generally more tax-efficient than mutual funds. Ultimately it’s not clear whether to prefer the AQR Style Premia Alternative Fund or a long/short strategy using the Alpha Architect ETFs, but either choice probably works fine.

This is how I personally would invest my altruistic money, given what I know right now:

I’d put about half in a long/short strategy where I buy QVAL/IVAL/QMOM/IMOM while shorting a US index fund and an international index fund, with the short position being about 1/3 to 1/2 as large as the long position.
I’d put the other half in KFA Mount Lucas Index Strategy ETF.⁵⁵
I’d use leverage—probably between 2:1 and 3:1.

Update 2020-12-14: I wrote a new essay that goes into more detail on asset allocation: Asset Allocation and Leverage for Altruists with Constraints. See the linked essay for further discussion.

There are many details to consider when making investments in practice, and I can’t cover all of them. If you’re considering building a factor portfolio, do more research or speak to a good investment advisor⁵⁶ to ensure you understand the implementation details.

I will briefly mention a few such details:

Different funds will have different tax consequences, which can substantially impact the long-run return of a portfolio.
Some ETFs, including the Alpha Architect funds, have fairly low volume, so traders must be careful to avoid paying large bid/ask spreads.
Wealthy investors can ask ETF issuers to issue new shares directly to them, rather than having to buy shares on the open market. This can avert liquidity issues.
You could pay an investment manager to develop a custom factor portfolio, which might come closer to ideal than the funds I proposed using. (I am sure Alpha Architect would provide this service for a reasonable fee—they already run custom strategies for some clients.) This approach seems especially appealing for extremely wealthy investors.
Altruists might prefer to use leverage, perhaps a lot of leverage. Investors who want leverage need to decide how to get it.
Altruists might prefer high-risk, high-return funds, especially if they can’t use leverage. For example, AQR Style Premia Alternative Fund targets 10% volatility, which is low enough that investors who can’t use leverage might prefer to invest in something more volatile.

Behavioral concerns

Factors like value and momentum probably arise from behavioral biases²⁸ that most investors share. People naturally compare their investments’ performance to the performance of the stock market. But a long/short factor portfolio (or really any sort of market-neutral portfolio) can deviate from the market for a decade or more at a time. Almost everyone naturally tracks their performance against the stock market, and doesn’t like to deviate too much. Even long-term-oriented funds such as endowments face heavy criticism after a single bad year. According to a survey by State Street, 40% of institutions would fire a manager after a single bad year, and 89% would fire them after two bad years (more).

Most people don’t actually want zero correlation to the market. We want something that goes up when the market goes down, but that doesn’t go down when the market goes up. (Please let me know if you ever discover such a mythical investment!⁵⁷)

Factor portfolios are extremely painful. Unless you are an alien.

This raises a practical problem: we’re not aliens. So how do we invest in factor portfolios without giving up when times get tough?

Trendfollowing fund manager Eric Crittenden tells a story about this:

I would show people the calendar-year returns of a managed futures index and let them compare those calendar-year returns to the US stock market and ask them, how would you feel about making a 5% allocation to this managed futures index? And the vast majority of them enthusiastically declined to make an allocation to managed futures. And I said, are you sure? Not even 5%? And they said no way, my clients are gonna fire me, it underperforms, this thing sucks. Why would I do that? I said OK, alright, no problem, let’s kick the managed futures out.

And then I would show them this thing that I call “all-weather” and I didn’t tell them what it was. But between you and me, it’s just a 50/50 split of managed futures and US stocks. But they don’t know. So they do the same thing—they scroll, down they look at the years, and they say, “All right, now we’re talking. This is something I can get behind.” […] And then I reveal to them it’s just a 50/50 split of the thing you kicked out and the thing you [already invest in]—and they’re baffled. Absolutely baffled. To the point where they don’t believe me so I have to give them the spreadsheet and let them look at it and see the math.⁵⁸

(Crittenden ended up starting a firm that invests in both equities and trendfollowing—much like the 50/50 split he describes.)

This story suggests that the best way to manage our behavioral issues might be to give up on the idea of investing in a strategy that’s totally uncorrelated to the market, and instead do something that’s only partially correlated, such as a mix between trendfollowing/momentum/value and a conventional equity index.

Some more suggestions, taken from talks by Meb Faber⁵⁹ and Wesley Gray:⁶⁰

Create a set of rules and then stick to them. If possible, automate execution—that way, you never have to force yourself to make an uncomfortable trade.
Share your investing plan with someone who can help hold you accountable.
Hire an investment advisor to keep you on track/provide behavioral coaching.
Make sure you understand what you’re investing in and why you expect it to work.
Study the history of your investment strategy, particularly the times when it’s performed badly. Cliff Asness of AQR has a rule of thumb on what to expect from a strategy during bad times: take the biggest historical drawdown and double it.

The way I think about it: if I’m losing money during a market upswing, then I can’t donate as much. But that’s okay because other altruists will donate more. I don’t think of my altruistic investments as my own money, I think of it as the world’s money. And I’m happy as long as the world gets richer overall.

Ultimately, I don’t believe there’s a great way to overcome our behavioral biases—if there were, factor investing would stop working. But I’m hoping the sort of person who’s crazy enough to donate 10% of their income to cash transfers or AI safety or wild animal welfare is also crazy enough to stick with a weird, unpopular investment portfolio even during bad times.

Appendix

Appendix A: Factors other than value/momentum/trend

In this essay, I focused on value, momentum, and trendfollowing because these are the strongest, most robust factors. But it might make sense to diversify across more than three factors. As shown in Blitz (2011)³⁸ and in my own analysis above, even a factor with only slightly positive expected return should still receive a substantial allocation. That suggests we should invest in as many factors as we can, as long as the factors are reasonably credible.

The funds I listed do include exposure to factors other than value and momentum. AQR Style Premia Fund includes the carry and “defensive” factors; QVAL/IVAL use quality; and QMOM/IMOM use low-volatility.

It might make sense to invest at least a little money in every factor listed by Berkin and Swedroe, with the largest allocations going to the value, momentum, and trend factors. Any alleged factors not included in Berkin and Swedroe’s list probably don’t have strong enough supporting evidence to justify investing in them.

Appendix B: Reducing correlation to income

Technically, we don’t just want to find investments with low correlation to a traditional stock/bond portfolio. Lots of altruists donate money out of their income, so we also want investments not to be correlated with income. In aggregate, income grows with the economy,⁶¹ so we want to reduce correlation to economic growth. Can we get that with the value, momentum, and trend factors?

Yes. Sheth and Tim (2017)⁶² found that the value and momentum factors both performed worse in recessions, but still had positive performance on average. (During recessions, the market had substantially negative performance.) Additionally, during recessions, the pairwise correlations (market, value) and (market, momentum) both dropped below zero, suggesting that they provide even stronger diversification benefits in bad times. In an informal article, Berkin (2020)⁶³ reproduced this result over a longer time horizon.

Hutchinson and O’Brien (2015)⁶⁴ found a similar result for trendfollowing: it performed worse during recessions than expansions, but still showed positive return during both types of economic environment, and provided diversifying power during downturns. I was able to reproduce this result using AQR’s publicly available time series momentum factor data.⁶⁵

Appendix C: Changelog

2020-12-04:

Add a reference to Babu et al. (2020), with explanation.
Update Funds I like based on some new analysis I’ve done since writing this essay. Previously, I wrote that I personally would invest in a fully market-neutral strategy. I now believe it makes more sense to have some market exposure because beyond a certain point, the cost of short-selling (in terms of reducing expected return) exceeds the diversification benefit.

2021-01-29:

Update Funds I like to replace AXS Chesapeake Strategy Fund with KFA Mount Lucas Index Strategy ETF (KMLM).
Change reference to “a forthcoming essay” to link to the actual essay.

2021-02-08:

Add a reference to Lettau et al. (2018) as a corroboration of Blitz (2017).

2022-05-31:

Minor wording improvements.

Notes

I performed these calculations using leverage.py, adding the following code:

rra = 1
print(LeverageEnvironment(
    rra=rra, mu=0.03, sigma=0.12
).certainty_equivalent_return())
print(LeverageEnvironment(
    rra=rra, mu=0.9999*0.03 + 0.0001*0.01,
    sigma=np.sqrt((0.9999*0.12)**2 + (0.0001*0.12)**2)
).certainty_equivalent_return())

↩

The linked article primarily reviews a paper: Kizer and Grover (2017). Are REITs a Distinct Asset Class?

It also provides some analysis that’s not covered in Kizer and Grover. ↩
Levine, Ooi, Richardson, and Sasseville (2016). Commodities for the Long Run. ↩
For interested readers, it is worth noting that commodity futures had positive excess return even while in contango (although returns were higher during times of backwardation, with arithmetic means of 1.8% and 7.7%, respectively). ↩
At least according to Levine et al., commodities provide more value when, rather than holding them long-only, we go long on commodity futures in backwardation and short commodities in contango. This is essentially a long/short carry factor strategy. In a later section, I will discuss in more general terms why I believe factor investing makes sense. ↩
Bogle and Nolan (2015). Occam’s Razor Redux: Establishing Reasonable Expectations for Financial Market Returns. ↩
Radha (2020). Using CAPE to Forecast Country Returns for Designing an International Country Rotation Portfolio. ↩
I know we haven’t gotten to the section on factor investing yet, but this strategy is a type of value factor strategy. Emerging equities are cheap and US equities are expensive, so a long/short strategy gains exposure to the value factor while mostly eliminating market beta exposure. ↩
Notice that getting uncorrelated positive return and beating the market are equivalent—you can convert one into the other. If a manager beats the market but still has exposure to the market, you can get uncorrelated positive return by shorting the market. And if a manager provides uncorrelated positive return, you can beat the market by investing in both the market and that manager. ↩
According to Carhart (1997), On Persistence in Mutual Fund Performance, some mutual funds do persistently beat the market, but their performance is almost entirely explained by their exposure to the size, value, and momentum factors. ↩
Fama and French (1992). The Cross-Section of Expected Stock Returns. ↩
Jegadeesh and Titman (1993). Returns to Buying Winners and Selling Losers: Implications for Stock Market Efficiency. ↩
Fama and French (2004). Profitability, Growth, and Average Returns. ↩
Introduction, page 24. ↩
Quote is edited for brevity. ↩
Berkin and Swedroe classify the factors a little differently. They identify seven factors: market beta, term, momentum, value, size, carry, and profitability/quality. They consider trendfollowing a strong investment strategy, but they don’t classify it as a separate factor. They also identify three commonly discussed factors that they don’t believe hold up: dividend yield, low-volatility, and default/credit. Gray does not include dividend yield on his list, presumably because he considers it sub-bronze (I would agree with that assessment).

Berkin and Swedroe acknowledge that trendfollowing has substantial predictive power on top of momentum, so I’m not sure why they don’t consider it a separate factor. ↩
Asness, Moskowitz, and Pedersen (2013). Value and Momentum Everywhere. ↩ ↩²
Geczy and Samonov (2016). Two Centuries of Price Return Momentum. ↩
Baltussen, Swinkels, and van Vliet (2019). Global Factor Premiums. ↩
Hou, Xue, and Zhang (2017). Replicating Anomalies. ↩
Wesley Gray (2017). Want to Learn More About Factor Investing? Read This. ↩
Asness, Frazzini, Israel, and Moskowitz (2014). Fact, Fiction and Momentum Investing. ↩
Frazzini, Israel, and Moskowitz (2013). Trading Costs of Asset Pricing Anomalies. ↩
Frazzini, Israel, and Moskowitz (2018). Trading Costs. ↩
Moskowitz, Ooi, and Pedersen (2011). Time Series Momentum. ↩ ↩²
Hurst, Ooi, and Pedersen (2017). A Century of Evidence on Trend-Following Investing. ↩
Babu, Levine, Ooi, Pedersen, and Stamelos (2018). Trends Everywhere. ↩
Lakonishok, Shleifer and Vishny (1993). Contrarian Investment, Extrapolation, and Risk. ↩ ↩²
Lewellen (2011). Institutional Investors and the Limits of Arbitrage. ↩
Shleifer and Vishny (1997). The Limits of Arbitrage. ↩
Ilmanen, Israel, Moskowitz, Thapar, and Wang (2019). How Do Factor Premia Vary Over Time? A Century of Evidence. ↩
McLean and Pontiff (2016)⁶⁶ examined 82 published market anomalies and found that on average, they worked 35% less well post-publication. Ilmanen et al. (2019) re-examined the McLean and Pontiff data along with new pre-sample data and found that the anomalies on average decayed about as much in the pre-sample period as post-sample, suggesting that of the performance decay was attributable to overfitting, not arbitrage by sophisticated investors. They “acknowledge, however, that the power to detect arbitrage crowding into a factor and its effect on prices is challenging […]. For this reason, different tests and different samples may yield different inferences and hence may be why our conclusions differ from Mclean and Pontiff (2016)”.

My own take is that even though the post-sample decays did not statistically significantly differ from the pre-sample decays for any of the tested factors (with the exception of two), the factors did overall perform worse post-sample, which offers Bayesian evidence that arbitrage activity influenced returns (even if only weakly). ↩
Blitz (2017). Are Exchange-Traded Funds Harvesting Factor Premiums? ↩
Lettau, Ludvigson & Manoel (2018). Characteristics of Mutual Fund Portfolios: Where Are the Value Funds? ↩
Arnott, Harvey, Kalesnik, and Linnainmaa (2020). Reports of Value’s Death May Be Greatly Exaggerated. ↩
Babu, Levine, Ooi, Schroeder, and Stamelos (2019). You Can’t Always Trend When You Want. ↩
Technically, it identified three components, but these two are the relevant ones for our purposes. ↩
Blitz (2011). Strategic Allocation to Premiums in the Equity Market. ↩ ↩² ↩³
Asness (2015). How Can a Strategy Everyone Knows About Still Work? ↩
Hurst, Ooi, and Pedersen (2013). Demystifying Managed Futures. ↩
AQR (2018). It Was the Worst of Times: Diversification During a Century of Drawdowns. ↩ ↩²
I used the data series titled “Portfolios Formed on Book-to-Market” and “10 Portfolios Formed on Momentum”. ↩
I used the data series titled “Fama/French 3 Factors”, “Portfolios Formed on Book-to-Market”, and “10 Portfolios Formed on Momentum”. ↩
Asness, Moskowitz, and Pedersen (2013). Value and Momentum Everywhere. ↩
“Optimal” is defined as the allocation that maximizes the Sharpe ratio. ↩
Calculated using mvo.py ↩
DeMiguel, Garlappi, and Uppal (2007). Optimal Versus Naive Diversification: How Inefficient is the 1/N Portfolio Strategy? ↩
AQR has two different managed futures funds, the only major difference being that one targets relatively low volatility, and the other targets high vol. Here I linked the high-volatility fund because the expenses are lower on a vol-adjusted basis, and altruists probably prefer higher volatility. ↩
In a few papers, AQR defines “defensive” as betting against beta, e.g., Ilmanen et al. (2019).³¹ Presumably, this fund uses the same definition. The details of the betting-against-beta factor are specified by Frazzini and Pedersen (2013), Betting Against Beta. This is related to the low-volatility factor discussed by Berkin and Swedroe. ↩
I originally read their book in 2016, and forgot most of its content. I identified these two AQR funds without any memory of what funds Berkin and Swedroe had recommended. Then, in the process of researching other parts of this essay, I re-read Berkin and Swedroe and saw that they recommended the two funds that I had already listed. ↩
Eric Crittenden – All-Weather Portfolios with Trend Following (S3E7). Flirting with Models podcast. 2020. Around 9:30. ↩
When this essay was originally published, I wrote that I invested in the AXS Chesapeake Strategy Fund. In February 2021, I moved my investment to KMLM, a new fund that launched shortly after I initially published this essay, primarily because (1) it charges lower fees on a vol-adjusted basis and (2) it’s an ETF instead of a mutual fund, which is more tax-efficient. There exist a few managed futures ETFs with lower fees, but I don’t invest in them because they have too little volatility. Investing in a cheap but low-vol fund means you need to use more leverage, so you end up paying more than if you had invested in the more expensive high-vol fund.

As of 2022, I invest in managed futures through a separately managed account instead of an ETF. But if I did use an ETF, I’d use KMLM. ↩
Coincidentally (or perhaps not so coincidentally), both Wesley Gray and Cliff Asness, co-founders of Alpha Architect and AQR respectively, got their PhDs under Eugene Fama—one of the most influential living figures in economics and widely described as the father of the efficient market hypothesis.

Does this prove that, to run a really good asset management firm, you have to study under Eugene Fama? ↩
I calculated this by combining Alpha Architect’s data with market data from the Ken French data library, specifically the series titled “Fama/French 3 Factors”. ↩
Instead of a 50/50 split, it might be better to use risk parity such that the portfolio is exposed to the same amount of volatility from each of the three factors. ↩
Unfortunately, many (most?) investment advisors are kind of clueless, even about basic things like minimizing fees. So it’s important to find a good one. ↩
Actually, I do know of an investment like this: Renaissance Technologies’ Medallion Fund. Unfortunately, they won’t accept my money. ↩
Eric Crittenden – All-Weather Portfolios with Trend Following (S3E7). Flirting with Models podcast. 2020. Around 54:00. ↩
Faber (2020). The Best Investment Opportunity in 40 Years. Interview at the Prime Quadrant conference. ↩
Gray (2019). Factor Investing, Simple but Not Easy. ↩
Not really, but it’s an okay approximation. ↩
Sheth and Tim (2017). Fama-French Factors and Business Cycles. ↩
Berkin (2020). Recessions and Factor Performance: What History Tells Us. ↩
Hutchinson and O’Brien (2015). Time Series Momentum and Macroeconomic Risk. ↩
Time Series Momentum: Factors, Monthly. AQR Data Library. AQR Capital Management, LLC. ↩
McLean and Pontiff (2016). Does Academic Research Destroy Stock Return Predictability? ↩

employer matching (%)
total investment return including dividends (nominal) (%)
dividend yield (%)
401(k) fee (%)
capital gains tax rate (%)
income tax rate today (%)
income tax rate in retirement (%)

Protest movements can have significant short-term impacts	Strong
Protest movements can achieve intended outcomes in North America and Western Europe	Strong

More wealth is better	\(\displaystyle\frac{dU}{dW} > 0\)
More of the mission target is worse	\(\displaystyle\frac{dU}{db} < 0\)
More of the mission target makes wealth more valuable	\(\displaystyle\frac{d^2U}{dW db} > 0\)

Wealth has diminishing marginal utility	\(\displaystyle\frac{d^2 U}{dW^2} < 0\)
Sufficient wealth can eliminate almost all of the mission target, but never quite all of it	\(\displaystyle\lim\limits_{W \rightarrow \infty} \displaystyle\frac{dU}{db} = 0\)
We have constant relative risk aversion with respect to wealth	\(-W \displaystyle\frac{d^2 U}{dW^2} / \frac{dU}{dW} = \gamma\)

	\(g\)	\(\mu\)	\(\sigma\)	\(\delta\)
\(\eta\)				\(\eta\)=0.5	\(\eta\)=1	\(\eta\)=1.5	\(\eta\)=2	\(\eta\)=3
AUS	3.37	5.47	7.23	3.9	2.1	0.5	-1	-3.6
BEL	2.26	5.90	9.11	4.9	3.6	2.6	1.8	0.8
CHE	2.45	5.29	6.82	4.1	2.8	1.7	0.6	-1.1
DEU	3.83	6.14	6.32	4.3	2.3	0.4	-1.3	-4.6
DNK	2.74	6.93	6.43	5.6	4.2	2.9	1.7	-0.5
ESP	3.02	4.10	8.62	2.7	1.1	-0.3	-1.6	-3.5
FIN	3.91	8.03	15.52	6.4	4.1	2.5	1.4	1.1
FRA	2.38	4.48	8.75	3.4	2.1	1	0.1	-1.1
GBR	2.32	4.28	8.02	3.2	2	0.9	0	-1.4
ITA	3.12	4.62	6.72	3.1	1.5	0	-1.4	-3.8
JPN	4.29	5.04	7.45	3	0.8	-1.3	-3.3	-6.7
NLD	3.48	5.29	6.92	3.6	1.8	0.1	-1.4	-4.2
NOR	3.28	6.64	7.26	5.1	3.4	1.8	0.3	-2.1
PRT	2.94	5.04	8.88	3.7	2.1	0.7	-0.4	-2.2
SWE	3.06	6.77	8.62	5.3	3.7	2.3	1	-0.9
USA	3.12	5.51	8.39	4	2.4	0.9	-0.4	-2.4

Philosophical Multicore

Do Small Protests Work?

Contents

Evidence from nationwide natural experiments

Direct evidence from lab experiments

Indirect evidence from lab experiments

Non-experimental evidence

Conclusion

Future work

Appendix: Table of papers from Orazani et al. (2021)

Notes

My Third Caffeine Self-Experiment

Contents

Experimental procedure

Results

Alternative experimental procedures that I’m not going to do

A story about how I thought my experiment failed, but actually I was just being stupid

Notes

Things I've Become More Confident About

Notes

Will Welfareans Get to Experience the Future?

Notes

The Next-Gen LLM Might Pose an Existential Threat

Mechanisms Rule Hypotheses Out, But Not In

Why?

Some more examples

Notes

How Much Does It Cost to Offset an LLM Subscription?

The model

The inputs

Revenue-to-valuation ratio

Valuation-to-expenditures ratio

Frontier AI company expenditures

Total cost to offset AI company harm

The answer (according to my model)

Limitations of the model

Conclusion

Notes

I Made an Emacs Extension That Displays Magic: the Gathering Card Tooltips

Usage

Card legality

Exporting to HTML

Table utilities

AI Safety Landscape and Strategic Gaps

Contents

Introduction

Prelude

Some positions I’m going to take as given

Definitions

Prioritization

Why not technical safety research?

Why not AI policy research?

Downsides of AI policy/advocacy (and why they’re not too big)

What kinds of policies might reduce AI x-risk?

Some AI policy ideas I like

Maybe prioritizing post-TAI animal welfare

Why not prioritize digital minds / S-risks / moral error / better futures / AI misuse x-risk / gradual disempowerment?

Who’s working on them?

Some relevant research agendas

General recommendations

Advocacy should emphasize x-risk and misalignment risk

Prioritize work that pays off if timelines are short

Top project ideas

Talk to policy-makers about AI x-risk

Write AI x-risk legislation

Advocate to change AI training to make LLMs more animal-friendly

Develop new plans / evaluate existing plans to improve post-TAI animal welfare

Honorable mentions

Directly push for an international AI treaty

Organize a voluntary commitment by AI scientists not to build advanced AI

Peaceful protests

Media about dangers of AI

Message testing

Host a website for discussion of AI safety and other important issues

List of other project ideas

AI-for-animals ideas

Neartermist animal advocacy

Using TAI to improve farm animal welfare

Lobby governments to include animal welfare in AI regulations

Traditional animal advocacy targeted at frontier AI developers