# Evaluation Frameworks (or: When Importance / Neglectedness / Tractability Doesn't Apply)

Let’s look at how we use frameworks to prioritize causes. We’ll start by looking at the commonly-used importance/neglectedness tractability framework and see why it often works well and why it doesn’t match reality. Then we’ll consider an alternative approach.

## Importance/neglectedness/tractability framework

When people do high-level cause prioritization, they often use an importance/neglectedness/tractability framework where they assess causes along three dimensions:

1. Importance: How big is the problem?
2. Neglectedness: How much work is being done on the problem already?
3. Tractability: Can we make progress on the problem?

This framework acts as a useful guide to cause prioritization. Let’s look at some of its benefits and problems.

Posted on

# Quantifying the Far Future Effects of Interventions

Part of a series on quantitative models for cause selection.

## Introduction

In the past I’ve written qualitatively about what sorts of interventions likely have the best far-future effects. But qualitative analysis is maybe not the best way to decide this sort of thing, so let’s build some quantitative models.

I have constructed a model of various interventions and put them in a spreadsheet. This essay describes how I came up with the formulas to estimate the value of each intervention and makes a rough attempt at estimating the inputs to the formulas. For each input, I give either a mean and σ1 or an 80% confidence interval (which can be converted into a mean and σ). Then I combine them to get a mean and σ for the estimated value of the intervention.

This essay acts as a supplement to my explanation of my quantitative model. The other post explains how the model works; this one goes into the nitty-gritty details of why I set up the inputs the way I did.

Note: All the confidence intervals here are rough first attempts and don’t represent my current best estimates; my main goal is to explain how I developed the presented series of models. I use dozens of different confidence intervals in this essay, so for the sake of time I have not revised them as I changed them. To see my up-to-date estimates, see my final model. I’m happy to hear things you think I should change, and I’ll edit my final model to incorporate feedback. And if you want to change the numbers, you can download the spreadsheet and mess around with it. This describes how to use the spreadsheet.

Posted on

# A Complete Quantitative Model for Cause Selection

Part of a series on quantitative models for cause selection.

Update: There’s now a web app that can do everything the spreadsheet could and more.

Quantitative models offer a superior approach in determining which interventions to support. However, naive cost-effectiveness estimates have big problems. In particular:

1. They don’t give stronger consideration to more robust estimates.
2. They don’t always account for all relevant factors.

We can fix the first problem by starting from a prior distribution and updating based on evidence–more robust evidence will cause a bigger probability update. And we can fix the second problem by carefully considering the most important effects of interventions and developing a strong quantitative estimate that incorporates all of them.

So that’s what I did. I developed a quantitative model for comparing interventions and wrote a spreadsheet to implement it.

Posted on

# GiveWell's Charity Recommendations Require Taking a Controversial Stance on Population Ethics

Update 2016-12-14: GiveWell’s 2016 cost-effectiveness has updated the way it handles population ethics. It now explicitly takes the value of saving a 5-year old’s life as input and no longer assumes that it’s worth 36 life-years.

GiveWell claims that the Against Malaria Foundation (AMF) is about 10 times as cost-effective as GiveDirectly. This entails unusual claims about population ethics that I believe many people would reject, and according to other plausible views of population ethics, AMF looks less cost-effective than the other GiveWell top charities.

## GiveWell’s Implicit Assumptions

A GiveWell-commissioned report suggests that population will hardly change as a result of AMF saving lives. GiveWell’s cost-effectiveness model for AMF assumes that saving one life creates about 35 quality-adjusted life years (QALYs), and uses this to assign a quantitative value to the benefits of saving a life. But if AMF causes populations to decline, that means it’s actually removing (human) QALYs from the world; so you can’t justify AMF’s purported cost-effectiveness by saying it creates more happy human life, because it doesn’t.

You could instead justify AMF’s life-saving effects by saying it’s inherently good to save a life, in which case GiveWell’s cost-effectiveness model shouldn’t interpret the value of lives saved in terms of QALYs created/destroyed, and should include a term for the inherent value of saving a life.

GiveWell claims that AMF is about 10 times more cost-effective than GiveDirectly, and GiveWell ranks AMF as its top charity partially on this basis (see “Summary of key considerations for top charities” in the linked article). This claim depends on the assumption that saving a life creates 35 QALYs.

To justify GiveWell’s cost-effectiveness analysis, you could say that it is good to cause existing people to live longer, but it is not bad to prevent people from existing. (Sean Conley of GiveWell says he and many other GiveWell staffers believe this.)

In particular, you’d have to assume that:

Posted on

# Songs of the Day

Starting about seven years ago, every time I heard a song that I really liked that stuck with me for the rest of the day, I recorded it in my journal on a list of “Songs of the Day”.

This list shows how my musical tastes have shifted over the years. It isn’t entirely representative because there are plenty of songs I love that never made it onto this list.

Here’s the complete list up to the time of this writing. An asterisk means that I liked the song before it was Song of the Day, but I gained a new appreciation for it on that day. I have a corresponding Spotify playlist, although it only includes songs up to November 2015.

Posted on

# On Priors

Part of a series on quantitative models for cause selection.

## Introduction

One major reason that effective altruists disagree about which causes to support is that they have different opinions on how strong an evidence base an intervention should have. Previously, I wrote about how we can build a formal model to calculate expected value estimates for interventions. You start with a prior belief about how effective interventions tend to be, and then adjust your naive cost-effectiveness estimates based on the strength of the evidence behind them. If an intervention has stronger evidence behind it, you can be more confident that it’s better than your prior estimate.

For a model like this to be effective, we need to choose a good prior belief. We start with a prior probability distribution P where P(x) gives the probability that a randomly chosen intervention1 has utility x (for whatever metric of utility we’re using, e.g. lives saved). To determine the posterior expected value of an intervention, we combine this prior distribution with our evidence about how much good the distribution does.

For this to work, we need to know what the prior distribution looks like. In this essay, I attempt to determine what shape the prior distribution has and then estimate the values of its parameters.

Posted on

# How Should a Large Donor Prioritize Cause Areas?

## Introduction

The Open Philanthropy Project has made some grants that look substantially less impactful than some of its others, and some people have questioned the choice. I want to discuss some reasons why these sorts of grants might plausibly be a good idea, and why I ultimately disagree.

I believe Open Phil’s grants on criminal justice and land use reform are much less effective in expectation1 than its grants on animal advocacy and global catastrophic risks. This would naively suggest that Open Phil should spend all its resources on these more effective causes, and none on the less effective ones. (Alternatively, if you believe that the grants on US policy do much more good than the grants on global catastrophic risk, then perhaps Open Phil should focus exclusively on the former.) There are some reasons to question this, but I believe that the naive approach is correct in the end.

Why give grants in cause areas that look much less effective than others? Why give grants to lots of cause areas rather than just a few? Let’s look at some possible explanations for these questions.

Posted on

# How Sentient Are Farm Animals?

I wrote this as a quick explanation of why I value non-human animals the way I do. It’s not particularly thorough, and my explanation has some clear holes; this is just a general outline.

When we’re considering charitable interventions that help animals, it’s important to have some sense of how valuable it is to help those animals, which means we want to know how sentient they are.

How sentient an animal is–that is, how strongly it experiences pleasure and pain–almost certainly relates to how its brain works. I see four reasonably plausible ways that sentience could relate to brain size:

1. Suffering is caused by certain fixed brain structures, and for certain types of physical pain (like what chickens experience on factory farms), humans and chickens have the same brain parts and therefore experience this pain equally.
2. Sentience is linear with brain size.
3. Sentience is sub-linearly related to brain size; for example, sentience may be logarithmic with brain size.
4. Less intelligent animals are generally more sentient because they “are [not] capable of intelligently working out what is good for [them], and what damaging events [they] should avoid”, so they need a stronger pain response to compensate.
Posted on

# The Myth that Reducing Wild Animal Suffering Is Intractable

Lots of people accept that wild animal suffering is a big problem, but they believe it’s completely intractable. I even see some people claim that it’s one of the biggest problems in the world, but we still shouldn’t try to do anything about it. Wild animal suffering is in fact much more tractable than most people believe.

If we think wild animal suffering is a pressing problem and we want to do something about it, what can we do?

Posted on

# Preventing Human Extinction, Now With Numbers!

Part of a series on quantitative models for cause selection.

## Introduction

Last time, I wrote about the most likely far future scenarios and how good they would probably be. But my last post wasn’t precise enough, so I’m updating it to present more quantitative evidence.

Particularly for determining the value of existential risk reduction, we need to approximate the probability of various far future scenarios to estimate how good the far future will be.

I’m going to ignore unknowns here–they obviously exist but I don’t know what they’ll look like (you know, because they’re unknowns), so I’ll assume they don’t change significantly the outcome in expectation.

Here are the scenarios I listed before and estimates of their likelihood, conditional on non-extinction:

*not mutually exclusive events

(Kind of hard to read; sorry, but I spent two hours trying to get flowcharts to work so this is gonna have to do. You can see the full-size image here or by clicking on the image.)

I explain my reasoning on how I arrived at these probabilities in my previous post. I didn’t explicitly give my probability estimates, but I explained most of the reasoning that led to the estimates I share here.

Some of the calculations I use make certain controversial assumptions about the moral value of non-human animals or computer simulations. I feel comfortable making these assumptions because I believe they are well-founded. At the same time, I recognize that a lot of people disagree, and if you use your own numbers in these calculations, you might get substantially different results.