How Can Donors Incentivize Good Predictions on Important but Unpopular Topics?

Altruists often would like to get good predictions on questions that don’t necessarily have great market significance. For example:

• Will a replication of a study of cash transfers show similar results?
• How much money will GiveWell move in the next five years?
• If cultured meat were price-competitive, what percent of consumers would prefer to buy it over conventional meat?

If a donor would like to give money to help make better predictions, how can they do that?

You can’t just pay people to make predictions, because there’s no incentive for their predictions to actually be accurate and well-calibrated. One step better would be to pay out only if their predictions are correct, but that still incentivizes people who may be uninformed to make predictions because there’s no downside to being wrong.

Another idea is to offer to make large bets, so that your counterparty can make a lot of money for being right, but they also want to avoid being wrong. That would incentivize people to actually do research and figure out how to make money off of betting against you. This idea, however, doesn’t necessarily give you great probability estimates because you still have to pick a probability at which to offer a bet. For example, if you offer to make a large bet at 50% odds and someone takes you up on it, then that could mean they believe the true probability is 60% or 99%, and you don’t have any great way of knowing which.

You could get around this by offering lots of bets at varying odds on the same question. That would technically work, but it’s probably a lot more expensive than necessary. A slightly cheaper method would be to determine the “true” probability estimate by binary search: offer to bet either side at 50%; if someone takes the “yes” side, offer again at 75%; if they then take the “no” side, offer at 62.5%; continue until you have reached satisfactory precision. This is still pretty expensive.

In theory, if you create a prediction market, people will be willing to bet lots of money whenever they think they can outperform the market. You might be able to start up an accurate prediction market by seeding it with your own predictions; then savvy newcomers will come and bet with you; then even savvier investors will come and bet with them; and the predictions will get more and more accurate. I’m not sure that’s how it would work out in practice. And anyway, the biggest problem with this approach is that (in the US and the UK) prediction markets are heavily restricted because they’re considered similar to gambling. I’m not well-informed about the theory or practice of prediction markets, so there might be clever ways of incentivizing good predictions that I don’t know about.

Anthony Aguirre (co-founder of Metaculus, a website for making predictions), proposed paying people based on their track record: people with a history of making good predictions get paid to make more predictions. This incentivizes people to establish and maintain a track record of making good predictions, even though they don’t get paid directly for accurate predictions per se.

Aguirre has said that Metaculus may implement this incentive structure at some point in the future. I would be interested to see how it plays out and whether it turns out to be a useful engine for generating good predictions.

One practical option, which goes back to the first idea I mentioned, is to pay a group of good forecasters like the Good Judgment Project (GJP). In theory, they don’t have a strong incentive to make good predictions, but they did win IARPA’s 2013 forecasting contest, so in practice it seems to work. I haven’t looked into how exactly to get predictions from GJP, but it might be a reasonable way of converting money into knowledge.

Based on my limited research, it looks like donors may be able to incentivize donations reasonably effectively with a consulting service like GJP, or perhaps by doing something involving predictions markets, although I’m not sure what. I still have some big open questions:

1. What is the best way to get good predictions?
2. How much does a good prediction cost? How does the cost vary with the type of prediction? With the accuracy and precision?
3. How accurate can predictions be? What about relatively long-term predictions?
4. Assuming it’s possible to get good predictions, what are the best types of questions to ask, given the tradeoff between importance and predict-ability?
5. Is it possible to get good predictions from prediction markets, given the current state of regulations?

Discuss on the Effective Altruism Forum.

Posted on

Should Global Poverty Donors Give Now or Later? An In-Depth Analysis

Disclaimer: I am not an investment advisor and nothing in this essay serves as investment advice.

Introduction

Robin Hanson: If More Now, Less Later

The rate of return on investment historically has been higher than the growth rate–or, as they say, r > g. If you save your money to donate later, you can earn enough interest on it that you eventually have the funds to donate a greater amount. Because r > g, you should invest your money for as long as you can before donating1–or so the argument goes.

Traditionally, we’d apply a discount rate of g to future donations, because that’s the rate at which people get richer and therefore the rate at which money becomes less valuable for them. But this ignores some important factors that affect how much we should discount future donations, and we can create a much more detailed estimate. This essay will explore that in detail. Exactly what factors determine the investment rate of return and the discount rate on poverty alleviation? Can we gain any information about which is likely greater?

Posted on

Why Do Small Donors Give Now, But Large Donors Give Later?

Some people have observed that small and large donors follow different giving patterns. Small donors who give out of their salary—that is, most people—tend to donate money more or less as soon as they earn it (usually within a year). Large donors—e.g., extremely wealthy people and foundations—tend to slowly distribute their money and hold on to most of it1. For example, large foundations typically donate little more than the legally required 5% of assets each year. Why do they behave differently?

I don’t believe this difference is surprising, and actually it’s not really even a difference.

Posted on

Where Some People Donated in 2017

This is a collection of writings on where people are donating. It only includes writings that I am aware exist (obviously) and that are written by effectiveness-minded people.

My descriptions are paraphrased from the linked writings as much as possible. The writing in this post includes combinations of my own and the linked writers’ words. My summaries often do not do the original writers justice, so I recommend reading all of the linked articles if you are interested.

Posted on

Newcomb's Problem and Efficient Markets

Summary: The stock market can be modeled as Omega in Newcomb’s problem. On average, an asset will only outperform if the market predicts that you won’t buy it. So you cannot say “if I had bought that, I would have made a lot of money”, just as in Newcomb’s problem you can’t say “if I had taken both boxes, I would have gotten more money than if I only took one”.

Posted on

All Expected Utility Distributions Have One of Two Big Problems

Summary: All expected value distributions must either (1) have a mean of infinity, or (2) have such thin tails that you cannot ever reasonably expect to see extreme values. When we’re estimating the utility distribution of an intervention, both of these options are bad.

Posted on

New Comment System

I have removed Disqus and replaced it with built-in static comments. Disqus comments are disabled, but still visible on any old posts1. New posts going forward will only use the new static comment system.

I had been wanting to switch off Disqus for a while. It has a few disadvantages:

1. I have no control over comments except for the moderation tools Disqus provides.
2. I have no control over how comments are displayed.
3. I don’t know what Disqus is doing or might do with commenters’ personal information.

The new comment system does exactly what I want it to do and nothing more.

Edited to add: If anyone’s interested, I’m using the Jekyll Static Comments plugin by Matt Palmer, with a few personal modifications.

Notes

1. A lot of old posts don’t have any comments as of this writing, so I removed Disqus from those posts. I left the Disqus comment section only on posts that actually had comments.

Posted on

An Idea for How Large Donors Can Support Small Charities

Summary: Large donors may create better incentives for both charities and small donors if, rather than providing fixed funding to a charity, they offer to match all donations to that charity over a relatively long time horizon.

Posted on

Do Investors Put Too Much Stock in the U.S.?

Summary: Investment advisors typically recommend that you put somewhere between 50% and 75% of your stock investments into US stocks and the rest into international markets. Most individual investors have 70% or more of their stock money in the US or their home country, a phenomenon that’s aptly called home country bias. But there are reasons to believe that even 50% is too much, and most people should really have more like 0-30% of their stock investments in the United States12.

Disclaimer: I am not an investment advisor and this should not be taken as investment advice. Please do your own research or seek professional advice and otherwise take reasonable precautions before making any significant investment decisions.

Posted on

Ideas Too Short for Essays

Ever since I’ve been writing essays, I have always accumulated many more essay ideas than I end up actually writing. I frequently have an idea, write a paragraph or two, and then realize I have nothing left to say. Rather than leaving these unpublished, I am trying an experiment. This post contains a compilation of some of these ideas that were too short for essays.

In this issue, we discuss: