Many people support stricter regulations on frontier AI, and some of those people donate to political campaigns for pro-regulation candidates. Not many people oppose AI regulations, but that group includes AI companies and execs who have giant piles of cash to throw at super PACs.

This situation is made possible by three court decisions:

1. Buckley v. Valeo (1976) held that limits on independent political spending violate the First Amendment (while limits on direct contributions to candidates do not).
2. Citizens United (2010) extended those First Amendment protections to corporations and unions, thus allowing them to spend unlimited money on political ads.
3. SpeechNow.org v. FEC (2010) built on Citizens United to allow unlimited donations to independent-expenditure-only committees—i.e., super PACs—as long as they don’t directly work with political candidates.

Donations straight to a candidate are still capped at a few thousand dollars per person; donations to a super PAC supporting that candidate are not. If SpeechNow were overturned, or a law passed overriding it, the $5,000 cap on super PAC contributions would apply again, which means the popularity of a position would matter more than how rich its supporters are.

(I often hear “Overturn Citizens United” as the slogan, but SpeechNow allows super PACs to exist, and Buckley v. Valeo enables unlimited spending by individuals. What the advocacy groups in the space really want to do is restore the power of Congress and the states to enforce spending limits.)

Restoring spending limits would prevent unpopular but rich anti-regulation special interests (like Leading the Future) from dominating the political funding landscape. This would make the AI regulation situation better, but it would also improve the situation on any policy issues where unpopular, wealthy special interests wield disproportionate (and arguably un-democratic) influence.

It would also worsen any situation in which rich special interests are in the right, and the majority is in the wrong. But there aren’t many of those, and anyway there’s good reason to dislike an un-democratic process even if it produces a good outcome in a particular case.

An important caveat: Before writing this post, I was confident that imposing campaign spending limits would be a good thing. After doing some more research, I find it hard to dismiss the free speech defense of unlimited campaign spending (I’ll say more about this in the next section). Both sides of the debate have points in their favor. My thesis in this post should be taken as the narrow claim that campaign spending limits would have positive first-order effects on AI extinction risk, and they’re worth exploring on that basis. I don’t have an established view on whether stricter spending limits would be good or bad all-things-considered.

Continue reading

TLDR: Take the population ethics quiz here: https://mdickens.me/pop-ethics/

Population ethics is an oft-overlooked subfield within ethics. Many people hold views that they don’t realize contradict each other, or that have strange implications that they wouldn’t endorse if they thought about it more.

Not just that—population ethics is a BIG DEAL. A lot of ethical decisions hinge on how you think about changes in future populations.

I’ll give concrete examples below the fold, but reading the examples might change your answers, so you should take the quiz first. It has 13 to 17 short questions. (Questions are skipped when they can’t affect the results.)

Continue reading

This post is a submission to the Cluelessness Critiques Essay Competition.

Credence: Likely.¹

Anthony DiGiovanni’s sequence, The challenge of unawareness for impartial altruist action guidance (henceforth “the sequence”), argues that impartial altruists have no non-arbitrary reason to prefer any decision over any other. The central problem is unawareness: “many possible consequences of our actions haven’t even occurred to us in much detail, if at all.”

I disagree. Impartial altruists should act as if we have precise probabilities for beliefs and precise expected utilities of decisions, even if we (usually) do not explicitly state them. It’s okay to act on heuristics, but those heuristics are in service of maximizing the expected value of a (non-literal) utility function, rather than being “terminal” heuristics.

“Act as if” means: either I give an explicit precise credence; or I don’t, but I operate under the assumption that I could give a precise credence if I spent enough time reasoning through my beliefs.

This post starts by explaining the sequence’s central arguments in my own words. Then it offers five defenses of acting on your best guess.

Continue reading

I continue to believe we should pause frontier AI development. Any discussion of alternative strategies should be thought of as planning for contingencies.

A unifying driver behind many post-alignment risks—catastrophic risks that remain even if we solve the alignment problem—is that by strong default, ASI would end liberal democracy. Liberalism—in which people have individual rights, autonomy, and the ability to choose their own destiny—is an important force protecting human welfare.¹ When people are free, we are reasonably good at making our lives better of our own volition.

Many post-alignment risks have a certain flavor. AI-empowered terrorism; coups; permanent dictatorships; concentration of power. Those risks already exist today (and existed 20 years ago), but they’re mitigated by the fact that power is relatively evenly distributed across people. The most powerful person in the world doesn’t have an extraordinary advantage over the 10th-most-powerful person. ASI could change that.

If people still have civil liberties post-ASI, that will only be because the controllers of ASI allow us to have them.²

One way of thinking goes: AI will be extremely powerful. If everyone had their own personal AI, we could each use it to protect our own interests, and things will turn out okay for us. But how do you get there? It’s not going to happen automatically, but it may be possible to set up a gradual process to keep power balanced.

Continue reading

I continue to believe we should pause frontier AI development. Any discussion of alternative strategies should be thought of as planning for contingencies.

“Super-persuasive” AI is dangerous because a misaligned ASI could persuade humans to help it take over. But setting that aside, even if we manage to make ASI friendly, it may provide super-persuasive but mistaken guidance that permanently sets us down the wrong path.

This post focuses on the danger of an aligned super-persuasive AI that simply comes up with the wrong answers.

Continue reading

An argument goes: If we don’t build aligned artificial superintelligence, we risk driving ourselves extinct for some other reason. We should rush to build ASI quickly, in spite of the risks—the longer we wait, the more vulnerable we are to extinction from a different cause.

Other than ASI, the biggest extinction risk is synthetic biology. Some lab could (accidentally or on purpose) develop a highly transmissible, 100% fatal super-plague that wipes out humanity.

An aligned ASI could stop that from happening by shutting down dangerous biological research, or by developing advanced countermeasures that stop the spread of deadly infections. So the argument goes: We need to build ASI to save us from non-AI extinction risks.

However, that argument doesn’t work. In the near term, AI will make biological risks worse, not better. AI will accelerate scientific research, which will bring us closer to the level of knowledge necessary to build extinction-level pathogens. And in the long term, the way ASI eliminates biological x-risk is by taking control of the world.

Continue reading

Say you work at a private company that gives you stock options or RSUs. How should you value your stock?

• If you have a choice between getting more stock or more cash salary, how do you decide which to get?
• If you have the chance to sell some stock, should you do it?

Stock is risky and inflexible (especially if you work for a private company where you can’t easily sell shares), but you might be able to get it at a discount to its true value. How do you estimate how much it’s worth?

One heuristic you can use is to compare the stock against a risk-matched index fund. What would happen if you used the cash to buy a leveraged index fund with the same level of risk as the company stock? If the leveraged index fund has a higher expected return than the company stock, that means cash is probably better. (The reverse is not necessarily true because company stock can have other downsides, which I will get into later.)

Continue reading

Image credit: Jebulon

To prevent superintelligent AI from killing everyone, I would like there to be a strong international agreement banning the development of ASI until it can be proven safe. But that sort of agreement requires a lot of political buy-in and coordination. In the meantime, it may be easier to get light-touch AI safety regulations passed. To what extent do weak regulations decrease extinction risk?

In this post:

• Part I discusses routes by which weak regulations can reduce extinction risk. [More]
• Part II considers some downsides of weak regulations. [More]
• Part III reviews specific categories of weak regulation and how they might reduce risk. [More]

Continue reading

Depth-first plans lay out a path from here to aligned superintelligent AI. We need those kinds of plans. But depth-first plans depend on many assumptions: “We will make AI safe by doing step 1, then step 2, then step 3.” Step 1 only works under condition A, step 2 requires condition B, step 3 requires condition C. If A or B or C is false, the whole plan fails (and there’s a good chance we all die).

Consider Google’s safety plan from April 2025. To my knowledge, this is the best among the frontier AI companies’ plans.¹

Google’s plan depends on a series of conditions:

Continue reading

They say exercise improves sleep quality. Is that true for me?

To test this hypothesis, I took my daily calorie expenditures from the Apple Health app and correlated them with that night’s sleep time.¹ I also included caffeine intake as a potential confounding variable.

The hypothesis: when I exercise more, I’ll get better rest that night, and therefore wake up earlier.

The results:²

name	coef	t-stat	p-value
intercept	9.0134	65.072	0.0000
calories	-1.6844	-6.967	0.0000
caffeine	0.4157	9.404	0.0000
R²	0.2409

I sleep 10 minutes less for every additional 100 calories of exercise. Exercise plus caffeine explained 24% of the variance in my sleep time; exercise alone explained 6.6%.

The trend shows up whether or not I have caffeine:

Data are binned into increments of 100 calories. Any bins with fewer than 5 data points are not displayed. Vertical lines show the 95% confidence intervals for each bin.

Continue reading