Epistemic status: Speculating with no central thesis. This post is less of an argument and more of a meditation.

A decade ago, before there was a visible path to AGI and before AI alignment was a significant research field, I figured the solution to the alignment problem would look something like Coherent Extrapolated Volition. I figured we’d find a way to get the AI to internalize human values. I had problems with this approach (why only human values?), but I still felt reasonably confident that the coherent extrapolation of human values would include concern for the welfare of all sentient beings. The CEV-aligned AI would recognize that factory farming is wrong, and that wild animal suffering is a big problem.

Today, the dominant research paradigms in AI alignment have nothing to do with CEV, and I don’t know what to think.

Continue reading

Last September, I wrote:

1. There’s a (say) 80% chance that an aligned(-to-humans) AI will be good for animals, but that still leaves a 20% chance of a bad outcome.
2. AI-for-animals receives much less than 20% as much funding as AI safety.
3. Cost-effectiveness maybe scales with the inverse of the amount invested. Therefore, AI-for-animals interventions are more cost-effective on the margin than AI safety.

Today, I’m fleshing out this argument with a cost-effectiveness model. The model estimates how much it costs to make progress on AI alignment—the general problem of getting ASI to achieve any goal without subsequently killing everyone—compared to how much it costs to make progress on aligning AI to animal welfare specifically.

The model is on SquiggleHub: https://squigglehub.org/models/AI-for-animals/alignment-to-animals-EV-simple

Continue reading

AI alignment is typically defined as the task of aligning artificial superintelligence to human preferences. But non-human animals, future digital minds, and maybe other sorts of beings also have moral worth; ASI ought to care for their interests, too.

In broad strokes, if we place all alignment techniques on a spectrum between

getting AI to do things that their users expressly want in the immediate term

and

embedding in AI the generalized notion of respecting beings’ preferences

then things more like the latter are better for non-humans, and things more like the former are worse.

In this post, I review 12 categories of AI safety research based on how likely they are to be good for non-human welfare.

Continue reading

(Or: Why I don’t see how the probability of extinction could be less than 25% on the current trajectory)

AI developers are trying to build superintelligent AI. If they succeed, there’s a high risk that the AI will kill everyone. The AI companies know this; they believe they can figure out how to align the AI so that it doesn’t kill us.

Maybe we solve the alignment problem before superintelligent AI kills everyone. But if we do, it will happen because we got lucky, not because we as a civilization treated the problem with the gravity it deserves—unless we start taking the alignment problem dramatically more seriously than we currently do.

Think about what it looks like when a hard problem gets solved. Think about the Apollo program: engineers working out minute details; running simulations after simulations; planning for remote possibilities.

Think about what it looks like when a hard problem doesn’t get solved. Consider the world’s response to COVID.

When I look at civilization’s response to the AI alignment problem, I do not see something resembling Apollo. When I visualize what it looks like for civilization to buckle down and make a serious effort to solve alignment, that visualization does not resemble the world we live in.

Continue reading

Introduction

Most people who write about AI and investing fall into one of two camps: traditional investors who see the high valuations of AI stocks and say it’s a bubble;¹ or AGI-pilled investors who will buy AI stocks at any price, regardless of fundamentals. There’s only a tiny intersection of people who understand that AGI is not a normal technology while also recognizing that fundamentals matter.

I’m not an expert (or even a journeyman) on AI or fundamental analysis, but I do know a little bit about both.

The basic thesis of value investing is that the market over-rates expected future growth and under-rates present-day fundamentals. Stocks that are poised to benefit from AGI tend to be growth stocks—people have high expectations for them, and they’re priced expensively relative to present-day fundamentals. That suggests that we shouldn’t buy AI-related stocks.

At the same time, the market does not appear to expect AGI, which suggests we should buy them. Which of these two forces is stronger?

My current thinking is that value investing probably won’t work in light of AGI, but there is some reason to believe it might work even better; and value investing is a useful hedge in case AI progress slows.

Continue reading

Value investing had a singularly bad run from 2007 to 2020. (And it hasn’t done great since 2020, either.) Is that because value investing is broken, or did it simply hit a streak of horrendous luck?

Skeptics of value investing have made many claims about why value investing doesn’t work anymore, but these claims tend to be light on evidence.¹ Value investing proponents have empirically researched most of these claims and found that they don’t stand up to scrutiny.²³⁴

The poor performance of the value factor was not primarily driven by weakening fundamentals, but by the widening of the value spread. A wider value spread makes value investing look more attractive going forward, not less.

What's the value spread?

Value stocks are defined using the ratio of a stock’s price to some fundamental metric—for example, earnings, book value, or cash flow. If we use earnings as the metric, then value stocks are those with low P/E ratios and growth stocks are the ones with high P/Es.

The value spread is the ratio of price-to-fundamental ratios between growth stocks and value stocks. For example, if growth stocks have an average P/E of 30 and value stocks have an average of 15, then the value spread is 30/15 = 2.

All else equal, a wider value spread is good for value because you’re buying the same fundamentals at a lower price. However, a widening spread is bad for value because it means value stocks are declining (relative to growth stocks). This is analogous to how bond investors like when bond yields are high, but they lose money when yields are increasing.

I wouldn’t dismiss value investing on the basis of poor recent performance.

However, there’s a potentially strong argument against value investing that remains unrefuted.

Historically, the structural return of the value factor—the component of return that comes from company fundamentals, rather than changes in the value spread—was about 4–6%.⁴ But over the past two decades, that number has averaged a mere 1%. Unlike with the value spread, a muted structural return does not imply higher future expectations for value investing.

Continue reading

Summary

Time series momentum (TSMOM) is an investment strategy that involves buying assets whose prices are trending upward and shorting assets that have a downward trend. In 2012, Moskowitz, Ooi & Pedersen published Time Series Momentum¹. They analyzed a simple version of the strategy that buys assets with positive 12-month returns and shorts assets with negative 12-month returns. They found that the strategy had statistically significant outperformance in equity indexes, currencies, commodities, and bond futures from 1985 to 2009.

However, others have raised doubts. Huang, Li, Wang & Zhou (henceforth HLWZ) criticized the strategy in Time Series Momentum: Is It There? (2020)², concluding that the evidence for TSMOM is not statistically reliable.

Some of their criticisms have merit, but TSMOM remains an appealing strategy.

The abstract of Time Series Momentum: Is It There? reads:

Time series momentum (TSMOM³) refers to the predictability of the past 12-month return on the next one-month return and is the focus of several recent influential studies. This paper shows that asset-by-asset time series regressions reveal little evidence of TSMOM, both in- and out-of-sample. While the t-statistic in a pooled regression appears large, it is not statistically reliable as it is less than the critical values of parametric and nonparametric bootstraps. From an investment perspective, the TSMOM strategy is profitable, but its performance is virtually the same as that of a similar strategy that is based on historical sample mean and does not require predictability. Overall, the evidence on TSMOM is weak, particularly for the large cross section of assets.

To rephrase, HLWZ make two central arguments:

1. Moskowitz, Ooi & Pedersen (2012)¹ did a pooled regression and found a statistically significant correlation, but this methodology is flawed: it finds a strong correlation even when time series momentum cannot predict future prices.
2. TSMOM performed similarly to a strategy that simply buys assets with positive long-run historical returns and shorts assets with negative historical returns. The authors call this strategy Time Series History or TSH.

My responses to the two arguments:

1. I agree that a pooled regression is flawed. But a statistically significant correlation on a pooled regression is not what convinced me that TSMOM works. [More]
2. TSMOM and TSH indeed have similar(ish) historical returns. However:
   1. TSMOM’s positive performance cannot be explained by TSH alone. [More]
   2. TSMOM provided better diversification to an equity portfolio. [More]
   3. TSMOM has large unexplained returns when regressed onto a Fama-French factor model. [More]
   4. TSMOM still performed well on a much larger sample going back a century. [More]

TSMOM looks strong in the historical data. TSMOM probably survives fees and trading costs, but the evidence for that is less clear. [More]

Continue reading

You may have seen this graph from Chris Olah illustrating a range of views on the difficulty of aligning superintelligent AI:

Evan Hubinger, an alignment team lead at Anthropic, says:

If the only thing that we have to do to solve alignment is train away easily detectable behavioral issues…then we are very much in the trivial/steam engine world. We could still fail, even in that world—and it’d be particularly embarrassing to fail that way; we should definitely make sure we don’t—but I think we’re very much up to that challenge and I don’t expect us to fail there.

I disagree; if governments and AI developers don’t start taking extinction risk more seriously, then we are not up to the challenge.

Continue reading

Many people in AI safety, especially AI policy, want to increase government expertise. For example, they want to place people with AI research experience in relevant positions within government. That may not be a good idea.

People who better understand AI can write more useful regulations. However, people with relevant expertise (such as ML researchers) tend to be less in favor of strong regulations and more in favor of accelerating AI development.¹ We need regulations to prevent misaligned AI from killing everyone, and to prevent other kinds of catastrophes. If government expertise goes up, all else equal we will get fewer such regulations, not more.

Continue reading

As of a few days ago, my website supported comments via Commento. If you click on that link, you will find that the page doesn’t load. Unfortunately, that website was also hosting my website’s comments, so all the comments are gone now, and I have no way to recover them.¹ Some of y’all left some good comments, but future readers will never know what they were.²

(I knew Commento was no longer actively supported, but in my foolishness, I thought to myself, well, the comments still work, so I’ll keep using it. Too bad I didn’t back up the comments while I had the chance.)

Commento was my third comment system. Originally I used Disqus, but I didn’t like how it impacted page load times, and I didn’t like how it disrespected my readers’ privacy. So I switched to a janky basic HTML commenting system that required me to manually copy/paste people’s comments into a text file so my website could serve them statically. That system was annoying³, so I switched to Commento, which was lightweight, privacy-respecting, and didn’t require manual effort on my part.

Commento is dead. My website now uses Comentario, which is basically the same as Commento except that (1) it still exists and (2) it’s self-hosted, which means even if Comentario stops existing and the website disappears, the comments on my website will still work.

(Commento had an option for self-hosting, but it looked like a lot of work so I didn’t do it.)

(Setting up Comentario self-hosting was a lot of work, confirming my suspicions. It took me about 10 hours, although to be fair, 8 of those 10 hours were spent trying to upgrade my server’s operating system because it was too old to be compatible to Comentario, and also it reached end-of-life in 2021 and probably had a lot of security vulnerabilities, oops. Anyway I hope y’all appreciate all the work I’m doing to prevent Disqus from spying on you.)

Notes