AI companies want to bootstrap weakly-superhuman AI to align superintelligent AI. I don’t expect them to succeed. I could give various arguments for why alignment bootstrapping is hard and why AI companies are ignoring the hard parts of the problem; but you don’t need to understand any details to know that it’s a bad plan.

When AI companies say they will bootstrap alignment, they are admitting defeat on solving the alignment problem, and saying that instead they will rely on AI to solve it for them. So they’re facing a problem of unknown difficulty, but where the difficulty is high enough that they don’t think they can solve it. And to remediate this, they will use a novel technique never before used in history—i.e., counting on slightly-superhuman AI to do the bulk of the work.

If they mess up and this plan doesn’t work, then superintelligent AI kills everyone.

And they think this is an acceptable plan, and it is acceptable for them to build up to human-level AI or beyond on the basis of this plan.

What?

Continue reading

I’ve been playing a lot of MTG Arena lately, but I refuse to spend any money on it, which means I can’t craft many rare cards. When I look up meta decklists, they always include a lot of rares and mythic rares. I don’t want to spend all my rare wildcards on one deck!

That’s sort of what the Pauper format is for. Pauper decks are only allowed to use common cards, which makes them cheap. But that format isn’t quite what I’m looking for, for four reasons:

1. There is no Pauper ladder on Arena, there are only tournaments.¹ I don’t want to play a tournament, I just want to be able to hop on and play a few games.
2. I have some rare wildcards that I can use to craft rare cards; I don’t have to limit myself to commons only. I just don’t have very many rare wildcards, so I want to spend them judiciously.
3. All the strongest Pauper decks are aggro decks. What if I don’t want to play aggro?
4. There are thousands of MTG cards that aren’t playable on Arena, so I can’t build most Pauper decks anyway.

There’s the Artisan format which is Arena-specific (so it fixes problem 4), but it still has the other three problems.

What I really want is to build a Standard deck using only 4–8 wildcards to craft the most important rares, and then if I decide I like the deck enough, I can craft some more. Which means I want to know which rares I really need, and which ones I can replace with common or uncommon substitutes.

Continue reading

Inspired by this post by Tomás Bjartur, which is an allegory; but I’m not writing an allegory, I’m writing about the rules of Quidditch.

The rules of Quidditch have a big problem. The game ends when a seeker catches the snitch, and the snitch is worth 150 points. So most of the players on the field don’t matter; in almost all games, the only thing that matters is who catches the snitch.

This also makes it a bad spectator sport because you can’t see the snitch, so nobody knows what the hell is going on.

I propose some rule changes:

Continue reading

Sometimes I’m talking about lifting weights and someone asks me, “What’s your goal weight?” I don’t understand why I would have a goal weight.

Say I want to bench press 300 pounds. What happens when I reach 300? I just give up on the bench press now? That would be silly. If I can keep getting stronger, I should.

What happens if I fall short of my goal? Say I haven’t been able to bench more than 285.¹ Should I start eating 5000 calories a day to put on as much muscle as possible? No, I’m not going to do that, I don’t want to get fat. Realistically, if I fall short of my goal, the answer to the question of what I should change is “nothing”.

The point of a goal is to make tradeoffs between objectives. But when you set goals, you have less information about your costs than when you’re trying to implement them. At implementation time, you have new information that might change how you prioritize things, which may result in failing to achieve a goal; and that’s perfectly fine.

Sometimes a goal turns out to be easier than you thought; that doesn’t mean you should give up after you achieve it.

Sometimes a goal turns out to be harder than you thought; that doesn’t mean you should sacrifice everything else for it.

Continue reading

A curiosity stopper is an answer to a question that gets you to stop asking questions, but doesn’t resolve the mystery.

There are some curiosity stoppers that I’ve heard many times:

• Why doesn’t cell phone radiation cause cancer? Because it’s non-ionizing radiation.
• Why are antioxidants good for you? Because they eliminate free radicals.
• Why do bicycles stay upright? Because of gyroscopic forces.
• Why do solids hold together? Because of intermolecular forces of attraction.

For the first three, those answers confused me because I didn’t know what those words meant. I guess I know what an ion is (it’s an atom with an electrical charge) but why do I care whether radiation is ionizing? And what makes radiation ionizing or non-ionizing?

What’s a free radical? Why is it bad?

What’s a gyroscopic force? (What even is a gyroscope? It’s some sort of top, right?) How on earth does a bicycle generate a gyroscopic force?

The fourth curiosity stopper—”intermolecular forces of attraction”—is even more of a non-answer. Of course solids hold together because a force holds them together. That’s what a force is. But what is the force, and where does it come from?

Another genre of curiosity stopper is the out-of-context number:

• “The Dow is down 600 points today.” (How much is that?)
• “My proposed policy will create two million jobs.” (What percentage is that? What are the odds that I, personally, get a new job?)
• “This product has 7 grams of protein per serving!” (How big is a serving? How much would I need to eat to meet my daily protein requirement?)

Answers (sort of)

Continue reading

Vizzini: Inconceivable!

Inigo: You keep using that word. I do not think it means what you think it means.

What did Inigo mean by this?

(Don’t laugh, this is serious.)

Continue reading

Classically, according to the Abrahamic religions, God is a man.

According to some more recent depictions, God is a woman. Which is a nice subversion.

But like, y’all are both a bit crazy. If there is an omnipotent Creator of the universe, then it definitely doesn’t have a gender.

When people call God “he” or “she”, this is what they’re saying happened:

Continue reading

An investor is considering putting her money into a mutual fund. “I will just invest some money for the next six months,” she says, “and see how it goes.”

A philanthropist is considering donating to a charity. “I will donate some money and see how it goes.”

Harvard University is considering whether SAT scores are all that important for admissions. “Let’s make SAT scores optional and see what happens.”

A child climbs to the top of a slide and is about to jump off the edge. “Don’t jump off of that,” his mom says, “you’ll get hurt.” He jumps off the slide. He gets hurt.

Not-invented-here syndrome is when an organization unnecessarily re-invents products or tools that already exist elsewhere. The cousin of this phemonenon is not-discovered-here syndrome, in which people refuse to consider evidence unless they’ve collected it themselves.

“A wise man learns from his mistakes, but a wiser man learns from the mistakes of others.” Not-discovered-here syndrome is what happens when you insist on making mistakes for yourself.

Continue reading

One day, I was at my grandma’s house reading the Sunday funny pages, when I suddenly felt myself getting sucked into a Garfield comic.

Continue reading

If we are correct about the laws of physics, then ghosts can’t exist. But some people are insistent that they’ve directly interacted with ghosts. Is there a way ghosts could exist if we modified the laws of physics a bit?

Continue reading