x-risk-themed

Sometimes, a friend who works around here, at an x-risk-themed organisation, will think about leaving their job. They’ll ask a group of people “what should I do instead?”. And everyone will chime in with ideas for other x-risk-themed orgs that they could join. When they make their suggestions, a lot of the conversation will be about who’s hiring, what the pay is, what the work-life balance is like, how qualified for the role the person is and other features of the job. Certainly, they’ll be trying some amount to pick things that match their impact models, or those of the person they’re advising. But I feel like not as the highest crux. Not as a node at the top of the search tree that disqualifies everything else.

To be clear, I think it’s good that people are open to trading off impact for other resources. Most people aren’t in crunch mode, super short timelines mode; even if their models would license that, I think they don’t know how to do it without throwing their minds away or Pascal’s mugging themselves. And if they’re not playing a super short term game, it can’t be the plan to run super unsustainably for an arbitrary length of time. And people probably make better plans if they’re honest about their limits.

But, given the breadth of tradeoffs that are being considered, given my suspicion that impact models are coming weirdly far down the consideration order, I’m surprised that basically no one ever mentions working for or starting a non-x-risk org. And when it is brought up, it’s just like that: “you could work at a non-x-risk place”, “maybe you could just do a startup?”. Almost never concrete things like “you could convince farmers to plant hawthorne trees, maybe offering them financing so they’ll take the long-lead time high-payoff strategy”.

And I think that’s pretty bad. I think it means that people want to stay x-risk-themed even when it doesn’t make x-risk-sense.

But, if you don’t get an x-risk-themed job, if you go and work for Google rather than Anthropic, you can’t go to Constellation, you can’t get an office at Lighthaven, you are judged by some, you’re somewhat less connected to your social scene, invited to a few less things, and maybe that snowballs into more isolation.

Listen, it makes a lot of sense to want to be around people who are woke to x-risk. There are some kinds of orienting that it’s hard to do alone. It can be a good idea to “work with your door open” à la Hamming, so you passively get exposed to new ideas and opportunities, ones that actually make sense. And it can be alienating to spend time with people who aren’t woke to x-risk. Also, the x-risk crowd is positively selected in a bunch of ways — ways that you may also be, so it’s kind of reasonable to want to join that positively selected crowd. And, if you’ve worked around here for a while, then you have a bunch of personal and professional connections with people here, ones that you probably want to keep enjoying.

If you work on something x-risk-themed, it can help you think about x-risk, even if you don’t believe in your day-to-day work. You’re working on something related everyday. It might be easier to turn the problem over in your mind if your life’s shape is in part a reaction to the problem. If you feel dissatisfied with your work, the shape of your dissatisfactions might tell you something about what you’d rather do instead.

But people underestimate the dangers of working on x-risk-themed stuff.

Here’s one. Crucial considerations and sign flips are common. Maybe sneakily trying to do compromise policy is destroying all the credibility we’ll need at the one critical juncture. Or, maybe being weird on the internet is ruining our ability to get taken seriously in DC. Maybe, working at labs is the only way to have actual leverage over the important decisions. Or, maybe it provides a fig leaf that makes it easy for them to fob off complaints, while you get corrupted and advance their agendas.

Here’s another danger, which I think may be worse. if you insist on working somewhere x-risk-themed, you’re basically asking for someone to make you a sucker.

When I was in college, I was mugged. A few evenings later, I was hanging out at my friend S’s house, and it was getting late. My friend W was about to leave to walk home, when he remembered the mugger. Enjoying the feeling of being a bit scared, he decided he needed to be able to defend himself. “C’mon, S,” he said, “You’ve got to give me something! I can’t go back without something to defend myself.” Eventually, S scrounged up a hammer. A hammer is not a weapon. And it’s not a good idea to defend yourself from a phone theft with a hammer anyway. A phone is cheap to avoid prison time and mental scarring.

I think sometimes, I feel like I need something to defend myself from x-risk. It’s a little bit like when you’re performing, and you don’t know what to do with your hands. And it’s a little bit like W’s “you’ve got to give me something!”. How could I be unarmed when I face the end? And so I’m looking around, looking for something to pick up in both hands, to have something I’m doing about it all.

I think a lot of people feel something like this. It makes it a good market to sell blunderbusses and levers and pitchforks, as long as they’re labelled “x-risk”. In my mind, I sometimes see this theming as a sky blue lick of paint, and someone is offering me a bunch of levers and pipes that lead off invisibly into the heavens. The handles I’d operate are right next to the x-risk spot, so that seems promising. “Operate these,” they say, “it’s part of a plan to make things better.”

For a long time, I felt envious of people with big visions of how they were going to tackle x-risk. I didn’t think their visions were good. But at least they had them! “One day,” I thought, “I will join their ranks. I will enter The Reference Class.” In my mind, I had a good vision, unlike all the others. It didn’t occur to me to think that the visions being bad was a defining characteristic of The Reference Class.

It can feel better to do something x-risk-themed than to live a life on the farm, knowing about x-risk, caring about it, but knowing you don’t have a good idea of something to do.