EA vs rats vs post rats vs EA adjacent

These are loose distinctions, based on my experience.

  • EAs are people and orgs motivated by using reason and evidence to improve the world. At least thats what the initial motivation was.
    • Exemplified by people like Peter Singer and Will MacAskill.
    • Orgs: CEA, Open Philanthropy, now Coefficient Giving.
  • Rats, aka rationalists, are folks mainly hanging around forums such as LessWrong, Slate Star Codex, and Overcoming Bias.
  • Post rats: TPOT (this part of Twitter) and some older rat folks who seem relatively enlightened to me.
    • They were usually rats themselves at some point, then took a more pragmatic or spiritual approach.
    • Part of that comes from learning the limits and perils of the epistemic and instrumental rationality espoused by rats.
    • TPOT is a loose social label too. I wouldnt treat everyone around it as a post rat.
  • EA adjacent: ex-EA folks, or those who sympathise with EA but do not identify as one.

From my experience, P(EA | rat) >= P(rat | EA), because EA seems to have a higher base rate than rats. I havent counted this. Post rats seem to be an even smaller minority.

What is my stance on EA?

  • I want to distinguish the people from the institutions.
  • From my interactions, EA folks are some of the smartest and most prosocial people I have met.
    • They are some of the densest high trust groups I’ve found.
    • I’ve made a lot of friends, even though we dont agree on many things.
  • AFAIK there isnt a single EA high command. But following the paper trail gives you a rough idea of who is pulling the strings.
  • Like other institutions, EA institutions suffer from power seeking and moral mazes.
    • Being in positions of power, capital, and influence has made the stakes higher.
    • The baseline risk of fumbling with such stakes is high enough that idt being an EA affects it much.
    • The closest analogy is a destroyer vs a yacht trying to cross a vortex. One needs to be careful in assigning causality to the behaviour of both vessels.
    • But you could ask: why would you steer towards a vortex if you knew it was on the way? Which is a sensible position.
    • I think the usual argument involves massaging utility functions and probability distributions until you get Pascal-wagered into taking the bet.
  • People and institutions coevolve, and its hard to separate them. But I want to separate the role from the individual who is instantiating it.
    • This reduces the agency placed on the individual, but thats how our institutions are structured given the power levels.
    • Hence the idea of gaining power as a means to do impactful things.
    • But it sidelines the pitfalls of amassing power for its own sake.

So how can you remain sane?

  • Dont be forced to groupthink or fit in.
  • Maintain financial autonomy.
    • People dont realise how much of their epistemics is affected when their career or livelihood is at stake.
  • Speak up if you notice grifters or status seekers.
    • The culture of institutions changes through distillation.
  • Look at actions over words.
    • EAs are good at writing and coming up with arguments, so its easy to get yourself confused.
    • When in doubt, take a step back, forget all the reasoning, and look at the actions.
    • Given how much consequentialist reasoning there is in EA, I find this useful for avoiding motivated reasoning.
  • Take your own time to question assumptions. Dont take everything you read for granted.
  • In the earlier EA spaces I encountered, near term causes such as global health and animal advocacy were more prominent.
    • AI safety was one of the cause areas. Routes included an ML PhD or working on agent foundations at MIRI.
    • AI safety felt like a fringe idea back then, but there was ongoing work on alignment.
    • Iterated amplification and debate were being developed before ChatGPT. ELK came in 2021. RLHF was another strand of work.
  • As recently as 2022, before ChatGPT, longtermism was a major focus of EA discussion.
  • Open Philanthropy has funded academics working on AI safety since well before the 2020s. Its 2017 grants already included academic AI safety research.
  • MIRI’s history goes back to 2000, with a shift towards alignment around 2003. FLI was founded in 2014.
  • Wealthy donors have funded large parts of this ecosystem.
    • Dustin Moskovitz, a Facebook cofounder, and Cari Tuna established Good Ventures. It became the main philanthropic partner behind what is now Coefficient Giving.
    • Jaan Tallinn, a Skype cofounder, has supported FLI, FHI, and MIRI. FLI’s 2015 conference biography describes this funding.

Related: How do we understand the current AI safety landscape?.