Resources
Key Ideas
MIRI's introduction to why smarter-than-human AI could be an existential risk.
The Center for AI Safety's overview of the major pathways by which AI could cause catastrophe.
Leopold Aschenbrenner's essay series on the trendlines pointing toward AGI this decade and their national security implications.
A detailed scenario forecast of how superhuman AI could arrive within a few years, and what happens if it does.
Robert Miles on why almost any goal produces the same dangerous sub-goals: self-preservation, resource acquisition, and resisting shutdown.
Holden Karnofsky on how systems trained with today's methods could end up with goals that put them at odds with people.
AI Safety in the news
Careers
Some curated links we think students and researchers will find most helpful for finding career and funding opportunities in AI safety. To schedule a consultation with us, please email us your CV/resume and clearly describe what types of opportunities you are searching for.
Comprehensive resources
A live directory of AI safety programs, fellowships, and openings.
The most complete directory of AI safety jobs, programs, communities, and funding.
Curated job board filtered to AI safety and policy roles.
Career guidance for people entering US technology policy.
Encode AI's guide to what advanced AI means for early careers, and how to prepare for it.
Resource center for university AI safety groups and their members.
Free facilitated courses on AI safety, alignment, and governance, plus grants for career transitions and in-person programs and placement support that move talent into the field.
Getting started in...
Bootcamps and part-time programs for testing your fit with policy work.
Government, think tank, and congressional internships in technology policy.
Longer-term fellowships placing technical people into policy roles.
Charbel-Raphael Segerie argues AI safety is limited less by unsolved research than by the political will to act on what is already known.
An intermediate course on AI verification: the technical, institutional, and legal mechanisms that make international AI agreements mutually trustable and enforceable.
Highest capacity, most entry-level friendly, runs triannually. Typical outcome is a research blogpost or workshop paper. Apply to your top 2-3 projects. AISI helped get this off the ground and it is currently run by our friends at Kairos.
Lower capacity, but has the best mentors (e.g., Anthropic, OpenAI, DeepMind). May require research and engineering experience.
Full-time summer program, low capacity. Good fit for students who can execute on their own projects and ideas.
London-based full-time research programme running small teams toward a published paper over roughly three months.
Full-time summer fellowship in London supporting independent technical and governance research.
Constellation's full-time fellowship placing researchers alongside mentors from frontier labs and safety organisations.
A catalogue of unclaimed, high-leverage projects for people willing to go solve tasks nobody's job description covers.
A three-month summer residency from Constellation and Kairos placing 15-30 generalists inside core AI safety organisations.
A write-up of what the first Generator Residency cohort actually worked on in Berkeley.
Funding
Major grantmaker for work on risks from advanced AI, and a supporter of AISI.
Grants for career transitions, projects, and work that strengthens AI safety and biosecurity.
Proposal requirements, selection criteria, and timeline for Thinking Machines Lab's safety research grant program.