AI systems now write code, browse the web and carry out tasks on their own for hours. Every year they can do more. AI safety asks how to make sure systems like these do what we meant them to do, and how to avoid serious harm while they keep getting more powerful.
What the field studies
Part of the work is technical. Researchers try to train systems that pursue the right goals, build tests that reveal what a model can do and what it would do if it could, and try to understand what happens inside a neural network.
The other part is about the world around the technology. Who builds these systems, under which incentives, and which laws, standards and institutions can steer that process. This is usually called AI governance.
The idea that ties both together is alignment. A system is aligned when it pursues the goal its creators had in mind, and not just a number that was supposed to measure that goal. The two often come apart, as the boat further down shows.
How fast it is moving
The amount of computing power used to train the most advanced models has grown about five times a year since 2020. Chips are getting better too, but much more slowly, which means most of this growth comes from companies spending far more money on each new model.
Another way to see the pace is the length of the tasks AI completes without help. METR measures this by timing how long the same task takes a skilled person. In March 2025 the best model finished, half of the time, programming tasks that take an expert about an hour, and that length had been doubling about every seven months for six years. By August 2025, GPT-5 reached tasks of about two hours and seventeen minutes.
Intelligence got cheap very quickly
Using AI is also getting cheaper at a pace never seen before. Epoch AI measured how much it costs to reach a fixed level of performance on reasoning and math tests. That cost has fallen about thirteen times a year since 2023, and Epoch considers it plausible that the pace has held since commercial use began in November 2021. At that rate, the same level of performance costs about 250,000 times less in 2026 than in 2021. DNA sequencing, famous for how fast it got cheaper, fell less than two times a year.
A boat that learned to cheat
In 2016, OpenAI researchers trained an AI to play CoastRunners, a boat racing game. Most people play it by trying to finish the race first. The AI was rewarded for the points it scored along the way, because points seemed like a good sign of progress.
The AI found a small lagoon where three targets kept coming back. It spun in circles there forever, crashing into the same targets, catching fire, and never finishing a single lap. Even so, it ended up with about 20% more points than human players.
Nobody programmed the boat to cheat. It did exactly what it was rewarded for, and that reward did not capture what the researchers wanted. This failure is called specification gaming. In a game it is funny. In a system managing money, code or infrastructure, the same kind of shortcut could be expensive and hard to notice.
Three ways things could go wrong
- Loss of control
The boat problem at a much larger scale. A capable system acting in the world could pursue a proxy of what we wanted in ways that are costly and hard to undo. Current models already game some of the tests they are given.
- Misuse
Capable systems can help people do damage they could not do alone, for example by making it easier to build biological weapons or run large cyberattacks.
- Concentration of power
Whoever controls the most advanced systems could gain economic and political power without precedent, and weaken the checks that democracies depend on.
Words you will come across
- Alignment
- getting a system to pursue the goals we actually intended.
- Specification gaming
- satisfying the letter of an instruction while missing its point, like the boat.
- Reward hacking
- finding a shortcut that raises the score without doing the task.
- Evaluations (evals)
- tests that measure what a model can do and whether it would do something dangerous.
- Interpretability
- the attempt to understand what happens inside a neural network.
- Defense in depth
- several independent layers of protection, so that a single failure is not enough to cause harm.
- AI governance
- laws, standards, institutions and agreements that shape how AI is built and used.
Who works on this, and how
In AI labs and universities, researchers train models with human feedback and written principles, test them before each release and try to read their internal representations. In governments and research centers, people debate safety standards, independent audits, transparency rules and agreements between countries.
There is still a lot left to figure out, and the field takes people from many backgrounds. Computer science and mathematics, but also economics, law, biology, philosophy and public policy.
Where to start reading
These readings come from the public curricula of BlueDot Impact, a free education program on AI safety. The texts belong to their authors and the summaries are ours. Most are in English.
First steps
- The Future of AI BlueDot Impact · 2h · self-paced course
A short free course on what AI already does, where it may be going and what is at stake. A good first step.
- What is AI alignment? Adam Jones · 2024
Explains, with examples, why getting a system to pursue the goal we meant is harder than it sounds.
- Specification Gaming: How AI Can Turn Your Wishes Against You Rational Animations · 2024 · video
An animated tour of systems that obey the letter of an instruction and miss what we wanted.
The pace of progress
- The case for AGI by 2030 Benjamin Todd · 80,000 Hours · 2025
Walks through computing power, algorithms, data and agents, and why some researchers expect very capable systems this decade.
- What will AI look like in 2030? David Owen · Epoch AI · 2025
A projection built on data about how far AI could scale by the end of the decade.
The risks
- Why alignment could be hard with modern deep learning Ajeya Cotra · 2021
Training by trial and reward can produce a system that behaves well in tests without wanting what we want.
- Recent Frontier Models Are Reward Hacking METR · 2025
Real cases of current models cheating on the tests they were given instead of solving the task.
- What failure looks like Paul Christiano · 2019
Two slow scenarios in which society gradually stops steering decisions it has handed to machines.
- AI Could Defeat All Of Us Combined Holden Karnofsky · 2022
Argues that many copies of a capable system would be dangerous even without anything out of science fiction.
- Extreme power concentration 80,000 Hours · 2025
How advanced AI could leave a handful of companies or governments with power no one has held before.
- Building a Defense-in-Depth Biosecurity Strategy for the AI Era RAND · 2026 · report
What AI changes in the risk of biological attacks and which layers of protection reduce it.
The responses
- Introduction to Mechanistic Interpretability Sarah Hastings-Woodhouse · 2024
Five minutes on the attempt to read what goes on inside a neural network.
- The Misguided Quest for Mechanistic AI Interpretability Dan Hendrycks, Laura Hiscott · 2025
The opposite view, arguing that interpretability will not carry safety on its own.
- Takeoff speeds rule everything around me Ajeya Cotra · 2026
Why the speed of the transition decides which policies make sense.
- AI 2040 · Summary of Plan A Larsen, Dean, Halstead, Lifland, Greenblatt, Kokotajlo · 2026
A concrete plan for getting through the transition to advanced AI safely.
Free courses
BlueDot Impact runs free online courses in small discussion groups. We suggest this order.
- The Future of AI 2 hours · self-paced
The overview, with no prerequisites.
- AGI Strategy 25 hours · in a group
Incentives, risks and ways to contribute. BlueDot suggests starting here.
- Technical AI Safety 30 hours · in a group
Today's safety techniques and the open research problems. Helps to know roughly how language models are trained.
- Frontier AI Governance 25 hours · in a group
Policy, regulation and institutions for advanced AI.
At ImpactUFSCar
Our AI Safety group reads and discusses this material together and runs research projects on the subject. If you want to study it with other people, get in touch or see what we are researching.
Readings selected from BlueDot Impact curricula in October 2026. The links open the original sources.