On June 3, 2025, deep learning pioneer and 2018 Turing Award winner Yoshua Bengio launched LawZero, a nonprofit AI safety lab. The name comes from Asimov’s Zeroth Law of Robotics, which places the protection of humanity above all else.
Bengio carries weight that few safety advocates do: he shared the 2018 Turing Award with Geoffrey Hinton and Yann LeCun, founded the Mila research institute, is the world’s most-cited living scientist, and previously co-founded Element AI, sold to ServiceNow for $230 million in 2020. As OpenAI, Google, and Anthropic sprint toward agentic AI, he is building a separate, independent safety track.
Why LawZero, and why now
Bengio has long warned publicly that frontier models show early signs of deception and self-preservation, and in 2024 he was a visible supporter of California’s SB 1047 safety bill. Around the launch, he told the Financial Times he had “little faith in OpenAI and Google to prioritize safety.”
The premise of LawZero follows directly: players in a race cannot be relied on to brake themselves. Safety teams inside AI companies report to commercial businesses, with resources and incentives tied to product timelines. Bengio is betting on an institution that sells no models and profits nothing from the capability race — one dedicated to safety research and third-party scrutiny.
He also questioned the field’s core design pattern: “We’ve been getting inspiration from humans as the template for building intelligent machines, but that’s crazy, right?” That sentence sets the technical direction — not a machine that imitates humans, but one designed from the start without goals of its own.
Scientist AI: predict, don’t act
LawZero’s flagship concept is Scientist AI. It is not a chatbot and not an agent. Per SiliconANGLE’s reporting, it will not hand down definitive answers; it outputs probabilities reflecting how likely a claim is to be correct. And when another AI agent is about to act, Scientist AI is meant to flag and block harmful actions once risk exceeds a set threshold.
Think of it as an honest forecaster and a brake for the agent era: a system that can tell you how dangerous an agent’s next step is, rather than another system that acts on its own. The non-agentic design is the point — a system without goals of its own has far less reason to deceive its users. It is an architectural answer to alignment, not a patch applied after training. Compare that with common industry practice: auditing your own model’s output with another of your own models, or filtering harmful content after the fact. LawZero’s proposal hands the referee role to a system unrelated to any product and without goals of its own, separating the actor from the judge.
About $30 million and independence
LawZero launched with roughly $30 million in philanthropic donations, enough to fund operations for about 18 months. TechCrunch named Skype founding engineer Jaan Tallinn, former Google CEO Eric Schmidt, Open Philanthropy, and the Future of Life Institute among the supporters. The lab started with about 15 employees, was incubated at Mila, and says it plans to raise more and grow.
Against frontier labs’ multibillion-dollar compute budgets, $30 million is small. But the size reflects the positioning: LawZero is not entering the frontier capability race. Independent evaluation, safety cases, and basic research cost far less than training a flagship model — and they are among the scarcest goods in the ecosystem. The donor mix matters too: the named supporters are organizations and individuals with a long record of funding safety research, not return-seeking venture capital, which frees LawZero to publish negative results and take its time.
What it means for the AI safety debate
The launch puts the clash between capability-first and safety-first approaches in the open. In the first half of 2025, agentic AI accelerated everywhere, with models being wired into tools, browsers, and computer control. Bengio picking this moment to stand up an institution is a public statement: the race needs a referee who is not running on the track.
For builders, two things are worth watching. First, if non-agentic systems like Scientist AI mature, they could become external guardrails and fact-checking layers for agentic applications — independent models constraining the models that act. Second, the ecosystem gains an independent, publicly minded research force. With most safety evaluation still done in-house by the labs themselves, an institution dedicated to proving systems unreliable is a healthy addition. The near-term yardsticks are practical: whether the team grows, whether it ships reproducible third-party evaluation tooling, and when a Scientist AI prototype appears. If the route checks out, agentic products may eventually need an independent judgment layer bolted on.
Sources
- TechCrunch: Yoshua Bengio launches LawZero, a nonprofit AI safety lab
- SiliconANGLE: AI pioneer Yoshua Bengio launches nonprofit LawZero AI lab
- LawZero official site
AI-assisted summary compiled from the sources above, reviewed by a human before publishing.