Don’t let Big Tech write the AI rules, warns AI godfather

• An AI godfather says we should all be worried about the concentration of power in the AI sector.
• Yoshua Bengio said the potential issue was the “number two problem” after existential risks.
• Bengio said the control of powerful AI systems was a central question for democracy.

The concentration of power in the AI arena is one of the main risks facing the industry, an AI godfather says.

Yoshua Bengio, a Turing Award-winning Canadian computer scientist, told Insider the potential for the AI sector to be controlled by a few companies was the “number two problem” behind the existential risks posed by the technology.

“We are on a trajectory to build more and more powerful systems and the central question for democracy is who controls these systems – is there a risk of excessive concentration of power,” he told Insider in an interview.

“These are important questions I’ve been thinking about for years but the arrival of ChatGPT and these systems made me worry even more about the concentration of power,” Bengio added.

Addressing concerns raised by fellow AI pioneer Yann LeCun, Bengio said he disagreed with LeCun’s suggestion that prominent tech leaders including OpenAI’s Sam Altman were trying to control the sector by pushing for stronger regulation.

Last week, LeCun accused AI founders such as DeepMind’s Demis Hassabis and Anthropic’s Dario Amodei of attempting to perform a “regulatory capture of the AI industry.”

“We should avoid the situation where the rules are written by the companies – that’s obvious,” Bengio said. “But this whole discussion about regulatory capture I think it’s just not a fact.”

Regulation, at least in its current form, will not be the boost for big tech companies that some industry experts have suggested it could be, he added.

“All the various regulations that are being discussed are generally with the idea that the oversight regulatory burden will be larger for these bigger systems that those large companies build,” Benigo said.

“So in other words it’s actually going to put more pressure and cost more,” he added. “But the small guys that are either doing narrow AI, or just designing applications on top of the biggest systems, will not need to be under the same kind of scrutiny.”