- Google has contributed some of the most important research to the field of AI.
- Still, the company has been slow to turn some of its biggest innovations into actual products.
- Top AI researchers have been leaving for startups where their work can have more impact.
Editor's note: This story was originally published on March 23, 2023 and most recently updated on July 13, 2023 to reflect recent developments.
Google has found itself in a defensive position as it faces questions over whether it's lost its lead in AI, especially as top researchers leave for competitors.
Now, many of the major contributors to Google's most seminal AI papers have left to join competitors like OpenAI or start their own firms. In June, Llion Jones, a researcher who helped write a foundational AI paper for Google, said he was leaving the company, Bloomberg first reported. He was the last researcher on the landmark paper, titled "Attention is All You Need," who remained at Google.
That paper was about Transformers, core to the technology behind OpenAI's ChatGPT. ChatGPT hit the scene in late 2022 and took the internet by storm with its ability to respond to questions on just about any topic with complex, humanlike responses. Some managers at Google pulled a "code red" internally to rally the company around a response and allay fears that its search business could be at threat, The New York Times reported.
While Google may be in defense mode, it didn't have to be that way. The company itself created much of the foundational technology that powers products like ChatGPT. It also made its research available as open source, which somewhat ironically enabled OpenAI's rapid ascent.
CEO Sundar Pichai has said that Google continues to hire top researchers in the AI field, and there's no doubt the company still retains much strong talent.
Now Google is moving at breakneck speed to inject all its key products with generative AI. It continues to expand the availability of Bard, a chatbot similar to ChatGPT that can respond to some search queries. It also plans to infuse its productivity software with generative AI, the technology that allows Bard and ChatGPT to understand and generate new text. Microsoft meanwhile has invested billions into OpenAI and has also been launching generative AI products.
Google long hesitated to release a chatbot similar to ChatGPT over concerns that the technology could cause reputational harm to its business. Daniel De Freitas and Noam Shazeer, two researchers behind Google's large language model LaMDA, left the company, frustrated by its hesitation to release a chatbot like ChatGPT, The Wall Street Journal reported.
That frustration over Google's slow movement has been corroborated by other former Google researchers who spoke to Insider. One former employee said that at such an exciting time for AI, startups offer the opportunity for researchers to have more ownership over their work and possibly make a greater impact.
Below are some of the most notable papers in the field of AI, the papers' researchers who have departed Google, and where they are now.
Sequence-to-Sequence Learning with Neural Networks
Published in 2014, the sequence-to-sequence paper explored training language models to convert word sequences from one domain to sequences in another — for instance, converting an English sentence to one in French.
Ilya Sutskever led the research on this paper. He left Google in 2015 after serving for almost three years as a research scientist. Sutskever cofounded OpenAI and continues to work there as its chief scientist.
Attention Is All You Need
"Attention Is All You Need" is about Transformers. Considered a breakthrough in natural-language processing, the Transformer model helps an AI understand meaning by looking at each word in a sentence simultaneously and weighing the importance of each word to glean contextual nuance. It laid the foundation for today's chatbots: the "T" in ChatGPT stands for Transformer, as the chatbot builds on Google's work. All the authors of this paper have left Google.
Ashish Vaswani left Google Brain after five years to start Adept, which recently raised $350 million to build generative AI tools that help people use productivity software more efficiently. He departed Adept in November to cofound a stealth startup.
Noam Shazeer is now the CEO of Character.AI after spending most of his 21+ year career as an engineer Google.
Niki Parmar left Google Brain after five years to serve as a cofounder and CTO of Adept, though in November, she left to found a stealth startup.
Jakob Uszkoreit spent 13 years at Google working on neutral networks and deep learning. He is now a cofounder of Inceptive, a startup using deep learning to design new therapeutics.
Aidan Gomez is the cofounder and CEO of Cohere, which has raised approximately $270 million to help developers incorporate generative AI into their apps and websites. He spent a year and a half as a researcher at Google Brain. His cofounder at Cohere, Nick Frosst, spent four years as a researcher at Google Brain.
Lukasz Kaiser left Google Brain after working there for more than seven years to join OpenAI in 2021. Kaiser was recently cited in OpenAI's whitepaper for GPT-4 as a core contributor to its long-context ability, allowing the chatbot to hold longer conversations before forgetting the context of the discussion.
Illia Polosukhin spent three years working on deep learning and natural-language understanding at Google. He left in 2017 to start NEAR, a Web3 system.
Llion Jones joined Google in 2015 and has worked on machine intelligence and natural language understanding. Previously, he also spent three and a half years at YouTube. He announced this month he would leave Google and plans to start his own company, Bloomberg reported. His departure is the last of the authors of the "Attention Is All You Need" paper.
Toward a Humanlike Open-Domain Chatbot
This paper introduces Meena, Google's initial chatbot. While "Attention Is All You Need" laid the foundation for a language learning model that can understand writing, this paper introduces dialogue. It explores how a chatbot can learn to talk conversationally about any topic by studying data scraped from public social-media conversations. It also introduces a test created by Google to grade how well the chatbot talks like a person. The paper was considered another major milestone in large-language modeling, with the authors arguing they could make a large-language model that could generate a humanlike response to any question without hard-coded training.
Daniel De Freitas serves the cofounder and president of Character.AI after spending five years as a researcher at Google Brain.
De Freitas' colleague at Character.AI, Romal Thoppilan, also contributed to this paper. He joined Character.AI as a founding researcher after spending seven years at Google.
LaMDA: Language Models for Dialogue Applications
LaMDA stands for Language Models for Dialogue Applications. It powers Bard, the chatbot that Google plans to incorporate into search. First demoed in 2020 as Meena, Google never released the chatbot to the public (LaMDA is more powerful than Meena's model). Former employees of Google's AI research division told Insider that the search giant feared a public-relations nightmare if the bot made any comments that could be harmful.
Several of the lead researchers behind LaMDA have decamped for Character.AI. Daniel De Freitas and Noam Shazeer formed the company last year and has raised roughly $200 million to create chatbots that speak in the form of various characters, ranging from Elon Musk to a therapist or life coach.
Romal Thoppilan also contributed to this paper.
Alicia Jin joined Character.AI in 2022 as a research engineer. She spent nearly four years at Google.
BERT
BERT (Bidirectional Encoder Representations from Transformers) is built on the Transformer model for natural-language processing and pretrained to accomplish two tasks well: masked-language modeling and next-sentence prediction. In other words, BERT tries to predict hidden or "masked" words, forcing the algorithm to try and learn more about the surrounding text and better predict hidden words.
Google provides the example that if you type in a query such as "Can you get medicine for someone else pharmacy," it will better understand that "someone else" is an important part of the query.
Google began incorporating BERT into Search back in 2019. It's one of the biggest advancements in search accuracy since another machine-learning algorithm, RankBrain, was incorporated in 2015.
Jacob Devlin led this paper. He joined OpenAI shortly before the launch of ChatGPT, the Information reported, but recently returned to Google in a surprise boomerang.
T5
The T5 paper, formally known as "Exploring the Limits of Transfer Learning With a Unified Text-to-Text Transformer," builds on BERT and is considered well-suited for tasks like translation and summarizing.
Colin Raffel, who led this paper, spent roughly five years as a research scientist at Google Brain before leaving in 2021. Raffel is currently an assistant professor at UNC Chapel Hill and spends one day a week as a researcher at Hugging Face, where users can share large language models and datasets. Hugging Face announced it raised $100 million in May 2022, valuing the company at $2 billion.
Sharan Narang, another contributor to the T5 paper, left Google Brain in 2022 after four years there. He's now an AI researcher at Meta.
A graph placement methodology for fast chip design
Led by Google scientists Azalia Mirhoseini and Anna Goldie, the paper "A graph placement methodology for fast chip design" finds that AI can complete the design process for chips faster than a human expert. The two led another paper, "Chip placement with deep reinforcement learning," that offers a method for using AI in chip design to maximize performance while minimizing area and power usage.
The findings helped Google design TPU chips, specifically designed for machine-learning tasks.
Both Mirhoseini and Goldie left Google in 2022 to join Anthropic, a competitor to OpenAI developing its own large language models and a chatbot called Claude. The pair were at the center of a controversy within Google after a senior engineering manager was fired for trying to discredit their work on the two papers. Google has continued to defend the research amid an ongoing lawsuit.
DeepMind
Mustafa Suleyman cofounded and served as the chief product officer of DeepMind, an artificial-intelligence lab that was acquired by Google in 2014 and became one of the company's "other bets" (an assortment of businesses that are owned by Alphabet but function independently from Google). The lab developed AlphaGo, a machine-learning program that beat a world-champion professional at the strategy game Go.
Alphabet, the parent company of Google, announced in April that DeepMind would be combined with its Brain AI unit, which has been renamed Google DeepMind.
Suleyman has been a vocal proponent of ensuring safety in new AI products. During his time at DeepMind, Suleyman set up a research unit called DeepMind Ethics & Society to study the real-world impacts of AI. He was placed on leave from DeepMind in 2019 over allegations that he bullied employees. He then joined Google as a vice president.
Suleyman is cited in numerous research papers pertaining to machine learning. In February 2022, he cofounded the AI startup Inflection with the creator of LinkedIn, Reid Hoffman. The company has since launched a chatbot, named Pi.
Got a tip about Google? You can reach Thomas via email at tmaxwell@businessinsider.com, encrypted messaging app Signal at 540.955.7134, or Twitter at @tomaxwell. You can reach Hugh via encrypted email (hlangley@protonmail.com) or encrypted messaging apps Signal/Telegram (+1 628-228-1836).