Attention Is All You Need
2017 research paper by Google
"Attention Is All You Need" is a 2017 research paper by Google.[1] Authored by eight scientists, it was responsible for expanding 2014 attention mechanisms proposed by Bahdanau et. al. into a new deep learning architecture known as the transformer. The paper is considered by some to be a founding document for modern artificial intelligence, as transformers became the main architecture of large language models.[2][3] At the time, the focus of the research was on improving Seq2seq techniques for machine translation, but even in their paper the authors saw the potential for other tasks like question answering and for what is now called multimodal Generative AI.
The paper's title is a reference to the song "All You Need Is Love" by the Beatles.[4]
As of 2024,[update] the paper was cited more than 100,000 times. By 2023, all eight authors had left Google.[5]
References
- ^ Vaswani, Ashish; Shazeer, Noam; Parmar, Niki; Uszkoreit, Jakob; Jones, Llion; Gomez, Aidan N; Kaiser, Łukasz; Polosukhin, Illia (2017). "Attention is All you Need" (PDF). Advances in Neural Information Processing Systems. 30. Curran Associates, Inc.
- ^ Toews, Rob (3 September 2023). "Transformers Revolutionized AI. What Will Replace Them?". Forbes. Archived from the original on 26 September 2023. Retrieved 3 December 2023.
- ^ Murgia, Madhumita (23 July 2023). "Transformers: the Google scientists who pioneered an AI revolution". Financial Times. Archived from the original on 28 December 2023. Retrieved 22 March 2024.
- ^ Levy, Steven. "8 Google Employees Invented Modern AI. Here's the Inside Story". Wired. ISSN 1059-1028. Retrieved 20 March 2024.
- ^ "Meet the $4 Billion AI Superstars That Google Lost". Bloomberg. 13 July 2023 – via www.bloomberg.com.
- v
- t
- e
Google AI
- Google Brain
- Google DeepMind
| AlphaGo |
| ||||||
|---|---|---|---|---|---|---|---|
| Other |
|
| Neural networks |
|
|---|---|
| Other |
| Chatbots |
|
|---|---|
| Language models |
- "Attention Is All You Need"
- Future of Go Summit
- Generative pre-trained transformer
- Google Labs
- Google Pixel
Category
Commons
 | This Google-related article is a stub. You can help Wikipedia by expanding it. |
- v
- t
- e
