AI / ML / LLM / Transformer Models Timeline Details

Viktor Garske @vemgar, Last update: Tue Dec 26 15:23:35 2023
← Back to the full graph

Attention / Transformers

This graph is clickable!

timeline 06/2017 06/2017 06/2018 06/2018 06/2017->06/2018 10/2018 10/2018 06/2018->10/2018 04/2019 04/2019 10/2018->04/2019 09/2019 09/2019 04/2019->09/2019 10/2019 10/2019 09/2019->10/2019 04/2020 04/2020 10/2019->04/2020 05/2020 05/2020 04/2020->05/2020 06/2020 06/2020 05/2020->06/2020 05/2021 05/2021 06/2020->05/2021 04/2022 04/2022 05/2021->04/2022 05/2022 05/2022 04/2022->05/2022 09/2022 09/2022 05/2022->09/2022 11/2022 11/2022 09/2022->11/2022 02/2023 02/2023 11/2022->02/2023 05/2023 05/2023 02/2023->05/2023 09/2023 09/2023 05/2023->09/2023 12/2023 12/2023 09/2023->12/2023 MegatronLm Megatron-LM Gpt GPT Llama LLaMA T5 T5 Ul2 UL2 Lamda LaMDA Whisper Whisper Bloom BLOOM Palm PaLM Palm->Llama Bert BERT Mpt7bBase MPT-7B Base Falcon Falcon Mistral7b Mistral 7B Gemini Gemini Phi15 Phi-1.5 Attention Attention / Transformers Attention->MegatronLm Attention->Gpt Attention->Llama Attention->T5 Attention->Ul2 Attention->Lamda Attention->Whisper Attention->Bloom Attention->Palm Attention->Bert Attention->Mpt7bBase Attention->Falcon Attention->Mistral7b Attention->Gemini Attention->Phi15 DenoisingDiffusion Denoising Diffusion Attention->DenoisingDiffusion Rwkv RWKV Attention->Rwkv Longformer Longformer Attention->Longformer LongSeqSparseTransformers LSST Attention->LongSeqSparseTransformers Rag RAG Attention->Rag Longformer->Mistral7b LongSeqSparseTransformers->Mistral7b
Type
Architecture, Method
Paper name
Attention Is All You Need
Paper authors
Vaswani et al.
Paper link
https://arxiv.org/abs/1706.03762
Publish date
2017-06-12
Affiliation
Google