Transformers Revolutionized AI. What Will Replace Them?

Linking pages

AI That Can Invent AI Is Coming. Buckle Up. https://www.forbes.com/sites/robtoews/2024/11/03/ai-that-can-invent-ai-is-coming-buckle-up/ 45 comments

Linked pages

Anthropic | Introducing 100K Context Windows https://www.anthropic.com/index/100k-context-windows 474 comments
[1706.03762] Attention Is All You Need https://arxiv.org/abs/1706.03762 145 comments
Liquid Neural Networks, A New Idea That Allows AI To Learn Even After Training - YouTube https://www.youtube.com/watch?v=0FNkrjVIcuk 71 comments
What is RT-2? Google DeepMind’s vision-language-action model for robotics https://blog.google/technology/ai/google-deepmind-rt2-robotics-vla-model/ 69 comments
Forbes https://www.forbes.com/#79b70b222254 19 comments
[2209.10655] Mega: Moving Average Equipped Gated Attention https://arxiv.org/abs/2209.10655 14 comments
What is a liquid neural network, really? | TechCrunch https://techcrunch.com/2023/08/17/what-is-a-liquid-neural-network-really/ 10 comments
[2111.00396] Efficiently Modeling Long Sequences with Structured State Spaces https://arxiv.org/abs/2111.00396 8 comments
[2302.10866] Hyena Hierarchy: Towards Larger Convolutional Language Models https://arxiv.org/abs/2302.10866 3 comments
AI Startup Cohere Raises $270 Million in Round Backed by Oracle, Nvidia - Bloomberg https://www.bloomberg.com/news/articles/2023-06-08/ai-startup-cohere-raises-270-million-in-round-backed-by-oracle-nvidia 1 comment
Ex-Google, Stability AI Researchers Launch AI Startup That Takes Cues From Fish - Bloomberg https://www.bloomberg.com/news/articles/2023-08-17/ex-google-stability-ai-researchers-launch-ai-startup-that-takes-cues-from-fish 1 comment
The A.I. Industry’s Desperate Hunt for GPUs Amid a Chip Shortage - The New York Times https://www.nytimes.com/2023/08/16/technology/ai-gpu-chips-shortage.html 1 comment
[2007.14062] Big Bird: Transformers for Longer Sequences https://arxiv.org/abs/2007.14062 0 comments
[2001.04451] Reformer: The Efficient Transformer https://arxiv.org/abs/2001.04451 0 comments
[2006.04768] Linformer: Self-Attention with Linear Complexity https://arxiv.org/abs/2006.04768 0 comments
[1409.0473] Neural Machine Translation by Jointly Learning to Align and Translate http://arxiv.org/abs/1409.0473 0 comments
Microsoft Makes Multibillion Dollar Investment in OpenAI - Bloomberg https://www.bloomberg.com/news/articles/2023-01-23/microsoft-makes-multibillion-dollar-investment-in-openai 0 comments
Adept, a startup training AI to use existing software and APIs, raises $350M | TechCrunch https://techcrunch.com/2023/03/15/adept-a-startup-training-ai-to-use-existing-software-and-apis-raises-350m/ 0 comments
Top ex-Google AI researchers raise funding from Thrive Capital | Reuters https://www.reuters.com/technology/top-ex-google-ai-researchers-raise-funding-thrive-capital-sources-2023-05-04/ 0 comments
[2212.10544] Pretraining Without Attention https://arxiv.org/abs/2212.10544 0 comments