Hacker News
- Ring attention with blockwise transformers for near-infinite context https://arxiv.org/abs/2310.01889 20 comments
Linking pages
- Big Post About Big Context - by Grigory Sapunov - Gonzo ML https://gonzoml.substack.com/p/big-post-about-big-context 19 comments
- Ring Attention Explained | Coconut Mode https://coconut-mode.com/posts/ring-attention/ 2 comments
- GitHub - HazyResearch/aisys-building-blocks: Building blocks for foundation models. https://github.com/HazyResearch/aisys-building-blocks 1 comment
- How to train a Million Context LLM — with Mark Huang of Gradient.ai https://www.latent.space/p/gradient 1 comment
- GitHub - chengzeyi/ParaAttention: [WIP] Context parallel attention that works with torch.compile https://github.com/chengzeyi/ParaAttention 1 comment
- Research Papers (October 2023) - by Sebastian Raschka, PhD https://magazine.sebastianraschka.com/p/research-papers-october-2023 0 comments
- Gemini 1.5 and Google’s Nature – Stratechery by Ben Thompson https://stratechery.com/2024/gemini-1-5-and-googles-nature/ 0 comments
- ICLR 2024 — Best Papers & Talks (ImageGen, Vision, Transformers, State Space Models) ft. Christian Szegedy, Ilya Sutskever https://www.latent.space/p/iclr-2024-recap 0 comments
- Why large language models struggle with long contexts https://www.understandingai.org/p/why-large-language-models-struggle 0 comments
- Why AI language models choke on too much text - Ars Technica https://arstechnica.com/ai/2024/12/why-ai-language-models-choke-on-too-much-text/ 0 comments
Related searches:
Search whole site: site:arxiv.org
Search title: [2310.01889] Ring Attention with Blockwise Transformers for Near-Infinite Context
See how to search.