• State-space models can learn in-context by gradient descent https://arxiv.org/abs/2410.11687 58 comments 26/10/2024

• Supervised Pretraining Can Learn In-Context Reinforcement Learning https://arxiv.org/abs/2306.14892 5 comments 28/6/2023 reinforcementlearning
• Against LLM maximalism: Why in-context learning won't replace the need to train task-specific models https://explosion.ai/blog/against-llm-maximalism 8 comments 19/5/2023 languagetechnology
• llm_memory: A Ruby Gem for LLMs like ChatGPT to have memory using in-context learning https://github.com/shohey1226/llm_memory 2 comments 16/5/2023 ruby
• [R] General-Purpose In-Context Learning by Meta-Learning Transformers https://arxiv.org/abs/2212.04458 3 comments 9/12/2022 machinelearning
• [R] In-context Reinforcement Learning with Algorithm Distillation https://arxiv.org/abs/2210.14215 7 comments 26/10/2022 machinelearning
• [R] State-space models can learn in-context by gradient descent https://arxiv.org/abs/2410.11687 4 comments 7/11/2024 machinelearning
• Transformers learn in-context by gradient descent [R] https://arxiv.org/pdf/2212.07677 5 comments 22/8/2024 machinelearning
• [R] ICLERB: A better way to evaluate embeddings and rerankers for in-context learning https://arxiv.org/pdf/2411.18947 10 comments 4/12/2024 machinelearning
• LLM In-Context Learning (with a few enhancements) seems to outperform SetFit for Text Classification with limited labelled data availability https://medium.com/@sumanthprabhu.104/self-training-llms-for-text-classification-using-dqc-toolkit-d1d63fc5e97c 3 comments 10/6/2024 datascience
• [2402.00795] LLMs learn governing principles of dynamical systems, revealing an in-context neural scaling law https://browse.arxiv.org/abs/2402.00795 4 comments 2/2/2024 machinelearning
• [R] Transformers Learn Higher-Order Optimization Methods for In-Context Learning: A Study with Linear Models https://arxiv.org/abs/2310.17086 15 comments 2/11/2023 machinelearning
• "Transformers Learn Higher-Order Optimization Methods for In-Context Learning: A Study with Linear Models", Fu et al 2023 (self-attention learns higher-order gradient descent) https://arxiv.org/abs/2310.17086 16 comments 3/11/2023 reinforcementlearning
• [R] Symbol tuning ( i.e finetuning on input-label pairs where natural language labels (e.g., "positive/negative sentiment") are replaced with arbitrary symbols (e.g., "foo/bar") ) improves in-context learning in language models, with much stronger results for algorithmic reasoning benchmarks. https://arxiv.org/abs/2305.08298 2 comments 17/5/2023 machinelearning