Hacker News
- Is In-Context Learning Sufficient for Instruction Following in LLMs? https://arxiv.org/abs/2405.19874 1 comment
- "MLPs Learn In-Context", Tong & Pehlevan 2024 (& MLP phase transition in distributional meta-learning) https://arxiv.org/abs/2405.15618 0 comments reinforcementlearning
- Supervised Pretraining Can Learn In-Context Reinforcement Learning https://arxiv.org/abs/2306.14892 5 comments reinforcementlearning
- Against LLM maximalism: Why in-context learning won't replace the need to train task-specific models https://explosion.ai/blog/against-llm-maximalism 8 comments languagetechnology
- llm_memory: A Ruby Gem for LLMs like ChatGPT to have memory using in-context learning https://github.com/shohey1226/llm_memory 2 comments ruby
- [R] General-Purpose In-Context Learning by Meta-Learning Transformers https://arxiv.org/abs/2212.04458 3 comments machinelearning
- [R] In-context Reinforcement Learning with Algorithm Distillation https://arxiv.org/abs/2210.14215 7 comments machinelearning
- [2402.00795] LLMs learn governing principles of dynamical systems, revealing an in-context neural scaling law https://browse.arxiv.org/abs/2402.00795 4 comments machinelearning
- [R] Transformers Learn Higher-Order Optimization Methods for In-Context Learning: A Study with Linear Models https://arxiv.org/abs/2310.17086 15 comments machinelearning
- "Transformers Learn Higher-Order Optimization Methods for In-Context Learning: A Study with Linear Models", Fu et al 2023 (self-attention learns higher-order gradient descent) https://arxiv.org/abs/2310.17086 16 comments reinforcementlearning
- [R] Symbol tuning ( i.e finetuning on input-label pairs where natural language labels (e.g., "positive/negative sentiment") are replaced with arbitrary symbols (e.g., "foo/bar") ) improves in-context learning in language models, with much stronger results for algorithmic reasoning benchmarks. https://arxiv.org/abs/2305.08298 2 comments machinelearning