Linking pages
- The History of Open-Source LLMs: Better Base Models (Part Two) https://cameronrwolfe.substack.com/p/the-history-of-open-source-llms-better 1 comment
- The History of Open-Source LLMs: Imitation and Alignment (Part Three) https://cameronrwolfe.substack.com/p/the-history-of-open-source-llms-imitation 0 comments
- Dolma, OLMo, and the Future of Open-Source LLMs https://cameronrwolfe.substack.com/p/dolma-olmo-and-the-future-of-open 0 comments
Linked pages
- Llama 2 - Meta AI https://ai.meta.com/llama/ 820 comments
- [2205.01068] OPT: Open Pre-trained Transformer Language Models https://arxiv.org/abs/2205.01068 318 comments
- Introducing ChatGPT https://openai.com/blog/chatgpt/ 296 comments
- [2101.00027] The Pile: An 800GB Dataset of Diverse Text for Language Modeling https://arxiv.org/abs/2101.00027 81 comments
- BLOOM https://bigscience.huggingface.co/blog/bloom 46 comments
- Understanding and Coding the Self-Attention Mechanism of Large Language Models From Scratch https://sebastianraschka.com/blog/2023/self-attention-from-scratch.html 44 comments
- OpenAI API https://openai.com/blog/openai-api/ 25 comments
- BigScience Research Workshop https://bigscience.huggingface.co 11 comments
- Mosaic LLMs (Part 2): GPT-3 quality for <$500k https://www.mosaicml.com/blog/gpt-3-quality-for-500k 7 comments
- Fully Sharded Data Parallel: faster AI training with fewer GPUs Engineering at Meta - https://engineering.fb.com/2021/07/15/open-source/fsdp/ 2 comments
- GitHub - facebookresearch/metaseq: Repo for external large-scale work https://github.com/facebookresearch/metaseq 2 comments
- [2001.08435] The Pushshift Reddit Dataset https://arxiv.org/abs/2001.08435 0 comments
- [1801.06146] Universal Language Model Fine-tuning for Text Classification https://arxiv.org/abs/1801.06146 0 comments
- EleutherAI https://www.eleuther.ai/ 0 comments
- The BigScience RAIL License https://bigscience.huggingface.co/blog/the-bigscience-rail-license 0 comments
- [2204.06745] GPT-NeoX-20B: An Open-Source Autoregressive Language Model https://arxiv.org/abs/2204.06745 0 comments
- metaseq/OPT175B_Logbook.pdf at main · facebookresearch/metaseq · GitHub https://github.com/facebookresearch/metaseq/blob/main/projects/OPT/chronicles/OPT175B_Logbook.pdf 0 comments
- metaseq/projects/OPT/chronicles at main · facebookresearch/metaseq · GitHub https://github.com/facebookresearch/metaseq/tree/main/projects/OPT/chronicles 0 comments
- LLaMA: LLMs for Everyone! - by Cameron R. Wolfe https://cameronrwolfe.substack.com/p/llama-llms-for-everyone 0 comments
Related searches:
Search whole site: site:cameronrwolfe.substack.com
Search title: The History of Open-Source LLMs: Early Days (Part One)
See how to search.