[2211.04325] Will we run out of data? Limits of LLM scaling based on human-generated data

Linking pages

AI models could devour all of the internet’s written knowledge by 2026 | Live Science https://www.livescience.com/technology/artificial-intelligence/chatbots-could-devour-all-of-the-internets-written-knowledge-by-2026 332 comments
I. From GPT-4 to AGI: Counting the OOMs - SITUATIONAL AWARENESS https://situational-awareness.ai/from-gpt-4-to-agi/ 91 comments
AI Appears to Rapidly Be Approaching Brick Wall Where It Can't Get Smarter https://futurism.com/the-byte/ai-running-out-data-smarter 84 comments
Why transformative artificial intelligence is really, really hard to achieve https://thegradient.pub/why-transformative-artificial-intelligence-is-really-really-hard-to-achieve/ 63 comments
The World Is Running Out of Data to Feed AI, Experts Warn : ScienceAlert https://www.sciencealert.com/the-world-is-running-out-of-data-to-feed-ai-experts-warn 51 comments
AI Needs Us More Than We Need It | Washington Monthly https://washingtonmonthly.com/2024/10/29/ai-needs-us-more-than-we-need-it/ 28 comments
What Happens When AI Has Read Everything? - The Atlantic https://www.theatlantic.com/technology/archive/2023/01/artificial-intelligence-ai-chatgpt-dall-e-2-learning/672754/ 18 comments
10 AI Predictions For 2023 http://curated.content.curateus.com/r/11709 7 comments
NVIDIA vs. AMD vs. Intel: Comparing AI Chip Sales https://www.visualcapitalist.com/nvidia-vs-amd-vs-intel-comparing-ai-chip-sales/ 4 comments
Software²: A new generation of AIs that become increasingly general by producing their own training data https://thegradient.pub/software2-a-new-generation-of-ais-that-become-increasingly-general-by-producing-their-own-training-data/ 2 comments
First principles on AI progress https://dynomight.substack.com/p/scaling 1 comment
RLHF: Reinforcement Learning from Human Feedback https://huyenchip.com/2023/05/02/rlhf.html 1 comment
Introduction to Large Language Models - Abi Aryan https://abiaryan.com/posts/intro-llms/ 1 comment
AI Companies Are Running Out of Training Data https://futurism.com/ai-companies-training-data 1 comment
Missed Shot at Artificial General Intelligence - Libove Blog https://blog.libove.org/posts/missed-shot-at-artificial-general-intelligence/ 1 comment
10 AI Predictions For 2023 https://www.forbes.com/sites/robtoews/2022/12/20/10-ai-predictions-for-2023/ 0 comments
The Next Generation Of Large Language Models https://www.forbes.com/sites/robtoews/2023/02/07/the-next-generation-of-large-language-models/ 0 comments
A social network for AI | Nature Machine Intelligence https://www.nature.com/articles/s42256-023-00769-4 0 comments
Techno-optimism for 2024 - by Noah Smith - Noahpinion https://www.noahpinion.blog/p/techno-optimism-for-2024 0 comments
Is Synthetic Data the Key to AGI? - by Nabeel S. Qureshi https://digitalspirits.substack.com/p/is-synthetic-data-the-key-to-agi 0 comments