AI Progress Driven by New Data

AI’s four major breakthroughs reused existing methods and were driven by tapping new datasets.

Each breakthrough corresponded to a new data source: ImageNet images, web text, human feedback labels, and programmatic verifiers.

Technical and architectural innovations matter less than the data; scaling and diversifying training data is the main driver of progress.

The next AI paradigm shift will likely come from harnessing new data sources like YouTube video or embodied sensor data rather than novel algorithms.