When ChatGPT came out, like many of my colleagues, I was disoriented. What sur‐
prised me wasn’t the model’s size or capabilities. For over a decade, the AI commu‐
nity has known that scaling up a model improves it. In 2012, the AlexNet authors
noted in their landmark paper that: “All of our experiments suggest that our results
can be improved simply by waiting for faster GPUs and bigger datasets to become
available.”

What surprised me was the sheer number of applications this capability boost
unlocked. I thought a small increase in model quality metrics might result in a mod‐
est increase in applications. Instead, it resulted in an explosion of new possibilities.