Between these kinds of optimizations, improved data center efficiency, and smaller models being more capable, I wonder how long it will be before someone manages to make a profitable AI business. Maybe when they race to train better models slows down and they don't need to constantly upgrade capacity.
Reminds me of the early days of cloud computing. It was very pricey, but once the tools caught up in 5 or so years, it went from "omg cloud is so expensive" to "omg cloud is only expensive when its worth building your own data center"
