Why don't frontier AI model providers continuously improve their models?

Question

Just what the title says: I'm wondering why we're still, years after ChatGPT, having to wait weeks or months for "the next version" of a model when so much else in the software world has moved toward continuous improvement?

PaulHoule · Accepted Answer

It just doesn't work that way.To some extent you can append some knowledge to a model with low-rank adaptation and other techniques but if you want to train a model which is substantially better than your old model you need to train a new model which is much bigger and/or more efficient than your old model and it learns a whole new representation.See https://en.wikipedia.org/wiki/Catastrophic_interference for one problem.