Why Some Researchers at OpenAI, Thinking Machines and Amazon Want to Change How LLMs Are Trained

It seems like every AI researcher these days has lots of ideas around how to combat major shortcomings in the field, such as the “split-brain problem” (in which a model can incorrectly answer a question just because of the way the question is phrased), by developing new models that do “continual learning,” for instance.

AI researchers also tell me they’re interested in revamping the model training process. Some researchers, including Amazon’s David Luan, say the current order of steps of pretraining (giving it a broad, general knowledge of the world) before posttraining (honing its knowledge in specific domains) don’t make much sense. 

This order of steps has become the norm in AI because, in some ways, it makes intuitive sense: humans similarly require a general base of knowledge you learn during childhood before learning more specific skills later on in life.

But Luan says that if you know a model will be used for coding or helping with customer refunds, why spend so much effort teaching the model about completely irrelevant topics, like poetry or horticulture?