Thread

Keep thinking about the different memory levels in LLMs. There is the low level "muscle memory" that takes ages to train, "finetuning" that you can update with acceptable effort, and the short-term memory of explicitly provided history tokens. Not so different from human brains.

The difference however is that the transition from short term memory to longer term memory takes place automatically. You read an article and still remember it the next day, but with LLMs you need to be quite explicit about all this.