A new “Sleep” framework proposes letting LLMs accumulate short-term memories in a buffer/cache during use, then rank and consolidate the useful bits into model weights during a sleep cycle using low-rank structures and mixture-of-experts, followed by “dreaming” via synthetic routing and RL refinement. The approach is described as on-policy distillation, where an expanded student learns token-specific feedback from teacher distributions on self-sampled sequences, with discussion highlighting both its technical promise and audit/lineage challenges for tracking how and when model knowledge changes.