AWS priming cuts hybrid model pretraining cost by over 99%
Amazon Web Services researchers introduced priming, a method that reuses pretrained Transformer weights to build hybrid state space models, cutting pretraining cost by more than 99% and using under 0.5% of the source model's token budget. A 32-billion-parameter Gated KalmaNet hybrid matched its source Transformer on reasoning while decoding up to twice as fast at 128,000-token context. Code and models were released under Apache 2.0.
Why it matters: Cheap, controlled architecture experiments could accelerate the shift away from pure Transformers for long-context agents, where KV-cache memory costs dominate serving economics.