Baidu: ERNIE 5.5 Flash Base officially open-sourced

Today, Baidu officially open-sourced its latest foundation model, ERNIE 5.5 Flash Base. ERNIE 5.5 Flash Base adopts a MoE architecture, with 280 billion total parameters and 14 billion parameters activated per token. It was pre-trained on 33.5 trillion tokens, with a pre-training context length of up to 128K, and supports a context length of up to 1 million tokens during fine-tuning and inference.