upstage

A 250B-A15B MoE supporting English, Korean, and Japanese, trained on 12T tokens.

Upscaling Phi 3 14B to 22B parameters and applying a new post-training recipe. It's designed specifically for efficient inference on 80GB GPUs.