Xiaomi's Second-Generation Large Language Model MiLM2: Comprehensive Upgrade with 45% Performance Boost, On-Device Deployment Breakthrough, Empowering the Human-Vehicle-Home Ecosystem

Xiaomi's Second-Generation Large Language Model MiLM2: Comprehensive Upgrade with 45% Performance Boost, On-Device Deployment Breakthrough, Empowering the Human-Vehicle-Home EcosystemMiLM2 expands the parameter scale both downward and upward, covering multiple levels from 0.3B to 30B, adapting to different resource constraints and demands across various scenarios

Xiaomi's Second-Generation Large Language Model MiLM2: Comprehensive Upgrade with 45% Performance Boost, On-Device Deployment Breakthrough, Empowering the Human-Vehicle-Home Ecosystem

Xiaomi recently announced the second generation of its self-developed large language model, MiLM2, showcasing significant advancements in performance, on-device deployment, and application scenarios. As a key component of Xiaomi's "Human-Vehicle-Home" ecosystem strategy, MiLM2 aims to deliver smarter and more convenient experiences for users.

Comprehensive Performance Enhancement: Expanded Parameter Scale, Enhanced Capabilities

MiLM2 expands the parameter scale both downward and upward, covering multiple levels from 0.3B to 30B, adapting to different resource constraints and demands across various scenarios. Compared to the first generation, MiLM2 achieves an average improvement of over 45% in key capabilities such as instruction following, translation, and casual conversation. These advancements are comprehensively evaluated through the self-built general capability evaluation set Mi-LLMBM2.0. For instance, both MiLM2-1.3B and MiLM2-6B models demonstrate significant improvement in ten key capabilities. Notably, the MiLM2-6B model exhibits superior performance in casual conversation and translation compared to other models within the same parameter scale.

On-Device Deployment Breakthrough: Lightweight Model Deployment, Enabling "Cloud-Edge-End" Integration

Xiaomi's self-developed large model team has made significant progress in on-device deployment, bringing large model capabilities to more terminal devices. The release of the MiLM2-4B model marks Xiaomi as the first company in the industry to successfully run a 4B-parameter-scale large model on mobile devices. To achieve lightweight deployment, Xiaomi has developed the "TransAct Large Model Structural Pruning Method," which achieves a 4B model from a 6B model using only 8% of the training computation, significantly improving training efficiency. Additionally, Xiaomi has developed the "Weight Transfer-Based On-Device Quantization Method" and the "Outlier Separation-Based On-Device Quantization Method," effectively reducing accuracy loss during on-device quantization. Compared to the industry standard Qualcomm solution, the quantization loss is reduced by 78%.

Model Matrix Expansion: Multi-Scenario Applications, Empowering the "Human-Vehicle-Home" Ecosystem

To meet the diverse business needs within the group, MiLM2 has built and continuously expanded its self-developed large model matrix, covering multiple parameter scales including 0.3B, 0.7B, 1.3B, 2.4B, 4B, 6B, 13B, and 30B, adapting to the demands of different scenarios.

- 0.3B~6B: On-device Scenarios, suitable for low-cost, specific tasks, adapting to terminal devices with different chips and storage spaces. After fine-tuning, the performance is comparable to open-source models within the hundred-billion parameter range.

- 6B, 13B: Task-Specific Scenarios, requiring more zero-shot/in-context learning capabilities than models below 6B parameters, supporting multi-task fine-tuning. After fine-tuning, they can achieve the performance of hundreds of billions of open-source models, showcasing the potential of LLM emergence capabilities.

- 30B: Cloud Scenarios, equipped with robust zero-shot/in-context learning capabilities and generalization abilities. The model has strong inference capabilities, capable of completing complex multi-tasks, reaching the level of general large models.

Furthermore, the MiLM2 series incorporates models with MoE (Mixture of Experts) structure, such as MiLM2-0.7B8 and MiLM2-2B8. By parallelizing multiple "expert" models, it improves the overall prediction accuracy and efficiency. For example, MiLM2-2B8 achieves performance comparable to MiLM2-6B while improving decoding speed by 50%, enhancing operation efficiency while maintaining performance.

Application Landing: Empowering Pangu OS, Xiao Ai Assistant, Intelligent Cockpit, etc.

The achievements of Xiaomi's second-generation self-developed large model are starting to penetrate real business scenarios and user needs. It not only helps address diverse internal business demands and improve work efficiency, but has also begun to be applied in products like Pangu OS, Xiao Ai Assistant, intelligent cockpits, and intelligent customer service. For example, MiLM2's instruction following capabilities can help Xiao Ai Assistant better understand user instructions, improving the user experience. MiLM2's translation capabilities can provide more accurate language translation services for intelligent cockpits, enhancing the driving experience.

Summary: MiLM2 marks a significant breakthrough in Xiaomi's large model research and development. It not only showcases Xiaomi's technological leadership but also provides strong driving force for the implementation of the "Human-Vehicle-Home" ecosystem strategy, propelling intelligent living to a new height.