Pangu Ultra MoE by Huawei

• Blockchain Council
• May 21, 2025

Huawei’s Pangu Ultra MoE is a powerful AI model that runs faster, uses less power, and handles complex tasks like math, code, and reasoning. It uses a unique design called Mixture-of-Experts (MoE), where only a part of the model is active at a time. This means it can do more with less compute. This design choice helps reduce cost while improving both speed and accuracy—something most traditional models struggle with.

In this guide, you’ll learn what makes Pangu Ultra MoE different, how it works, and how it compares to other models like GPT-4.

What Is Pangu Ultra MoE?

Pangu Ultra MoE is a large language model created by Huawei. It has 718 billion parameters, but only 39 billion are active during any one task. This is possible because of its sparse architecture. The model is trained on Huawei’s Ascend NPUs and uses smart techniques to make training fast and efficient.

Huawei didn’t just build a bigger model—they made it smarter. They used simulations to plan its design before training. The result is a model that runs fast and performs well in areas like math, medical reasoning, and logic. It’s one of the most scalable AI models built specifically to work with dedicated hardware.

Key Features of Pangu Ultra MoE

Here are the top features that make Pangu special:

• 718 billion total parameters with 39 billion active at once
• Uses 8 experts per token out of 256 in each layer
• Built with 61 transformer layers and a hidden size of 7680
• Trained with 6000 Ascend NPUs using advanced parallelism
• Fast token processing: 1.46 million tokens per second
• High efficiency: 30% Model Flops Utilization (MFU)
• Pre-training simulations to reduce trial-and-error during real training

How Pangu Ultra MoE Stands Out

Pangu’s biggest advantage is its balance of size and efficiency. It performs like top models but uses less compute. Here’s a quick look at how it compares.

Pangu Ultra MoE Model Comparison

Model	Total Parameters	Active Parameters	MFU	Tokens/Second
Pangu Ultra MoE	718B	39B	30%	1.46M
DeepSeek R1	685B	Not Available	N/A	N/A
GPT-4	175B	175B	N/A	N/A

Pangu is larger than GPT-4 and designed for efficiency. It gives competitive results without using all parameters at once.

Pangu Ultra MoE Performance Analysis

Huawei tested Pangu on different types of questions—math, medical, logic, and language. The model showed strong results across all benchmarks.

Task	Score (%)
MATH500	97.4
AIME2024	81.3
MMLU	91.5
CLUEWSC	94.8
MedQA	87.1
MedMCQA	80.8

These results show that Pangu is not only efficient—it’s accurate too. It handles complex math and medical questions while using fewer resources.

Innovations That Power Pangu

Huawei used smart training strategies to make Pangu efficient:

• Simulation-based design: Tested designs before training to avoid trial and error
• Adaptive pipeline overlap: Reduced wait time during training
• Fine-grained recomputation: Recalculated only small parts to save memory
• Tensor swapping: Moved data in and out of memory smartly
• Hierarchical communication: Made data sharing between chips faster

These methods allow the model to train faster and scale better, even when using fewer computing resources than competitors.

Training Efficiency and Cost Benefits

One of the biggest advantages of Pangu is that it saves cost without cutting performance. Sparse models like this activate fewer parts during training, which means:

• Less energy use per token processed
• Lower hardware demands for fine-tuning
• Faster training cycles with optimized compute planning

The result? You can train and deploy powerful AI tools even if you don’t have massive GPU clusters. Huawei’s use of Ascend NPUs with optimized scheduling makes this even more efficient.

How Enterprises Can Use Pangu

Pangu isn’t just a research project. It’s already being used in real-world applications:

• Healthcare: For diagnosis support and medical question answering
• Education: To help students understand math step-by-step
• Finance: To automate decision-making in analysis tools
• Code generation: To write and fix programs across languages
• AI agents: For multi-turn conversations and complex problem solving

Because of its design, Pangu is easy to scale and adapt to different industries. Enterprises can build custom agents, chatbots, or recommendation tools without needing to retrain from scratch.

Future Outlook for MoE Models

Mixture-of-Experts models are becoming more popular because they solve a major problem in AI: balancing performance with cost. Instead of turning on the whole model every time, MoEs like Pangu let you activate only the “experts” you need.

That means more personalized and efficient AI systems. In the future, we may see:

• More open-source MoE frameworks
• Better cross-platform support (GPUs, NPUs, TPUs)
• Custom MoE models for niche tasks (legal, engineering, etc.)
• Wider adoption in edge devices and cloud platforms

Pangu is a strong example of what happens when hardware and software evolve together.

Should You Care About Pangu Ultra MoE?

Yes—especially if you’re building or working with AI models. Pangu shows that big models don’t have to be slow or expensive. Its efficient design and high scores make it a great option for anyone building smart tools.

If you want to learn how models like Pangu are built and trained, check out the AI Certification. If you’re managing data or training workflows, go for the Data Science Certification. If you’re in business or marketing, the Marketing and Business Certification helps you apply AI strategies in your field.

Final Thoughts

Pangu Ultra MoE is a smart and scalable model that pushes AI forward. It proves you can build a massive model that’s also efficient and useful. With 718 billion parameters and fast performance, Pangu is one of the most exciting AI models right now.

Its design shows what’s possible when you match hardware and software from the ground up. The future of AI isn’t just bigger—it’s smarter, and Pangu is leading the way.