Introduction
DeepSeek-Prover-V2 is a breakthrough in neural theorem proving, showcasing how large language models can master formal mathematics. This guide walks you through the innovative training pipeline that made it possible, from cold-start data generation to reinforcement learning. By understanding these steps, you can replicate or adapt the methodology for your own projects in Lean 4. Whether you're a researcher or a math enthusiast, this structured approach reveals the secrets behind state-of-the-art performance.
What You Need
- Familiarity with Lean 4 – Basic knowledge of the theorem prover and its syntax.
- DeepSeek-V3 model access – Required for generating initial reasoning data (API or local deployment).
- A 7B parameter prover model – Used for proof search on subgoals (e.g., DeepSeek-Prover-V2 base).
- Computational resources – High-performance GPUs for training the full 671B parameter model.
- A dataset of mathematical theorems – Preferably formalized in Lean 4, such as MiniF2F or PutnamBench problems.
Step-by-Step Guide
Step 1: Generate Cold-Start Reasoning Data
Begin by prompting DeepSeek-V3 to decompose complex theorems into a series of manageable subgoals. This leverages the model’s powerful mathematical reasoning. Simultaneously, have DeepSeek-V3 formalize each high-level proof step in Lean 4, creating a structured sequence of sub-problems. This process produces a rich dataset of paired informal reasoning and formal code.
Step 2: Decompose Theorems into Subgoals
For each theorem, ensure the decomposition is exhaustive. Use chain-of-thought prompts to guide DeepSeek-V3 into breaking down the proof into logically connected lemmas. The goal is to create subgoals that are individually provable yet collectively solve the original problem. Save the decomposition as a list of Lean 4 proof obligations.
Step 3: Formalize Subgoals in Lean 4
Convert each subgoal into a Lean 4 theorem statement. Verify that the formalization captures all necessary hypotheses. This step is critical because the subsequent proof search will operate on these precise representations. Store the formalized subgoals alongside the original chain-of-thought reasoning from Step 1.
Step 4: Prove Subgoals with a Smaller Model
Use a smaller 7B parameter prover model to attempt proofs for each subgoal. The computationally intensive nature of proof search makes a smaller model practical. Run the search iteratively, allowing the model to use tactics and rewrite rules. Once all subgoals of a given theorem are proven, merge their proofs into a complete formal proof of the original problem.
Step 5: Combine Proofs and Chain-of-Thought
For every theorem that the 7B model fully solves, pair the final Lean 4 proof with the original chain-of-thought reasoning from DeepSeek-V3. This creates a unified training example that demonstrates both the high-level reasoning and its formal realization. This synthetic dataset is the foundation for fine-tuning.
Step 6: Fine-Tune with Synthetic Data
Curate a set of challenging problems that the 7B model could not solve end-to-end but for which all subgoals were proven. Combine the subgoal proofs to form a complete proof, then link it with DeepSeek-V3’s decomposition chain-of-thought. Fine-tune the prover model on this synthetic data to improve its ability to generalize from informal reasoning to formal proofs.
Step 7: Apply Reinforcement Learning
After supervised fine-tuning, enter the reinforcement learning stage. Use a binary reward signal: correct or incorrect final proof. This feedback loop incentivizes the model to refine its proof search strategies. The model learns to bridge the gap between informal mathematical intuition and rigorous formal steps, effectively exploring more reliable proof paths.
Step 8: Achieve State-of-the-Art Performance
Scale up to the full 671B parameter model (DeepSeek-Prover-V2–671B). Test it on benchmarks like MiniF2F and PutnamBench. With this pipeline, the model achieves an 88.9% pass ratio on MiniF2F-test and solves 49 out of 658 Putnam problems. The proofs for MiniF2F are publicly available for verification and further research.
Tips for Success
- Prioritize decomposition quality – The success of the entire pipeline depends on how well DeepSeek-V3 breaks down theorems. Experiment with different prompts and few-shot examples.
- Handle incomplete subgoal proofs – If the 7B model fails on some subgoals, consider re-decomposing or using a stronger model for those specific parts.
- Use diverse training data – Including problems from multiple sources (e.g., MiniF2F, PutnamBench) improves generalization.
- Monitor reinforcement learning stability – Binary rewards can be noisy; use techniques like reward shaping or curriculum learning to guide convergence.
- Share your proofs – The authors released proofs for MiniF2F, fostering community improvement. Consider open-sourcing your own results.
- Iterate on model size – The final 671B model sets the record, but smaller variants may suffice for less complex domains. Scale according to your computational budget.
By following this guided pipeline, you can harness the power of recursive proof search and data synthesis to train a neural theorem prover that excels in formal mathematics. The methodology detailed here is not only reproducible but also adaptable to other formal systems beyond Lean 4.