Warning

THIS PROJECT IS ARCHIVED
Intel will not provide or guarantee development of or support for this project, including but not limited to, maintenance, bug fixes, new releases or updates.

Patches to this project are no longer accepted by Intel.

Caution

This project has been identified as having known security issues.

DeepMath: A Lightweight Math Reasoning Agent for LLMs

Large language models (LLMs) have made impressive strides in reasoning tasks, yet mathematical problem-solving remains a challenge. Traditional "chain-of-thought" reasoning often produces verbose explanations and error-prone arithmetic. DeepMath tackles this by combining a small Python executor with a fine-tuned LLM, enabling concise, computation-driven reasoning.

DeepMath

DeepMath is built on Qwen3-4B Thinking and fine-tuned with GRPO (Group Relative Policy Optimization). Instead of verbose text, the model emits short math snippets in a restricted Python-like language. A bounded interpreter evaluates approved mathematical operations and folds the results back into its reasoning.

The interpreter does not expose general Python imports, arbitrary file/network operations, or SymPy's string-expression evaluators. Its worker is not an OS sandbox: it runs with the service account's operating-system privileges unless deployment-level isolation is configured. The inference server also does not authenticate requests by default; do not expose it to untrusted networks without adding access controls and isolation.

We evaluate DeepMath on four math datasets: MATH500, AIME, HMMT, and HLE, and show that:

• The math agent alone improves accuracy and reduces verbosity.

• GRPO training alone biases outputs toward brevity and correctness.

• Combining the agent with GRPO yields the largest gains.

Check out:

• 🔗 Blog: https://huggingface.co/blog/intel-deepmath

• 👉 Model: https://huggingface.co/Intel/deepmath-v1

Why DeepMath?

LLMs often struggle with numeric precision and produce unnecessarily long reasoning chains. Two opportunities stand out:

1. Offload deterministic computation to a restricted math executor.

2. Train models to prefer concise, computation-oriented traces over verbose text.

DeepMath implements both. The model learns to generate short math snippets, which are interpreted by a restricted math executor and reintegrated into the context. GRPO fine-tuning encourages this behavior by rewarding correctness and encouraging shorter outputs.

How It Works

• Base model: Qwen3-4B Thinking.
• Executor constraints: restricted math syntax and fixed operation registry, bounded inputs/outputs and per-call worker deadline. This is not an OS sandbox; production deployments need OS-level isolation.
• Inference: based on SmolAgents, a math agent was created. vLLM is used as the inference engine.
• Training: based on the GRPO trainer in TRL, we modified TRL's vLLM client and server to generate GRPO completions using our DeepMath agent.

Figure 1: The vLLM client and server were modified to use the DeepMath agent in generating the candidates, while using the vLLM backend.

• Agent Interface: During inference, the model can output normal tokens or special agent calls containing restricted math snippets.

• Execution: Snippets run in a restricted interpreter. The worker retains its service account's OS privileges; filesystem, network, credentials, and deployment isolation must be controlled outside the interpreter.

• Design Goals:

  • Concision: Replace multi-line textual calculations with short, focused snippets.

    • Bounded computation: Enforce interpreter-level source, operation, collection, output, and execution limits.

  • Interpretability: Snippets are readable and auditable.

Figure 2: Output example where python code is generated, evaluated and the answer is inserted into the trace and used for context.

Training with GRPO

We fine-tune the model using GRPO, a reward-based optimization that balances:

• Accuracy Reward: +1 for correct answers.

• Using code snippets: +1 for generating code snippets, weighted 10:1 vs. the accuracy reward.

• Length reduction: shorter lengths are encouraged by limiting the GRPO completion candidates to 5k tokens.

• Temperature Scheduling: We implemented linear temperature scheduling (T=1.2 → T=0.7) to balance exploration and stability during training. This approach aims to enhance experimentation during the initial training phases, subsequently reducing the temperature as we refine our proficiency in the skill.

• In-context Learning: we include 4 solved examples where the trace contains agent calls and executor outputs, so the model learns the syntax and the call/response pattern.

• Dataset: we used OpenMathReasoning dataset, the tool-usage subset. Note that GRPO only uses the problem, not the solution in the data. Choosing this dataset ensures problems benefit form tool use.

Evaluation

We benchmarked DeepMath against baselines on four datasets. Metrics include:

• majority@16 (robustness across samples).

• Mean output length (brevity).

• We compare a baseline configuration (Qwen3-4B-Thinking-2507, no agenting) with our DeepMath model. As ablation, we evaluate the agentic framework we developed running with the untrained Qwen3 model, denoted by +Agent. Additionally, we examine whether the GRPO training (for agentic use) improves non-agentic inference, denoted by +GRPO. Thus the two ablations are independent, not additive.

• We observe the agentic inference reduces output lengths, with mixed accuracy results. The DeepMath model is both GRPO-trained and run in agentic mode, and shows the highest accuracy with shortened traces. We conclude both GRPO training and agentic inference are needed for best results.

Key Insight: DeepMath reduces output length by up to 66% while improving accuracy on challenging datasets.

Why It Matters

• Accuracy: Offloading computation reduces arithmetic errors.

• Efficiency: Shorter outputs mean faster inference and easier interpretability.

  • Safety: Restrict the supported math language; use deployment-level OS isolation for defense in depth.

Conclusion

DeepMath demonstrates a practical and lightweight way to combine a small executor with an LLM and to train the model to prefer short, computation-driven traces. Offloading deterministic computation reduces arithmetic and numerical errors and shortens traces, and GRPO fine-tuning further encourages concise, correct answers. The result is a more accurate and more interpretable math-solving agent without requiring a massive model or heavyweight external tools.

Try It Yourself

Check out the GitHub repository and give DeepMath a try. Feedback and contributions are welcome. 🚀

Installation

DeepMath uses uv for Python environment and dependency management.

1. Install uv

If uv is not already installed:

curl -LsSf https://astral.sh/uv/install.sh | sh

2. Clone the repository

git clone https://github.com/intellabs/DeepMath.git
cd DeepMath

3. Install DeepMath

Create the project's virtual environment and install the standard dependencies:

uv sync

uv will automatically create a .venv virtual environment, install the locked dependencies, and install DeepMath in editable mode.

You can run commands directly in the project environment with uv run:

uv run python --version

Or activate the virtual environment:

source .venv/bin/activate

GPU / CUDA Support

DeepMath's GPU training and vLLM dependencies are optional because they require a supported GPU environment and are not available on all platforms.

For a Linux x86-64 NVIDIA/CUDA environment with Python 3.12, install the GPU dependencies with:

uv sync --extra gpu

The gpu extra includes dependencies such as vLLM, FlashAttention, DeepSpeed, and bitsandbytes.

A standard uv sync is sufficient for development environments that do not require these GPU-specific features, including macOS.

Standard development still installs PyTorch because the training libraries depend on it. On macOS, PyTorch comes from PyPI and does not require CUDA; CPU and supported Apple MPS operations remain available. The CUDA 13.0 PyTorch source and the pinned FlashAttention wheel are restricted to the Linux x86-64 gpu extra. This configuration does not guarantee a CPU-only PyTorch installation on Linux.

uv resolves all extras when updating the lockfile, so GPU packages can appear in the lockfile without being installed by a standard macOS sync.

Usage

The example.sh script provides complete examples for training, inference, and evaluation. Below are the main workflows:

Training

Training involves two components running in parallel:

1. Start the vLLM server (for generating completions during GRPO training):

CUDA_VISIBLE_DEVICES=0,1,2,3,4,5 \
    python -m deep_math.vllm_serve_agent \
    --model Qwen/Qwen3-4B-Thinking-2507 \
    --tensor_parallel_size 1 \
    --data_parallel_size 6 \
    --config configs/vllm_agent_server_4B.yaml \
    --max_steps 50 \
    --max_agent_output 5000

2. Run the GRPO trainer:

CUDA_VISIBLE_DEVICES=6,7 \
    accelerate launch --config_file configs/zero2.yaml \
    --num_processes 2 \
    training.py

Inference

Run inference with different configurations. Examples use the MATH500 dataset:

Baseline (no agent, no fine-tuning):

python inference.py \
    +model.use_vllm=true \
    model.generation.max_new_tokens=20000 \
    model.generation.do_sample=true \
    model.generation.temperature=0.6 \
    model.model_name_or_path=Qwen/Qwen3-4B-Thinking-2507 \
    hf_tag=HuggingFaceH4/MATH-500 \
    generated_file=baseline-MATH500.jsonl

With agent (adds Python code execution):

python inference.py \
    +model.use_vllm=true \
    +model.math_agent=true \
    +model.examples=deep_math/fewshot.txt \
    model.generation.max_new_tokens=3000
    +model.max_agent_output=20000 \
    +model.max_steps=50 \
    model.model_name_or_path=Qwen/Qwen3-4B-Thinking-2507 \
    hf_tag=HuggingFaceH4/MATH-500 \
    generated_file=agent-MATH500.jsonl

With fine-tuned LoRA adapter:

python inference.py \
    +model.use_vllm=true \
    +model.math_agent=true \
    model.lora_path=/PATH/TO/CHECKPOINT \
    +model.vllm_params.max_lora_rank=32 \
    model.generation.max_new_tokens=3000
    +model.max_agent_output=20000 \
    +model.max_steps=50 \
    model.model_name_or_path=Qwen/Qwen3-4B-Thinking-2507 \
    hf_tag=HuggingFaceH4/MATH-500 \
    generated_file=trained-agent-MATH500.jsonl

Evaluation

Compare multiple inference runs:

python evaluation.py \
    generated_file=baseline-MATH500.jsonl,agent-MATH500.jsonl,trained-agent-MATH500.jsonl

See example.sh for complete command examples with all parameters.

Citation

If you use DeepMath in your research, please cite:

@software{deepmath2025,
  author = {Fleischer, Daniel and Berchansky, Moshe and Wasserblat, Moshe},
  title = {DeepMath: A Lightweight Math Reasoning Agent for LLMs},
  year = {2025},
  publisher = {Intel AI Labs},
  url = {https://github.com/IntelLabs/DeepMath}
}

Limitations & Future Work

• Scope: we focused on a small model and on mathematical reasoning.

• Generalization: evaluated on contest-style math; results may not transfer to open-ended mathematical creativity or formal proofs.

• Generated snippets are interpreted by a restricted math executor, not run as arbitrary Python. The worker is not an OS sandbox and the server routes are unauthenticated by default; do not expose the service to untrusted networks. Production operators must add authentication, OS-level isolation, resource controls, and rate limits. See restricted math execution.