Skip to main content
This walkthrough shows how to use Synth in production to continually optimize your prompts in light of production results. We’ll do this by running MIPRO Online on the Banking77 intent classification task. In online mode you run rollouts locally, while the backend proposes and selects prompt candidates reactively in real time, with under ~500ms of latency added from retrieving the cached candidates.

​
Prerequisites

  • Python 3.11+
  • uv package manager
  • SYNTH_API_KEY set in your environment
  • Access to a Synth backend (default is production)

​
Run the demo locally as a script

From the synth-ai repo: 
cd /Users/joshpurtell/Documents/GitHub/synth-ai

export SYNTH_API_KEY="sk_live_your_key"
export SYNTH_URL="https://api.usesynth.ai"

uv run python demos/mipro_banking77/run_online_demo.py \
  --rollouts 100 \
  --train-size 200 \
  --val-size 10 \
  --min-proposal-rollouts 20

​
What the flags do

  • --rollouts: Number of online rollouts to run
  • --train-size: Number of training seeds (0..train-size-1)
  • --val-size: Number of validation seeds (train-size..train-size+val-size-1)
  • --min-proposal-rollouts: Minimum rollouts before generating new proposals

​
What happens

  1. The script starts a local task app and health-checks it.
  2. A MIPRO online job is created on the backend.
  3. The backend returns a proxy URL for prompt candidate selection.
  4. The script runs rollouts locally, calling the proxy URL for each LLM call.
  5. Rewards are reported back and proposals evolve in real time.

​
Tips

  • To use a different backend, set SYNTH_URL (preferred). SYNTH_BACKEND_URL and RUST_BACKEND_URL are also supported for compatibility.
  • You can change the policy model with --model gpt-4.1-nano (or another supported model).
  • The script auto-generates ENVIRONMENT_API_KEY if it is not set.

​
Production usage

When you move this flow to production, the loop is the same. You just swap the backend URL, send rewards back to the online MIPRO system, and rely on the proxy URL to perform prompt substitution.

​
1) Set the backend URL

Point to the production backend: 
export SYNTH_API_KEY="sk_live_your_key"
export SYNTH_URL="https://api.usesynth.ai"
The demo resolves the backend URL from SYNTH_URL (preferred), then SYNTH_BACKEND_URL, then RUST_BACKEND_URL.

​
2) Send reward updates

After each rollout, report the reward to the backend system. The demo uses: 
POST /api/prompt-learning/online/mipro/systems/{system_id}/status
{
  "rollout_id": "<id>",
  "status": "reward",
  "reward": <float>
}
Then it sends a final "done" status for the rollout: 
POST /api/prompt-learning/online/mipro/systems/{system_id}/status
{
  "rollout_id": "<id>",
  "status": "done"
}
This is what push_status() does in the demo.

​
3) Prompt substitution (what happens behind the scenes)

The backend returns a proxy URL (e.g. mipro_proxy_url) for each online job. You call it like: 
{proxy_url}/{rollout_id}/chat/completions
Behind the scenes:
  1. The proxy selects the current best candidate prompt for the rollout.
  2. It substitutes that candidate into the prompt template (system/user message patterns).
  3. The proxy forwards the request to the model provider with the substituted prompt.
  4. You receive the model response and compute a reward locally.
  5. Reward updates drive the next round of proposals.

​
See Also