Skip to main content
RL training often targets a remotely hosted task app so the trainer cluster can reach it. Synth ships commands to run locally, deploy to Modal, and manage secrets consistently. For detailed CLI flag references, see Run Task Apps Locally, Run Task Apps on Modal, Deploy Task Apps, and Run Evaluations.

​
Local development

# Start the task app on localhost:8001
uvx synth-ai serve your-task-id \
  --port 8001 \
  --trace traces/v3

# Run a quick smoke rollout
uvx synth-ai eval \
  --app-id your-task-id \
  --model Qwen/Qwen3-0.6B \
  --seeds 1-3
Keep local iteration tight until /rollout responses look good and traces contain the information you need.

​
Deploy to Modal

uvx synth-ai deploy your-task-id \
  --name my-task-app \
  --trace traces/v3
The CLI bundles your task app, uploads it to Modal, and returns the public URL (for example https://my-task-app.modal.run). Use the same .env file so the deploy command can forward ENVIRONMENT_API_KEY. The CLI loads this .env automatically if you previously ran uvx synth-ai setup; only reach for --env-file when you need to override the detected values.

​
Managed secrets

Modal secrets keep your environment key out of the container image: 
modal secret list
modal secret create synth-env-key
modal secret set synth-env-key ENVIRONMENT_API_KEY=$ENVIRONMENT_API_KEY
Then reference the secret inside your task app’s Modal wrapper. If you stick to .env, the Synth deploy command encrypts ENVIRONMENT_API_KEY before shipping it to Modal.

​
Post-deploy checklist

curl -H "X-API-Key: $ENVIRONMENT_API_KEY" https://my-task-app.modal.run/health
curl -H "X-API-Key: $ENVIRONMENT_API_KEY" https://my-task-app.modal.run/task_info

uvx synth-ai eval \
  --task-url https://my-task-app.modal.run \
  --model Qwen/Qwen3-4B \
  --seeds 1-2
Only proceed to RL training once these checks succeed. The trainer will use the same URL listed in your RL TOML (services.task_url).

​
Updating deployments

Whenever you change prompts, reward shaping, or default models: 
uvx synth-ai deploy your-task-id --name my-task-app
Modal keeps versioned revisions, so the command swaps the live deployment atomically. Remember to update your RL config if the URL or default model changes. Tip: the CLI auto-loads the .env written during setup. Reach for --env-file only when you need to override or layer additional secrets.