MIPRO

Multi-Instruction Prompt Optimization - Bayesian optimization for prompts

MIPRO: Multi-Instruction Prompt Optimization

MIPRO uses Bayesian optimization (TPE) to jointly optimize instructions and few-shot demonstrations for multi-stage LLM programs.

Paper: Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programs (EMNLP 2024)

When to Use

• Instruction-following tasks with clear structure
• Tasks where few-shot examples improve performance
• When you want faster convergence than genetic methods
• When demo selection matters

Config Reference

[prompt_learning]
algorithm = "mipro"
task_app_url = "https://your-tunnel.trycloudflare.com"
task_app_api_key = "$ENVIRONMENT_API_KEY"
task_app_id = "your-task"

[prompt_learning.initial_prompt]
id = "my_prompt"
name = "My MIPRO Prompt"

[[prompt_learning.initial_prompt.messages]]
role = "system"
pattern = "You are a classifier."
order = 0

[[prompt_learning.initial_prompt.messages]]
role = "user"
pattern = "{query}"
order = 1

[prompt_learning.policy]
model = "gpt-4o-mini"
provider = "openai"
temperature = 0.0
max_completion_tokens = 128

[prompt_learning.mipro]
env_name = "my-task"
num_iterations = 20
num_evaluations_per_iteration = 5
batch_size = 32
max_concurrent = 20
meta_model = "gpt-4o-mini"
meta_model_provider = "openai"
few_shot_score_threshold = 0.8
proposer_effort = "LOW"
proposer_output_tokens = "FAST"

# Seed pools
bootstrap_train_seeds = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
online_pool = [10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
test_pool = [20, 21, 22, 23, 24]

[prompt_learning.mipro.tpe]
gamma = 0.25
n_candidates = 32
n_startup_trials = 5
epsilon = 0.1
alpha = 1.0

[prompt_learning.mipro.demo]
max_few_shot_examples = 3
sets_per_size = 4
include_empty = true

[prompt_learning.mipro.grounding]
n = 8
temperature = 0.7
max_tokens = 600

[prompt_learning.mipro.meta_update]
enabled = true
every_iterations = 3
topk_success = 3
topk_failure = 3
validate_on_batch = 16
keep_k = 8
dedup_token_overlap = 0.8
regen_temperature_decay = 0.95

Top-Level Parameters

Parameter	Type	Default	Description
algorithm	string	-	Must be "mipro"
task_app_url	string	-	URL of your task app (tunnel URL)
task_app_api_key	string	-	Environment API key for auth
task_app_id	string	-	Identifier for your task

MIPRO Section [prompt_learning.mipro]

Parameter	Type	Default	Description
env_name	string	"banking77"	Environment/task name
num_iterations	int	20	TPE optimization iterations
num_evaluations_per_iteration	int	5	Evaluations per iteration
batch_size	int	32	Batch size for evaluation
max_concurrent	int	20	Max concurrent rollouts
meta_model	string	"gpt-4o-mini"	Model for instruction generation
meta_model_provider	string	"openai"	Provider for meta model
meta_model_inference_url	string	null	Custom inference URL
few_shot_score_threshold	float	0.8	Threshold for demo selection
proposer_effort	string	"LOW"	Model quality: "LOW_CONTEXT", "LOW", "MEDIUM", "HIGH"
proposer_output_tokens	string	"FAST"	Token limit: "RAPID", "FAST", "SLOW"
metaprompt	string	null	Custom metaprompt
max_wall_clock_seconds	float	null	Max runtime in seconds
max_total_tokens	int	null	Max tokens across all rollouts
max_token_limit	int	null	Total token budget
max_spend_usd	float	null	Maximum spend in USD
token_counting_model	string	"gpt-4"	Model for token estimation
enforce_token_limit	bool	true	Halt if limit exceeded

Seed Pools

Parameter	Type	Description
bootstrap_train_seeds	list[int]	Seeds for initial instruction generation
online_pool	list[int]	Seeds for TPE optimization
test_pool	list[int]	Held-out seeds for final evaluation
val_seeds	list[int]	Seeds for top-K validation
reference_pool	list[int]	Seeds for meta-prompt context

TPE Config [prompt_learning.mipro.tpe]

Parameter	Type	Default	Description
gamma	float	0.25	Quantile for good/bad split
n_candidates	int	32	Candidates for Expected Improvement
n_startup_trials	int	5	Random trials before TPE
epsilon	float	0.1	Exploration probability
alpha	float	1.0	KDE smoothing parameter

Demo Config [prompt_learning.mipro.demo]

Parameter	Type	Default	Description
max_few_shot_examples	int	3	Max demos in prompt
sets_per_size	int	4	Demo sets per size
include_empty	bool	true	Include zero-shot option

Grounding Config [prompt_learning.mipro.grounding]

Parameter	Type	Default	Description
n	int	8	Number of instruction proposals
temperature	float	0.7	LLM temperature for proposals
max_tokens	int	600	Max tokens per proposal

Meta-Update Config [prompt_learning.mipro.meta_update]

Parameter	Type	Default	Description
enabled	bool	true	Enable periodic regeneration
every_iterations	int	3	Regenerate every N iterations
topk_success	int	3	Top success examples for grounding
topk_failure	int	3	Top failure examples for grounding
validate_on_batch	int	16	Batch size for validation
keep_k	int	8	Max instruction variants to keep
dedup_token_overlap	float	0.8	Deduplication threshold
regen_temperature_decay	float	0.95	Temperature decay per update

Policy Config [prompt_learning.policy]

Parameter	Type	Default	Description
model	string	-	Model name (e.g., "gpt-4o-mini")
provider	string	-	Provider: "openai", "groq", "synth"
temperature	float	0.0	Sampling temperature
max_completion_tokens	int	128	Max output tokens
inference_mode	string	"synth_hosted"	Inference mode

Returns

from synth_ai.sdk.api.train.prompt_learning import PromptLearningJob

job = PromptLearningJob.from_config("mipro.toml")
job.submit()
result = job.poll_until_complete()

# Get results
results = job.get_results()
print(f"Best Score: {results['best_score']}")

# Get best prompt text
best_prompt = job.get_best_prompt_text(rank=1)
print(best_prompt)

Results Structure

{
    "best_prompt": {...},     # Optimized instruction + selected demos
    "best_score": 0.92,       # Accuracy on validation
    "top_prompts": [...],     # Top K configurations by score
    "optimized_candidates": [...],  # All TPE candidates
    "attempted_candidates": [...],  # All evaluated candidates
    "validation_results": {...},    # Per-seed validation scores
}

Download via CLI

# Download best prompt as JSON
uvx synth-ai artifacts download pl_71c12c4c7c474c34

# Download as YAML
uvx synth-ai artifacts download pl_71c12c4c7c474c34 --format yaml

# Save to file
uvx synth-ai artifacts download pl_71c12c4c7c474c34 --output prompt.json

Related

• GEPA — Alternative prompt optimization method
• Artifacts CLI — Download prompts
• Prompt Optimization SDK — Job events reference