> ## Documentation Index
> Fetch the complete documentation index at: https://docs.usesynth.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Optimize Your Model's Prompts using GEPA

> Full walkthrough of GEPA prompt optimization with Synth AI

GEPA (Generalized Evolutionary Prompt Adaptation) is a reflective evolutionary optimizer that automatically improves your prompts through LLM-guided mutations and multi-objective selection. This walkthrough covers everything from setup to retrieving your optimized prompts.

**Reference:** Agrawal et al. (2025). "GEPA: Reflective Prompt Evolution." [arXiv:2507.19457](https://arxiv.org/abs/2507.19457)

## Why GEPA?

GEPA outperforms GRPO by 10% on average (up to 20%) while using up to 35x fewer rollouts.

**Best for:**

* Classification tasks (Banking77, intent classification)
* Multi-hop QA (HotpotQA)
* Instruction-following tasks
* When you want diverse prompt variants (Pareto front)

**Typical results:** 60-75% baseline accuracy → 85-90%+ after 15 generations

## Prerequisites

Before starting, ensure you have:

```bash theme={null}
# Required environment variables in .env
GROQ_API_KEY=gsk_...          # For policy model inference
SYNTH_API_KEY=sk_...          # For backend authentication
ENVIRONMENT_API_KEY=sk_env_... # Optional - auto-generated if not set
```

Install the Synth AI SDK:

```bash theme={null}
pip install synth-ai
```

## How GEPA Works

GEPA uses evolutionary principles to explore the prompt space. Understanding the algorithm helps you configure it effectively.

### The Optimization Flow

1. Initialize
   * Split seeds into pareto\_seeds and feedback\_seeds
   * Evaluate baseline transformation
   * Generate initial population via proposer
   * Evaluate & add to Pareto archive
2. Evolve (for each generation)
   * For each child:
     * Select parent (instance-wise Pareto sampling)
     * Generate feedback from parent trace
     * Mutate via proposer (LLM-guided)
     * Minibatch gating (quick eval)
     * Full Pareto evaluation (if gating passed)
     * Update archive if non-dominated
3. Terminate
   * Budget exhausted OR
   * Generation limit OR
   * No improvement for N generations
4. Return best transformation, by accuracy

### Key Components

#### 1. Pattern-Based Transformations

GEPA represents prompt changes as transformations that can be applied to your baseline:

```python theme={null}
# A transformation replaces text in your prompt
TextTransformation(
    old_text="You are a helpful assistant.",      # Original text
    new_text="You are a banking classification expert...",  # Optimized text
    apply_to_role="system"  # Only apply to system messages
)
```

#### 2. Pareto Archive

GEPA maintains a Pareto front of non-dominated solutions, balancing multiple objectives:

* **Accuracy** (primary) – Task performance
* **Tool call rate** – Function calling frequency (for agentic tasks)

Solutions are kept if they're not dominated by any other solution across all objectives.

#### 3. Instance-Wise Parent Selection

Unlike traditional selection that uses aggregate scores, GEPA counts how many individual seeds each prompt "wins" on:

```python theme={null}
# Parent selection weights prompts by per-seed wins
wins = count_seeds_where_prompt_is_best(prompt)
selection_weight = (wins + ε) ** selection_pressure
```

This favors prompts that excel on specific example types, encouraging specialization.

#### 4. LLM-Guided Mutations

The proposer (meta-model) generates new prompts by analyzing:

* Current instruction (baseline)
* Rollout examples (input/output/feedback for each seed)
* Trace feedback (e.g., "model under-utilizes tools")
* Dataset and program context

The proposer uses **instruction typology** to structure outputs with: input descriptions, core task, premises, heuristics, constraints, rules, and output descriptions.

## Step 1: Create a LocalAPI

Your LocalAPI evaluates prompts by running rollouts and returning scores. See [LocalAPI Guide](/sdk/localapi/in-process) for details.

**Example Banking77 LocalAPI structure:**

```python theme={null}
from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()

class RolloutRequest(BaseModel):
    seed: int
    run_id: str
    # ... other fields

@app.post("/rollout")
async def rollout(request: RolloutRequest):
    # 1. Load example for this seed
    example = load_example(request.seed)

    # 2. Call your LLM with the prompt (interceptor handles substitution)
    prediction = await call_llm(example.query)

    # 3. Score the prediction
    correct = prediction == example.expected_label

    return {
        "metrics": {"correct": correct},
        "outcome": 1.0 if correct else 0.0
    }
```

## Step 2: Deploy Your LocalAPI

The Synth AI backend needs to reach your LocalAPI over the internet to send rollout requests. Use the Python SDK's `InProcessTaskApp` for seamless deployment with automatic tunneling:

```python theme={null}
import os
from synth_ai.sdk import InProcessTaskApp

# Import your task app
from my_task_app import app

async with InProcessTaskApp(app=app) as task_app:
    print(f"Task app running at: {task_app.url}")
    # task_app.url is a SynthTunnel URL like https://st.usesynth.ai/s/rt_...
```

This:

1. Starts your LocalAPI locally
2. Creates a SynthTunnel URL (or another tunnel backend if configured)
3. Returns the tunnel URL via `task_app.url`

### Verify the Deployment

Check that your LocalAPI is accessible:

```bash theme={null}
curl https://st.usesynth.ai/s/rt_.../health
```

### Alternative Tunnel Backends

SynthTunnel is the default and recommended backend. If you need a Cloudflare tunnel instead:

```python theme={null}
# Cloudflare quick tunnel (ephemeral, requires cloudflared binary)
async with InProcessTaskApp(
    app=app,
    tunnel_backend="cloudflare_quick",
) as task_app:
    print(f"URL: {task_app.url}")

# Cloudflare managed lease (stable hostname)
async with InProcessTaskApp(
    app=app,
    tunnel_backend="cloudflare_managed_lease",
) as task_app:
    print(f"Stable URL: {task_app.url}")
```

## Step 3: Create the Configuration

Create a TOML file defining your optimization parameters. The `task_app_url` should match the URL from Step 2 (stored in your `.env` as `TASK_APP_URL`):

```toml theme={null}
[prompt_learning]
algorithm = "gepa"
task_app_url = "https://my-company.usesynth.ai"  # From TASK_APP_URL in .env
task_app_id = "banking77"

# Training seeds (used during optimization)
evaluation_seeds = [50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79]

# Validation seeds (held-out for final evaluation)
validation_seeds = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49]

# Initial prompt template
[prompt_learning.initial_prompt]
messages = [
  { role = "system", content = "You are a banking intent classification assistant." },
  { role = "user", pattern = "Customer Query: {query}\n\nClassify this query into one of 77 banking intents." }
]

# GEPA-specific configuration
[prompt_learning.gepa]
num_generations = 15              # Evolutionary cycles to run
children_per_generation = 5       # Mutations per generation
pareto_set_size = 20              # Seeds for Pareto evaluation
minibatch_size = 3                # Seeds for quick gating
rollout_budget = 1000             # Total rollouts allowed
archive_size = 64                 # Max Pareto archive size
```

### Configuration Parameters

| Parameter                 | Description                                | Default | Recommended Range |
| ------------------------- | ------------------------------------------ | ------- | ----------------- |
| `num_generations`         | Evolutionary cycles                        | 10      | 5-20              |
| `children_per_generation` | Mutations per generation                   | 5       | 3-10              |
| `pareto_set_size`         | Seeds for Pareto evaluation                | 20      | 15-30             |
| `minibatch_size`          | Seeds for gating evaluation                | 3       | 2-5               |
| `rollout_budget`          | Total rollouts allowed                     | 1000    | 200-2000          |
| `archive_size`            | Max Pareto archive size                    | 64      | 32-128            |
| `feedback_fraction`       | Fraction of seeds for feedback             | 0.3     | 0.2-0.5           |
| `proposer_mode`           | Proposer type (`synth`, `gepa-ai`, `dspy`) | `synth` | -                 |

## Step 4: Launch the Optimization Job

```python theme={null}
import os
from synth_ai.sdk import PolicyOptimizationJob

async def run_optimization():
    client = PromptLearningClient(api_key=os.environ["SYNTH_API_KEY"])

    # Create job from TOML config
    job = await client.create_job_from_toml("configs/prompt_learning/banking77_gepa.toml")
    print(f"Created job: {job['id']}")

    # Start the job
    await client.start_job(job["id"])

    # Poll until completion
    result = await client.poll_until_terminal(job["id"])
    print(f"Best score: {result['best_score']}")
    return result
```

The SDK will:

1. Validate your TOML configuration
2. Verify the task app is reachable
3. Submit the job to Synth AI
4. Poll for completion

### Understanding the Output

During optimization, you'll see progress updates:

```
[18:35:37]    0.0s  Status: running
[18:35:42]    5.2s  Status: running | Best: 0.500
[18:35:48]   11.4s  Status: running | Best: 0.625
[18:35:54]   17.6s  Status: running | Best: 0.750
[18:36:00]   23.8s  Status: running | Best: 0.875
...
[18:38:50]  175.9s  Status: succeeded | Best: 0.875
```

Your LocalAPI logs will show rollout requests:

```
[TASK_APP] INBOUND_ROLLOUT: run_id=prompt-learning-74-5bec8a6f seed=74
[TASK_APP] PREDICTION: expected=card_arrival predicted=card_delivery_estimate correct=False
[BANKING77_ROLLOUT] run_id=prompt-learning-74-5bec8a6f reward=0.0
```

## Step 5: Understanding the Optimization Process

### Generation-by-Generation Progress

| Generation   | What Happens               | Expected Accuracy |
| ------------ | -------------------------- | ----------------- |
| 0 (baseline) | Evaluate initial prompt    | 60-75%            |
| 1-3          | Explore diverse mutations  | 70-80%            |
| 5-10         | Convergence begins         | 80-85%            |
| 10-15        | Fine-tuning best solutions | 85-90%+           |

### How Mutations Are Generated

The proposer receives:

1. **Baseline instruction**: Your current system prompt
2. **Rollout examples**: Input/output pairs with feedback (correct/incorrect, error messages)
3. **Trace statistics**: Tool call rate, trajectory length, etc.
4. **Feedback hints**: Rule-based suggestions like "model under-utilizes tools"

It generates a new instruction following **instruction typology**:

```
[Input Description]
You will be given a customer banking query.

[Core Task Description]
Your task is to classify the query into one of 77 banking intents.

[Premises]
Banking queries often contain domain-specific terminology.
Multiple intents may seem applicable; choose the most specific.

[Heuristics]
Look for keywords indicating the customer's primary need.
Consider the emotional tone to distinguish complaints from inquiries.

[Constraints]
Avoid defaulting to generic intents when specific ones apply.

[Rules]
Output only the intent name, nothing else.

[Output Description]
Return exactly one intent from the predefined list.
```

### Minibatch Gating

Before full evaluation, GEPA performs a quick check:

1. Evaluate child on a small minibatch (3 seeds)
2. Compare to parent's score on the same seeds
3. If child is worse → skip full evaluation (saves budget)
4. If child is promising → proceed to full Pareto evaluation

This saves significant compute by filtering out poor mutations early.

## Step 6: Retrieve Optimized Prompts

After completion, fetch your results using the Python SDK:

```python theme={null}
import os
from synth_ai.sdk import PolicyOptimizationJob

API_KEY = os.environ["SYNTH_API_KEY"]
JOB_ID = "pl_abc123"  # From the job submission output

# Get all results
results = get_prompts(job_id=JOB_ID, api_key=API_KEY)
print(f"Best Score: {results['best_score']:.3f}")

# Get top 5 prompts from Pareto front
for rank in range(1, 6):
    prompt = get_prompt_text(job_id=JOB_ID, api_key=API_KEY, rank=rank)
    print(f"Rank {rank}: {len(prompt)} chars")
    print(prompt[:200] + "...")

# Get scoring summary
summary = get_scoring_summary(job_id=JOB_ID, api_key=API_KEY)
print(f"Train={summary['best_train_accuracy']:.3f}")
print(f"Validation={summary.get('best_validation_accuracy', 0.0):.3f}")
print(f"Candidates Tried={summary['num_candidates_tried']}")
```

### Understanding the Pareto Front

GEPA returns multiple prompts representing different trade-offs:

| Rank | Accuracy | Token Count | Trade-off                   |
| ---- | -------- | ----------- | --------------------------- |
| 1    | 92%      | 450         | Highest accuracy            |
| 2    | 90%      | 280         | Good accuracy, shorter      |
| 3    | 88%      | 150         | Efficient, still performant |

Choose based on your latency/cost requirements.

## Step 7: Use the Optimized Prompt

Replace your baseline prompt with the optimized version:

```python theme={null}
# Before: baseline prompt
system_prompt = "You are a banking intent classification assistant."

# After: optimized prompt (rank 1 from GEPA)
system_prompt = get_prompt_text(job_id=JOB_ID, rank=1)

# Use in your application
response = await openai.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": system_prompt},
        {"role": "user", "content": f"Customer Query: {query}"}
    ]
)
```

## In-Process Optimization

For development and testing, run everything from a single Python script:

```python theme={null}
import os
from synth_ai.sdk import InProcessTaskApp
from synth_ai.sdk import PolicyOptimizationJob

# Start LocalAPI in-process (handles tunneling automatically)
async with InProcessTaskApp(app=my_task_app) as task_app:
    # LocalAPI is now accessible via tunnel
    task_app_url = task_app.url

    # Submit optimization job
    client = PromptLearningClient(api_key=os.environ["SYNTH_API_KEY"])
    job = await client.create_job(config=my_config)
    await client.start_job(job["id"])

    # Poll until complete
    result = await client.poll_until_terminal(job["id"])
    print(f"Best score: {result['best_score']}")
```

See [In-Process Task App Walkthrough](/sdk/localapi/in-process) for a complete example.

## Termination Conditions

GEPA stops when any condition is met:

| Condition              | Description                      | Configuration              |
| ---------------------- | -------------------------------- | -------------------------- |
| `rollout_budget`       | Total rollouts exhausted         | `rollout_budget = 1000`    |
| `max_spend_usd`        | USD budget limit                 | `max_spend_usd = 5.0`      |
| `num_generations`      | Generation limit reached         | `num_generations = 15`     |
| `patience_generations` | No improvement for N generations | `patience_generations = 5` |

## Supported Models

See [Supported Models for Prompt Optimization](/sdk/hosted-optimizers) for the full list of policy models.