Reinforcement Learning Task App

Contract checklist for RL task apps served through the Synth SDK.

Authenticated routes require an X-API-Key header set to your ENVIRONMENT_API_KEY (run uvx synth-ai setup to provision credentials).

RL task apps must supply a fully populated TaskAppConfig so trainers can enumerate environments, request seeds, and execute rollouts.

Core contract

• app_id, name, description: identifiers surfaced by / and /info.
• base_task_info: TaskInfo describing the task, environment, dataset split, rubrics, inference target, limits, and any task-specific metadata.
• describe_taskset() → returns the taskset descriptor consumed when /task_info is called without a seed.
• provide_task_instances(seeds) → yields a TaskInfo for every requested seed; expect seeds 0..9 during validation.
• rollout(request, http_request) → executes the environment and returns a RolloutResponse.

Rollout expectations

• pipeline_metadata.inference_url must include a ?cid= parameter.
• Every step’s info.meta.inference_url must reproduce the same correlation id so traces join with inference logs.
• Populate legacy trajectories alongside the v3 trace payload until trainers drop the old format.
• Set metrics, branches, and ops_executed accurately; downstream evaluation relies on these aggregates.

Tracing knobs

• Honour TASKAPP_TRACING_ENABLED; the CLI toggles this when users request tracing.
• When TASKAPP_SFT_OUTPUT_DIR is present, emit JSONL snapshots via unique_sft_path to support offline analysis.
• Respect TURSO_LOCAL_DB_URL / SQLD_DB_PATH when persisting traces to a local database.

Optional vendor proxying

• If you expose OpenAI/Groq proxy routes, configure proxy on TaskAppConfig and ensure vendor keys are available via normalize_vendor_keys().

Modal deployment (optional)

• Provide a Modal app module (modal.App, @modal.asgi_app) plus a ModalTaskAppConfig (CLI binary, app name, mounts, secrets) when deploying to Modal.
• The Modal image must mount the repo and receive ENVIRONMENT_API_KEY via inline secret or Modal Secret.

FastAPI endpoints:

• /
  • Method: GET
  • Returns:
  { "status": "ok", "service": str }
  

• /health
  • Method: GET
  • Returns:
  {
      "healthy": bool,
      "auth": {
          "required": bool,
          "expected_prefix": str
      }
  }
  

• /info
  • Method: GET
  • Returns:
  {
      "service": {
          "task": {
              "id": str,
              "name": str,
              "version": str
          }, 
          "version": str
      },
      "dataset": {
          "id": str,
          "splits": list[str],
          "default_split": str
      },
      "rubrics": {
          "outcome": { "sections": list },
          "events": { "sections": list }
      },
      "inference": {
          "url": str,
          "model": str
      },
      "limits": {
          "max_llm_calls": int,
          "max_turns": int
      }
  }
  

• /task_info
  • Method: GET
  • If no seed param, returns:
  {
      "taskset": {
          "id": str,
          "split": str,
          "cardinality": int,
          "metadata": { "difficulty": str }
      }
  }
  

  • If seed param, returns:
  {
      "task": {
          "id": str,
          "name": str,
          "description": str,
          "version": str
      },
      "environment": str,
      "dataset": {
          "id": str,
          "name": str,
          "split": str,
          "version": str
      },
      "rubric": {
          "id": str,
          "name": str,
          "sections": list
      },
      "inference": {
          "model": str,
          "base_url": str
      },
      "limits": {
          "max_llm_calls": int,
          "max_turns": int,
          "timeout_seconds": int
      },
      "task_metadata": {
          "seed": int,
          "difficulty": str
      }
  }
  

• /rollout
  • Method: POST
  • Returns:
  {
      "run_id": str,
      "trajectories": [
          {
              "env_id": str,
              "policy_id": str,
              "steps": [
                  {
                      "obs": object,
                      "tool_calls": list,
                      "reward": float | null,
                      "done": bool,
                      "truncated": bool | null,
                      "info": {
                          "meta": {
                              "inference_url": str
                          }
                      }
                  }
              ],
              "final": object | null,
              "length": int,
              "inference_url": str,
              "decision_samples": list | null
          }
      ],
      "branches": object,
      "metrics": {
          "episode_returns": list[float],
          "mean_return": float,
          "num_steps": int,
          "num_episodes": int,
          "outcome_score": float | null,
          "events_score": float | null,
          "details": object
      },
      "aborted": bool,
      "ops_executed": int,
      "trace_correlation_id": str | null,
      "trace": object | null,
      "pipeline_metadata": {
          "inference_url": str
      }
  }