# LLM serving pilot specification

**Unrun protocol using synthetic tasks.** No endpoint, model, vendor account or GPU has been executed for this guide.

## Prerequisites

Choose the exact candidate service/model pair, version/configuration, region, API surface, credentials policy and allowed budget. Record mandatory quality, latency, data and feature requirements. Adapt pilot-fixtures.json to the selected API; it is a task-level fixture, not a vendor request body. Use no real customer records.

## Workload profiles

- Bursty interactive: first one support-draft request, then a controlled twelve-request burst during a chosen active window. Record acceptable first-useful-output and completion times before running. Values are example test cases, not capacity claims.
- Scheduled batch: first one extraction job, then four concurrent jobs using the synthetic records. Define batch size/deadline and acceptance rules before scaling. Measure actual tokens rather than assuming input length from text characters.

## Protocol

1. Establish one accepted response; validate field values as well as output structure.
2. Record warm-state observations and a separate documented idle/scale-to-zero wake-up case. Verify actual endpoint state; do not infer it from a short pause.
3. Increase load within approved provider/account and spend limits. Log raw samples, concurrency, queue time where available, first useful output, full completion and failures.
4. Inspect timeout/retry behavior and supported provider status/idempotency features. A local request ID does not universally prevent duplicate work.
5. Reconcile provider usage/charges and internal logs over the same window, including failed attempts and operating effort.
6. Preserve sample counts and configuration differences. Do not infer stable p95 latency or a universal winner from a few requests.

## Decisions

Reject candidates that fail mandatory compatibility, quality or boundary requirements. For remaining candidates, compare accepted throughput, latency behavior, actual complete cost and operational responsibility. Use unknown/not_run where evidence is absent. Record owner, date, budget consumed, unresolved limitations and the next bounded check.