AI Scheduling Agent Service Level Objectives: Measure Booked Meetings, Not Tool Calls

Define outcome-level scheduling SLOs in eight steps

Use a corpus of hard meetings with three to eight participants, required and optional roles, Google and Microsoft calendars, unconnected participants, time-zone boundaries, approvals, retries, reschedules, and cancellations. Do not choose a target percentage until the event model and baseline can tell success from a false green.

1. 1

   Define the meeting contract and terminal states

   Persist authenticated organizer, purpose, duration, bounded date range, deadline, IANA zones, required and optional participants, approved substitutes, connected sources, outreach policy, approval mode, organizer calendar, and terminal proof. Define terminal states such as booked-and-verified, canceled-under-authority, declined, expired, reclaimed, and bounded-unresolved exception.
   • A poll winner, suggested slot, drafted message, or tool response is not terminal success.
   • Required attendance and optional decisions are visible in the completion test.
   • Every failure state names the participant, provider, policy, or authority dependency that owns it.
2. 2

   Choose service-level indicators along the whole execution path

   Instrument accepted intent, participant identity resolution, authorized outreach, connected free/busy freshness, conversational reply freshness, required overlap, proposal integrity, approval validity, booking idempotency, provider reconciliation, invitation coverage, and terminal-state accuracy. Keep both counts and latency distributions where time matters.
   • Each indicator has a numerator, denominator, inclusion rule, and event source.
   • Unknown provider or participant state cannot be counted as success.
   • A duplicate booking fails booking integrity even when one event is correct.
3. 3

   Separate machine processing from human and policy waits

   Track compute, queue, provider, participant, approval, and exception time independently. A participant taking two days to answer should not hide a worker that spent forty minutes retrying the wrong calendar. A fast API should not hide an outreach message that never reached the required person.
   • The clock pauses only under an explicit published measurement rule.
   • Deadline miss remains visible even when most elapsed time belonged to a person.
   • Operators can see which wait resumes automatically and which requires judgment.
4. 4

   Segment by hard-meeting shape before averaging

   Break out required-person count, optional-person count, Google-only, Microsoft-only, mixed provider, cross-company, connected-only, optional-sync fallback, IANA-zone span, recurrence, approval mode, and reschedule or cancellation paths. Overall averages can stay green while the durable product wedge fails.
   • Mixed-provider meetings have their own completion and recovery view.
   • Unconnected-participant results are not blended away by easy internal meetings.
   • Low-volume segments carry confidence and sample-size context.
5. 5

   Set objectives from evidence, not a round-number reflex

   Run the current path against controlled fixtures and real consented traffic, establish a baseline, choose the user harm each objective limits, and document the measurement window. Avoid invented speed or completion claims. Raise targets only when event quality, sample size, and operational response can support them.
   • Every objective names why it matters to the organizer or participant.
   • The target excludes no hard cases merely to protect the number.
   • Editorial claims remain separate from internal reliability objectives.
6. 6

   Build an error budget around failed meeting outcomes

   Spend budget on stale evidence bookings, missing required invitations, duplicate events, unauthorized outreach, unresolved provider side effects, wrong time zones, consent breaches, and terminal-state errors. Define which releases pause when budget is exhausted and which repairs take priority. Raw 429s matter when they threaten these outcomes.
   • A recovered provider throttle need not consume the same budget as a missed invitation.
   • Security, consent, and privacy failures can have zero-tolerance policies outside ordinary budget tradeoffs.
   • The release rule is written before a launch deadline applies pressure.
7. 7

   Trace one meeting from instruction to read-back proof

   Emit correlated events for intent revision, identity, free/busy source and retrieval time, participant outreach and reply, time-zone normalization, proposals, approvals, provider writes, idempotency keys, retries, reconciliation, invitations, and terminal state. Redact calendar and message content while preserving causal evidence.
   • Metrics, logs, and traces join through durable meeting and operation identities.
   • Duplicate worker deliveries remain visible without double-counting progress.
   • An operator can reconstruct why a meeting was marked complete.
8. 8

   Run failure drills and publish a completion scorecard

   Inject stale calendars, throttling, duplicate replies, ambiguous zones, unconnected required people, revoked consent, expired approval, hidden event-write responses, provider changes after proposal, and worker restarts. Report booked-and-verified, bounded exceptions, false success, duplicate write, missing invitation, and recovery results by segment.
   • Every drill ends with one verified invitation set or an accurate non-success terminal state.
   • Error-budget policy produces an actual release or repair decision.
   • The scorecard cannot be satisfied by more tool calls or shorter model turns alone.

Compare reliability by the boundary each team operates

The right SLO matches the promise. If the promise is a booked hard meeting, tool availability and model latency are supporting indicators.