AI Scheduling Assistant vs AI Scheduling Agent: The Completed-Booking Test

Run the completed-booking test in eight passes

Use a controlled fixture instead of a vendor demo. Record every human touch, external side effect, state transition, and failure. A good result is not a polished chat response. It is one correct event, the right people invited, and a recoverable history.

1. 1

   Create a hard but ordinary meeting fixture

   Use an organizer and four participants across two companies. Put two required people on Google, one required person on Microsoft, one optional observer on Microsoft, and leave one required external person unconnected. Give the meeting a duration, date window, preferred hours, one excluded day, and a booking deadline.
   • Use test calendars and test identities with approval for outreach.
   • Mark required and optional people before the run.
   • Seed at least two plausible overlaps so slot policy matters.
2. 2

   Give one natural-language instruction

   Say: ‘Book a 45-minute project review next week with these people. Jordan is optional. Keep Friday clear. Morning is preferred. Confirm before the deadline.’ Do not add hidden instructions through the admin panel. The product should either produce a typed brief or ask only for the facts it genuinely lacks.
   • Record how required and optional roles are represented.
   • Record the organizer’s authority and approval mode.
   • Fail the pass if the system silently invents a time zone or deadline.
3. 3

   Inspect connected free/busy behavior

   Confirm the product reads only the approved free/busy evidence from Google and Microsoft. Private titles, descriptions, locations, and attendee lists should not appear in model output, logs, or participant messages. Introduce one provider error and see whether the system says incomplete rather than treating the participant as free.
   • Availability includes source and freshness.
   • Provider failure has a stable reason code or clear operator state.
   • Time zones survive conversion around the chosen slot.
4. 4

   Test the participant with no calendar connection

   The product should use an approved conversational or private response path, state the organizer, duration, date window, and time zone, then normalize the answer into the same availability model. Reply with ‘Tuesday afternoon works’ and omit the time zone. The correct next move is a narrow clarification, not a guess or a generic booking link.
   • Outreach requires consent and offers an opt-out.
   • Reminder count is bounded.
   • The unconnected participant does not need an app account to share availability.
5. 5

   Force a required decline and an optional conflict

   Have one required participant decline the leading slot and make the optional observer busy for the fallback. The product should preserve the required person, accept the optional absence when policy allows, and explain the tradeoff. A system that picks the prettier slot by dropping a required person has failed the test.
   • Required attendance remains a hard constraint.
   • Optional attendance is visible in the proposal.
   • No-overlap produces a bounded exception and a path to change policy.
6. 6

   Replay the booking command

   Approve the selected slot, then repeat the approval or force a worker retry after the calendar provider receives the write. The system should return the same booking result, not create another event. Inspect the idempotency key, meeting revision, provider event identifiers, and reconciliation path for an uncertain timeout.
   • One revision creates at most one canonical event.
   • Duplicate webhooks do not trigger duplicate messages.
   • A timeout after an uncertain write reconciles before retry.
7. 7

   Change one calendar after confirmation

   Create a new conflict for a required participant and request a reschedule. The system should open a new revision, keep the original history, re-use still-valid evidence, collect only what is missing, and update or replace the canonical event according to a documented rule. The organizer should not have to reconstruct the meeting from scratch.
   • The original and revised event relationship is visible.
   • The participant communication uses one canonical update path.
   • The old slot is not left as a ghost event.
8. 8

   Score completion and operator control

   Count human interventions, but do not reward hidden risk. Verify the final URL or event ID, title, start, end, time zone, organizer, required attendees, invitation state, and optional-attendee outcome. Review traces by meeting request ID. A pass needs either confirmed or a specific exception with a safe next action.
   • Every side effect maps to an approved command or policy.
   • Every terminal state has evidence.
   • A human can pause, approve, deny, or reclaim the workflow.

What each category actually completes

Use this table as a product requirement. A tool can be excellent inside one column and still be wrong for an agent that promises a completed group meeting.