Claude Certified Architect — Foundations · Free practice question 10 of 10
Shadow-mode rollout for prompt changes
You are about to ship a major change to your support-classification prompt. Engineering and product agree the change is right, but no one wants to risk a regression hitting all users. The team has logging on classification accuracy with a roughly one-day lag. Which rollout strategy fits these constraints best?
- A.Replace the prompt all at once, since the team agrees the change is right.
- B.Run the old and new prompts in shadow mode (both run, only the old is acted on) for one day, compare metrics, then cut over.
- C.Make the prompt configurable per user and let users opt in.
- D.Run the new prompt on 50% of traffic immediately; if metrics look fine, expand to 100% the next day.
Show answer and explanation
Correct answer: B. Run the old and new prompts in shadow mode (both run, only the old is acted on) for one day, compare metrics, then cut over.
Why: Shadow mode is the lowest-risk option: no user is exposed to the new prompt's outputs until you have confirmed metrics match, which fits the one-day measurement window. Canary or 50% rollouts expose real users to potential regressions before metrics arrive; opt-in selection is biased.
More free Claude Certified Architect — Foundations questions
- Narrowing label definitions to stop over-triggering
- Severity and confidence metadata for review findings
- Tool-based verification to prevent hallucinated APIs
- Precision vs recall trade-offs in extraction
- Evidence fields for auditable structured output
- Tool schemas for strict JSON conformance
- Routing scarce human review capacity
- Execution-based evals for SQL generation
- Prompt caching for repeated context