Assuring a supplier-led migration: the questions to ask
When a supplier migrates a service for you, the risk stays with you. These are the questions a client-side architect should be asking, and when.
Why assurance is your job
A supplier is accountable for delivering what the contract says. You remain accountable for the service, its users and its data. Assurance is how you find out, early enough to act, whether what is being built will meet the need. It works best as a steady conversation through delivery, with findings in writing, and worst as a single review just before go-live.
Before designs are approved
- What does success look like? Ask for measurable criteria: performance, availability, recovery times, cost to run. Without them, nobody can say whether the migration worked.
- What has been left out? Integrations, batch jobs, reporting and support tooling are the usual gaps. Ask for the list of everything the current service does and where each item goes.
- How are the security controls evidenced? Ask to see how each control is implemented and tested. A statement that the design is compliant is where the conversation starts.
- Who runs it afterwards? If your own team will operate the service, they should be reviewing the design now.
Before cutover
- What is the rollback plan, and has it been tested? This is the question that matters most. A rollback that has never been run should be treated as not existing.
- How will the data be reconciled? Agree how you will prove that what arrived matches what left: counts, checksums and sampled records.
- Has the cutover been rehearsed? A full rehearsal shows how long each step really takes and where the plan depends on one person.
- What are the go and no-go criteria, and who decides? Agree them before the day, while everyone is calm.
After go-live
- Is it meeting the criteria you set? Check against the measures agreed at the start, using real monitoring data.
- What is it costing to run? Compare actual spend with the estimate in the first full month, while there is still appetite to tune it.
- Has the knowledge transferred? Runbooks, diagrams and decision records should be in your hands, and your team should have used them.
How to raise a concern
Raise it early, in writing, with the evidence and a suggested way forward. State the consequence in terms the programme board cares about: delay, cost, risk to users. A concern raised this way gives a good supplier the chance to fix it, and gives you a record if it isn't fixed.