Check the model
Before you trust a model, check two things: does it do what you meant, and does it match reality? This is what lets you defend the answer in front of a client.
Two different questions
Simulation people call these verification and validation. The words are easy to mix up, so think of them this way:
| Term | Plain question | Who judges |
|---|---|---|
| Verification | Did I build what I meant to build? | You |
| Validation | Does what I built behave like the real process? | You and the people who know the process |
Do them in that order. Comparing a model with reality is pointless until you know it does what you intended.
Check it behaves as intended
Clear the errors
The chip at the top of the Model editor shows ✓ when all is well. A ⚠ with a number counts errors that stop a run, and What-if Analysis stays disabled until you clear them. A ✓ with a number counts warnings. Click the chip to read them, and use Go to source to jump to the problem where it's offered. Read the warnings as well. They don't stop the run, but they may be telling you something.
Do the arithmetic
Work out by hand what the model should do, then compare. Utilization should be close to arrival rate × processing time ÷ capacity. In our check-in example, that is 1 patient every 5 minutes × 4 minutes ÷ 1 desk, or 80%. The model gave 79%. If those had been far apart, we'd have gone looking for the reason.
Count what goes in and out
On Overview, Throughput shows how many entities finished and how many were created. Entities left over at the end of a run are normal, because work is still in progress. A large gap means work is piling up somewhere. Is that what you expect?
Push it to extremes
Break the model on purpose. These are quick tests with obvious right answers:
| Try | You should see |
|---|---|
| Slow arrivals right down | Almost no waiting. Time in system close to the sum of the processing times. |
| Give a step a very large Capacity | The queue at that step disappears, and the constraint moves somewhere else or vanishes. |
| Make arrivals faster than the process can handle | The queue keeps growing. Verdict shows Overloaded or Fell behind. |
| Send all the work down one path | The other path sees nothing. |
Try these as what-ifs, so your model stays as it is. See Test your changes.
Watch it
Run with Include animation ticked and watch work move through your diagram on the Animation tab. Wrong routing and steps nobody visits are easy to spot by eye, even when they don't stand out in a table.
Compare it with what really happens
Set your targets first
Before you look at results, write down what the model should reproduce. Use numbers the client already trusts. Agree how close is close enough, and write down the conditions, such as "typical weekdays in March".
| Measure | Real value | Close enough | Source |
|---|---|---|---|
| Patients per weekday | 140 | Within 5% | Registration system, March |
| Average door-to-discharge | 95 minutes | Within 10% | Registration system, March |
| Busiest step | Provider exam | Same step | Charge nurse |
Use data you didn't use to build the model where you can. If you set the inputs from March data, check against April. A model will always look good against the numbers it was built from.
Run the Baseline and compare
Look at the spread across replications, the repeated runs of the same period, not only the average. Real days vary too. A real value inside the model's range is a much better sign than one average that happens to match. Also compare the pattern: the right step should be the constraint, and queues should build where staff say they do.
Show it to the people who do the work
Sit down with a supervisor or a front-line expert and play the animation. Ask, "Does this look like your Tuesday?" They'll spot things no dataset shows. You'll also gain allies for when the results come out.
When it doesn't match
Resist the urge to nudge numbers until the output looks right. A model tuned that way matches for the wrong reasons and won't predict the effect of a change. Look for the cause, in this order:
- Inputs. Does a processing time include waiting? Are the arrivals demand or completions?
- Missing logic. Is there a shared resource, rework loop, batch or break the model leaves out? Shared resources need the Intermediate view.
- A flat assumption. Does one capacity number stand in for shifts that really change through the day?
- Start-up. See below.
- The real data. Sometimes the model is right and the client's report is wrong. It happens, and it's useful to find out.
Settings that affect the comparison
These live under Advanced Settings on the Model editor's Basic tab, and need the Intermediate view.
- Replications. How many times the model repeats the run. New models use 10. Use more when results vary a lot from one replication to the next. See Randomness and replications.
- Warm-up Period. A model starts empty. A clinic that opens empty each morning is fine, but a process that runs all the time isn't really empty at the start. The warm-up is the first part of each replication: the model runs but nothing is counted, and statistics restart when it ends. See Warm-up.
When you edit a model after a run, older results still reflect the older version. To bring a scenario up to date, re-run it and tick Use latest model definition.
What you should have now
- A record of what you checked, and what you found
- A table of targets against model results, with sources
- A list of known gaps, and what they mean for the decision
- A Baseline your client recognizes as their process
You don't need a perfect model. You need one that is good enough for this decision, and you need to be able to show why.