PIM demo scorecard: weights, a 0 to 4 scale and scoring sheets

How does the PIM demo scorecard work?

The scorecard has three layers: a 0 to 4 score per scenario, a weight per scenario taken from requirement priority, and a roll-up into four domains. It turns half a day of demo per vendor into a comparison the board can audit.

The scorecard belongs to the scenario-based demos step in weeks 8 and 9 of the 12-week PIM selection process. It assumes you have validated, prioritised requirements and a script of 8 to 12 scenarios, for example from the Demo Theatre scenario library.

Four rules apply throughout:

1. Every scorer scores each scenario immediately, individually and before discussion.
2. Every score has a one-line reason based on what the scorer saw.
3. Scorers score only the scenarios in their area.
4. Knock-out requirements are marked before the demos, never after.

Copy the tables below into a spreadsheet. Each vendor gets its own copy of the scoring sheet, and the roll-up compares them side by side.

The 0 to 4 scoring scale

Use five points with a written anchor for each, so every scorer applies the same meaning. The anchors focus on what the vendor showed live, and on who could do it.

Score what you saw. A promise to show it later scores as not shown. If a vendor completes the scenario in a follow-up session, rescore it then and note the date.

How are weights linked to requirement priority?

Each scenario's weight is the sum of the weight points of the requirements it tests, so business priority drives the ranking. This prevents an impressive extra from outweighing a failed core capability.

An illustrative example: a scenario linked to two must-haves and one should-have has a weight of 5 + 5 + 3 = 13. A scenario linked to two could-haves has a weight of 2.

The weighted score per scenario is the score multiplied by the weight. The maximum per scenario is 4 times the weight. The requirements prioritisation page explains how to set priorities that hold up.

Knock-out rule: if a vendor scores 0 or 1 on a scenario linked to a knock-out requirement, the vendor leaves the shortlist, whatever its total.

Scenario scoring sheet

Fill in one row per scenario per scorer, during the session, and consolidate after each vendor. The sheet below is the template for one vendor.

After each vendor, the selection lead consolidates:

Use the median of the individual scores as the consolidated score. When scores differ by two points or more, discuss the evidence and record any change with a reason.

Domain roll-up and vendor comparison

Roll weighted scores up into the four PIMvendors domains, so the comparison shows where each vendor is strong or weak. A single total hides that a vendor may lead on integration and trail on the data model.

Assign each scenario to the domain of its main requirement. Then sum per domain:

Compare vendors side by side:

Add the count of unresolved red flags from the Red Flag Register. Scores measure capability, and red flags capture behaviour around it.

Combining the demo with the RFP, references and proof of concept

The demo carries the largest share of the evaluation because it is evidence on your own data, and the other components confirm or adjust it. Agree the weights before the RFP goes out.

Keep cost separate from the quality score. Mixing price into the weighted total lets a cheap offer hide a weak fit. Compare total cost of ownership side by side with the quality ranking, and decide on both in week 12.