PIM reference checks: questions that reveal real performance

Why do vendor references rarely reveal problems?

Vendor references rarely reveal problems because the vendor chooses them, and the vendor chooses its happiest customers. A hand-picked reference is a sales asset. It confirms that the platform can work, but says little about how it will work for you.

Three biases shape a typical reference list. The customers went live recently and still feel the launch energy. The contact is the sponsor who championed the decision. And the vendor often joins or prepares the call.

Reference checks run in week 10 of the 12-week PIM selection process, for the top 2 vendors only. They complement the proof of concept: the proof of concept tests one risk in depth, and references test how the platform and the vendor behave over years.

The goal is to correct the three biases. Choose at least part of the references yourself, speak to daily users as well as sponsors, and keep the vendor off the call. The rest of this page shows how.

How do you get references the vendor did not hand-pick?

Get independent references by setting the criteria for the vendor's list and by adding customers you find yourself. Both routes are standard practice, and serious vendors accept them.

Set criteria for the vendor-proposed references:

• Same industry or a comparable product data model.
• Similar catalogue size, number of locales and channels.
• Live for more than three years, including at least one major upgrade.
• Running the exact edition and version in your quote.
• Implemented by the partner proposed for your project, if possible.

Then find at least one reference yourself. Sources include the vendor's public case studies, user groups, implementation partners that work with several vendors, industry events and professional networks. Events such as Product Content Europe, co-organised by PIMvendors, bring product data teams from many platforms together.

Tell the vendor you will contact references directly. A vendor that resists independent references is showing a red flag worth recording.

Which questions reveal real PIM performance?

The questions that reveal real performance ask about effort, numbers and specific events, never about general satisfaction. "Are you happy with the platform" gets a yes. "How many hours a week does your team spend fixing failed imports" gets evidence.

Follow every answer with a request for an example. A specific event, a number of weeks or a cost turns an opinion into evidence you can compare.

Who should you speak to at a reference customer?

Speak to three roles at each reference: the sponsor, a daily user and the integration owner. Each sees a different side of the platform.

The sponsor explains why the customer chose the vendor, how the business case played out and how the vendor behaves commercially. Ask about renewal, account management and escalations.

The daily user, usually a product data manager or data steward, knows performance, usability and workarounds. Ask about imports, bulk edits, workflows and how long routine tasks take.

The integration owner, often an IT architect or developer, knows the APIs, connectors, error handling and upgrade effort. Ask how failures surface and how much maintenance the integrations need each year.

Three short calls are better than one long call with the sponsor. If the customer offers only the sponsor, ask once for a daily user. Persistent refusal is information in itself, because it often suggests the daily users have a different story.

How do you run and record a reference call?

Run each call with two interviewers, a fixed question set and a written record scored the same day. Consistency across references matters as much as the answers.

Book 45 to 60 minutes per call (PIMvendors estimate). One interviewer asks, the other writes. Start with context: the customer's industry, catalogue size, channels, go-live date and partner. Then work through the questions relevant to the role, ending with the overall verdict.

Record each call in a short template:

1. Reference profile: industry, scale, version, partner, years live.
2. Answers per question, with examples and numbers.
3. Risks confirmed or removed for your project.
4. Open points to raise with the vendor in negotiation.

Score each reference on the same areas as the scorecard: data model fit, process fit, integration, platform and vendor. Feed confirmed risks into the negotiation, where they become contract terms or price arguments.

Thank the reference afterwards, and offer to return the favour. Product data communities are small, and goodwill pays back.

How do you check the implementation partner's references?

Check the implementation partner's references separately, because the partner determines delivery quality as much as the platform. A strong platform with a weak partner produces a weak project.

Ask the partner for two references on the same platform, delivered by the team proposed for your project. Ask how the partner handled scope changes, data migration and the first months after go-live. Ask whether the named consultants stayed on the project to the end.

Compare the partner's estimates with the reference's real figures. A partner whose projects consistently ran longer than planned will likely do the same for you. The choosing a partner page covers partner selection in detail, and the partner scorecard gives criteria to score them.