Pitfalls of benchmarking

1. Not taking the KPI relevance into account.  One KPI in itself will not tell you anything.  It’s always about the interrelations of the different KPI’s and how they influence each other.  The KPI and benchmarking information must be interpreted by an expert, like the way an X-ray needs to be interpreted by a medical specialist.  A framework like the VDM Control Panel that aggregates the information in a comprehensive and organized set of KPI’s is very useful in the analysis.

2. Using the KPI’s can be like comparing apples and oranges.  Analyses using benchmark data are possible only if companies are compared with peer companies in the same industry - a chemical company with a chemical company and a paper plant with a paper plant.  Even within the same industry there can also be differences.  A fine paper plant has far higher benchmark for technical availability than a cardboard plant, for example. So it is important to find the right level for conducting the benchmark. This level will differ from company to company.

3. The absence of definitions for the KPI’s being reviewed.  Benchmark data is often gathered without clear prior specification of the data that must be gathered.  After all, everybody surely knows what maintenance costs are, or how technical availability must be calculated?  In reality, the opposite is true.  Every company applies its own particular definitions invalidating the results of the benchmark. Utilizing a templated tool like myVDM.com uses standardized definitions ensuring that all of the data can be compared accurately.

4. Blindly accepting the output of the study. The benchmark study is a means to an end, not a goal in its own right.  It is like an initial X-ray being used to determine the patient's overall health.  Further investigation must be carried out before any conclusions may be drawn.  Using the medical example this could include talking to the patient and conducting targeted diagnostic tests.  Therefore, benchmark studies must always be accompanied by further analyses.  Methods include interviews with management, but also with representatives of the shop floor.  This enables validation and explanation of measured performances.  A validation of this kind will strengthen the report and increase its acceptance.  This avoids a benchmark report being seen as a grading of performance and ensures it is used more as an improvement recommendation.