Why did this happen? Is the backlog value a puzzle where everyone tries to guess an answer? It must be a straight-forward job to get the backlog value, but sometimes the work practices make it a PUZZLE.

What happened here initiated many key questions for key performance indicator (KPI) calculations.What is the definition of a specific KPI? Meaning and characterization included.

1. What are the prerequisites before calculating a specific KPI? Specific preparations in the work processes, flow, practices, or computerized maintenance management system (CMMS) to be ready to measure.
2. Why are we calculating this KPI? What is the value added to the business if we know this value and what will we miss if we don't?
3. What is the calculation methodology? Parameters and equation explained.
4. What are the precise sources for obtaining the accurate data? Persons, CMMS, log files, etc.
5. Who are the contributors and owners of a specific KPI? Stakeholders whose work affects the value.
6. In what stage of the process and how frequently should a specific KPI be measured? Daily, weekly, monthly?
7. What is the relation between a specific KPI and other indicators?
8. What is the plan for overcoming out-of-range values of this KPI? Remedy actions required to reverse the trend, if any.

Let's go back to the maintenance orders backlog example and try to find out how it became a puzzle, guess a solution and review what other puzzles can be found in a maintenance workshop.

• The primary key in a KPI development is that everyone in the organization must have the same understanding. It must be well communicated so everyone is working to achieve the targeted value according to a common shared definition.

For example, let's say we have two categories of backlog: ready and total. The ready backlog includes the jobs that are planned and all job requirements are available, in other words, jobs are ready to be inserted in the schedule. The total backlog includes the ready jobs plus other jobs that are in planning or planned, but still miss some of the required resources (spares not available, awaiting vendor, etc.), in other words, all approved maintenance requests.

What creates the puzzle?
The main reasons that created the differences in the reported values of backlog between the two supervisors are:

• Not all the staff has the same understanding of what the backlog is because it was not well communicated or explained to them. One supervisor calculated the ready backlog and the other calculated the total, regardless of the accuracy of calculations, which will be covered later in this article.While it seems both supervisors used the same source of data, the data source is not reliable enough to produce an accurate report.

• Having multiple data sources can misinform the parties and lead them the wrong way. These sources are within CMMS, private Excel or Access data sheets, private log books and the memory of each person. Imagine if each staff member has all these data sources, what type of accuracy, quality, or integrity can you expect?
• Data corruption factors that created the puzzle, among them:
  • One of the supervisors considered the preventive maintenance records and the other did not. This reflects a poor understanding of what exactly is required about reporting.
  • Job is done, but the work order is not closed in the CMMS. If both supervisors used the CMMS record as is, the backlog is not accurate. If one used the CMMS and the other knows from his Excel data sheet or logbook that the job is already done and excluded this work order from the value, this makes it worse.
  • One of the supervisors downloaded the backlog work from the CMMS to an Excel sheet and made some changes in the planned man-hours and durations because he didn't believe in the estimates of the CMMS. The other supervisor used the data as is; the puzzle started becoming tricky.
  • Some duplicate jobs with different descriptions for the same equipment or the same description used for the equipment and its children.
  • Same preventive maintenance jobs that are issued before completing the previous ones (fixed plan) are included many times by one supervisor, while the other supervisor considered only one event; the puzzle now is a difficult challenge.
  • Some long-pending jobs that are no longer required were considered by one supervisor and ignored by the other; the puzzle becomes more complicated.
  • Some jobs still in the request phase, not yet approved for planning were totally counted by one supervisor and partially by the other; more confusion is added to the puzzle.
  • One of the supervisors had purposely done some data manipulating to either:
    
    • Reduce the backlog value to show maintenance performance was in good shape in order to get personal advantages or avoid blaming.
    • Increase the backlog value to give justification for overtime or additional resources.
  • Many jobs with unclear status are lost in the two versions of the backlog report.
  • Non-official jobs are in progress in the site, but there is no record in the CMMS, making the puzzle cloudier.
  • You can imagine many more scenarios to add mystery to the backlog from the four data sources mentioned above. Try it and see what you find.

At this stage, the big question that comes to mind is this: Is there a maintenance organization that knows what the exact value of backlog is at the end of any working day? I have great doubt about this, but you can obtain a closer to real value backlog using some controls. I've used the backlog as a clear example of the inaccuracies of performance measures in the maintenance function due to poor practices, lack of control and unaccountability on the part of the staff.

The puzzle can be easily expanded to many performance indicators; the inaccuracy of the backlog itself is a symptom of inaccuracies in most of the indicators tied to the work management process.

Add to the previous scenarios other corruption factors as described below:

• Jobs in progress are not the jobs in the approved schedule
• Inaccurate man-hour estimates and durations
• Actual work hours are a "copy and paste" from the planned hours
• Detailed work done is not recorded
• Failure codes and reasons are not entered in the work order
• Materials used from hidden stock are not recorded in the order
• Unused materials are not returned to the store
•  Inaccurate dates of order closing and material transactions
• Blanket work orders that contain a mix of data for different jobs.

You can then forecast that most of the work management indicators or cost indicators are not functioning to your expectations. Samples of these suspected KPIs are:

• PM compliance/schedule compliance/planning accuracy 
• Maintenance costs (for all cost elements and maintenance types)
• Failure and reliability reports

Remember that no single performance indicator works in isolation; all the input and output measures of a process are linked together. Keeping in mind that all maintenance internal processes are interrelated, you can't skip the accuracy in one process because it will pull down all the others. As such, no right decisions can be taken to improve any of the maintenance functions. Imagine the impact of these decisions using these suspected values.

How to solve the puzzle?
Whatever the complexity of the puzzle may be; there are always solutions. I recommend adopting a two-prong approach to solve the puzzle and avoid its re-creation.

1. One focus is on the cultural change of the staff. They need to realize the value of the measures and be accountable when working with the maintenance data. They have to understand that achieving the true target of indicators will impact not only the organization, but its individuals also. Cultural change must also extend to managers so they understand that measures are not a blaming tool. Everyone has to know that work measurements are used to promote improvements in the maintenance process and practices.
2. The second focus is on introducing controls to your work process and data management. Start with KILLING all other data sources and make the CMMS the official and only data source. Then work to improve data accuracy and completeness in the CMMS.

• Enhance your CMMS configuration so it is able to receive and restore the specific data you need. Confirm full implementation with full capacity of the CMMS.
• Build standard reports in the CMMS for all your KPIs, with customized selection criteria that prevents different interpretations of the indicator and individual tailoring. The goal is for everyone to do the same clicks to get the same report.
• Implement regular data audits and validation. They are key to ensuring data integrity of the CMMS and verifying that no maintenance, procurement, or inventory transactions are executed outside CMMS.
• Schedule refresher training for the staff to fill gaps in system utilization.
• Introduce some controls of access and authorization to the system and enforce some mandatory data fields. Too many optional fields sometimes lead to blank records. Automated work order flow is also useful for imposing accountability.
• Consider applying some incentives or enforcement rules; sometimes they are necessary.

Working in these two directions will definitely help to solve the puzzle. Start now by taking count of how many puzzles are in your maintenance function. Perhaps one solution is valid for all, or it may be that each puzzle requires its own solution.

Whatever the case may be, one aspect remains constant. All the puzzles in maintenance are from one root and all the solutions are also from one root. One puzzle will lead to another and one solution will lead to another. ENJOY SOLVING!

Tarek Atout, CMRP is currently working with Qatar Gas in the maintenance excellence team, has 20 years experience working with many international organizations in the Middle East as a consultant in maintenance planning, work management, inventory control and CMMS development and implementation where he participated in improving the maintenance functions by developing the maintenance strategies, planning process and work management flow and roles with associated performance indicators.

Excel™ and Access™ are registered trade marks