Sunday, December 24, 2017

Move from too many metrics to few right metrics

Many of us wonder when we get non-conformance report (NCR) during process audits regarding not following the documented process. NCR like "Collection and analysis of metrics is not as per documented process as in QMS" many times common in internal or external audit. Where are we going wrong? Is it wrong metrics proposed to be collected? People are not clear why these data required? or all metrics collected and dumped in some manager's hard disk and no analysis? or metrics collected are inaccurate and looked like collected for the sake of compliance to stated process?

If we do this as survey, you may get equal response to all the above questions when you count number of "yes". How to improve this situation so that people see metrics program in the organization is effective? From my experience metrics are collected to know about where we are with respect to a specific goal set for an attribute? Where we would like to be is the expected improvement goal? What we need to do is the outcome of analysis of metrics trend observed across teams?

Let me give an example why metrics program not implemented correctly which may confuse everyone

Organization has one goal which states "We wanted to grow our business every quarter for all business units". The brainstorming was done and they arrived at let us measure following metrics to see how we perform this year against this goal. The list of measurements selected finally were:
1. No. of new customers added  every quarter
2. Revenue % growth quarter by quarter

CEO was reviewing the reports from two divisional heads of the organization after three quarters, the report summary high-lighted looks like this.
Div 1: 
We added 2 new customers every quarter in the last three quarters
The revenue % growth from quarter to quarter is maintained at 10%

Div 2:
We were able to add 2 new customers in Q1 only but no new customers were added in next 2 quarters
The revenue % growth from quarter to quarter was 15% with existing customers giving more additional business

Which division is doing well?

I know you need some more accurate data to answer this. The key point here is the right metrics are required to measure progress towards team goal and metrics are not the right candidate to reward someone or some team. Normally such metrics program to be used for measuring improvement from baseline value. Division one added customers who may give more revenue in coming quarters but revenue growth % was 10% quarter to quarter. Division 2 though gave 15% growth in revenue quarter to quarter but not adding customers in Q2 and Q3 may not go well with CEO. Also 10% and 15% are not apple to apple comparison as baseline revenues of a division may be different.

May be a goal like min 5 customers/per qtr and revenue growth of 15-25% quarter to quarter as organization goal for this year will really bring meaning to the metrics collection program.

More than someone spending time to collect these metrics the tool they use must help in computing them. Also many times we feel the questions which come during management review had no answers at that point. A good dashboard and reports in tool also is a great advantage covering metrics planned. Sometime helps to derive on te spot answers to management questions. 

Appreciate your views and comments.