KPI for performance evaluation in ISO 9001

14 Replies

KPI for performance evaluation in ISO 9001

Posted by Seyid Aghayev on Oct 10, 2018 10:43 am

Some companies especially metal construction, define KPI for their welding process as "keep the monthly rework rate below or equal to 2%". They also present it as a quality objective. They monitor this data monthly and if rate becomes over the 2% they will take corrective actions. Within the 3 year period they tried to keep the rate below 2% and achieved it. When we investigate their rework rate data, observed that sometimes there was increasing or decreasing trends in monthly rework rates but all was below 2%. They stated that they will not take any improvement to decrease this rate or define less rework rate as a target, justify that it may be not achievable.
Question: Is this situation ensure continuous improvement? (the target is rework rate below 2%), if not how can we ensure continuous improvement for welding rework rates? May it quality objective?
What are your opinions?

Re: KPI for performance evaluation in ISO 9001

Posted by Amanda Foster on Oct 11, 2018 9:52 am

Seyid Aghayev‍, I have been considering this question for you for the past day. Here's what I think. It appears to be an appropriate objective for the process, but I don't see how it is evidence of continuous improvement unless the rates are actually regularly improving. I may be wrong on this, but that's what I think.
Amanda Foster, ASQ CQA

Re: KPI for performance evaluation in ISO 9001

Posted by John Chivers on Oct 11, 2018 10:32 am

Continuous improvement is just that, continuously improving.

What you are describing is effective performance evaluation in the eyes of ISO 9001. There is an established target to meet business plans, it is monitored and countermeasures are put in place if it goes over the threshold.

Remember, there is a business to be run here. You have to balance the cost of poor quality with the cost of preventative quality. In this case it appears management don't want to just throw more resource at improving this specific metric and the return is not enough to warrant continuous CI work.

"Is the process capable of achieving a lower target?" you can work that out yourself using statisical analysis. But do be mindful of focusing on what matters most, it is clear that your management don't think that this is a critical issue that requires more work/review than is already happening.

Re: KPI for performance evaluation in ISO 9001

Posted by John Vandenbemden on Oct 11, 2018 11:25 am

With regard to C.I. there are two common metrics:  
1) The first is very very weak, and that is the number of C.I. activities or projects that have been completed or are in process.  My opinion, I personally view  this is as an attempt to meet the requirements without really making change.
2) The second is tangible results, which are normally qualitative, such as 
  • The project is to reduce scrap by 2% before the end of the year.  At the end of the year the scrap rate was reduce by 2.1%.  
  • Increase employee satisfaction from the last survey this year.  Numerous activities were defined and implemented satisfaction on a 5 point rating, 5 being excellent went from 2.6 to 3.5
These measures can be monitored weekly, monthly or quarterly depending on the organization.  The implementation plans can also be tracked for the number of successful tasks completed, creating a sub-level of monitoring

 
John Vandenbemden

Re: KPI for performance evaluation in ISO 9001

Posted by Seyid Aghayev on Oct 11, 2018 3:02 pm

...John Chivers:
"Is the process capable of achieving a lower target?" you can work that out yourself using statisical analysis. But do be mindful of focusing on what matters most, it is clear that your management don't think that this is a critical issue that requires more work/review than is already happening.

Thank you for your reply,
"Is the process capable of achieving a lower target?" this will direct us to pursue this issue.

Re: KPI for performance evaluation in ISO 9001

Posted by Seyid Aghayev on Oct 11, 2018 3:44 pm

Thank you all, for your valuable replies. 

Re: KPI for performance evaluation in ISO 9001

Posted by Ernest Phoon on Oct 11, 2018 7:31 pm

Dear Seyid Aghay,

I have experimented with a linear regression line plotted on an statistical  process control framework to demonstrate the overall continual improvement trend. Preferably you don't minus off the trend and plot it on a standard spc chart as it tends to confuse the auditors... or 

 devildo it if you want to spend a little more time eating up their auditing time ... 

Then when the trend line begins to plateau (bottom out) or as John Chivers commented

"Remember, there is a business to be run here. You have to balance the cost of poor quality with the cost of preventative quality."

We accept the risk of the kpi metric and put it on tracking mode.

A new kpi will then be used in its place.

Yours sincerely,
Ernest




 

Re: KPI for performance evaluation in ISO 9001

Posted by Paul Ipolito on Oct 11, 2018 9:49 pm

I am curious about what the 2% represents. Rework cost as a percentage of sales? A welding-related metric? No matter. I would not be satisified with a supplier who is comfortable with this steady state. Especially in an area such as welding operations where there are so many controllable variables and opportunities for almost immediate feedback on process changes.

Re: KPI for performance evaluation in ISO 9001

Posted by John Chivers on Oct 12, 2018 2:03 am

John Vandenbemden:
With regard to C.I. there are two common metrics:  
1) The first is very very weak, and that is the number of C.I. activities or projects that have been completed or are in process.  My opinion, I personally view  this is as an attempt to meet the requirements without really making change.
2) The second is tangible results, which are normally qualitative, such as 

  • The project is to reduce scrap by 2% before the end of the year.  At the end of the year the scrap rate was reduce by 2.1%.  
  • Increase employee satisfaction from the last survey this year.  Numerous activities were defined and implemented satisfaction on a 5 point rating, 5 being excellent went from 2.6 to 3.5
These measures can be monitored weekly, monthly or quarterly depending on the organization.  The implementation plans can also be tracked for the number of successful tasks completed, creating a sub-level of monitoring
For the first metric, we have moved on to a savings/avoidance-based measure. Here are some suggestions:
  • You can use ROI to measure the macro of a 6 sigma project outcomes,
  • Proportion of qualified belts per head for coverage
  • For target setting you can pick a percentage of turnover as a target (2% turnover in savings over the year)
  • During projects you can use Project Management metrics such as earned value (in the case of booked projects) or a simple measure of schedule (how far ahead/behind you are in man-hours)

Re: KPI for performance evaluation in ISO 9001

Posted by Ernest Phoon on Oct 12, 2018 4:53 am

Dear Seyid Aghay,

Sometimes we should take reference from the industry experts on the state of the art of their craft on the benchmarks and norms...

I look to the American Welding Society for the reference point in this instance... This particular forum post appears to clarify the rejection rate experienced depending on the type of welding... Luckily, I am not a welder as I did particularly badly when attempting overhead welding...

https://app.aws.org/forum/topic_show.pl?tid=5445

Depending then on the competency of the welders and the actions taken versus the correlation with the improvement trend: from the auditing perspective - was continual improvement sincerely carried out.

Hope this clarifies,
Ernest
 

Re: KPI for performance evaluation in ISO 9001

Posted by Seyid Aghayev on Oct 12, 2018 8:54 am

Paul Ipolito:
I am curious about what the 2% represents. Rework cost as a percentage of sales? A welding-related metric? No matter. I would not be satisified with a supplier who is comfortable with this steady state. Especially in an area such as welding operations where there are so many controllable variables and opportunities for almost immediate feedback on process changes.

Dear Paul, the 2 % is the re-welding rate, calculation of this is changing depends on type of the welding work type. For example if the work is pipe welding "length of re-welding on total welding length, if work is structural welding, number of joints re-welded to total number of welded joints, taken into consideration. Re-welding means it doesn't pass from inspection, and conveying to fabrication to rework.

Re: KPI for performance evaluation in ISO 9001

Posted by Seyid Aghayev on Oct 12, 2018 8:57 am

Ernest Phoon:
Dear Seyid Aghay,

I have experimented with a linear regression line plotted on an statistical  process control framework to demonstrate the overall continual improvement trend. Preferably you don't minus off the trend and plot it on a standard spc chart as it tends to confuse the auditors... or 

 devildo it if you want to spend a little more time eating up their auditing time ... 

Yours sincerely,
Ernest




 

Dear Ernest, By the way I am a Lead Auditor:))

Re: KPI for performance evaluation in ISO 9001

Posted by Seyid Aghayev on Oct 12, 2018 9:04 am

Amanda Foster:
Seyid Aghayev‍, I have been considering this question for you for the past day. Here's what I think. It appears to be an appropriate objective for the process, but I don't see how it is evidence of continuous improvement unless the rates are actually regularly improving. I may be wrong on this, but that's what I think.

This issue was appeared during the audit, when I audited one of the metal construction company, it was also confusing for me to keep main indicator of the process constant e.g below 2%, I couldn't get reasonable reply on continuous improvement.

Re: KPI for performance evaluation in ISO 9001

Posted by Seyid Aghayev on Oct 12, 2018 9:27 am

Please, look at the issue form auditor's view point.
There are many indicator or KPIs that may be defined to monitor and control welding process.
Many of them were reasonable, specific, tangible. But my confusion was, trying to keep target below defined figure (2%).
If improvement is not achievable such as to decrease average re-welding rate monthly (2%), ok, keep it as it was, but for my opinion: monitoring standard deviation of re-welding rates on monthly or weekly base and trying to minimize it may be a good point for continuous improvement.  And using attribute control charts to control process, 

Re: KPI for performance evaluation in ISO 9001

Posted by Joe Wojniak on Oct 14, 2018 12:24 pm

Hi Seyid,
thanks for presenting this engaging topic!  I think we need to first keep in mind the purpose of C.I.  The purpose of C.I. is not necessarily to continuously improve all KPI's.  The strategic objective is that the company remain in business given a competitive marketplace.  I recommend posing this question to the Auditee's executive management: which areas of performance have you chosen to improve that will ensure the longevity of the company?  what is your baseline (current state or current performance) and where do you want to be (future state or goal)?
Joe Wojniak ASQ CMQ/OE, CQE