My DOE Didn't Work - Now What?
I'm pretty sure most practitioners have run across this problem sometime in their careers - I know I have!  You plan the DOE well by having a meaningful response, good factors and levels--and yet, when the results are analyzed, the results are quite disappointing.  What to do now?  Well, there are several options.

The first option is to make sure you have validated your measurement system.  There is no sense rushing into a DOE without doing this step first, because uncertainty in the measurement system only adds to the noise in the experiment and makes it more difficult to discern differences.

The second option is to rethink both the factors and the levels of those factors that were studied in the DOE.  Very possible is the situation in which a factor was overlooked.  Review the manufacturing process in person, query the process engineer and operator again for input, and make sure you looked at factor settings that are more than just the process operating parameters.  That last case might likely be your culprit because the process is meant to be robust and stable withing the operating parameters.

The third option is partly related to the second option, but for a different reason.  This option involves overlooking the statistical significance and instead seeking some logical trending.  What I mean by this is to look at the longest bars in your Pareto of Effects plot.  There are times when a factor has influence on the response, but the influence is not strong enough within your factor levels to trigger statistical significance.  Therefore, the bars for these factors will be long in the Pareto of Effects plot, but not cross the line of significance.  Of course, this may be noise within the experimental data, but if the factor, through physics, was expected to be significant but was not, expand your study by running some trials past the DOE setting that looks like it is making your response trend in the right direction.

The last option is that your process might be at a point of "process entitlement."  This simply means that the process is producing results to the best of its ability with the current design.  At this point, as much as I despise this option, your last recourse would be to change the design in order to meet your performance goals.  I use this option as an absolute last resort because it is the most expensive and takes the most time.  I like free, and adjusting process parameters is often free.

Hopefully, one of these options will help you answer the question about why your first crack at solving your problem with a DOE did not work.  Until next time, I wish you good and effective problem solving.
2 Replies
Certainly "spot on" advice!  I see Belts and others always assuming the measurement system is good and jetting right by this.  I believe this has impacted many DOE outcomes and projects. (See how I went right by the need for a solid measure much less a validated measurement system....). 

Additionally, a weak Analyze phase contributes to not understanding factors and levels. How often have all experienced where a less than rigorous Analyze phase was employed?

Amazing how each part of Six Sigma is needed to arrive at the right answer. "Tailoring" may be needed in large projects utilizing formal project management processes but that concept doesn't seem to work as well in Six Sigma.

Marnie Ham
13 Posts
I would add, that you need to look at the DOE execution - was it completed correctly, was it randomized, do the factors all make sense?  some times just asking the questions can help improve the DOEs.

I reviewed a DOE (post execution) where all of the above were issues.
All of this could have been avoided had they had a pre-execution consultation with me.

I try to emphasize during my GB and BB classes - one session on DOEs doesn't make you an expert.  You are just learning - I have tried to get my students to call me (or email or something else) before they run their DOEs.