CMQ/OE Written Section

4 Replies

CMQ/OE Written Section

Posted by Jeremiah Genest on Jul 24, 2019 1:15 pm

Over on this thread - https://my.asq.org/discuss/viewtopic/39/1308 - it is mentioned that the written section is going away. This seems very counterproductive to me. One of the things that makes the CMQ/OE exam and certification worthwhile was this written which truly go to the heart of the methodology.

I would appreciate hearing more information and justification on this decision and just what this means for the certification.

Re: CMQ/OE Written Section

Posted by Mustafa Pasha on Jul 25, 2019 2:45 am

Hello Jeremiah,

Im glad you brought this up. I myself believe this will be counterproductive as the essay section really tests your knowledge and concepts in quality management system.

I hope they introduce eassys in SSBB exam as well, as giving answers in only MCQs doesnt really test your knowledge.

What do other members think?

regards,
Mustafa

Re: CMQ/OE Written Section

Posted by Carmen O'Neill on Jul 25, 2019 9:32 am

Hello,

Constructed response (CR) items are not providing the differentiation that was once thought by researchers to convey creative and critical thinking. In fact, the very nature of a constructed response is limiting in creativity and critical thinking (e.g. you can only respond in a certain manner based on the scenario presented) and even then, they are limiting in relation to real world job practice. Additionally, there is a multitude of modern psychometric research disputing the merits of this type of testing and they point to many pitfalls including the memorization of common scenarios or tools and limited scope which can create issues with fairness. In addition to pitfalls in testing CRs, there is also that old enemy of testing called rater reliability and its direct impact to scoring & fairness in testing. The inherent subjectivity of human scored responses (no matter how good we try and calibrate them to limiting rubrics), is a major fairness issue and rightly so. Moving to multiple-choice stimulus item scenarios will test equivalent cognitive abilities, improve the reliability of exam forms and increase fairness in testing – that is the bottom line.

The new testing design changes to the CMQ/OE are of the result of increasing fairness in testing. To recap, Constructed response (CRs) will be replaced by multiple/stimulus item scenarios that will challenge candidates at an equivalent cognitive level. These scenarios will be used to answer approximately 10-15 multiple choice questions. Additionally, the total number of questions on the exam has been increased from 150 to 165. (CBT exams will have 15 pre-test/unscored questions for a total of 180 questions).

Kind regards,
Carmen O'Neill
ASQ Psychometrician

Re: CMQ/OE Written Section

Posted by Carmen O'Neill on Jul 25, 2019 10:20 am

Please check out our exam development guide for information about what all goes into developing ASQ exams.

Thank you, Carmen 
Attached files

Re: CMQ/OE Written Section

Posted by Jeremiah Genest on Jul 25, 2019 11:50 am

Constructed response (CR) items are not providing the differentiation that was once thought by researchers to convey creative and critical thinking. In fact, the very nature of a constructed response is limiting in creativity and critical thinking (e.g. you can only respond in a certain manner based on the scenario presented) and even then, they are limiting in relation to real world job practice. Additionally, there is a multitude of modern psychometric research disputing the merits of this type of testing and they point to many pitfalls including the memorization of common scenarios or tools and limited scope which can create issues with fairness.


Carmen,

Thank you for the thorough response. Can you please provide some good references as this is certainly relevant to a team I am involved with at work.

I think some better messaging on this is a good idea, and I know I would appreciate hearing more on the direction for increasing the rigor of the more advanced certifications.

Jeremiah