In my last blog post I continued the series on the exam development process at Microsoft Learning. I began by asking if you had ever wondered how a Microsoft certification exam comes into being? Following are steps 7-12 (Steps 1-3 were in Part 1 and steps 4-6 were in Part 2). As always, I thank the fine folks at: http://borntolearn.mslearn.net for the information for this post..
Phase 7: Exam Item Selection
The results of the beta exam are analyzed to determine which items should be included in the live item pool.
Phase 8: Standard (Cut-Score) Setting
Once the final item pool has been established, the cut score is set using industry recognized methodologies. Regardless of the method used, the cut score is always based on SME input of some kind—cut scores for MCP exams are never set in an arbitrary manner.
Phase 9: Forms Assembly
After the item selection and standard setting meeting, the final item pool and cut score information are used to assemble forms, or instances, of the exam. If an exam has multiple forms, these forms are designed to be psychometrically equivalent in terms of difficulty, meaning that one form is not harder than another.
Phase 10: Exam Publication
This phase is self explanatory. Prometric publishes the exam, and it becomes available for candidates to take.
Phase 11: Translations/Localization
Localization is handled by an external vendor. Microsoft Learning exams are typically translated into French, German, and Japanese; additional languages are added based on market research, reach, and/or other strategic initiatives.
Phase 12: Sustained Engineering
This is technically not a phase of exam development. Rather, it’s a phase of the exam lifecycle that is critical to maintaining the quality of the exam, which is why I like to include it as part of our process.
Exams are psychometrically analyzed at least annually. Microsoft Learning reviews the results of the candidate satisfaction survey (the survey you are invited to complete via email after you take an exam—your feedback drives change, so take a minute to complete it the next time you take an exam) and other sources of candidate feedback to identify ways to improve our exams.