Assessments of multiple-choice questions (MCQs) are often viewed with skepticism in higher education. I have always questioned its value until recently, when I began to see its usefulness in certain educational contexts for checking student progress. Non-native speakers often find essay tests to test their English language skills more than their understanding of the content, and they value using exam time to think rather than write.
Companies use MCQs for critical internal compliance training. If we accept the validity of MCQs for important governance issues, I see no reason to separate them from the reach of higher education. I think MCQs have a place, provided this is done we will.
The first step is to understand how to build effective MCQs. I’ve shared extensive advice on this in a previous advice resource, Creating Worthwhile Multiple-Choice Questions for Higher Education Assessment.
Once you have a bank of appropriate questions, the problem becomes how to rank them in an effective test. When using MCQs, I distinguish four main uses:
appearance – This is something students fill out either online or on paper. It’s something they give me to monitor their progress. For example, if I submit a structured assignment, I will ask them to complete a form as they go. This allows me to compare the progress of the different groups and see how much time we need. It is unclassified and helps manage session flow. I usually use Google Forms for MCQ forms. Given that this is without grades, I also include open-ended questions, where students write in their answers.
Test – This is intended to help students check their learning. They can be used at the end of a session to cover recent concepts, or at the beginning of a session to allow time for reflection. They are unrated, low risk and can be fun. I usually use Kahoot! For MCQ competitions.
Test This is primarily to allow students to practice for the test. It has a similar format and uses similar types of questions. I usually use Google Forms, which is set as “Test”, for MCQ tests.
Exam – This is a formal assessment and forms part of the student’s grade. To integrate with the Gradebook, I use whatever learning platform the software uses, such as Blackboard or Canvas.
Note that in this article I focused on the latter two. Here are the main considerations for creating a test or exam with MCQs:
Made a set of difficulty – There is no reason to assume that questions should be of equal difficulty, even if they count for the same number of points. On an essay test, students can get the first 50 percent of the score fairly easily, but every percentage point over 90 percent is progressively harder to achieve. This helps create a normal distribution.
A risk in the MCQ is that teachers do not have the discretion to present a curve by retrospectively modifying the scoring scheme, and therefore great care must be given to the set of questions. When teachers grade essays, they usually limit themselves to a narrow range, ensuring proper distribution. In MCQ, you bring up both ends. In the 10-question MCQ exam, making sure five questions are relatively easy reduces the risk of a large number of failures. Ensuring that at least one question is too difficult prevents scores at 100 percent, denying strong students the ability to distinguish themselves from the rest of the group.
Assigning different points to different difficulty levels will also help with this. For example, many simple questions are worth one point, and many more difficult questions are worth two or more.
Keep the questions independent Having questions follow sequentially creates greater risks for students, as missing one question severely affects the next, and can provide information to smart students reducing their need to use and apply course content, as the information provided in one question can be used as an input answer to another solution.
Multiple questions related to the same exhibit can be used by repeating the exhibit in each question.
Take steps to reduce cheating The main danger of cheating is communication with students. Introducing a mixed order of questions and a mixed order of alternatives within each stem reduces this.
Don’t reveal the answers When using MCQs as a practice test, it is important for students to see their scores, and I think it is helpful to let them know which questions they got right and which were wrong. However, in my experience, revealing the correct answers provides a very easy shortcut for students to take on and off learning actions.
Don’t deny them the chance to pass the test by giving up the solutions!
Do not reveal the grades Ensure that you have a chance to review the distribution of scores, correct any errors in the marking scheme, and finalize any partial credit decisions, before students see their scores.
The points above have convinced me that MCQ tests are a useful assessment method to use. However, I remain cautious and adopt the following rules:
1. Give students a chance to take a practice test before any graded test.
2. Provide clear instructions and communicate them effectively.
3. Monitor results. It is important for teachers to confront their intuition about what constitutes a good or bad MCQ exam. If a large number of students have misunderstood a question, it means that it was incorrectly worded. If all students get the same answer true or false, consider whether it serves its purpose.
The first few times I used MCQs, I misunderstood them. But I try to be a quick learner.
Anthony C. Evans is Professor of Economics at the University of ESCP Business School.