Ds Scholarship

Creating worthwhile multiple choice questions for higher education assessment

I remember encountering multiple-choice questions (MCQs) as a student – I assumed the teacher was neglecting his duties as a teacher, either by outsourcing assessment or choosing comfort in getting grades over effectiveness in learning. I’m still instinctively skeptical about their use.

However, three things have changed recently. First, after taking several online courses, I came to see its undisputed benefit in asynchronous learning. Second, I have noticed its effectiveness in the classroom as a fun way for students to check their learning and for me to monitor their progress. And third, the shift to online assessment has removed the traditional proctored test as an option. Now that we have better tools, greater experience, and weaker alternatives, it seems reasonable to be open minded.

Advantages of MCQs

MCQs offer benefits to students and teachers, including:

  • Can be clearer and less ambiguous than open-ended questions
  • They can cover more course content than the number of open questions
  • It removes discretion from labeling and thus generates greater consistency and transparency
  • They automate grading thus eliminating human error.

Note that I do Not Include “Reduce Teacher Commitment Time” in the list above because although MCQs are faster to grade, building and implementing them is more cumbersome than traditional test questions. Thus MCQs benefit trainers who feel comfortable moving labor from Tags exams for Design With them.

Suitability of MCQs

I divided my assessment questions into three categories (based on the work of American psychologist J.B. Guilford):

Driving questions: This tests convergent thinking, where students aim to arrive at the same answer as each other. Solutions must be unambiguous and objectively evaluated. Different teachers should be expected to give identical grades. Convergent thinking verbs include: ChooseAnd DetermineAnd DetermineAnd CalculateAnd Appreciation And Label.

Discussion Questions: These tests test divergent thinking, as students are expected to provide original answers. What constitutes a good answer can be communicated via the marking scheme, but there is plenty of room for students to deviate from one another in their work. Divergent thinking verbs include: CreateAnd I write And Present.

Presentation questions: This can be a command or discussion style question, but students present their solution in the form of a visual. For example, students should create a graph or complete a worksheet.

With regard to Bloom’s taxonomy of learning, the upper two levels, namely, construction and evaluation, are more widely associated with discussion questions, and the lower two levels, comprehension and recall, are more suitable for guiding questions. I believe that when well built, MCQs can also occupy the two middle positions, which are Analysis and Application. But this is not a necessary condition for its usefulness.

Provided that other assessment methods are used to check higher level thinking, this article explores the use of MCQs in their appropriate field.

MCQs structure

An MCQ well consists of two components:

1) stem This is the ‘question’ and it should present a problem or situation.

2) alternatives – Options menu for students to choose, containing the correct answer and several Distractors.

Some of the best practice principles include:

The trunk should be brief and meaningful It should contain only relevant information and focus attention on the learning objective without testing reading skills. It should be meaningful when read on its own, and should seek to provide a direct test of students’ understanding, rather than calling for vague thinking on a topic.

The material should relate to the course content – Questions should find a balance in their relevance to the course content and not a trivial memory test. If the question is about a simple definition, or anything that can be searched on Google, it calls into question the value of the course. Or, if the teacher is using a set of questions from a textbook, perhaps the students should study the textbook directly. Having a set of general and widely used questions creates an unfair advantage for students with prior background knowledge. The questions should include some nuance regarding what was taught, to discover if the students are actively involved. For example, there is a difference between the question “Which of the following are stakeholders” and “Which, according to the lectures, should be considered stakeholders”.

Consider taking the two-step Texas Top Level Assessment As instructional designer Mike Dickinson explains, this is a way to increase the ability of MCQs by offering a higher level of thinking. The idea is that although students cannot “describe” a concept if they have predefined alternatives, they do can Select the best description. MCQs do not allow them to provide an explanation, but they can “select the most accurate explanation”. Think about what you want the students to do and what verb is associated with it, i.e. describe or evaluate, then change this to the noun form i.e. describe or evaluate when writing the MCQ.

Avoid negative language Provide a list of options and ask which one is Not Correction adds a layer of complexity and therefore difficulty, but especially for non-native English speakers, it can move the focus toward reading comprehension rather than subject understanding. If negative phrasing is necessary, using an emphasis slash makes sense, for example, “Any of the following statements False: “

Avoid primary or internal voids Asking students to fill in the missing words moves the cognitive load far Mastery of knowledge of a particular topic. In many cases, stems can be rewritten to retain the intent of the question.

All alternatives must be reasonable There is nothing wrong with using dispersants as bait. Instead of listing a correct answer and many random words, consider every distraction. Bram argues that “common student errors provide the best source of distraction” and as long as they are real errors and not evidence of poorly formulated questions, this is true. If you ask “Which of the following, according to the course material, is an element of justice? Strength; b) reciprocity. c) respect. d) equivalence” A student who chooses “c) respect” may be frustrated if he does not score a point and cite With multiple articles online arguing that the concept of respect is closely related to the understanding of justice. As it already is. But in my class we use a framework that looks at four One of the elements of justice is b) reciprocity and none of it c) respect. The fact that one can make a reasonable argument as to why respect is such an important component of what constitutes justice is what makes it such a good distraction.

The alternatives must be reasonably homogeneous The presence of significantly different options can serve as clues about the correct answer, so the alternatives offered must be reasonably similar in language, form, and length. Brilliant and savvy students should not outperform the most naive.

B does not always make the correct answer – When I create MCQs, I remember a former student “revealing” his strategy of always choosing option B, since he was in a graduate course, this strategy must have proven to work.

Use caution when using “all of the above” or “none of the above” as options – This is not considered a best practice as it allows students with partial knowledge to infer correct answers. However, when used intentionally, it can lead to a closer interaction with the question by forcing students to read it multiple times. This is particularly true with alternatives such as “a) a and b; b) b, c) none of the above. d. all of the above.

Change the number of distractions Providing four options throughout means that random guessing equals 25 percent of the exam. Thus, introducing more distractions forces students to confront each one and choose something. When presented with more options, students are more likely to try to answer the question themselves, and then see if their answer is listed, rather than reverse engineering each option, to see if it is correct. However, the plethora of options makes it difficult to maintain reasonableness and homogeneity. So varying the distractors, from 2 to 5 depending on the question, is quite appropriate.

Anthony C. Evans is Professor of Economics at the University of ESCP Business School.

This tip is based on his blog, “Creating Good Multiple Choice Questions Exams”.

LEAVE A REPLY

Please enter your comment!
Please enter your name here