Connect with us

Teaching

10 Tips For Creating More Effective Multiple-Choice Questions

Published

on

10 Tips For Creating More Effective Multiple-Choice Questions

contributed by Dr. Stephen Murphy, Measured Progress

One of the critical elements to understanding what students know and are able to do is using spot-on assessment items that provide fair and valid data.

Good data helps teachers make instructional decisions, informing student groupings and identifying needs for additional instruction or stretch strategies for continued learning and growth. To better understand what goes into developing great assessment items, I’ve developed a guide that introduces the basic steps of the process. Here are nine steps, in the order they occur from creation to presentation to the student.

Assessment Quality? 9 Steps To Creating Effective Multiple-Choice Questions

1. Pinpoint the Purpose for Developing the Item

In other words, begin with the end in mind. What new information do I want about the student? What data do I want to collect? What information might I want to report—to the student or parent?

For example, if the goal is to assess grade 3 students’ knowledge of using “multiplication and division within 100 to solve word problems,” the item must be developed and structured to generate specific data about students’ grasp of this content.

2. Determine the Standard(s) to Which the Item Should Align

The purpose of writing an item should be clear; that is, every item should be written to a specific standard that you are interested in assessing. If an educator is interested in knowing whether a student has retained the desired understanding of a specific series of lessons, each item could be written to address one standard from this sequence of instruction.

It’s important for the item author to have clear expectations of the knowledge, skills, and abilities a student must have to demonstrate success with a particular standard.

3. Identify the Appropriate Structure for the Item

The most common structure for items is multiple-choice (MC), consisting of an item ‘stem’ that provides the information necessary to elicit a response, and a number of response options from which the student selects. Typical items for grades 3 through 8 have four response options and a single ‘key,’ or correct response.

In some instances, the content standards require more complex item types such as constructed-response or technology-enhanced items, or direct writing prompts. In choosing the item type, you need to identify the type of response that will allow students to demonstrate their understanding of the content, and determine the resulting scoring rules.

4. Target the item to your state’s Achievement Level Descriptors

Achievement Level Descriptors (ALDs, also called Performance Level Descriptors or Proficiency Level Descriptors) describe what students should know and be able to accomplish to be classified into a meaningful level of achievement. Targeting your items to ALDs can help you address a range of difficulty and give you an idea of what level your students might achieve.

For example, eMPower Assessments™ by Measured Progress have four achievement levels: Below Basic, Basic, Proficient, and Advanced. An item aligned to Proficient would be one that students in Proficient and Advanced should get correct, but not students who have only a basic or below basic understanding of the content.

In general, the verbs we use suggest the level of complexity of an item. For example, items that target advanced skills might ask students to analyze, construct, or interpret. To target proficiency, items might ask students to describe or represent.

At the basic level, students recall and compute. In eMPower, a college and career readiness assessment, students who are proficient in mathematics by the end of grade 4 can “solve multi-step mathematical problems … represent and compare fractions … identify and describe geometric properties … [and] use models to represent and solve nonstandard problems…”

5. Create Strategic Multiple-Choice Items

MC items may seem simple but they need careful development and can provide significant insight if the distractors are strong. Good distractors are response options that reflect common errors in student thinking and therefore are plausible options for students to select if they don’t understand the content being assessed.

Advertisement

Effective distractors help detect students’ specific knowledge gaps. Used in formative practices, these items can help teachers focus instruction and bridge gaps. One approach to creating good distractors is to start with common misconceptions and create options that represent them. Finally, make sure that the wording of the item and distractors doesn’t create a second plausible correct answer.

6. Eliminate Barriers for Students

To provide all students with the best opportunity to demonstrate their mastery of the content standard being assessed, items must be free of bias or sensitive issues that might detract from a student’s ability to engage in the item.

In addition, the item must be written to the specific targeted grade and vocabulary level and be clear and concise, avoiding unnecessary reading burden where reading is not being assessed.

7. Include all Necessary Supports for Generating a Response

While this might seem obvious, it’s important to include all the tools and information that a student needs to interpret and respond to an item. This applies to all item types. Here are some examples.

Does the item require a stimulus, a passage, or a graphic? If so, is all the necessary information provided, and is it appropriate and accessible to the student? Passages should reflect appropriate vocabulary and text complexity. Graphics need to be clear and should not include extraneous information.

If the mathematic content standard allows a student to use a calculator, that must be evident, and the appropriate tool must be available to the student.

8. Use Engaging Material

Students are more motivated to respond carefully if the items are based on contexts students are likely to encounter in their day-to-day lives. If using graphics or illustrations make sure they are appropriate for the grade level and familiar to students from the world around them.

The contexts should be authentic—not far-fetched or unlikely.

9. Make Sure the Students’ Responses will Provide the Right Kind of Information

Time is a precious commodity for educators, so it is important to start this checklist (and now conclude it) with the end in mind. As you plan and write items, continue to reference them against the kinds of information you need. Especially for formative assessment practices, focus on identifying students’ strengths and needs, and on using the items to help you adjust instruction.

BONUS STEP: Implement Quality Review.

Once the items are written, undertake these quality checks.

1. Have another grade/content expert confirm that the content (and the identification of a single correct option for MC items) is correct. This reviewer can solve each problem and identify any gaps or errors.

2. Have someone you trust as an editor make sure that the grammar is correct and the wording is appropriate.

3. Finally, conduct a review of each item in its final published form to verify that it has all come together correctly and is something a student can indeed respond to.

Dr. Stephen Murphy is Vice President, Measurement Services, at Measured Progress. He leads the groups responsible for all aspects of assessment design and development, including psychometrics, content development, publishing, and content management.

Assessment Quality? 9 Steps To Creating Effective Multiple-Choice Questions

Continue Reading
Click to comment

Leave a Reply

Your email address will not be published. Required fields are marked *

Teaching

The Selflessness Of Pedagogy: What Every Teacher Gives

Published

on

denisekrebs

The Selflessness Of Pedagogy: What Every Teacher Gives

by Terry Heick

It’s better to give than to receive–something that holds true for gift-giving and education.

In fact, the act of giving isn’t just a matter of ultimately receiving more, or even an act of altruism and selflessness. It has more to do with seeing the big picture itself — a healthy, robust, functioning system of careful human performance — that becomes the goal. It’s in this simple paradigm shift that we, as teachers, can find a new level of performance as professionals, contentment in our craft, and changed lives in the communities we serve.

So how might we focus on that service not as a characteristic of my job, but as the goal itself–the most macro product of all? For teachers, this would obviously be built around the idea of service-oriented teaching, where we gain strength and perspective from serving others. It would lead to ‘other growth,’ including, on a broader level, the formation of a personal and professional ecology than can sustain us through the challenges of teaching.

What would this look like?

Other teachers who need us, and who we need

Parents who need us, and who we need

Students who need us for something other than clarifying instructions, providing credit, and letter grades

The community of our school, including grade levels above and below ours, that needs us for our performance, collaboration, and ideas, just as we need that community.

In a word, this looks like interdependence. So what can I give to begin?

They give themselves.

Teaching is martyrdom. So often, educators feel the need to give themselves up to be feasted upon until there’s nothing left. Giving yourself is a different kind of gift, though. Here, it means truly putting your self aside — your need to be the best, your insecurities, professional goals, need for affirmation, and so on — and instead give in to the act of teaching.

But more crucially, this giving of yourself implies that you give your whole self to the act of teaching — your creativity, affection, background knowledge, contacts, networks, dreams, hopes, and so on — in the whole merging of you and your work.

They give others the benefit of the doubt.

As a teacher, you’ll see a lot moving upstream and down — struggling readers that always seem to come from that school; that family that doesn’t seem to care; that co-worker who seems to challenge you at every chance; that administrator who always seems to find a way to poke holes in your teaching. That assessment. That law. That policy.

Never, ever stop questioning the things happening around you. Be a critical educator, ask tough questions, and ring the bell when you’re concerned. Just do so from a position of positivity — give others the benefit of the doubt. Use positive presuppositions, such as: “We’ve always been strong supporters of literacy here, so I’m confused why. . . ”

Advertisement

They give themselves the opportunity to learn new things continuously.

Okay, perspective change to first-person: I’m going to give myself the gift of learning. As a teacher, it makes sense to learn endlessly, not just to model it for students, but to keep my own curiosity and tendency for play stirring and alive.

I may learn a new set of literacy strategies. Maybe it’ll be a variation on the Socratic Seminar, or I’ll mash Fish Bowls with Agree/Disagree. I may bring new education technology into my classroom, or reach for new learning models such as project-based learning, sync teaching, or self-directed learning. I may throw out my desk and go paperless, mobile, or completely back to basics.

But I’m never going to stop learning. That is my gift to myself.

They give students a chance.

Maybe that’s a chance to surprise themselves. I want to give the gift of inspiration. Who doesn’t? And what better way can we inspire than by designing learning experiences that let students do things they didn’t think they were capable of? My gift would be giving students opportunities to surprise themselves through their own skills, critical thinking, creativity, and deep understanding of important ideas.

They give parents authentic opportunity to get involved.

It’s tempting to complain about parents that aren’t involved in their child’s schooling. Who on earth wants to be involved in schooling? Learning is a different matter — that’s something with a foothold allowing parents to engage meaningfully. This isn’t grades and homework, but rather understanding and the need to understand.

You may or may not get anything more from parents, but at least you’ve given them more tempting access than they’ve ever had in the past.

They give others chances to create new measures of success.

Teaching — properly done and measured as we do today — is impossible. It can’t be done. You cannot bring every single child to proficiency in every single standard while, at the same time, meeting their needs as human beings and helping them both see and reach their full potential. If this is your goal, you’re only going to disappoint yourself endlessly until you either burn out or realize that you’re lying to yourself.

Teaching, though, with new metrics of success — well, that’s suddenly a whole lot easier.

They give themselves a break.

The preceding represents a lot of giving, and also a lot of complexity, interdependence, opportunity, work, and chances to fail. So above all, I’m going to give myself a break.

I will give all of myself. I will give the benefit of the doubt. I’ll learn new things, promote self-discovery, connect with communities, and establish new measures of success. And when things go wrong, I’ll have a short memory. I’ll give myself a break and push on, excited about what tomorrow might bring.

Image attribution flickr user denisekrebs and cloudboard; What Every Good Teacher Gives

Continue Reading

Teaching

How The Lesson Study Approach Can Improve Your Teaching

Published

on

How The Lesson Study Approach Can Improve Your Teaching

contributed by Vicki Collet

Making teaching a shared and public practice supports improved instruction.

When teachers have opportunities for collaboration, reflection, and inquiry, they “can find better ways to answer the learning needs of students” (Lieberman & Wood, 2002, p. 42). Teacher inquiry makes us question practices that have become stale, generate innovation, and create dynamic learning for ourselves and our students.

Over the past decade, I’ve participated in teacher inquiry using a process called Lesson Study. I’ve seen scores of teachers engage with authentic professional learning through this work. Lesson Study is a structure for teacher inquiry that is as straightforward as it sounds: the study of a lesson. However, Catherine Lewis, who learned about Lesson Study in Japan and has been instrumental in spreading its use in the United States, points out that during Lesson Study,

(Teachers) improve the lessons not as an end in
itself, but as a way to deepen their own content knowledge, their knowledge of
student thinking, their understanding of teaching and learning, and their
commitment to improvement of their own practice and that of colleagues…Lesson
study is not about discovering the one right way to teach a lesson, but about
building knowledge of many teaching strategies and habits of observation,
inquiry, and analysis of practice (Lewis & Hurd, 2011, p. 24).

When participating in Lesson Study, you will collaboratively plan lessons that become the focus of inquiry on effective teaching practices. Then you’ll observe one another teach lessons and collaboratively revise and refine your instructional plans, as shown in the figure below.

How The Lesson Study Approach Can Improve Your Teaching

Inquiry is the essential element of Lesson Study, so the clear focus you identify for a Lesson Study cycle should propel your desire to investigate, to analyze and study together. Pursuing the questions your group has identified provides challenge and authentic purpose, which will increase your team’s motivation and engagement. Because you have identified the problem for yourself, the process will have immediate value. 

The goal of Lesson Study is not to create the ideal lesson that is universally useful. During teacher preparation and professional development, you have doubtless been taught and encouraged to use ‘best practices.’ Although it is true that some practices are more effective than others (and research has much to teach us), it is also true that there is no single, perfect lesson. This is true because you and I, and the students we teach, are diverse in so many ways. Lesson Study focuses on “the why of teaching: why teaching methods work in particular ways in particular settings” (Smagorinsky, as cited in Garcia & O’Donnell-Allen, 2015, p. 5). 

Grow your school with TeachThought PD Collaborative Lesson Study Workshops.

In our quest for better ways of doing things, Lesson Study can help us identify places where we are frozen in old practices and can help us warm to new ones. Lesson Study uncovers new approaches that are informed by the examination of evidence gained through our professional inquiry. Collaboration with colleagues who share similar contexts and concerns makes the insights gained especially helpful. By being the guides to our own professional learning, teachers resist stagnation and become mobile, active agents of change. With our colleagues, we take a learning stance by asking questions, taking risks, and reflecting on both successes and failures.

Lesson
Study uses knowledge you gain from your own classroom to stimulate professional
conversations about teaching and learning. Shared experiences through joint
planning and observation support the conversation, helping you develop your own
understanding and share it with others.

Advertisement

If you are eager to investigate and improve instruction and increase your understanding about teaching, I invite you to learn more about Lesson Study. Lesson Study creates continuous learning cycles that put teachers in the driver’s seat of their own professional development.

———————

Book: Collaborative Lesson Study: ReVisioning Teacher Professional
Development

Website: vickicollet.com

Request to join Collaborative Lesson Study Book Club on Facebook est (next book study begins January 13, 2020)

Vicki’s blog: mycoachescouch.blogspot.com

Vicki on Facebook: facebook.com/mycoachescouch/

Vicki on Twitter: @vscollet

References

Garcia,
A., & O’Donnell-Allen, C. (2015). Pose, wobble, flow: A culturally
proactive approach to literacy instruction. Teachers College Press.

Lewis,
C. & Hurd, J. (2011). Lesson study step by step: How teacher learning
communities improve instruction. Portsmouth, NH: Heinemann.

Lieberman,
A., & Wood, D. R. (2002). From network learning to classroom
teaching. Journal of Educational Change, 3, 315-337.

Continue Reading

Teaching

18 Inconvenient Truths About Assessment Of Learning

Published

on

18 Inconvenient Truths About Assessment Of Learning

by Terry Heick

I. In terms of pedagogy, the primary purpose of an assessment is to provide data to revise planned instruction. It should provide an obvious answer to the question, “So? So what? What now?

II. It’s an extraordinary amount of work to design precise and personalized assessments that illuminate pathways forward for individual students–likely too much for one teacher to do so consistently for every student. This requires rethinking of learning models, or encourages corner-cutting. (Or worse, teacher burnout.)

III. Literacy (reading and writing ability) can obscure content knowledge. Further, language development, lexical knowledge (VL), and listening ability are all related to mathematical and reading ability (Flanagan 2006). This can mean that it’s often easier to assess something other than an academic standard than it is knowledge of the standard itself. It may not tell you what you want it to, but it’s telling you something.

IV. Student self-assessment is tricky but a key matter of understanding. According to Ross & Rolheiser, “Students who are taught self-evaluation skills are more likely to persist on difficult tasks, be more confident about their ability, and take greater responsibility for their work.” (Ross & Rolheiser 2001)

V. Assessments of learning can sometimes obscure more than they reveal. If the assessment is precisely aligned to a given standard, and that standard isn’t properly understood by both the teacher and assessment designer, and there isn’t a common language between students, teacher, assessment designer, and curriculum developers about content and its implications, there is significant “noise” in data that can mislead those wishing to use the data, and disrupt any effort towards data-based instruction.

VI. Teachers often see understanding or achievement or career and college-readiness; students often see grades and performance (e.g., a lack or abundance of failure) (Atkinson 1964).

VII. Self-evaluation and self-grading are different. ‘Self-evaluation’ does not mean that the students determine the grades for their assignments and courses instead of the teacher. Here, self-evaluation refers to the understanding and application of explicit criteria to one’s own work and behavior for the purpose of judging if one has met specified goals (Andrade 2006).

VIII. If the assessment is not married to curriculum and learning models, it’s just another assignment. That is, if the data gleaned from the assessment isn’t used immediately to substantively revise planned instruction, it’s at best practice, and at worst, extra work for the teacher and student. If assessment, curriculum, and learning models don’t ‘talk’ to one another, there is slack in the chain.

IX. As with rigor, ‘high’ is a relative term. High expectations–if personalized and attainable–can promote persistence in students (Brophy 2004). Overly simple assessments to boost ‘confidence’ are temporary. The psychology of assessment is as critical as the pedagogy and content implications.

Advertisement

X. Designing assessment that has diverse measures of success that ‘speak’ to the student is critical to meaningful assessment. Students are often motivated to avoid failure rather than achieve success (Atkinson 1964).

XI. In a perfect world, we’d ask not “How you do on the test,” but “How’d the test do on you?” That is, we’d ask how accurately the test illuminated exactly what we do and don’t understand rather than smile or frown at our ‘performance.’ Put another way, it can be argued that an equally important function of an assessment is to identify what a student does understand. If it doesn’t, the test failed, not the student.

XII. The classroom isn’t ‘the real world.’ It’s easy to say invoke ‘the real world’ when discussing grading and assessments (e.g., “If a law school student doesn’t study for the Bar and fail, they don’t get to become lawyers. The same applied to you in this classroom, as I am preparing you for the real world.”) Children (in part) practicing to become adults is different than the high-stakes game of actually being an adult. The classroom should be a place where students come to understand the ‘real world’ without feeling its sting.

When students fail at school, the lesson they learn may not be what we hope.

XIII. Most teachers worth their salt can already guess the range of student performance they can expect before they even give the assessment. Therefore, it makes sense to design curriculum and instruction to adjust to student performance on-the-fly without Herculean effort by the teacher. If you don’t have a plan for the assessment data before you give the assessment, you’re already behind.

XIV. Every assessment is flawed. (Nothing is perfect.) That means that the more frequent, student-centered, and ‘non-threatening’ the assessment is (here are some examples of non-threatening assessments) the better. It’s tempting to overvalue each assessment as some kind of measuring stick of human potential. At best, it’s an imperfect snapshot–and that’s okay. We just need to make sure teachers and students and parents are all aware and respond to results accordingly.

XV. As a teacher, it’s tempting to take assessment results personal; it’s not. The less personal you take the assessment, the more analytical you’ll allow yourself to be.

XVI. Confirmation bias within assessment is easy to fall for–looking for data to support what you already suspect. Force yourself to see it the other way. Consider what the data says about what you’re teaching and how students are learning rather than looking too broadly (e.g., saying ‘they’ are ‘doing well’) or looking for data to support ideas you already have.

XVII. Assessment doesn’t have to mean ‘test.’ All student work has a world of ‘data’ to offer. How much you gain depends on what you’re looking for. (Admittedly, this truth isn’t really inconvenient at all.)

XVIII. Technology can help make data collection simpler and more effective but that’s not automatically true. In fact, if not used properly, technology can even make things worse by providing too much data about the wrong things (making it almost unusable to teachers).

The Inconvenient Truth About Assessment

Continue Reading

Trending