How To Develop An Effective Assessment Practice Through The Moodle Quiz Activity

2802
How To Develop An Effective Assessment Practice Through The Moodle Quiz Activity

The power of the Quiz Activity in Moodle can make it overwhelming. But going through all of the options does not need to be tedious. All it takes is for you define what you want from your students, before setting up a quiz.

WIRIS

The Moodle documentation is not only a technical guide for setting up and developing an installation and plugins. It is also a rich reference on pedagogy. Being an open source, moodlers from all over contribute to this corpus. As mostly developers and IT people create and maintain technical Moodle pages, mostly teachers, instructors and instructional designers add to the pedagogical ones.

As a teacher or instructor, there are two basic variables in which you should frame your assessment practice. We are calling them Goal and Scope, and you will use them to choose your ideal practice as we review the Moodle documentation for Effective Moodle practice. So let us begin.

Goal

You can create quizzes for the Goal of:

a) making sure your students have followed through the coursework.

b) verify they have acquired an operational level of skill.

c) validating the level of engagement and value the coursework provided.

Your preference over the particular Goal scenario defines the kind of questions you ask. Goal a takes a less in-depth assessment. It can just be some basic definitions of the most important concepts. Objectives b and c are not necessarily more specific, in fact outcomes can turn inevitably vague. Goal c implies your commitment to steer your class until you achieve the learning version of “product/market fit“.

This leads to a hard truth about assessment: the more specific your questions are, the more it becomes a reading comprehension test instead of a deep examination of the relationship between student and subject. To figure out if they are “getting it”, you need to accept broad answers. Over time, with work and practice, you will be able to figure out the unique mental model of knowledge for each of your students. This model receives in psychology and cognitive science the name of schema.

Scope

Did you know that there is no agreement on the length of Great Britain’s coastline? Would you believe me if I told you that no unanimous agreement will ever be mathematically possible?

Clarity is, funnily enough, a rather subjective term. Because no matter how clear you are, there is always a “clearer” way possible. To define the intended level of clarity in your assessment practice, you can set the Scope to be

α) a textbook or reference as “holy grail”. What is there, is the one and only truth. What is not there, does not exist.

β) a body of knowledge that includes a predefined list of authors and references.

γ) a real life problem where answers are valid as long as they help advancing a solution in a real way.

This gives you another trade-off. This time it involves control. Route α is easy, as long as you choose a reference you know from heart.

Never go this way if it is not the case!

Routes β and γ give your students more autonomy which again brings ambiguity. Then again, ongoing assessment should over time tell you if your students are on a journey of knowledge, or just “winging it”.

Route γ should be the most empowering one. Students will learn to get and refine skills as they face real life problems. This will help them set autonomous learning goals and an intrinsic appreciation for knowledge.

More often than not, however, route γ is impossible. It is time consuming, high maintenance and if rewarding, only after a long succession of failures (ill-designed experiments) and erred trials (sound experiments with unactionable outcomes).

What is my practice then?

Follow this by no means authoritative rule of thumb:

  1. Select a Goal and a Scope.
  2. Set values of Goal a and Scope α = 1, b and β = 2, c and γ = 3. Calculate your score by adding the values of your selected Goal and Scope.
  3. Your practice belongs to a level depending on your score, as follows:

Score 1, 2: Security Level

We could also call this the “low investment” level. In this level, quizzes are a way for you to show evidence that you performed your duties as requested. Most standardized testing belongs in this category. There is no schema to talk about here, but if you are training students for a given examination standard, your focus should be on reproducing the experience.

Admittedly, students tend to view Security Level assessment activities as pointless or lacking in formative value. Likely, some of them will try to bend the rules.

To mitigate cheating, the Moodle documentation suggests randomizing the order of questions and responses. Better yet, build a question bank and randomize the questions themselves so students don’t get the same ones. Set timers for question responses. These are all customization options available in the Moodle Quiz Activity.

Score 3, 4: Robust Testing Level

This practice leans upon ongoing testing. Having weekly quizzes lowers the stakes and your students’ anxiety, while giving you more data points. You can get the most value from a larger dataset if you design the questions in ways susceptible to statistical analysis.

The amount of work in creating a large dataset is larger without a doubt. To counter this to some extent, repeat questions along assessment events. This will also allow you to measure basic retention, at least for the duration of your academic period.

Cheating is still possible, but the data could show you performance anomalies that could signal wrongdoing with a higher degree of confidence.

Score 5, 6: Certainty Based Marking (CBM) Level

CBM involves applying mathematical scenarios to validate performance and consistency. As a sophistication of Robust Testing, it benefits from recurrence.

Barebones CBM consists of adding to each question, another one that asks: “how confident are you in the answer you just gave?”. You can penalize them for being overconfident and wrong, or right but unsure, and vice versa. This approach should be better known as “Declared” CBM.

The ideal version of CBM helps students realize, proudly if need be, the value and power of their own schemes. It is about encouraging them, among other things, about the reliability of their learning risk management skills. It is often associated with thoughtfulness, attentiveness and self-confidence. I have yet to found a real example, but I’m an optimist.

To close: not a straitjacket

If CBM is taking time and skills you do not have, scale it down. If you want to up the excitement with a bit of a challenge, add some flavor of Robust Testing to your Security Level. Your Goal and Scope, as the many other variables I did not mention, may in fact vary, even during the course. At the end of the day, this is about finding your own style, one that gives you strength and resonates with your students.

Moodle gives you many tools. Nobody expects you to know everything about all of them. I’ve certainly feel like I’m only scratching the surface of the Quiz Activity! This is about finding a pace of self-actualization that feels right.

No matter how nurturing you are, you were not born pedagogic. But the fact that you think about getting more useful outcomes from the assessments you give your students, is a positive quality. It will show and they will recognize it, sooner or later.

Read up on Effective Quiz Practice on docs.moodle.org.

Do you have a question, or something to add? Be my guest! Leave a comment below.