Blackboard Research Looks At Moodle Class Sizes Through ‘X-Ray’ Vision [UPDATED]

2545
Blackboard Research Looks At Moodle Class Sizes Through X-Ray Vision | Investigación de Blackboard estudia tamaños de clase en Moodle

Updates made on August 23rd.

WIRIS

With the help of the University College of Estate Management in Reading, England, the Blackboard research team asks:

Are large enrollment courses fundamentally different from small enrollment courses? What course facilitation practices can reduce the impact of class size on student engagement, learning, and success?

Across several dimensions of learning behavior and performance, results from an exploratory research using data from UCEM students suggests that, for the first question, the answer is no.

So is it time to get rid of walls and bundle every group of students into one? As outlandish as it sounds, researchers might have found at least one approach where there is no reason you should not. Comparing classes of up to 1,500 students with classes of between 10 and 30, they “did not find any meaningful difference in activity pattern by class enrollment.” Furthermore, the lack of variation found in activity also did not translate to differences in grades or risk of dropping out.

A controversial element of the research involved engagement and quality of contribution in online forums, the number of contributions to which also did not change with class size. Preliminary results find a correlation between group size and what the authors define as “social cohesion” and “linguistic complexity.” By the first term, they refer to the level of reciprocating replies between students. By the second, to word count and quantitative measurements for written texts.

UCEM’s secret engagement sauce?

The next arduous step is to make sure these results are not anecdotal and that the sustained engagement is a merit of UCEM, not just a fact of online learning life. Hopefully, a scaled up, replicated study will include some specifics about the research design and make the data produced by X-Ray openly available. This would benefit the EdTech research community, and Blackboard too, as it would allow the field to identify the incidence of factors affecting the results and help them make any necessary amendments. Some elementary questions linger:

  • The white paper suggests that sustained engagement in large groups is the result of UCEM’s efforts, but there is no information about the level of engagement prior to the intervention. That is, there is no baseline scenario reported.
  • There is also no information on whether control groups were established within variations on UCEM’s engagement strategies.
  • While the groups are reportedly organized at random, they all share a special trait: they are all admitted and enrolled UCEM students. This selection bias hinders the benefits of the randomization efforts.

A final question concerns the qualitative differences in forum contributions. As they did not affect the final outcomes, the value of “social cohesion” or “linguistic complexity” as learning practices are called into question. This fact alone deserves more scrutiny.

Among UCEM’s strategies claimed to “virtually eliminate the negative effects” of scale, the report prominently features a “tutoring program” based in Moodle, where each student is assigned a tutor. A second role, “Module Leader,” oversees the tutors to ensure consistency. The tutors help students review course materials, watch the forums, and provide feedback. The availability and length of interaction between tutors and students are not disclosed in the paper.

On a conclusive note, the paper reports “UCEM knows that sub-group tutoring works.

Blackboard and UCEM’s white paper, “Using learning analytics to understand student success in large enrollment courses,” is available at blog.blackboard.com. (Available for download through contact form fill-out.)

Updates made on August 23rd. Blackboard’s communication team reached out to offer the following comments:

“The research from Blackboard only pertained to UCEM and is currently not generalizable. We do not claim that there are no significant differences between large and small courses. The literature strongly suggests that there are significant differences in student engagement patterns on the basis on class size, and this is not something we wish to dispute. The fact that these differences were not found at UCEM is what prompted the researchers to question why.

The research report published by Blackboard did not state or imply that institutions should get rid of walls and bundle every group of students into one.  It merely found that, in the case of UCEM, large classes did not appear to suffer from many of the negative consequences that we would otherwise expect.  The fact that the university had a sub-group tutoring program in place appeared to be a reasonable and promising explanation for why this was the case.

It is true that this research did not follow a strictly experimental design, but this is not a weakness so much as a function of this kind of exploratory research.  With that being said, now that UCEM is collecting data using X-Ray Learning Analytics, more longitudinal and quasi-experimental studies will be possible in the future. We agree that the results are very promising, and that this exploratory work should prompt more controlled and generalizable studies on the efficacy of sub-group tutoring in online environments.”

Parts of this article have been edited to emphasize the exploratory and non-generalizable nature of the findings, and that neither the researchers, UCEM nor Blackboard advocate for larger class sizes.


eThink LogoThis Moodle Practice related post is made possible byeThink Education, a Certified Moodle Partner that provides a fully-managed Moodle experience including implementation, integration, cloud-hosting, and management services. To learn more about eThink, click here.