The following is a back-to-school recap of our series: The Learning Analytics Roadmap. This series is an effort to centralize the community’s understanding on analytics, organized in three vectors:
- Theory and conceptual models
- Technologies, specifically Moodle plugins
- Practical applications and pedagogy
Each of our stories has included the three elements: Some detail on one of the frameworks proposed by “Moodlers” or other educational researchers; a plugin review (with few exceptions), and its application within a context given by the conceptual models.
The models we have used so far
The theory behind analytics for learning is not consolidated. We have used two frameworks that show promise in further research and application, and have a minimum number of supporters within the Moodle community.
The Fosway Analytics Maturity Model. This tool lets professionals place the use of analytics in an organization along a level of sophistication. This could be in terms of data volume, computation complexity, or also the level of talent savviness or competency to carry out programs where analytics play central or important roles.
The ROI on Learning Analytics Pyramid. This pyramid offers a quantifiable method to link a “learning intervention” ―as measured by analytics―, with a desired set of operational or financial outcomes. The technical level to calculate an ROI depends on the organization, regarding the learning intervention that is taking place. If it is intended to primarily increase satisfaction, it belongs to a bottom level of the pyramid. If it is intended to ultimately have a positive effect on Key Performance Indicators, or the bottom line, the intervention goes up the levels.
Common questions. We have received three kinds of recurring questions, to be answer as definitely as possible below:
- No, these are not the only Learning Analytics models out there. We are open to discuss new ones as their need arises, and once we have extracted enough value from the current ones. At the end of the day, analytics decisions are not made, just guided by the theory.
- Yes, the models are compatible. Both have a “stepping stone” narrative that posit a benefit for organizations to start at the beginning and work upwards gradually. Organizations can think of the “passive reporting” and the “learning (reaction and) satisfaction” steps as starting points, to highlight value with low effort and get people excited about analytics. We have stressed how an “analytics culture” is almost a requirement for more advanced analytics-based interventions.
- No, the models are not correlated. Theoretically, an organization could measure the ROI of a learning intervention through “passive reporting” (small data). Likewise, machine intelligence can be modeled around satisfaction. It is of course wiser to move forward on the technical and the ROI path with balance.
The Moodle Analytics plugins
The diversity of analytics solutions is a blessing and a curse. Currently, there are dozens of analytics plugins for Moodle, with varying levels of depth and reported usage. We are committed to review as many as we can, as thoroughly as possible. We also welcome suggestions and observations in the comments.
As expected, commercial plugins are more “polished”, while free, Open Source ones take more effort to implement on a Moodle site. A question still in the open regards monetization and business models of analytics plugins and services, for Moodle and the LMS market as a whole.
These are the plugins reviewed so far:
- SmartKlass (Behavior and engagement tracking)
- IntelliBoard (Performance against goal)
- Dropout Detective (Specific, ―dropouts― risk propensity)
- Analytics Graphs Plugin (Graphical representation, Visual correlation)
- Zoola Analytics (Reporting)
The Learning Analytics Roadmap Stories of 2016
- Where Can SmartKlass Take Your Moodle?, published August 23
- Intelliboard for Moodle Student Data Empowerment, August 24
- An ROI on Dropout Detective, September 2
- Get Your Organization Enthusiastic With Analytics Graphs for Moodle, September 29
- Support Your Learning Case In Moodle With Zoola, October 7
- The Dalton Plan (MoodleMoot Australia 2016), November 16
- Do All Roads Lead To Forecasting?, December 1
- Comparing Paths and “Completing the Loop”, January 12, 2017
What’s cooking for 2017
We intend to follow with deeper explorations of the topics covered so far, particularly along the lines of the Analytics Maturity and the ROI on Learning Analytics pyramid. Furthermore, we will follow-up on some developments coming up during the year, within Moodle and in the LMS market as a whole. Examples:
- The Moodle Machine Learning API, briefly discussed by Dalton at MoodleMoot Australia 2016.
- moodle.org discussions and documentation.
What lays ahead in your Learning Analytics Roadmap? Share your thoughts and wishes in the comments!