Back in February, we ran an article about a new study completed by Eduventure on what drives LMS purchase decisions. Because our article was only really able to scratch the surface of this hot topic, I decided to go to the source – Jeff Alderson, the Principal Analyst at Eduventures – and learn more about the data behind the report and what Eduventures has learned that might be important to the future of Moodle development.
Jeff pointed out that this deeper dive is especially important to an open source community like Moodle because it is often difficult for the research that he performs and publishes to get to the right people (i.e. decision makers) so they can have the data to help them make more informed decisions.
I kicked off my interview with Jeff putting a few points on the table:
- All of the LMS vendors that are a part of their research are clients of Eduventures – this has allowed him to have a debrief with each of them about what the data shows and the reason behind their rankings on the scorecard.
- Jeff made it clear that the rankings do not influence Eduventure’s recommendations to their clients in higher ed about what system to use. Eduventure has a holistic approach to this process to make sure that each school is looking at the system that is right “for them.” In the case of Moodle, Jeff stated that he’d usually only recommend an open source solution if the institution really had the in-house capability to support it (or an identified 3rd party vendor), or if they truly needed the ability to tweak and customize to a very high degree, that they can’t get from another solution.
- The research was initiated because Eduventure was receiving many requests from higher ed clients about their intention to purchase an LMS and their desire to receive advice on which way to go – and how an LMS can fit into the greater technology strategy and ecosystem (think: ERP, SIS, CRM, CMS, etc.).
- Eduventures noticed that there are a number of outlets that discuss the LMS market – number of vendors, financial stability of vendors, pricing, market share, etc. – but two missing pieces were (1) whether or not the LMS would support the way an institution wants to deliver teaching and learning and (2) will the LMS fit within the current tech ecosystem.
Does the research consider the “users” within the ecosystem (i.e. the student)?
Vendors were asked to complete a fairly complex survey to understand if their LMS product covered a long list of use-cases. This list of use cases was drawn directly from higher ed clients and what they perceive their needs (and the needs of their students) to be. When considering Moodle, Jeff explained that you can’t download Moodle today and have the core software cover the majority of required use cases. You have to “go plugin shopping.” So in their scorecard, a yellow for Moodle doesn’t mean the functionality isn’t there – but that the institution will need to invest a significant amount of time and energy getting Moodle to where they want it to be. With other vendors, when you install the LMS, you go through an configuration package and decide what will be ‘on’ and ‘off’. Jeff offered this as one suggestion to the Moodle project about a way to improve the Moodle experience: Moodle would do well to provide some type of “turn key package” that would allow an administrator to select functionality out of the box that covers a majority of necessary use cases.
Is budget still the largest factor in an LMS purchase decision?
Jeff said that the phrase he hears most often now is “total cost of ownership.” Even if an institution’s budget for LMS license fees are “0,” there will be other costs associated with the system in terms of project management, professional services, maintenance, etc. TCO is somewhat difficult to compare when it comes to comparing Moodle to Canvas to Blackboard or other LMS because you need to do a much more comprehensive look at competencies, available resources and other factors. Other “alternative pricing models” is also a term that is on the street. A suggestion Jeff had for the Moodle project in this regard is for Moodle to create an ROI calculator and simply help institutions answer the question of how long it will take for them to realize a return after their investment in implementation, training and support. You could then do an apples to apples comparison on ROI between LMS, which would be powerful.
Are schools looking at their LMS as a value-add to their brands?
In some cases yes, but he often hears from institutions that it doesn’t matter what LMS the institution has because students and teachers will use it “because they have to.” It is sad but true – even from R1 research universities. He suggests that students and teachers should be a part of the evaluation process for LMS (especially faculty and instructional designers). This is so important that this will be the thrust of Eduventures next round of research.
This was a part of the current research: the client’s ability to drive the product roadmap. Jeff says very few vendors scored well in this area for specific reasons. Some vendors have their attention split between multiple audiences (e.g. K-12 and higher ed), some just have bad track records of responsiveness to support requests, and some are entirely closed systems. The vendors that scored well in this area had clear road maps, were very responsive to client requests and had raving fans (Jeff pull out a student quote at this point – “this feels like all of the other technology I’m used to using outside of the learning environment, and so I’m going to use this more often and its more responsive to how I learn.”)
Can you tell us more about each of the six criteria in the scorecard?
Plays well with the ERP/SIS: An LMS is only as good as the data to drive it – student records, teacher information, etc. The reason that Moodle scores a yellow is not because Moodle doesn’t connect with ERP/SIS systems. It is because the integration isn’t seamless; someone at the institution had to make the connection happen. And, usually, this connection isn’t a process that works right the first (or second) time easily.
Scales seamlessly to meet demand: This is not just about whether or not there is a cloud solution – this is about the LMS architecture, deployment model, hosting and provisioning of institutions. For example, Blackboard typically has to be pre-provisioned to ensure it can handle load surges (Jeff noted that Blackboards cloud solution came out the same month as his report). This is same for Moodle – there is no native cloud stack to scale the solution. Solutions that were built in the cloud from the get go received the highest scores because they were built with this already in mind.
Percent covered of the required use cases: Eduventure sent out an RFI to collect this data from institutions. (See the detailed discussion above on this). Jeff emphasized that they really looked at the ability of the core software to deliver the full experience without any add ons or plugins (including needing to go an download a mobile app to run the mobile version of the LMS).
Clients’ ability to drive product roadmap: The pro and con for an open source community project like Moodle is that there is no guarantee that any particular service will be delivered on any specific timeframe. The open source community tends to take a very long time to produce what are relatively straight forward commodity features (unless the feature is dedicated to one developer who is responsible for making it happen). It wasn’t always like this – but as the LMS market has matured and the standard feature set has become commoditized, the time for development and update of new features has slowed dramatically.
Integrates with nearly everything: Integration with LTI and other tools is an absolute must have. Also the availability of an app store/system that 3rd parties can contribute to. Almost all vendors scored well on this.
Training and support communities: Basically, if, as a user, you have a question, can you find documentation or a person who is authoritative about that documentation to get your questions answered? Moodle has a very vibrant community in this regard. This is whether or not you have a paid support option. In cases where users didn’t generally have a positive experience with support, the LMS was dinged in the scoring.
What are the most important differentiators between LMS?
- Schoology is known as a “social LMS” – it looks and feels differently than everyone else. They kill it in user experience.
- It is challenging to recommend Sakai to basically anyone.
- iTunesU is a viable option if a school has a one-to-one device program.
- In every presentation Jeff is in, they are recommending that institutions evaluation Blackboard learn, Brightspace, Canvas and Moodle.
Is there data that you wish were available to help schools make a more informed decision?
Outcome data. Efficacy data. Control groups. Many vendors make some claims about how their system create impact, but no one has the data to compare apples to apples. There isn’t a whole lot of functional difference between Blackboard, Brightspace, Canvas, and Moodle. On the fringe, maybe, but not in the core LMS functionality. Jeff would like to see the market share outcomes data anonymously so that it could be analyzed.
What are Moodle’s strengths and weaknesses in today’s marketplace?
Moodle’s biggest opportunity is with institutions that have looked at the LMS market and have come away with the notion that something off the shelf won’t fit their needs. Moodle is great for this because of its customization possibilities and ability to tightly control the user experience. One of Moodle’s greatest strengths is its ability to selectively turn off things. This is actually kind of hard in most of the other platforms. This could position Moodle as a limited feature, but powerful, LMS – rather than trying to do everything, do the things that your clients really need better than anyone else.
Jeff ending our interview noting that their research for Moodle was based on publicly available data and his conversations with institutions that have implemented Moodle. He welcomed the opportunity to speak with the project team at Moodle HQ.
If you’d like to Jeff’s blog post on the report and get more information visit: http://www.eduventures.com/2016/02/whats-driving-lms-buying-behavior/