How To Evaluate An LMS – Use Case Scenarios

If you are looking to purchase an LMS for the first time – you have begun on the right path. If you are looking to switch to a different LMS, it is great that you are actively using an LMS. Your IT- enabled learning initiative has matured enough for you to understand the gap between what you actually need and what you are getting. In both cases - for the current purchase exercise and for the long term satisfaction (or the lack of it) depends not only on the LMS you purchase, but also on HOW you went about evaluating WHICH LMS to purchase.

For those who are looking to switch, it may be easier, but not really easy, as they may have a list of things they want the new LMS to DO or NOT do. While for the ones looking to buy a new one, it may be a tad more difficult.

So how does one go about evaluating an LMS? The process (overall) seems to be standard: Send out RFPs, give each respondent scores on various aspects, shortlist a few, get demonstrations and proposals, compare and award contract. Yes, I agree, the process is indeed simple and logical. However, the success of the process depends on a critical thing – the phase where your team is audience to vendor LMS demonstrations. This post of mine focuses on this particular phase more than the others and this doesn’t intend to diminish the importance of any other phase in the process.

One may wonder what it is with vendor demonstrations. In most cases skilled sales people come with their presales/technical colleagues and demonstrate the system in detail. Well, yes! But isn’t this kind of one-way? You may have asked a few questions and those were answered but how do you ensure the single most important thing – that this LMS solution/service will work for you for a good number of years? Even though you had a detailed RFP, the chances that the vendor skipped a few things are high as there are almost invariably time constraints in demos. The perception of importance could be different for a few of your requirements as you see them, and as the vendors see them. So how do you ensure demonstrations capture what you are looking for to help you qualify (or disqualify) the LMS being demonstrated?

The answer lies in using USE CASE SCENARIOS for demonstrations. The good thing about using Use Case Scenarios is they are simple to employ. Use case scenarios are typically high level scenarios where the need is to define a work flow in the desired LMS system which includes:

a. End user experience (could be the learner, line manager, training manager, instructor, training administrator, system administrator or a mix of these profiles). All these workflows must map an existing or desired workflow in your organization as a process. The key idea here is to break down the requirements into logical workflows: e.g. Curriculum Request Workflow:- Learner views the catalog; In the catalog section learner needs an easy way to filter through the available learning curriculums to identify the one(s) s/he needs access to; Learner places the request along with a reason and forwards it to his/her line manager; an email is sent to the learner for notification and CC’ed to the line manager for action; and so on and so forth. The key is to describe these workflows to the vendor prior to demonstrations so the vendors could prepare data and also be ready to demonstrate how their LMS maps to these workflows. Ensure that you cover ALL critical workflows which will make or break your LMS system’s acceptance with the end users.

b. Data flow and visual representation. This part focuses on what and how the data flows across the workflow and how is it visually represented – you may be more comfortable with matrix reports or dashboards, you may want the end users to see confirmation messages, automated emails, etc.

c. MIS (Reports) Impact. Most workflows would impact one or the other report. How the reports are expected, at times, depict the gaps, if any, between your desired workflow(s) and what the LMS offers. In the context of the workflow example, you may need a report of all requests made with their current status and all iterations made on them. It is extremely critical to be able to view all these during the demonstrations.

As a summary, during the vendor demonstrations, apart from the usual aesthetic and usability evaluation parameters, the focus should be on allowing vendors to demonstrate the LMS not how they want to do it but how you would like to see your use case scenarios being handled by the system.

Once the use case scenarios are all demonstrated, the vendors and you will have a clear picture of how much the system offers out of the box, how much needs configuration and how much needs customization. This not only allows you to confidently evaluate the LMS systems effectively but also allows vendors to clearly understand the needs and helps them submit clear and accurate proposals. In the long run this also ensures a smooth implementation and a long haul of usage without hassle. Isn’t this what you were looking for in the first place – an LMS that meets your needs, doesn’t become a headache while implementing and when implemented doesn’t look like a misfit?

As I said earlier, there are many ways to go about the evaluation but using Use Case Scenarios in your method would make it easier to evaluate and select the right LMS.

Our Other Blog:

8 Key Reasons why LMS Learning is the Best