Once you’ve developed your RFP, it’s time to work through the steps of evaluating RFP responses, handling product demonstrations and making your final selection.
Your Selection Timeframe
From the time you kick off your project to the time you’ve made your final selection, you can expect to spend between four to six months. Yes, you can accelerate that timeframe but doing so is risky. People who try to rush things usually miss requirements and then they’re locked into a multi-year contract they didn’t fully understand or that doesn’t take into account their specific LMS needs.
It takes time to get everybody on board with the selection decision. Plus, there’s a benefit: by getting buy-in from a group of people, there’s more momentum to carry the project forward, even as specific individuals leave the company and new staff members join. And the likelihood that the software will work cross-functionally is greater, since more points of view will have gone into decision-making.
On the other hand, you don’t want to drag the selection process out too long or allow deadlines to slide. The complexion of the project team could change dramatically, and there’s a risk that you could end up plowing a lot of effort into a decision that never gets made. Technology changes, and the longer you wait from the time you’ve defined your requirements to the time you make your decision, the more likely specific features may become outdated. In the meantime, your company probably won’t have made any progress toward the goals for which you wanted the LMS in the first place.
How to Evaluate Proposal Responses
Not everybody invited to participate in the RFP will do so. There are all kinds of reasons companies don’t. They may decide their LMS technology isn’t appropriate for the goals you’ve set. Or they may believe you already have another solution picked out and you’re just going through the motions of the RFP process to show you’re following the rules.
But for all of the RFP responses you do get, a small team of people needs to sit down and go through each to make sure your must-have requirements have been addressed. That’s why it’s important in the RFP to highlight those. You don’t want vendors overlooking them. And you want the people doing the assessment on your end to know exactly what they’re looking for. Then, have the team go through the list of nice-to have features.
Using the various categories of your RFP, you can create a rubric or scoring sheet to use during this and the demonstration phase of the selection process.
You can either use a weighting system or, for simplicity’s sake, give each must-have function a value of, say, 10 points, and a nice-to-have feature a value of one point. Doing so helps to ensure that a bunch of unneeded bells and whistles won’t by their sheer numbers overwhelm the evaluation process.
By tallying the various scores—paying particular attention to those essential requirements—you should be able to whittle down the pile of RFP responses from a long list to a short list. The companies still in the running are the ones from which you’ll request demonstrations.
What about an RFI?
Government agencies will often put out a Request for Information (RFI) to help them identify industry-caliber features. You can try the same. This serves as a shortcut to developing an RFP on your own, having to use a generic RFP form you’ve downloaded from the internet or having to hire a consultant to manage this process. The RFI does not commit your organization to any deal, and it can inform your eventual RFP greatly.
Demo Essentials
What’s involved in a demonstration? I have participated in hundreds of them for my LMS company, and I recommend having two components on hand:
- Copies of a fresh version of the scoring sheet or rubric, which is handed out to every person sitting in on the demo and includes room for notes; and
- A script that’s use case-focused.
As the demo proceeds, those people viewing it should be able to follow along in their rubric and do check-offs. As they come across something they especially like, dislike or don’t understand, they can add notes and questions to the relevant section.
The checkoffs and note-taking are important for a few reasons. First, memories fail. After the first demonstration, the various aspects of individual LMS solutions will begin to pile up and people won’t be able to remember what they saw in one demo versus another. Second, people tend to be more attentive when they have a document in front of them to fill out, especially if they know that immediately after the demonstration, they’re supposed to hand in the scoring sheet to whoever is running the project. Third, later on, if you and the vendor disagree about the availability of a given feature, those scoring sheets can help settle the disagreement. Was it really checked off or noted or is somebody misremembering?
That script helps make sure vendors spend the bulk of their demo time on scenarios that reflect your company’s actual needs and less on features you don’t care about or “fluffy” company information that should have been provided way before now.
By giving the same script to every vendor for the demonstrations, you’ll be able to see and compare the same things. There will be those specific things that some vendors will say they can do in the response to your RFP, but if it’s a must-have for you, you need to see it being done in the system or they need to explain how they plan to accomplish it.
Then there are basics to cover. For example, the script should include coverage for each user persona—the new learner, the technical user, the administrator, and so on. For example, think about your typical channel partner learner and find out how that user would enter the LMS for the first time, identify a course, register, go through the lessons, take an assessment and receive his or her certificate of completion. Another use case might be to ask the vendor to demonstrate how reports are built: “Build a report from your demo site indicating the number of learner completions for a specific course over a three-month period.”
First, Virtual Demos, Then Live
While some customers will choose to go straight to in-person demos, we’d advise virtual or web-based demos in the first go- around. They’re more efficient, less resource intensive on your end (and the vendor’s side), and they can tell you a lot to help you continue in your winnowing process. Remember, your goal is to find The One—that LMS company that will meet your needs for helping you develop and grow your channel program.
A virtual demonstration will be set up by the vendor, which will provide a link for each person to connect to. If you have a large display (65 or 75 inches), you may choose to have everybody sit in the same room and do the activity as a group. Or it may be that everybody is watching from their own desktop computers with headsets on.
Live demos are definitely done as a group activity, also on a large display. Depending on the size of your company, you could easily have as many as eight to 15 people participating in each type of demo.
Try to schedule all demos of a particular kind within the same week, if possible. That’ll keep your evaluation work charging ahead and allow the group to converge after all the web-based or in- person demos are done to compare notes and discuss next steps.
If you start with a virtual demo, you’ll be able to whittle down a finalist list to just two or three companies. That’ll save you time on the in-person demonstrations, which last longer.
Virtual demos typically last between 30 minutes and two hours (which is on the long side). Longer than that and participants tend to get antsy. The best live demos run between an hour and three hours; any longer and, again, attention fades.
In both cases, that’s another advantage of providing a script— you corral the focus of the tasks being demonstrated to those that are most important to you and your organization versus the ones that are most important to the LMS vendor.
5 Do’s and Don’ts of Demos
- Do avoid—at all costs—the automated demo. You might sit through one of those when you’re compiling your RFP to get an idea of how some programs in the category work, but they’re not customized enough to help you gain an understanding of how it would work in your environment.
- Don’t bother with demos that are nothing more than slideshows. You don’t want to see screenshots of the product in action. You want to see how the product actually works from one step to the next.
- Do take phone calls from vendors prior to the demos. They want to pick your brain and understand what roles will be represented in the meeting, where the emphasis will be for the script and any other tips that will help make the demo more valuable for everybody concerned.
- Don’t just rely on virtual demos. You can use them to cut down to two to three vendors but then be sure to have onsite use case demonstrations. You want to meet your potential future partners face to face.
- Do get your C-level executives’ blessing on the selected vendor.
Demo Diversions
Even though we advise you to work through a script during demos, there are bound to be questions. These can be dealt with in a few different ways.
If they’re relevant to the specific part of the program being shown, then, by all means, get the question answered. But sometimes questions are “off-script” and have nothing do with the subject at hand—or, more likely, require far more detail than what’s available during the demo time. When that’s the case, put the topic into a “parking lot”—a list of subjects to be dealt with at a later time. I prefer to handle parking lot issues in a separate phone call or follow-up demonstration.
Occasionally, the top decision-maker will pop in long enough to watch a little of the presentation and then say, “I’m not interested in seeing any more of this demo until we know your software can do this…” When that’s the situation, you can expect the vendor to interrupt the scripted presentation and drill down on the given topic right then and there. It might be that the script was created by others and they weren’t aware of specific parts of the functionality that management viewed as critical. As vendors, we understand that. We just want to make sure everybody gets onto the same page.
Making the Final Cut
After the demo sessions, convene a group meeting to do the next round of winnowing. By the time your web-based demos are done, you should have whittled the finalist list to a handful of companies. By the time the lengthier in-person demos are done, the task force should have settled on one or—at most –two vendors.
At that point it’s time for the buyer to set up a separate session involving IT experts from each side to address issues such as bandwidth and storage requirements, integrations with other systems in use and other subjects using the technical lingua franca they understand. Those will often last no longer than an hour.
Keeping Vendors Informed
If you’re a person who avoids conflict, it may be uncomfortable to say, “No,” to a vendor, especially if you feel like they’ve put a lot of time and effort into winning your business. But if you started with 10 LMS companies on your original list, it’s unavoidable that nine of them will be disappointed in the final decision.
Just let them know as soon as they’ve been culled out of the herd. Being upfront will reduce the number of follow-up phone calls and emails you’ll have to deal with, and the vendors who didn’t make the cut can move on to other prospects.
Oftentimes, customers will hold off on talking with you about their decision until the contract has been signed with the chosen vendor. Avoid that if possible. If a company that’s acquiring the LMS needs to change its mind for legal reasons, there’s not a vendor in the world that won’t be happy to get back into the running. Personally, I like to know that we’re not a good fit so everybody can move on.