Four Steps to Building E-Learning Success

E-learning matures, the quality of courseware is improving, fringe content publishers are vanishing, and design and technical standards are escalating. But even now, there is no one-stop shop for the best e-learning content.

Those responsible for building off-the-shelf e-learning portfolios must choose from an overwhelming number of publishers and can rarely recommend a selection of courses from just one source. To make the best choice, training managers must quickly settle on a small pool of publishers and perform a thorough analysis of each.

The following ideas outline a proven four-step process for building the right e-learning solution under strict time and resource constraints.

Six weeks to do it
Let’s say the vice president needs sales-management training for a whole division — right now. Her voice mail says it all: “We’re launching our new competency framework under some serious pressure. No time to wait, and our budget is limited. Can you get a developmental program together within the next six weeks without making any major financial commitments? Custom content is out. It’s just too expensive and time-consuming in this economic environment.”

An off-the-shelf, Web-delivered e-learning portfolio may be the way to go. A collection of online courses from one or more vendors can give you high-quality content and cost-effective Web delivery without excessive development costs. However, in the slowly maturing e-learning industry, there are still many vendors, thousands of courses, and no best-of-breed solutions.

How do you get through the mess? Pick a single vendor and live with its limited course offering? Cherry-pick courses from all of the vendors?

Looking for the right off-the-shelf courseware without a clear plan will cost you time, money, and credibility. Try this four-step method for creating an off-the-shelf e-learning portfolio:

• Identify Selection Criteria and Constraints
• Create a Vendor Shortlist
• Select and Test Courses
• Package and Implement

Before you start
Take the time to determine if an off-the-shelf e-learning portfolio is right for you. If you answer yes to any of the following questions, it may not be the ideal solution.

• Are my learning objectives too specialized for generic content?
• Are there major impediments for more than 10 percent of my users, such as limited access to the Internet?
• Do I have very specific implementation requirements that cannot be met by vendors’ standard delivery models?

Once you have decided that an off-the-shelf portfolio is the way to go, make sure that your stakeholders are on board. Talk with your project sponsor and other key constituents to determine the answers to several important questions:

How will the e-learning courses support existing in-class or on-the-job training? Are there ongoing technology initiatives that might affect your recommendation? What resources can you rely on to perform your search and implement the program? Will you have an evaluation team or will you be solely responsible for making judgments? What was the fate of previous e-learning initiatives?

Having these discussions early in the process will raise your credibility, prevent embarrassing oversights down the road, and facilitate implementation.

Step 1: Identify selection criteria and constraints
User profiles are the compass that will guide you through the whole selection process. They can range in complexity from brief descriptions of typical end users to page-long biographies and checklists. Let the depth of your user profiles vary according to need.

Creating powerful user profiles takes a bit of time, research, and imagination. Describe the relevant user characteristics and how the courses will be used. Include “soft” issues relating to usability. How much time do users have available for each learning session? What is their level of education and computer literacy? Will they access courses from home or from work? What distractions will they face?

Also include “hard” issues relating to technology. At the very least, be sure that you know your user’s sound card, browser version, monitor size, connection speed, processor speed, and operating system. Consult with the IT department to determine if there are policies prohibiting the installation of plug-ins or software that vendors may require to run certain courses.

On the basis of your user profiles and stakeholder needs, identify objective technical and administrative criteria and constraints. Criteria are self-imposed. For example, “The vendor must offer an e-commerce purchase option that gives users course access within an hour of purchase.” On the other hand, constraints are defined by factors beyond your immediate control. For example, “The course must work on a computer with a Pentium 75 MHz processor and 64 MB of RAM.” To facilitate the quick elimination of less desirable vendors and courses, phrase each statement objectively. Some examples of criteria and constraints follow:

• The vendor must offer live telephone support during business hours via a toll-free number.

  
  

• The vendor must have a broad course selection (at least 20 soft-skill courses).

  
  

• The courses must not require the installation of any software or plug-ins that are not currently supported by the IT department.

  
  

• The courses must comply with all accessibility criteria in section 508 of the Rehabilitation Act.

  
  

• The vendor must comply with AICC and SCORM interoperability standards.

  
  

• The course must cost less than $200 per seat.

Step 2: Create a vendor shortlist
With your selection criteria and constraints in hand, it’s time to create a shortlist of vendors. Visit vendor Web sites, examine their marketing claims, and follow up with vendor sales representatives when you require additional information. If your users will purchase through a public e-commerce Web site, buy at least one course from each vendor to test the e-commerce experience.

On the other hand, if you require integration with your learning management system, installation on your intranet, or any other special administrative process, speak with vendor sales representatives about the feasibility and cost of such a solution. Also scan each vendor’s course list (which should state exactly what learners should get out of every offering) and list each course that meets one or more of your learning objectives.

Eliminate any vendor that fails to meet your list of criteria and constraints, and eliminate any vendor without a significant number of courses that coincide with your learning objectives. The more vendors that you eliminate at this early stage, the more time you will have for a rigorous course analysis later on. If your goal is to move through this four-step process in six weeks or less and save on implementation costs, create a shortlist of three or four vendors with very broad course offerings.

At this stage, you are assessing whether vendors meet your technical and administrative requirements and making a primary assessment of learning-objective fit. Avoid the temptation to evaluate the quality of vendors’ offerings. It’s simply too early in your process for subjectivity, especially when you have not yet had the opportunity to take a detailed look at vendor courses. Even if you do purchase and examine a single course from each vendor, avoid early conclusions about vendor quality. Many vendors are still working out their designs and offer courses that vary in quality.

Step 3: Select and test courses
Once you have a shortlist of vendors that meet your requirements and a list of course candidates for each vendor, it’s time to call your vendors and ask for demonstration access. But be sure to attain access to every course that you are considering, even if you have to make a purchase.

This is the fun part of the selection process. Your task is to make a careful hands-on analysis of each course and answer the following two questions: (1) Does the course content really meet your learning objectives? (2) Is the course appropriate for your organization and your audience? As you become familiar with each vendor’s basic design, you’ll be able to focus more on the course content and skim through courses efficiently. For courses that are two to three hours in length, budget an average of one hour for each review.

Eliminate any courses that fail to meet your learning objectives or that are not appropriate for your audience/organization. For example, your sophisticated e-learners may laugh at the reconstituted MS PowerPoint presentations in one course. Or maybe the content of another relies on too much jargon. Sometimes course content will conflict with important organizational practice.

Your list of course candidates will slowly shrink to a manageable size. And if you’ve chosen vendors with broad enough course offerings, you’ll still cover all or most of your learning objectives. This part of the process is more subjective. Therefore, it’s important to keep careful notes to defend your choices.

Before deciding on a final list of course candidates, perform a quick test with stakeholders and end users. Show your sponsor and other key stakeholders one or more representative courses to help set expectations and gather feedback before it’s too late to make adjustments to your recommendation.

Also, organize a field test for one typical course from each vendor. Field tests allow you to watch a typical user go through the whole process of acquiring and completing an online course. Carefully watch and take notes on how end users interact with the technology and the content. Pay equal attention to what they do and what they say about the experience. The results of these observation sessions will prevent bad course choices and enable you to write helpful user instructions.

When you’re comfortable with the outcome of your tests, you’re ready to settle on a final portfolio. In this last round of elimination, compare courses head to head and remove any inferior courses with learning objectives that are satisfied by stronger courses.

Ideally, you’ll be able to reduce your implementation costs by removing one or more vendors that are faring poorly while still covering your learning objectives. (Don’t select a vendor with only one or two suitable courses unless the courses are exceptional.) This last step is the most subjective part of your process. If you have an evaluation team, you’ll have to work closely and refer often to your written notes. After several hours of discussion, you’ll have a small group of courses that meet your learning objectives from a small number of vendors.

Step 4: Package and implement
The greatest portfolio in the world will flop without a solid implementation plan. Although the focus of this article is on selection, we would be remiss not to identify the four important issues you will face in preparing for rollout:

• How will you present the portfolio? In a printed booklet? In a page on your corporate intranet?

  
  

• How will you prepare people for e-learning, and how will you make the case for signing up and completing courses?

  
  

• What instructions will you provide to help people access and use the courses? What special instructions will you give to enable them to learn effectively? (You’ll have to refer back to your evaluation notes to identify where special instruction is required.)

  
  

• What feedback mechanisms will you employ, and how will you measure the success of your program?

If you can, recruit assistance from your creative service department and information technology to build a strong implementation plan. Also, seek out an internal change management expert for advice. Don’t underestimate the work required to successfully implement your carefully chosen portfolio.

Workforce, May 2002, pp. 42-46 — Subscribe Now!