Choosing the Right Assessment Vendor

Choosing an assessment vendor can be a confusing process. There are literallyhundreds of vendors selling all kinds of products to help you hire employees.The majority of these vendors will assure you that their products are perfectfor your company or can be easily modified to fit your needs. While theappropriate use of assessment tools can provide significant ROI, using the wrongtool can result in substantial losses of time and revenue. The key to ensuringthat you reap the benefits of assessment tools while avoiding the risks is touse a well-planned and thorough vendor-selection process.

    Effective vendor selection requires three basic steps:

Step 1: Create a team and a process
    Unless you are addressing a very simple and straightforward staffing need, itis a good idea to create a formal team and project plan to oversee thedeployment of assessment tools. The functions of this team include determiningassessment tool requirements, researching and gathering information fromvendors, reviewing and evaluating vendor solutions, and outlining processes forongoing use and support of assessment tools once they have been implemented. Theteam should include representatives from the major stakeholder groups that willbe affected by the use of assessment tools.

    In addition to HR, consider including personnel from IT, legal, andoperations. It is also useful to include someone with expertise in assessmentdesign and validation to help provide guidance during the process.

Step 2: Perform a needs analysis
    Before speaking with any vendors, clearly define your reasons for usingassessment tools. Outline specific business issues you wish to address and theoutcomes you expect. Consider operational issues such as where you plan to useassessments, how the assessments will be administered, how you will interpretthe results and handle assessment data, and who will provide ongoing support andtraining for the assessment tools. Define constraints such as budgets, personnelresources, and technology requirements.

    This needs analysis should clearly define what the assessment tools must doin order to be successful. Failure to conduct a good needs analysis will resultin vendors telling you what you need instead of allowing you to be in control ofthe process.

Step 3: Conduct a formal request for proposal (RFP)
    The assessment-tools market is rapidly changing, with new solutions beingintroduced virtually every month. It is always a good idea to get proposals froma range of vendors before deciding on any single solution. The RFP that you sendto vendors should detail the specific business objectives you want to achieve,which jobs you want to support with assessment tools, the number and location ofcandidates, and any technology requirements.

    The RFP should also ask vendors to supply some very specific information thatwill help you select the best one for your needs. The following 10 questions canbe used to evaluate a vendor’s solutions and their ability to help you meetyour goals.

1. What assessment techniques do you provide? It is important to understandthe types of assessments offered by a vendor. Do they sell primarilyqualifications screens, knowledge tests, talent measures, or background checks?Explore which assessments reflect their core capabilities and which ones theyoffer through partnerships or outsourcing. Ensure that the assessments offeredby the vendor align with your business objectives.

2. How do you determine what are the most critical aspects of jobperformance? To provide effective and legally sound assessment tools, a vendormust be able to clearly define the critical factors that influence job success.This process of defining job performance is commonly known as job analysis. Manyvendors will cut corners in this up-front work, so it is important that theyclearly demonstrate the processes they use to develop links between theirassessment tools and job performance.

3. What validity data can you provide for this product? Validity dataprovides the proof that an assessment tool actually predicts job performance.All assessment vendors should have summaries of validity data that are readilyavailable to prospective clients. This data should provide evidence of thetool’s performance on jobs similar to those for which you plan to use it. Lookfor “hard numbers” linked to well-defined measures of performance (e.g.,supervisor ratings, tenure).

    Avoid vendors that rely heavily on vague anecdotal statements about theirtool’s perceived effectiveness and impact. Ask for references from specificclients that you can contact to learn more about the effectiveness of theirassessments.

4. Do you have evidence regarding the legal defensibility of this tool? Youhave a right to know if a vendor’s product has ever been challenged in court.It is also important to determine if the tool displays adverse impact (i.e.,members of certain protected classes do not perform as well on it as members ofthe majority group).

5. How much consulting and customization is required to get your systemconfigured for our organization, and how much time will this take? Some systemsare built so that they can be used “off the shelf” with very littlecustomization, while others require more time and effort to configure. In mostcases, taking time to create a customized system offers advantages in accuracyof prediction and higher levels of legal defensibility. However, it alsoincreases development costs.

6. What delivery methods are available? Paper and pencil, telephone, and theInternet are the three main ways to deliver assessment tools to applicants. Manycompanies offer all three options, but differ widely in their level of expertisewith each method. The technology around telephone and Internet screening can bea major source of problems, but it can also offer significant benefits incomparison to paper-and-pencil delivery methods. It is a good idea to askvendors for specific examples of using similar technology to deliver theirassessment tools.

7. How easy will the results be for recruiters and HR personnel to interpret?You want a system that provides high-level results that are easy for non-expertsto understand but also allow in-depth information for the purposes ofdocumentation and more detailed investigation. Some assessments can even be usedfor both candidate evaluation and development of newly hired employees.

8. Does the system consider the needs and feelings of the test taker? Theexperience of applicants is a critical factor in defining the success of anassessment tool. Upsetting applicants is bad recruiting and can lead to legalaction. What steps has the vendor taken to ensure that candidates will perceiveits tools to be job relevant and culturally unbiased?

9. How is the system priced? It is important to clearly understand the feeschedule as it relates to the manner in which you plan to use the test. Askvendors to outline up-front fees for system design and configuration, as well asongoing usage fees.

10. What ongoing support will you receive? What services does the vendorprovide to ensure the ongoing effectiveness of its assessment tools? Does itprovide quarterly or semi-annual reviews of the performance of its assessmenttools, including EEOC summaries? What sort of ongoing customer and help-desksupport is offered? You want to look for vendors that will provide activeongoing support and will not simply disappear after you have bought their tools.

    You should also consider the financial and personnel stability of thecompany. Some assessment companies depend heavily on the expertise of one or twokey individuals. Try to find out if this is the case, and ask what transitionplans the companies have in place should these individuals leave.

    In general, the RFP process should be approached as a dialogue, not aninterrogation. Most assessment vendors are ready and willing to respond toformal RFPs. However, be respectful of the time it requires to respond to yourquestions, and do not ask for solutions that you are not serious aboutimplementing. It is also helpful to let vendors know in advance about what sortof budget constraints you have. It will save both parties a lot of time, and youmay be pleasantly surprised by the alternative solutions they will propose.

Workforce Online, December 2002 — Register Now!