Structured Interviews May Be More Valid and More Reliable

Matthew O’Connell, co-founder of Select International, gives his view on evaluating interviews and assessments:

“There are two important terms for evaluating assessment systems. The first is validity, which refers to how accurately a selection system predicts performance on the job. Does HR really care about anything else, with the possible exception of how long it takes to administer the tests? Validity essentially equals ROI, but in and of themselves, no tests are valid: it’s the way you apply the results that determines validity.

“The second term is reliability, which refers to how consistent the results are over time and on a wide range of people. If a system is reliable, two people assessing the same candidate should get the same results.

“The way you determine validity of an assessment system is to utilize a correlation coefficient. Correlation coefficients range between 0.0, which indicates that there is no relationship between test results and performance, and 1.0, which means that the test is 100% accurate in predicting performance. The best way to measure ROI on an assessment system is to ask about the validity of the system. Correlation coefficients between test results and job performance should be readily available when you ask.

“You should get the correlation as high as possible, and second, you necessarily have to do a lot of tests. You just have to administer the tests that most accurately predict performance. Unstructured interviews or semi-structured interviews are the most commonly used tools in the hiring process. The correlation between unstructured interviews and performance is only .2. It’s better than random, but only a little better.

“Unstructured interviews permit the interviewer to ask anything he or she wants, whether or not it’s valid. And unstructured interviews often don’t involved probing questions that yield further information about the reasons why a candidate responded as they did. If you don’t understand their thought process, their reasoning, then you don’t really understand how they might perform.

“Structured interviews, however, are twice as valid as unstructured interviews. When we have structured questions and standardized evaluation guidelines, we know we will get a .40 validity. Selection systems that test for the competencies that actually predict job performance have much higher validity.

SOURCE: From a speech by Matthew O’Connell, co-founder of Select International, at a conference on Recruiting & Hiring in a Tight Labor Market: New Practices in Recruitment & Selection, May 5-6, 1999.