How to Score a Behavior-Based Structured Interview

Much unlike traditional interview results, structured interviewing offers a planned scoring system for each question used in the interview. At the time questions are developed for the interview, the team works out a rating scale for a continuum of possible answers, ranging from a great answer to a poor one.

Such judgments as “excellent” and “poor” are tied directly to behavioral objectives in the job analysis, not to the flair or style with which the candidate can schmooze. An excellent answer is one that reflects probable success in performing the related job task. A marginal answer is one that reflects probable difficulty in performing the job task. A poor answer is one that reflects probable failure in performing the related job task.

But who is to say which answers are excellent, marginal, or poor? That judgment is up to the same team members who know the job best and are familiar with employees’ relative success in accomplishing it. In most cases, rating scales for interview questions are developed in brainstorming sessions by the job team. Excellent, marginal, and poor responses are specified as “anchors” on a five-point scale. Even though applicants may not hit any of these predicted answers on the nose, their responses can nonetheless be placed meaningfully at some point on the continuum marked by these anchors.

Here’s one example of an interview question with an accompanying rating scale. Note that these are not multiple choice questions. Only the interviewers see the suggested responses following the question.

Question:
“When reviewing a title search for a potential building site, you come across a reference in a recorded document to ‘other unrecorded covenants’ related to a subject property. What do you do in reporting on the condition of title?”

It’s worth emphasizing that interviewees are not expected to say the precise words suggested in the anchor responses. Interviewers simply use these benchmarks to determine the appropriate numeric score of the applicant’s actual answer. One or more interviewers mark scores for each question and also take notes on the content of the applicant’s answer. After all applicants have been interviewed, the job team or other authorized hiring managers in the company begin the task of comparing, compiling, and reconciling their scores for individual applicants.

If scores between raters differ by more than one number (for example, a “5” and a “3”), the team discusses the applicant’s response and seeks to bring the scores closer together. Ratings are then averaged for each question and totaled as a record of the applicant’s performance.

This scoring system makes it relatively easy to compare several candidates on the merit of their responses. It goes a long way toward eliminating distortions caused by interviewer bias, differences in questions, and interpersonal factors such as physical attractiveness, age factors, style of dress, and so forth.