Liam Healy & Associates

chartered occupational psychologists

Selection System Design

The correct match between job requirements and individual characteristics will produce happier employees and a more efficient organisation hence the need for an efficient and effective selection system.  The financial and functional consequences of ineffective selection can be considerable, and can drag a company to its knees.

The major part of our work involves designing selection systems and delivering selection services on behalf of clients, either for a specific job, or on an organisation wide basis.  This can entail either evaluating an existing system or developing a new system from scratch.

Many of the systems we develop are designed to be used on line, and because of our Human Factors/User Psychology expertise, we can combine high quality selection system and  practical on line user interface design under one roof. Typically we will design the selection process and the user interface, then we work closely with a small number of specialist IT solutions providers to provide the programming and code development framework. Effective risk management and control is always part of the staged delivery model we use.

The systems we design can involve all or some of the standard phases of the selection process. The first stage of the selection process involves carrying out an analysis of the job itself to identify the key elements or dimensions involved in it. From this  a person specification is produced. This stage also involves identifying the criteria that can be used to assess the effectiveness of the system as well as attracting suitable applicants. Next, the  tools available to the selector are assessed in terms of their psychometric proprieties (reliability and validity) and their suitability for the particular purpose before being  incorporated into a selection system and used to arrive at a decision. The final and possibly most important stage is that of evaluation of the system used in terms of its effectiveness and the effect it has upon the candidates.

1. Job Analysis

Job Analysis is carried out to determine the key tasks in a job. These then become the basis of the criteria to be predicted. This allows the tools subsequently used to be structured, that is, directly related to the knowledge, skills, abilities and other factors  that have been shown by job analysis to determine effective job performance.

There are two basic approaches to job analysis -

A logical or task oriented approach includes methods that involve only a small inferential task in the observation and report of task variables and then a larger inferential task in determining the attributes needed to do the task.

Person/theoretically oriented methods are more psychological because there is a larger inferential task involved in the report of worker variables from the observation of a task and then a smaller one between that and the deduction of the required attributes. Similar methods include observation and expert analysis, which can provide high quality information and are  well suited to  situations where there are only a few job holders.

2. Success Criteria

Before producing the person specification we must develop criteria, or the standards against which the predictive value of the selection system may be subsequently measured. Failure to choose criteria with care  is  the root of much poor prediction. In many cases expediency dictates the choice of criteria and that the convenient availability of a criterion is erroneously viewed  more important than its adequacy

By deciding upon the criteria before the system begins to operate, we can devise a method to collect the relevant information as the job goes on, and we can  evaluate the selection system fairly. Delaying the choice of criteria until the system is in use invites choosing those criteria which support the methods that have been used.

Criteria can be at three levels - immediate level, level of expected results, and ultimate or organisational level. We can also identify three different types of criteria - production, personnel or judgmental. Criteria must also be reliable i.e. consistent over time.  and valid i.e. measure true performance at work. We also need to take account of criterion coverage; contamination; dynamism; and interrelationships between criteria. 

3. Producing the Person Specification

If the purpose of a job analysis is to produce a description of the key tasks or elements in a job then the person specification is a description of the characteristics required of a person who can carry out those activities. The person specification is usually based on a plan, the points on which should  be relevant, independent, assessable and should have enough questions to avoid hasty conclusions but not so many so as to be laborious. There are a number of different models, one of the more straightforward and commonly used ones is  Rodger's Seven Point Plan. To reflect the fact some characteristics are more important than others, a distinction is usually made between those which are essential and those which are desirable.

We must be sure that the person specification is an accurate reflection of those individual human attributes that are actually required to do the job, this is difficult but it is possible and  must be done at   a detailed level since global estimates are unlikely to be sensitive enough.

Often, the behavioural i.e. competency, related part of the person specification is the most difficult to define. This is a very large area to cover, please read the discussion on competencies.

4. Attracting  and Dealing With Candidates

This is the most often overlooked part of selection, and a high candidate withdrawal rate is often a symptom that the organisation has failed to effectively engage with applicants.  We see selection  as a series of interactions between the organisation and the applicant, and at each stage both decide whether or not to continue the interaction. This continues until either party withdraws from the process or a job offer is made and accepted. This demonstrates the importance of the careful use of the   information  on which applicants base their decisions.

We know from research that that applicants are initially concerned with functional information, but  also place great weight on information which they obtain from personal sources,  when this is not displayed it can be offset by presenting a high level of information. Applicants also prefer selection  to be participative and non directive, so they feel that they can exert some control over the process, they like to be able to see the relevance of the procedures used, how information obtained will be used and  prefer methods which provide them with a lot of feedback on their performance.

5. Choosing and Using The Correct Assessment Tools

We pointed out earlier that the tools used must be to be structured, that is, directly related to the knowledge, skills, abilities and other factors that have been shown by job analysis to determine effective job performance.  They also need to meet minimum reliability and validity criterion. Depending on the outcome of the Person specification we have a wide range of existing tools.

Very often, clients wish us to develop new assessment tools which they, the client own and are free to use.

6. Making a Decision

There are two ways of making a decision; the clinical method, where a person uses their own expertise and judgement to make a decision, and the actuarial method where a number of factors are considered according to some model of decision making, this is generally the more reliable and valid method. While there were some cases of the clinical method equalling the actuarial, there a few none where it is actually better. However, the difference is not likely to be large, in cases such as one off appointments it can still be appropriate.

Some characteristics may be valued more highly than others, in which case each may be given a weighting, by which a mark can be multiplied to reflect the different values attached. Usually this is the most practical method, but in cases where there are a number of applicants (100 or more) and follow up data is available, then a multiple regression approach can be used to discover the best combination of  predictors of work performance. (NOTE: The multiple regression approach assumes that deficiencies in one area can be made up for by strengths in others but where this is not so minimum cut off points may be used to filter out those applicants who do not come up to scratch on the essential requirements of the person specification.)

A common approach which we are often asked to design  is worth  mentioning here is PROFILE MATCHING. In profile matching a profile of what the scores on  various measures of what an ideal candidate might look like  is produced, and  an attempt is made to match the profiles of candidates  to it. The candidate with the smallest sum of differences (which should be squared before adding to reflect the difference in importance between large and small differences)  is the one chosen. However, there are problems with what to use as criteria and  correlation cannot be used as a method of measurement since it is particularly sensitive to the relative rather than absolute shape of the profile.  For these reasons we tend to advise against this approach.

7. Evaluating the Selection System

The most overlooked part. Please read the sections on Selection System Audit and Validity for an overview of how we evaluate a selection system.