Liam Healy & Associates
chartered occupational psychologists
Selection System Design
The correct match between job requirements and individual characteristics will produce happier employees and a more efficient organisation hence the need for an efficient and effective selection system. The financial and functional consequences of ineffective selection can be considerable, and can drag a company to its knees.
The major part of our work involves designing selection systems and delivering selection services on behalf of clients, either for a specific job, or on an organisation wide basis. This can entail either evaluating an existing system or developing a new system from scratch.
Many of the systems we develop are designed to be used on line, and because of our Human Factors/User Psychology expertise, we can combine high quality selection system and practical on line user interface design under one roof. Typically we will design the selection process and the user interface, then we work closely with a small number of specialist IT solutions providers to provide the programming and code development framework. Effective risk management and control is always part of the staged delivery model we use.
The systems we design can involve all or some of the standard phases of the selection process. The first stage of the selection process involves carrying out an analysis of the job itself to identify the key elements or dimensions involved in it. From this a person specification is produced. This stage also involves identifying the criteria that can be used to assess the effectiveness of the system as well as attracting suitable applicants. Next, the tools available to the selector are assessed in terms of their psychometric proprieties (reliability and validity) and their suitability for the particular purpose before being incorporated into a selection system and used to arrive at a decision. The final and possibly most important stage is that of evaluation of the system used in terms of its effectiveness and the effect it has upon the candidates.
1. Job Analysis
Job Analysis is carried out to determine the key tasks in a job. These then become the basis of the criteria to be predicted. This allows the tools subsequently used to be structured, that is, directly related to the knowledge, skills, abilities and other factors that have been shown by job analysis to determine effective job performance.
There are two basic approaches to job analysis -
A logical or task oriented approach includes methods that involve only a small inferential task in the observation and report of task variables and then a larger inferential task in determining the attributes needed to do the task.
Person/theoretically oriented methods are more psychological because there is a larger inferential task involved in the report of worker variables from the observation of a task and then a smaller one between that and the deduction of the required attributes. Similar methods include observation and expert analysis, which can provide high quality information and are well suited to situations where there are only a few job holders.
2. Success Criteria
Before producing the person specification we must develop criteria, or the standards against which the predictive value of the selection system may be subsequently measured. Failure to choose criteria with care is the root of much poor prediction. In many cases expediency dictates the choice of criteria and that the convenient availability of a criterion is erroneously viewed more important than its adequacy
By deciding upon the criteria before the system begins to operate, we can devise a method to collect the relevant information as the job goes on, and we can evaluate the selection system fairly. Delaying the choice of criteria until the system is in use invites choosing those criteria which support the methods that have been used.
Criteria can be at three levels - immediate level, level of expected results, and ultimate or organisational level. We can also identify three different types of criteria - production, personnel or judgmental. Criteria must also be reliable i.e. consistent over time. and valid i.e. measure true performance at work. We also need to take account of criterion coverage; contamination; dynamism; and interrelationships between criteria.
3. Producing the Person Specification
If the purpose of a job analysis is to produce a description of the key tasks or elements in a job then the person specification is a description of the characteristics required of a person who can carry out those activities. The person specification is usually based on a plan, the points on which should be relevant, independent, assessable and should have enough questions to avoid hasty conclusions but not so many so as to be laborious. There are a number of different models, one of the more straightforward and commonly used ones is Rodger's Seven Point Plan. To reflect the fact some characteristics are more important than others, a distinction is usually made between those which are essential and those which are desirable.
We must be sure that the person specification is an accurate reflection of those individual human attributes that are actually required to do the job, this is difficult but it is possible and must be done at a detailed level since global estimates are unlikely to be sensitive enough.
Often, the behavioural i.e. competency, related part of the person specification is the most difficult to define. This is a very large area to cover, please read the discussion on competencies.
4. Attracting and Dealing With Candidates
This is the most often overlooked part of selection, and a high candidate withdrawal rate is often a symptom that the organisation has failed to effectively engage with applicants. We see selection as a series of interactions between the organisation and the applicant, and at each stage both decide whether or not to continue the interaction. This continues until either party withdraws from the process or a job offer is made and accepted. This demonstrates the importance of the careful use of the information on which applicants base their decisions.
We know from research that that applicants are initially concerned with functional information, but also place great weight on information which they obtain from personal sources, when this is not displayed it can be offset by presenting a high level of information. Applicants also prefer selection to be participative and non directive, so they feel that they can exert some control over the process, they like to be able to see the relevance of the procedures used, how information obtained will be used and prefer methods which provide them with a lot of feedback on their performance.
5. Choosing and Using The Correct Assessment Tools
We pointed out earlier that the tools used must be to be structured, that is, directly related to the knowledge, skills, abilities and other factors that have been shown by job analysis to determine effective job performance. They also need to meet minimum reliability and validity criterion. Depending on the outcome of the Person specification we have a wide range of existing tools.
Very often, clients wish us to develop new assessment tools which they, the client own and are free to use.
6. Making a Decision
There are two ways of making a decision; the clinical method, where a person uses their own expertise and judgement to make a decision, and the actuarial method where a number of factors are considered according to some model of decision making, this is generally the more reliable and valid method. While there were some cases of the clinical method equalling the actuarial, there a few none where it is actually better. However, the difference is not likely to be large, in cases such as one off appointments it can still be appropriate.
Some characteristics may be valued more highly than others, in which case each may be given a weighting, by which a mark can be multiplied to reflect the different values attached. Usually this is the most practical method, but in cases where there are a number of applicants (100 or more) and follow up data is available, then a multiple regression approach can be used to discover the best combination of predictors of work performance. (NOTE: The multiple regression approach assumes that deficiencies in one area can be made up for by strengths in others but where this is not so minimum cut off points may be used to filter out those applicants who do not come up to scratch on the essential requirements of the person specification.)
A common approach which we are often asked to design is worth mentioning here is PROFILE MATCHING. In profile matching a profile of what the scores on various measures of what an ideal candidate might look like is produced, and an attempt is made to match the profiles of candidates to it. The candidate with the smallest sum of differences (which should be squared before adding to reflect the difference in importance between large and small differences) is the one chosen. However, there are problems with what to use as criteria and correlation cannot be used as a method of measurement since it is particularly sensitive to the relative rather than absolute shape of the profile. For these reasons we tend to advise against this approach.
7. Evaluating the Selection System