To predict success, it is first important to decide what to measure and what combinations of factors will best enable one to predict success for a candidate in a certain position. What we are looking for is incremental validity. Incremental validity encompasses the amount of validity that any one measure has over and above another measure. It also relates to the combination of certain test tools best equipped to deliver the highest possible predictive validity for a particular group of predictive indicators e.g. a defined set of competencies, a required mix of personality traits, etcetera.
Based on the work of Nik Kinley and Schlomo Ben-Hur in their book “Talent Intelligence” (Kinley & Ben-Hur, 2013) and our experience within candidate assessment, Albright Life Sciences typically target five different kinds of match platforms on executive search assignments:
Candidate-job match: The level of match between a candidate’s competencies/characteristics and the position profile for which he/she is a candidate. Here we are looking to predict future performance, productivity, job satisfaction, etcetera.
Candidate-organization match: The level of match between a candidate’s competencies characteristics and the culture and working environment of a given company. This match often is recognized as a robust predictor of organizational citizenship behaviour, commitment and staff turnover.
Candidate-team match: The level of match between a candidate’s competencies/ characteristics and the colleagues he or she will be working with most closely. This match seems to predict the quality of relationships with co-workers.
Candidate-manager match: The level of match between a candidate’s competencies/ characteristics and the manager to which he/she will be assigned. This match seems good at predicting employees’ satisfaction levels with their manager.
Candidate-stakeholder match: The level of match between a candidate’s competencies/ characteristics and the stakeholders within which he/she must navigate. This match is often recognized as a robust predictor of quality of relationships with stakeholders and broader organizational behaviour.
As a headhunter, we are always seeking the right combination of tools and measures to help us predict success best in relation to our match platforms—we call this predictive validity. Predictive validity is a figure between 0 and 1 that indicates how robust the relationship between a particular factor (e.g. numeric ability) and a specified outcome is (e.g. job performance in relation to candidate-job match). According to S. Rothmann & E.P. Coetzer, (Rothmann, S. & Coetzer, 2003), Job performance is a multi-dimensional construct that indicates how well employees perform their tasks, the initiative they take and the resourcefulness they show in solving problems. Furthermore, it indicates the extent to which they complete tasks, the way they utilize their available resources and the time and energy they spend on their tasks.
Generally, a predictive validity of 0,30 is considered acceptable, and a validity of 0,5 is considered excellent. We would not expect to see a validity figure over 0,6 (Kinley & Ben-Hur, 2013).
Albright Life Sciences have identified six potential predictive indicators of high performance. They are generally supported by research performed in this area:
ability,
personality,
learning agility,
competencies,
past and current performance and,
prior experience
1. Ability
Ability is a combination of innate (e.g. intelligence) and learned skills.
Over the years, particularly cognitive ability—intelligence—has received much attention. Several measures of intelligence exist: IQ, GI (general intelligence), GCA (general cognitive ability), and GMA (general mental ability).
In line with research conducted, it is our experience that intelligence is a largely reliable predictor of future work performance. In fact, studies indicate that predictive validity of intelligence is approximately 0,38 (acceptable) for low-complexity jobs, 0,51 (excellent) for medium-complexity jobs, and 0,57 (excellent) for high-complexity jobs (Kinley & Ben-Hur, 2013). Many executive search firms—including Albright Life Sciences—recognize cognitive ability testing as a robust predictor of job performance, and we can see the validity of intelligence as a predictor rising as the complexity of the job increases. However, intelligence is not good at predicting performance in all jobs. For instance, when it comes to sales jobs research shows a significantly lower predictive validity for future high performance.
It is our experience that particularly when it comes to international headquartered companies with strong employer brands, cognitive abilities such as intelligence seem to be very highly prioritized. However, the extreme focus on intelligence sometimes backfires. A recent example is Nokia. A few years ago, Nokia boasted a very strong employer brand and were able to attract highly intelligent R&D people to the Copenhagen site. Over the span of a decade, the company employed hundreds of the most intelligent R&D and product management individuals that Denmark could muster. Unfortunately, with all these intelligent people gathered in one place, things seemed to become more and more intellectualized and theoretizised and less and less concrete, practical, productive and action-oriented. Many layers of bureaucratic work processes seemed to take over, creating decision vacuums and affecting overall performance negatively.
2. Personality
The predictive validity between personality and job performance has been a frequently studied topic in industrial psychology over the years (Rothmann, S. & Coetzer, 2003). From an overall perspective, research indicates that the predictive validity between personality and future performance ranges at approximately 0,30 (Kinley & Ben-Hur, 2013). This implies that personality is a good (acceptable) predictor, however, far from the levels seen with e.g. the cognitive ability of intelligence. Alternatively, unlike many measures of cognitive ability, personality measures typically do not have an adverse effect on disadvantaged employees, and thus, can enhance fairness in personnel decisions (Hogan, Hogan & Roberts, 1996).
Nonetheless, personality tests are probably the most widely used test tools when it comes to recruitment processes and with good reason, since as managers, we often forget that we typically start out with hiring for competencies yet end up firing for personality….
3. Learning agility
Learning agility is the ability and willingness to learn from experience, and subsequently apply that learning to perform successfully under new or first-time conditions. Learning agility is not a surrogate for IQ or personality variables. What often separates high performers from average performers is the ability to perform well under first time, challenging conditions. In fact, the innate ability to wrest meaning from experience (the essence of learning agility) often allows the high performer to deliver high performance faster than others do.
No clear measure of predictive validity exists for learning agility. However, when it comes to high performance it is our experience that learning agility is related closely to both current performance and longer-term potential and studies repeatedly have shown that the ability to learn from experience is what differentiates successful from unsuccessful executives. When we perform competency-based interviews and reference interviews, learning agility is one of the key high performance indicators we try to uncover, as time and again we find the best performing candidates having displayed impressive levels of learning agility. Especially when it comes to large, complex and politically driven organizations, it seems that learning agility is a particularly valued skill, as these organizations often are in a continuous state of flux and require the employees to speedily adapt and continually demonstrate improved learning and knowledge expansion.
4. Competencies
Competencies will help predict future high performance. However, only to a certain extend and only if uncovered with tools and processes that ensure an appropriate level of predictive validity.
Some executive search firms are happy only to employ structured, competency-based interviews to measure competencies. At Albright Life Sciences, we believe it is important also to employ a comprehensive assessment centre to accurately measure competencies. In fact, research supports that an assessment centre is the most effective method of measuring competencies. Assessment centres often involve multiple assessors who are observing a person or group of participants over a series of exercises and tests and rating them on a number of competencies. Assessment centres offer predictive validities of approximately 0,40. They are designed to give the assessors a well-rounded view of a person’s abilities and competencies—and include psychometric tests, presentations, written exercises, role plays, interviews, in-tray exercises/work sample tests, group exercises, business case studies, physical challenges or often a combination of some of them.
Job-simulating in-tray exercises tailored to a concrete job role or work sample tests (giving people a sample of work and then see how they perform) is a favourite of Albright Life Sciences and in fact delivers validities of up to 0,54 (excellent). Another form is situational judgment tests that encompass presenting people with realistic work scenarios and then asking questions about them. They are typically used for assessing job skills, like safety procedures, etcetera.
At Albright Life Sciences, we often employ comprehensive assessment centres when testing candidates for complex executive positions. A typical assessment centre will include the following mix of tests—of course, always taking the specific combination of factors that will best enable us to predict success for a candidate into consideration: 1) personality tests (we typically employ psychometric tests recognized for providing objective, detailed information about an individual’s predicted behaviours in the workplace; how they will fit into a work environment, work with others and to what roles they are most suited), 2) job-simulating assessment centre, tailor-made to test various competency-based hypotheses and the presence of a defined mix of key competencies, and 3) ability tests (psychometric tests typically measuring numeric, verbal and inductive abilities). In addition, we always employ unstructured and structured, competency-based interviews prior to the assessment centre.
5. Past and current performance
Future high performance will most probably emerge from today’s high performing employees. Thus, current and past performance is seen as predictors of future high performance. Private equity companies typically weigh historic performance of a company and management significantly higher than projected performance when evaluating risk and investment. This is simply because historic performance often is seen as the best predictor of future performance (if management has an historic track record of delivering on their budgets, there is a high probability that they will also deliver on future budgets).
No clear measure of predictive validity exists for past and current performance when it comes to high performance—and generally we have two opposite schools of thought—one claiming low predictive validity and the other claiming high predictive validity. However, it is our experience that the predictive validity of candidates with a continual track record of superior performance executed under conditions, contexts and cultures that are somewhat similar to that of a new employer, is relatively high when it comes to future performance—all other things being equal.
Moreover, as thorough reference checks are a critical part of the selection process at many executive search firms, it must largely be because past performance is seen as a good predictor of future behaviour.
6. Prior experience
It is the general assumption that prior experience in roles similar to a new role is predictive of future high performance in the new role. This assumption is supported by research. However, other things may dilute the effects of prior experience—e.g. organizational structures and processes (artefacts) in a new company that at first will be difficult to decipher for a newcomer, and values and basic underlying assumptions (perceptions, habits, beliefs, thoughts and feelings) that often are unconscious and taken for granted by the existing organization, but very difficult for a newcomer to decipher within the first months.
A headhunter has a tendency to raise an eyebrow when a person has a track record of holding many different jobs (especially with only a few years in between) as opposed to holding just a few jobs over a longer period. However, a recent study did not find a statistically significant relationship between how long a person has lasted in previous jobs and how long they will stay in their next one. Thus, prior experience will help to a certain level in predicting future high performance; however, other factors may contribute to reducing the predictive validity of this factor.
It is important to note that certain situational factors may affect job performance—hereunder the characteristics of the job, the culture, organisation, co-workers, etcetera. Moreover, dispositional factors may also affect job performance—hereunder attitudes, preferences, motives, etcetera. Both situational and dispositional factors may result in a person’s tendency to react to situations in a predisposed manner (House, Shane & Herrold, 1996) and should be considered in the overall evaluation.
As a headhunter, it is crucial to have the best intelligence platform available in order to know whether a candidate is a high performer, average performer or low performer in a future context. We need to generate an accurate understanding of the skills, expertise and qualities of the candidates. Various methods and tools are available to gather intelligence about candidates. There is no perfect formula; however, what we are looking for—and what any company should be looking for—is to employ a mix of tools and methods that in conjunction deliver the highest probable predictability of future overall performance.