Research Article: 2020 Vol: 23 Issue: 2
Patrycja Frontczak, Adam Mickiewicz University in Poznań
Katarzyna Pawlak, Adam Mickiewicz University in Poznań
Urszula Skorodzillo, Adam Mickiewicz University in Poznań
Milena Szymanowska, Adam Mickiewicz University in Poznań
Sandra Wilczynska, Adam Mickiewicz University in Poznań
Citation Information: Grobelny, J., Frontczak, P., Pawlak, K., Skorodzillo, U., Szymanowska, M., & Wilczynska, S. (2020). A conceptual model of the influence of résumé components on personnel decisions: a policy-capturing study on résumé screening. Journal of Management Information and Decision Sciences, 23(2), 51-72.
Based on a literature review of not only industrial and organizational psychology but also decision theory, we have developed a conceptual model of résumé screening. It postulates that personnel decisions concerning assignment to particular categories result from a gradual process with an underlying initial assumption, and the decision-making process varies depending on specific conditions. Under different conditions, decision makers utilize different résumé components (relevant, irrelevant and formal), whose impacts might interact with each other. We designed and conducted two policy-capturing experimental studies and employed a machine learning approach and a decision tree classification method to verify our conceptual model. The results indicate that it might be considered valid and might explain actual decisions regarding résumés. The data we have collected suggests that in a situation of certainty recruitment specialists make their decisions solely on the basis of information obtained from relevant résumé components and apply straightforward, i.e., noncompensatory, rules. However, when making decisions in a situation of uncertainty, recruitment specialists make an attribution and are influenced by the combined interactive effect of relevant, non-relevant and formal components of résumés. These decisions, in turn, are compensatory in nature. For example, positive personnel decisions regarding the appraisal of a résumé may be made if deficiencies in a relevant area are compensated for by an exceptional level of non-relevant or formal components.
Résumé Screening, Decision-making, Personnel Decisions, Policy-capturing Study.
Résumé screening is an initial step in the personnel selection process. Based on a review of submitted documents and their professional knowledge, recruiters (decision makers) must decide whether or not a certain applicant might be qualified and suitable for a given job. What is rather surprising is the relatively small number of studies on résumé screening given the fact that there is virtually no professional recruitment process in which this method is not employed (Cole et al., 2007; Thoms et al., 1999). The aim of this study was to further investigate the process underlying résumé screening and to attempt to explain some of the ambiguity that was not resolved by previous research. In order to achieve this, we first developed a theoretical model of résumé screening by integrating knowledge from both industrial and organizational (I/O) psychology, as well as from the judgment and decision-making (J/DM) fields. We then applied this model by designing and conducting two policy-capturing studies. Finally, we analysed the obtained results to verify whether or not the outcomes of the studies contradict the developed model.
This paper contributes to recent literature on a few counts. Firstly, we employed an experimental method and operationalization that better suits the nature of decision-making during résumé screening. Secondly, we drew upon knowledge from both I/O psychology and J/DM theories to develop our model. Finally, we emphasized the complex nature of J/DM related to résumé screening by a thorough analysis of interaction effects and conditions. We believe that we are able to clarify some of the remaining uncertainty regarding recent study results based on the aforementioned processes. Below, we firstly review and classify components of résumés which are the most common subjects of studies; we then describe our model with a description of the evidence that led to its development.
Over recent decades, researchers have examined the impact of a variety of résumé components (i.e. the formal elements or the information listed in these documents) on recruiters’ decisions. The subjects covered by such studies have varied greatly, but we believe they fit into three broad categories.
Relevant Components
The first two categories relate to the content of documents, i.e. the information listed by applicants. This information might be divided into two categories: one directly related to job requirements, and one insufficiently related to them. The basis of this division is the person–job fit theory (Kristof-Brown et al., 2005), according to which job performance is determined by a correlation of a variety of employee characteristics (such as knowledge, skills, abilities, traits, and other psychological predispositions) and the demands of a certain occupation. A group of résumé components that provide recruiters with direct cues and information as to whether or not an applicant possesses the characteristics that make him or her fit for the job will be classified into the first group (hereinafter referred to as relevant components). This group includes information about the applicants’ work experience, related coursework and education background, as well as their job-related skills and qualifications. In fact, the vast majority of résumé studies provide evidence that this information has a crucial impact on decision makers (Brown & Campion, 1994; Chen et al., 2011; Cole et al., 2007; Fritzsche & Brannick, 2002; Hakel et al., 1970; Oliphant & Alexander, 1982; Risavy, 2017; Singer & Bruhns, 1991; Thoms et al., 1999; Tsai et al., 2011).
Irrelevant Components
Interestingly, decision makers use not only job-related information to draw conclusions regarding applicants’ suitability for a position and to invite them to an interview. Previous research often addressed the impact of résumé components that have no clear association with the characteristics suggested by the job-fit theory. Researchers investigated components such as information about personal interests, hobbies, extracurricular activities (social, sports or volunteering), living arrangements, memberships of societies, or the use of certain (e.g. motivational) statements (Burns et al., 2014; Campion, 1978; Cole et al., 2007; Fritzsche & Brannick, 2002; Hakel et al., 1970; Harcourt et al., 1991; Hornsby & Smith, 1995; Hutchinson & Brefka, 1997; Knouse, 1994; Nemanick & Clark, 2002; Oliphant & Alexander, 1982; Renwick & Tosi, 1978; Rubin et al., 2002; Schramm & Neil Dortch, 1991; Tannous, 2012; Thoms et al., 1999). While it is debatable whether some of these factors (herein after referred to as irrelevant components) actually have some indirect or partial relation with job-related characteristics, there is no doubt that they do not correspond with the requirements listed in job descriptions and can hardly be recognized as job-related information? It is interesting that the results regarding the actual impact of these components on recruiters’ decisions are inconclusive. Some studies provide evidence that these components do not affect personnel decisions; however, some research shows that they have a direct impact on decision makers or affect personnel decisions under certain circumstances (Cole et al., 2007; Hutchinson & Brefka, 1997; Nemanick & Clark, 2002; Risavy, 2017).
Formal Components
Finally, the last category of résumé components we have distinguished does not refer to any of the information provided in these documents but rather to the way in which it is presented. Few studies focus on the topic of résumé length, fonts, colours and graphic elements, or their general visual appeal, aesthetic aspects or the general level of professionalism (Arnulf et al., 2010; Blackburn-Brockman & Belanger, 2001; Burns et al., 2014; Chen et al., 2011; Fritzsche & Brannick, 2002; Horn, 1988; Hornsby & Smith, 1995; Schramm & Neil Dortch, 1991; Schullery et al., 2009; Spinks & Wells, 1993; Thoms et al., 1999). These résumé components (hereinafter called formal components, as they do not directly provide any information) could at best have a partial, implied correlation with some characteristics of applicants, but it would be hard to provide any evidence about the actual connection with job-related qualifications or with job suitability level. Nevertheless, these formal factors could affect recruiters’ perceptions and decisions about applicants. As is also true in the case of irrelevant components, the results of previous research also reveal some ambiguity. For example, résumé length (not exceeding one page) proved to have a favourable impact on personnel decisions (Spinks & Wells, 1993), a negative impact (Blackburn-Brockman & Belanger, 2001), or a favourable impact only under certain circumstances (Thoms et al., 1999).
Interaction of the Impact of the Factors
We believe that this ambiguity in previous résumé research is of particular significance as its causes can be identified. That said, it should be noted that résumé studies frequently focus on the impact of a single (or a limited number of) factor(s), instead of considering more complex models that include interactions between factors (Cole et al., 2007; Graves, 1992), whereas J/DM literature repeatedly emphasizes that actual decision-making involves a combination and weighting of a variety of different cues and information (Born & Scholarios, 2004; Lindsay & Norman, 1984; Sekiguchi & Huber, 2011). Some researchers note that the interactions between the impacts of résumé components might play a crucial role in personnel decision-making. However, it must be emphasized that the exact nature of these interactions is far from clear (Born & Scholarios, 2004; Cole et al., 2007). We believe that the interactions between impacts of résumé components on personnel decision making are crucial in learning about the résumé screening process and play a prominent role in our conceptual model. We will attempt to demonstrate that the occurrence and nature of these interactions might be explained by reference to the conditions in which personnel decisions are made.
Based on a literature review of both I/O psychology and J/DM, we would like to propose a conceptual model of résumé screening that could serve as a descriptive decision theory (Hansson, 1994). Below, we present a description of this model along with evidence supporting each of its components.
The Decision
First of all, we would like to begin with the outcome of the résumé screening process, i.e. the personnel decision itself. The majority of previous studies used employability measures as an operationalization of a dependent variable (Brown & Campion, 1994; Burns et al., 2014; Campion, 1978; Cole et al., 2005; Knouse, 1994; Tsai et al., 2011; van Toorenburg et al., 2015). Even though employability might be a useful factor in other fields, e.g. labour market policies (McQuaid & Lindsay, 2005), we believe it is not a valid measure for JD/M studies. In fact, previous authors have indicated that decision makers’ judgements might be a different construct from employability (Rynes & Gerhart, 1990).
Employability measures are presented on interval or ratio scales, and the assumption here is that a personnel decision might be given as a quantitative value that represents an outcome of the decision process. This assumption is heavily grounded within classical, rational decision-making theories which presume that the decision maker acts in a rational and logical manner, taking into account all the available information and weighting it properly. Therefore, this process might be analysed with the use of a linear model (Elrod et al., 2004; Hansson, 1994). The problem is that these theories do not necessarily reflect the actual processes when it comes to making social judgements and decisions (Takemura, 2014).
As a matter of fact, actual outcomes of the résumé screening process have an ordinal or even dichotomous nature. In other words, decision makers either try to select a limited number of suitable employees or else they decide whether to accept or reject a certain applicant (Born & Scholarios, 2004; Risavy, 2017). This is the reason why many recent findings regarding résumé screening have probably lacked validity, as the employment of a methodology that fits for interval or ratio outcome variables in this particular field of study has been criticized (Born & Scholarios, 2004). Consequently, in our conceptual model the outcome of the résumé screening process is a decision of an ordinal nature. As a result of résumé screening, a certain applicant might be rejected from further consideration (a negative decision might be made) or accepted and invited to subsequent stages of the recruitment process (a positive decision). From a practitioner’s point of view, it might also be possible to make a partially positive decision, i.e. to accept an applicant as a consequence of their résumé screening, but only under certain circumstances (e.g. when no more suitable applicants can be found).
Initial Assumption
JD/M literature suggests that the majority of decision-making strategies begin with an initial assumption (Lindsay & Norman, 1984; Nosal, 1997). During the recruitment process, the cost of an incorrect hiring decision (i.e. hiring a person who does not fit) is substantial (Jagacinski, 1991). For this reason, it is reasonable to expect that decision makers initially assume that applicants whose résumés are screened are not fit for the job in question. Thus, the initial assumption involves a negative decision. Subsequently, by collecting evidence or using other strategies, decision makers might try to reject this initial assumption in favour of the applicant. Fritzsche and Brannick (2002) provide evidence for this mechanism with a negative evaluation model (decision makers try to abide by their negative decisions). Furthermore, the results of numerous studies regarding the prime importance of missing information in the résumé screening process provide important evidence for the occurrence of this initial assumption (Cole et al., 2007; Garcia-Retamero & Rieskamp, 2009; Highhouse & Hause, 1995; Jagacinski, 1991; Motowidlo, 1986; Sekiguchi & Huber, 2011; Stone & Stone, 1987). This missing information might result in fewer rejections of the decision makers’ initial assumption and therefore more frequently result in negative decisions.
Cognitive Schemas and Job Descriptions
An important assumption drawn from JD/M literature is that when a decision is being made, it is done so according to some standard (Hansson, 1994). Cognitive schemas might serve as such a standard, since many authors emphasize their role in the résumé screening process (Cole et al., 2007; Graves, 1992; Thoms et al., 1999). Cognitive schemas are representations of one’s knowledge on a certain topic that help one to understand and organize information received from the environment and make predictions (Cole et al., 2007; DiMaggio, 1997). In the case of occupations, these schemas might contain knowledge about an employee who fits the description well (a prototype employee) along with the characteristics responsible for his or her suitability and the attributes that one needs to demonstrate in order to be included in the cognitive schema of a suitable employee (Thoms et al., 1999). Certain résumé components might represent these attributes, i.e. they might inform decision makers whether an applicant possesses the requisite characteristics and therefore whether he or she could be classified into a particular cognitive schema. Cognitive schemas develop alongside experience (Graves, 1992), but the choice of a certain schema is determined by an external factor, which in this case is the job description. This is a matter of utmost importance, as the job description actually sets the boundaries for decision makers during résumé screening. Still, researchers have often failed to take this issue into consideration and have not included this element when trying to investigate personnel decision making (Sekiguchi & Huber, 2011). To summarize, based on the information described above, our conceptual model includes the following mechanism of résumé screening: every time a decision maker screens a résumé, he or she tries to decide whether a certain applicant could be classified into a cognitive schema that is determined by both the decision maker’s experience and a particular job description. The information derived from résumé components shows the decision maker whether or not the applicant possesses certain attributes defined by a cognitive schema. This process results in a decision about classifying a certain applicant into a cognitive schema of a fitting employee and consequently in positive or negative personnel decisions.
Initial Screening
Another important assumption that comes from JD/M literature is that the decision-making process is in fact gradual (Hansson, 1994; Nosal, 1997). In the case of résumé screening, two major steps can be identified: the first involves an initial screening, i.e. whether one should give the résumé a second look at all and screen it thoroughly later; the second involves making an actual personnel decision after an in-depth assessment of a résumé (Born & Scholarios, 2004; Nosal, 1997; Thoms et al., 1999). Few authors emphasize the complexity of the pre-screening step (Cole et al., 2007); however, we believe that decision makers get important information at this stage, and an important condition for a further more thoroughly considered decision is defined, i.e. whether a decision regarding a certain résumé (and therefore an applicant) is made in certainty or uncertainty. This initial screening is probably done solely on the basis of the relevant résumé components, primarily on the information about work experience. This is supported by study results that employed eye-tracking methods and which suggest that decision makers look at previous job titles and respective employment dates within the first few seconds (Evans, 2012; Pracuj Solution Group, 2015). Decision-making processes under certainty or uncertainty differ greatly (Lindsay & Norman, 1984) and it is a major drawback that little or no attention has been paid to this matter in previous research. In our opinion, the differences between these two conditions might explain the discrepancy in previous study findings as this situation would cause decision makers to employ different strategies and take different components into consideration.
Certain and Uncertain Conditions
There is evidence to suggest that decision makers could employ different strategies in the same decision task (Elrod et al., 2004), but it is still unclear when each of them is used. In our opinion, as the degree of certainty in a decision-making process determines its course, this degree could be used as an explanation. Two major conditions in résumé screening can be identified: if at the initial screening stage the decision maker recognizes a certain applicant as unquestionably suitable or not, the conditions could be assumed to be certain; if, on the other hand, it is difficult to determine at a glance whether an applicant is a good fit or not, the conditions should be assumed to be uncertain.
Certain Conditions
If all the attributes (included in a cognitive schema and the job description) related to relevant résumé components are present, it is sufficient to determine whether the particular applicant is suitable (e.g. if the applicant has listed enough work experience and sufficient education) or not. In such a case, there is no need for further information from other résumé components (Cole et al., 2007). In this certain situation, a decision might be made in accordance with, for example, the additive rule (Elrod et al., 2004), i.e. if each attribute of the decision object meets the minimum cut off criteria, the higher the rating of these attributes, the greater the chance of a positive personnel decision being made.
On the other hand, when recruiters deal with a clearly unsuitable applicant’s résumé, it is reasonable to assume that they employ some kind of non-compensatory rules in order to reach a decision as there is evidence of their use in this process (Cole et al., 2007; Sekiguchi & Huber, 2011; Webster, 1964). There could be a special use of the conjunctive rule, which stipulates that a negative decision towards an object that is based on an unfavourable appraisal of one attribute cannot be overridden by any number of positive evaluations of the other attributes (Elrod et al., 2004; Sekiguchi & Huber, 2011). For example, applicants with no experience at all but with superbly formatted professional résumés will still be rejected from the recruitment process. As the majority of résumés that need to be screened in a single recruitment process are from unsuitable applicants, this rule might be particularly useful as it is less cognitively demanding than the others (Sekiguchi & Huber, 2011) and is especially useful for reducing the number of options to choose from (Elrod et al., 2004).
Uncertain Conditions
Apart from the aforementioned situations of certainty, the applicant might fail to meet only one of a series of requirements; in other words, during a résumé screening the decision maker does not identify only one attribute (amongst many) required to include a certain applicant in the cognitive schema. Alternatively, the applicant might not quite meet some quantitative criteria, e.g. he or she might list only one year and ten months of certain work experience, but two years are required. Few authors emphasize that a more complex and demanding cognitive mechanism might be employed during this screening (Graves, 1992; Sekiguchi & Huber, 2011) and the purpose of our conceptual model is to provide an explanation of just such situations. We believe that in such uncertain situations, complex and non-linear rules for decision making which are based on the integration of various pieces of information could be employed. In fact, a few examples have been provided in the literature. Researchers have described the pervasive influence of cut offs, i.e. a situation in which an object that only just failed to meet the cut off criteria for a decision is evaluated in a different manner than those which failed to meet the criteria unambiguously (Elrod et al., 2004; Sekiguchi & Huber, 2011). Similarly, studies suggest that an applicant could make up for his or her single weakness (in other words, the lack of a certain attribute, i.e. the information provided in a résumé that would allow this candidate to be included in the cognitive schema of the fitted employee) with some strong asset, but this compensation would be impossible for general weakness (Knouse, 1994; Thoms et al., 1999).
Based on this, we assumed that decision makers in uncertain situations will screen résumés in accordance with compensatory rules. They specify that a negative decision towards an object that was a result of a lack of certain attributes might be compensated for by positive evaluation of another attribute (Elrod et al., 2004). For example, a résumé that listed fewer months of work experience than was required could still be accepted during recruitment if the applicant included other information, such as interests or extracurricular activities related to the occupation.
JD/M literature suggests that in uncertain situations the decision-making process is more time consuming as decision makers try to gather further evidence (Lindsay & Norman, 1984; Nosal, 1997). In fact there is also evidence that when recruiters do not find information related to work experience, they use other résumé components to elicit information, and they attribute other characteristics (allowing the inclusion in cognitive schemas) to applicants (Born & Scholarios, 2004; Cole et al., 2007). As we stated above, there is also evidence that résumé components interact when impacting recruiters’ decisions, but the results regarding this are dubious. In our opinion, the impact of the interaction between résumé components occurs particularly in situations such as the one discussed above in which conditions of uncertainty are present, i.e. the importance of irrelevant and formal components depends on the level of relevant résumé components. In uncertainty, decision makers try to attribute characteristics to applicants (or in other words, attributes that allow the applicant to be classified in the cognitive schema) by combining information from relevant, irrelevant and formal résumé components.
Conceptual Model and Hypotheses
Figure 1 presents the model that we have developed. It starts with a comparison of an applicant’s résumé against the cognitive schema of a suitable employee (determined by decision makers’ experience and the job description). This comparison is gradual and at the initial stage decision makers get information regarding whether the situation that they are dealing with is certain or not. Later, during a thorough evaluation, more complex strategies are employed depending on the conditions. In the case of certainty, the information about the applicant’s characteristics is drawn only from relevant résumé components. However, in uncertainty, the decision maker tries to attribute characteristics to the applicant in a complex, compensatory manner by utilizing all available information from relevant, irrelevant and formal components of the résumé (which in this scenario interact and therefore impact the recruiter’s decision). Both of these strategies lead to categorical, positive (and partially positive) or negative decisions regarding the acceptance or rejection of an applicant from further consideration in a given recruitment process.
To test this model, we designed experimental studies whose results allow us to verify whether personnel decisions are made in accordance with this model, or whether they are made differently and therefore the model is invalid. Hence, we expect that:
H1 Relevant résumé components impact decisions during the résumé screening process under certainty
H2 Relevant, irrelevant and formal résumé components impact decisions during the résumé screening process under uncertainty
To test our hypotheses, we conducted two studies according to the policy-capturing plan in which participants screened résumés. We employed an experimental, full-factorial design with within-subject manipulations. We also used between-subject manipulation to verify whether the effects were significant regardless of the job context. We created an orthogonal cue design by completely crossing three factors (résumé components) on three levels each, resulting in 27 different résumés being screened for each subject.
Participants’ Characteristics
The participants of this study were professionals experienced in personnel selection. To participate in the study, they had to have at least six months of experience. The average length of experience was 3.78 years and every participant was an employee of a recruitment agency. There were 42 participants, 37 female and 5 male, aged 25 to 39 (M = 27.29, SD = 6.15). All of the participants had a university degree, most of them in social and management sciences (n = 25). As each participant made 27 decisions, the actual sample size for this study was 1,134.
Sampling Procedures
The participants were recruited via professional social services and e-mails sent to recruitment agencies on the domestic market. We sent approximately 300 invitations with a brief description of the study procedure (without revealing the purpose of the study itself). After initial acceptance, the participants received thorough information about the study details. The participants did not receive any remuneration for their contribution and were free to leave the study at any stage of the procedure.
Measures and Factors
We gathered data on a single outcome measure, i.e. a personnel decision. Each participant made a number of decisions on a multi-attribute bundle of résumés. Every résumé contained three stimuli factors that represented our independent variables. Each of these factors appeared on different levels, and every possible combination of each factor on each level occurred in the résumé bundle exactly once. Therefore, we successfully employed a full-factorial design (Karren & Barringer, 2002). Appendix A provides examples of all factors, at each level that occurred in the study materials, for all bundles.
Personnel Decisions
Personnel decisions that could be made in response to a given résumé were represented by participants’ series of decisions that were based on the screening process. To reflect the real decision-making process, after looking through an applicant’s résumé as if it was an actual recruitment process, each participant had to decide whether or not to invite that person to a job interview: each résumé could be accepted (a positive decision), which meant that a participant decided to invite the applicant to a job interview; a résumé could be assigned to a reserve group (a partially positive decision), which meant that a participant would decide to invite an applicant for a job interview only if there was not anyone more suitable for the job or as a reserve if another candidate cancelled their appointment; or a résumé could be rejected (a negative decision), which meant that a participant was not sure whether to invite an applicant to a job interview. Therefore, every participant made a series of three possible decisions which operationalized the three levels of an ordinal, categorical, dependent variable.
Relevant Factor
The first factor consisted of relevant résumé components, specifically job experience, qualifications, education, and languages. This factor appeared on three ordinal levels, which concurrently represented certain and uncertain decision conditions. The first level, i.e. full suitability (certain conditions), meant that the résumé included all the information listed in the job description: sufficient job experience period that matched the requirements, proper education, all detailed qualifications (e.g. software and technology), and required foreign language mastery level. The second level, a partial fit, represented the uncertain condition and meant that the majority but not all information listed in the job description was included in the résumé. Specifically, this meant that the résumé included ca. 80% of the requisite job experience duration, proper education, ca. 80% of detailed qualifications, and knowledge of the required foreign language but at a level below the required one (with the use of CEFR). The last level meant that the résumé was submitted by a completely unsuitable applicant and reflected a condition of certainty (but a negative one). A résumé from this level presented less than 80% of the minimum experience period, incomplete proper education, less than 50% of the required qualifications and inadequate knowledge of a foreign language.
Irrelevant Factor
The second factor consisted of irrelevant résumé components. As recommended by policy-capturing methodology (Graves, 1992), to assure zero or near zero correlation between this and the previous factor we chose information about personal interests as the object of manipulations. This factor appeared on the following three categorical levels: typical interests (e.g. music, sport), untypical interests (creative, rare or humorous ones, e.g. “tamer of boredom”, “reading dictionaries” or “confusing my cats on a daily basis”), and job-related interests.
Formal Factor
The last factor consisted of formal résumé components and appeared on three ordinal levels which represented the level of professionalism of document formatting. Professional résumés were prepared without any inconsistencies or typos and with the use of graphic design and elements that increased clarity. Standard résumés were prepared without any inconsistencies or typos but without graphic design and other elements (as plain, well-formatted documents). Unprofessional résumés contained at least two inconsistencies (e.g. different fonts, indentations or date formatting), at least one typo, and no graphical design (as plain, poorly formatted documents).
Research Material
Research materials were prepared on the basis of real-life résumés and professional social network profiles in order to ensure the external validity of the research procedure. We gathered 255 actual résumés and purchased 20 professional résumé templates from a specialized company. Their level of formatting was then evaluated by peers (n = 9) with the use of the above criteria. Information about personal interests written by applicants themselves was gathered from the same résumés. They were then categorized and once again peer-evaluated (n = 9) to choose the most representative ones for their respective categories. On this basis we created the source material for the résumé bundles. Subsequently, the research team members chose an appropriate résumé template and interests and filled out the résumés with information related to the relevant components in a way that corresponded to the policy-capturing design described above. Relevant résumé components were copied from actual résumés and professional profiles to fit the particular job description. To avoid substantial confounders, there were no photos in résumés (just a blank place to indicate their position). Finally, each résumé was double-checked by another team member to verify whether it fitted the policy-capturing plan. The above procedure was carried out three times for three different job descriptions that were created on the basis of actual documents provided by a recruitment agency.
Study Procedure
The participants were provided with a laptop with a mail client installed and one of the two (randomly assigned) job description files available on the desktop (expert-level sales position or entry-level customer service position). Twenty-seven emails were available that contained job applications and appropriate résumés (in a random but fixed order). The participants were asked to read the job description and imagine that they were conducting the recruitment process for the job. They were told to select résumés in the same way they would normally, with no need to justify their decisions. Their task was to screen résumés and make one of three possible decisions regarding every single résumé. The participants were instructed to make decisions based on the degree of an applicant’s suitability for the job requirements that were defined in the job description. We also informed the participants that the résumés presented had only been prepared for the study but were based on real ones. This procedure was consulted with a group of professionals during the pilot study (n = 6) and verified as corresponding to real-life practices. After the study, the participants were debriefed and the study purpose was explained. Before the main procedure, the demographic data had been gathered. Each study was conducted by the main researcher and an observer who verified the compliance of the study with the plan. There were no major deviations from the procedure.
| TABLE 1: Descriptive Classification Results From Study 1 | ||||
| Decisions | ||||
|---|---|---|---|---|
| Factors' levels | Accepted | Reserved | Rejected | Mediana |
| Full fit | 284 (75%) | 67 (18%) | 27 (7%) | 1 |
| Partial fit | 68 (18%) | 190 (50%) | 120 (32%) | 2 |
| No fit | 2 (1%) | 13 (3%) | 363 (96%) | 3 |
| Professional format | 127 (34%) | 103 (27%) | 148 (39%) | 2 |
| Standard format | 105 (28%) | 81 (21%) | 192 (51%) | 3 |
| Unprofessional format | 122 (32%) | 86 (23%) | 170 (45%) | 2 |
| Untypical interest | 115 (30%) | 111 (29%) | 152 (40%) | 2 |
| Work-related interest | 136 (36%) | 62 (16%) | 180 (48%) | 2 |
| Typical interest | 103 (27%) | 97 (26%) | 178 (47%) | 2 |
| Overall | 354 (31%) | 270 (24%) | 510 (45%) | 2 |
Note: aValues for median statistics; 1–accepted; 2– reserve; 3– rejected
We employed the machine-learning approach and the purpose of the first study was to build a training model of the subjects’ decisions. The classification tree method was used with R’s rpart package (Therneau et al., 2018). Table 1 presents descriptive classification results, while Figure 2 presents the decision tree model that was built on the study data.
This model should be read as follows: at the top of the tree there is a condition which splits the tree into two branches: if the condition is met, follow the left branch; otherwise, follow the right one. The branch might lead to either a decision or to another condition. Therefore, the résumé-screening process we investigated should be described as follows: if the relevant résumé components do not fit at all, this leads directly and almost exclusively to a negative decision. Subsequently, if a relevant résumé factor is a “complete fit”, this mostly leads to a positive decision. However, if this condition is not met, branches lead to further conditions which include factors other than the relevant one. Therefore, if a résumé had an irrelevant factor with an “atypical” level, a partially positive decision was made. But if this condition was not met, the branch led to the last condition regarding the formal factor. If the formal factor was at the “professional” level, a partially positive decision was made. In other cases, the process resulted in a negative decision. This process corresponds very well to our conceptual model. As résumés presenting candidates who did not fit the job were almost exclusively rejected, it can be concluded that decisions about them were made in a non-compensatory manner and solely on the basis of relevant factors. Similarly, simple inference rules based solely on relevant components were applied in the case of résumés presenting fully fitted candidates. It can therefore be assumed that, consistently with H1, in a research situation that simulates decision-making in conditions of certainty, the participants used only information from relevant components. Importantly, formal and non-relevant résumé components were taken into account in decision-making only when the participants of the study considered partially fitted applications. This suggests that, in accordance with H2, all three résumé components interactively influenced the decisions of the participants only in a research situation that reflected conditions of uncertainty.
Table 2 presents the confusion matrix for the model; Table 3 presents the respective statistics. The overall accuracy of the model was 0.75 (with 95% CI 0.72 and 0.77, respectively). Null information rate (NIR) for this model was 0.45 (with p for accuracy to NIR comparison less than 0.001). NIR shows how accurate a model would be if one simply chose the dominant class as the predicted one; it is used to assess whether the actual model contributes to explaining the decision process beyond simply selecting the most common decision made by the participants as default. The comparison of accuracy and NIR (and the resulting p-value) shows that the model performs better than inferring without any additional information. Cohen’s Kappa was 0.60, indicating good agreement and similar decisions made by participants (Fleiss, 1981; Landis & Koch, 1977). With the exception of the sensitivity value for a ‘reserve’ decision, both sensitivity and specificity for all decisions were high and substantially higher than NIR, indicating that the model has performed well at predicting all decisions. Importantly, both positive (PPV) and negative predictive values (NPV) were high. For cases predicted to be a decision of a given type, PPV indicates what percentage was actually that kind of decision (e.g. of all decisions predicted by the model as “accepted”, what fraction was truly that one). Conversely, NPV shows the fraction of decisions that actually do not belong to specified classes, among those predicted not to by the model. These indicators are aimed at assessing decision model performance; their values obtained from the results of Study 1 indicate that where the model predicted a particular type of decision, there was a good chance that the participant actually made that decision. As NPV was higher, the model performed better at predicting when certain decisions would not be made. The analysis of the decision tree branches and metrics for the developed statistical model supports H1. The results suggest that in a situation of certainty the decisions of the research participants were made under the impact of relevant résumé components. Moreover, analysis of results and branches of the decision tree also provides data to support H2.
| Table 2: Confusion Matrix For Classification Results From Study 1 | ||||
| Actual decisions: | ||||
|---|---|---|---|---|
| Accepted | Reserved | Rejected | ||
| Predictions: | accepted | 284 | 67 | 27 |
| reserved | 32 | 128 | 50 | |
| rejected | 38 | 75 | 433 | |
| Table 3: Statics And Metrics For Tree Model From Study 1 | |||
| Metrics | Decisions | ||
|---|---|---|---|
| Accepted | Reserved | Rejected | |
| Sensitivity | 0.80 | 0.47 | 0.85 |
| Specificity | 0.88 | 0.91 | 0.82 |
| Positive predictive value | 0.75 | 0.61 | 0.79 |
| Negative predictive value | 0.91 | 0.85 | 0.87 |
| Prevalence | 0.31 | 0.24 | 0.45 |
| Detection rate | 0.25 | 0.11 | 0.38 |
| Detection prevalence | 0.33 | 0.19 | 0.48 |
| Balanced accuracy | 0.84 | 0.69 | 0.83 |
To validate this model, a random forest method was used with the ranger package for R (Wright et al., 2018). Table 4 presents the confusion matrix from the 10,000 trees used in this validation. Out-of-bag prediction error (OOB) was 0.24, which, compared to the model’s accuracy, indicates that it was not over-fitted and therefore should perform well at predicting the data outside of the training set. Overall, as this model proved to be robust and presented the decision process in accordance with our conceptual description, we believe that the study results support both of our hypotheses. Importantly, the use of both compensatory and more direct rules can be identified in the results of the study. Both rules were applied by all participants in the study, depending on the conditions in which they had to make decisions.
| Table 4: Confusion Matrix For Classification Results From The Random Forest Method (Study 1) | ||||
| Actual decisions | ||||
|---|---|---|---|---|
| Accepted | Reserved | Rejected | ||
| Predictions | Accepted | 284 | 67 | 27 |
| Reserved | 24 | 139 | 47 | |
| Rejected | 46 | 64 | 436 | |
To test the performance of our decision model, we conducted a second study. We used the same materials and a similar procedure, but this time the study was conducted online.
Participant Characteristics
As previously, the participants of this study included personnel selection professionals with at least six months of appropriate experience. There were 104 participants, aged 21 to 62 (M = 33.71, SD = 9.05), of which 78 were female and 26 male. As decisions were the actual subjects of this study, the total sample size was 2,808. The average length of experience was 6.63 years (SD = 5.93), and the participants were employed in recruitment agencies (46%), HR departments (50%), or other companies such as software houses (4%). All the participants had a university degree, mostly in social and management sciences (30%), psychology (25%) and HRM (21%).
Sampling Procedures
The participants were found via professional social services and mailing on the domestic market. We sent approximately 2,400 invitations with a brief description of the study procedure; 459 people agreed to participate and 104 completed the study. The participants did not receive any remuneration for their contribution and were free to quit at any stage of the procedure.
Measures, Factors and Research Materials
During the second study, we measured the same variables and used the same policy-capture design with three factors on three levels each, as in the previous study. Once again, we employed a full-factorial design with within-subject manipulations. The bundles of résumés that we had created before the first study were used (only this time we used all three sets).
Study Procedure
After giving initial acceptance, the participants were provided with information about the study details; they were also given a link to the survey in which they could make decisions regarding résumés, and a link to a dedicated cloud folder in which résumés and selected job descriptions were stored. The participants were informed that they would have two weeks to complete the study and after that period the cloud-based folders would be deleted. The participants were told to read the instructions, which directed them randomly to one of three job descriptions (expert sales position, entry customer service position or mid-level engineering position) and included the same information as previously. Subsequently, they were provided with a list of all the 27 applicants and a link to their résumés. They were supposed to make the same decisions (one of the three possible) regarding each résumé as in the previous study. After screening, they were provided with full information regarding the purpose of the study.
Results
As we employed a machine learning approach, the aim of this study was to test our previously built model and verify its performance on the new sample. Once again, R’s rpart package was used to test the classification decision tree model we had already tested. Table 5 presents the descriptive results; Table 6 shows the confusion matrix for this study; Table 7 presents the statistics and metrics. The overall accuracy of the model on the new data was 0.70 (with 95% CI 0.68 and 0.71, respectively) and NIR was 0.43 (p for the accuracy to NIR comparison was less than 0.001). Once again, the model had weak sensitivity for the ‘reserve’ decision, but both sensitivity and specificity for all the decisions were satisfactory (and considerably higher than NIR). As before, both PPV and NPV were high, and the latter was better. Cohen’s Kappa was 0.52, indicating fair inter-participant agreement. This value fell slightly compared to the previous study, but this was expected since the total sample size had more than doubled. Overall, the results indicate that the decision tree model was robust and performed well at predicting decisions based on the new data. Moreover, this demonstrates that the actual decision-making process does not invalidate our conceptual model. In our opinion, this provides further support for both hypotheses. Taking these results into account, it can again be assumed that the participants used less straightforward rules when making decisions in situations representing uncertain conditions. Based on the splits of the decision tree branches and the appearance of non-relevant and formal components as conditions for the split of branches only in the case of a specific level of relevant components (i.e. partial-fit), it can be concluded that these were compensatory rules. In the second study, in line with H2, the participants also utilized all types of components in interaction, but only in cases of uncertainty. Again, none of the additional stimuli (other than relevant components) improved the résumé decisions of unfit candidates, nor did they worsen the résumé assessments of fully fitted candidates. Therefore, the decisions made by the participants in certain situations were made only under the influence of relevant factors. This supports H1.
| Table 5: Descriptive Classifications Results From Study 2 | ||||
| Decisions | ||||
| Factors' levels | Accepted | Reserved | Rejected | Mediana |
| Full fit | 714 (76%) | 161 (17%) | 61 (7%) | 1 |
| Partial fit | 174 (19%) | 448 (48%) | 314 (34%) | 2 |
| No fit | 24 (3%) | 72 (8%) | 840 (90%) | 3 |
| Professional format | 330 (35%) | 244 (26%) | 362 (39%) | 2 |
| Standard format | 304 (32%) | 233 (25%) | 399 (43%) | 2 |
| Unprofessional format | 278 (30%) | 204 (22%) | 454 (49%) | 2 |
| Untypical interest | 312 (33%) | 216 (23%) | 408 (44%) | 2 |
| Work-related interest | 300 (32%) | 241 (26%) | 395 (42%) | 2 |
| Typical interest | 300 (32%) | 224 (24%) | 412 (44%) | 2 |
| Overall | 912 (32%) | 681 (24%) | 1215 (43%) | 2 |
Note: aValues for median statistics; 1–accepted; 2– reserve; 3– rejected.
| Table 6: Confusion Matrix For Classification Results From Study 2 | ||||
| Actual decisions | ||||
| Accepted | Reserved | Rejected | ||
| Predictions | Accepted | 714 | 161 | 61 |
| Reserved | 72 | 268 | 180 | |
| Rejected | 126 | 252 | 974 | |
| Table 7: Statics And Metrics For The Classification Decision Tree Model From Study 2 | |||
| Metrics | Decisions | ||
|---|---|---|---|
| Accepted | Reserved | Rejected | |
| Sensitivity | 0.78 | 0.39 | 0.80 |
| Specificity | 0.88 | 0.88 | 0.76 |
| Positive predictive value | 0.76 | 0.52 | 0.72 |
| Negative predictive value | 0.89 | 0.82 | 0.83 |
| Prevalence | 0.32 | 0.24 | 0.43 |
| Detection rate | 0.25 | 0.10 | 0.35 |
| Detection prevalence | 0.33 | 0.19 | 0.48 |
| Balanced accuracy | 0.83 | 0.64 | 0.78 |
We conducted two studies and employed the machine learning approach and the decision tree method to test our conceptual model. Our decision tree model performed well, predicting actual decisions made by professionals, and the course of the identified decision process does not invalidate our developed conceptual one. Therefore, we believe that these results might serve as proof of our conceptual model being a valid decision theory (Hansson, 1994). This theory states that the decisions made by professionals to evaluate application documents are determined by information derived from three different types of components of the résumés themselves: relevant, non-relevant and formal. However, the choice of these components and how to integrate the information from them depends on the degree of certainty in which decisions are made. In the case of certain conditions, i.e. when the initial résumé assessment demonstrates that the candidate fully meets the formal criteria described in the job description, or when he or she does not meet them completely, the decision is made solely on the basis of relevant information (e.g. length and type of work experience, professional courses, and language proficiency). The results of our study show that deficiencies in these areas cannot be compensated for in a situation of certainty by a high level in other components. However, in an uncertain situation, i.e. when the level of a candidate's fit for the job cannot be assessed unequivocally (for example, the criteria described in the job description are only partially met), the recruitment specialists attribute characteristics to candidates on the basis of the available résumé components, i.e. non-relevant (e.g. information about interests or intriguing, distinctive sentences included in the document) or formal (e.g. the appearance and professionalism of the document edition) factors. In such uncertain conditions, agents make decisions based on compensation rules; for example, they will produce a positive personal judgement if a lack of experience is compensated for by the professional appearance of a résumé or an amusing entry concerning personal interests.
We employed adequate methodology and introduced a series of procedural elements that improve the external validity and reliability of results, e.g. using a categorical outcome measure, engaging actual professionals as participants, not requiring them to explain their decisions, and giving them complete information regarding the origin of the résumés (Born & Scholarios, 2004; Elrod et al., 2004; Thoms et al., 1999). We consider this to be additional support for our conclusions. By conducting a study according to the aforementioned methodology and using a number of components in the policy-capturing plan, it is possible to explain some of the inconsistencies present in the existing research results. The literature review has shown that factors such as résumé length, personal interests or volunteering have a different impact on the actual decisions of participants. We believe that the results of our study show that these differences could be explained by the decision-making strategy chosen by the agents. These strategies, in turn, would depend on the conditions in which decisions were made, i.e. on the level of relevant work-related information described in these résumés.
Theoretical Implications
We believe that the conceptual model that we have developed and introduced might serve as both a useful and valid framework for further studies on the résumé screening process and, importantly, as a basis for the interpretation of results. This model adequately explains the impact on decision makers of multiple factors and the interactions between these factors. However, these interactions manifest themselves only in situations in which decision-making is most challenging for professionals, i.e. in situations of uncertainty. Since recruitment professionals are highly motivated to make accurate and firm decisions (Salgado et al., 2001; Struthers et al., 1998), they use other available sources in the absence of clear relevant information, integrating them on a compensatory basis. In such a situation, deficiencies in the actual job-fit of candidates are then compensated for by strengths presented in areas that would not otherwise be significant.
Interestingly, so far researchers have emphasized the rules according to which decisions are made and the way in which decision makers utilize a variety of decision objects’ attributes (Elrod et al., 2004; Sekiguchi & Huber, 2011). However, most of the previous studies failed to take into account the content of the decision objects’ attributes that serve as the basis of decision-making. In our opinion, the clear division of résumé components into three broad categories might help to understand the résumé screening process and to clarify some ambiguity. For instance, formal and irrelevant résumé components presented a significant, conditional or absolutely no impact on decision makers (Blackburn-Brockman & Belanger, 2001; Hutchinson & Brefka, 1997; Nemanick & Clark, 2002; Spinks & Wells, 1993; Thoms et al., 1999). We believe that if the degree of certainty during this decision process (reflected by relevant résumé components and therefore by the level of actual applicants’ suitability for the job) had been taken into account, these results might have been more consistent.
Despite our best efforts, naturally this study has some limitations that might be addressed in the future. First of all, as it has been proved that the context of decision-making (e.g. choosing the best or rejecting the worst applicants) affects the process itself (Born & Scholarios, 2004), it would be reasonable to determine whether our model applies equally to different screening scenarios. There might be some differences in the decision-making process when professionals are limited by time and resources (e.g. one might choose only very few applicants to interview). Furthermore, the decision process itself should be investigated in a broader perspective. Until now, studies have seen the decision process as non-recurring, i.e. decisions are investigated and measured once during a single study procedure. It may well be that this process is in fact more stretched over time and has a more sequential nature. This, however, would require the application of a more thorough methodology with a series of in-depth within-subject analyses. Next, to further prove the gradual nature of the résumé screening process and the occurrence of an initial assumption (which includes a negative decision), studies with chronometric and eye-tracking methods might be of use. Building on the two previous observations, it seems interesting to further develop our model and test it from the perspective of decision field theory (Busemeyer & Townsend, 1993) since it describes decisions as a dynamic process. Our model might also be further investigated with the use of a more complex factorial design, e.g. by including a series of factors for every component category. Finally, we believe it might be of particular interest to determine whether it would be possible to apply a similar model for the decision process regarding objects entirely different from résumés, e.g. financial behaviour.
| Relevant components | Full fit | More than 3.5 years of experience in managing a sales region ( out of the 3 required); more than 4 years of experience in retail (out of the 4 required); information about responsibilities such as managing points of sale in the region, building a distribution network, acquiring franchisees, approving promotional materials (all 4 listed in job description); English and German knowledge at C1 level (both required at that level) a |
|---|---|---|
| 6 months of experience as a customer service specialist (half a year to one year indicated in the job description as required) and 10 months of experience in selling call centre banking services (call center experience required); Bachelor's degree in marketing (university degree required); responsibilities listed includes after-sales service, customer record maintenance, office facilities and computer skills (all 4 indicated in the job description); English at B1 level (A2 level required) b | ||
| Degree in Mechanical Engineering (as required); 1.5 years’ experience as a mechanical engineer (one of the required positions, in accordance with the required length of experience); 2 years’ experience in the automotive industry (min. 1 year required). Responsibilities: fault removal, service and maintenance work, assembly and modernization, diagnosis and reporting (all 4 required in the job description); English level B2 (as required) c | ||
| Partial fit | 2.5 years of experience in sales region management (out of 3 required) and additional experience in point-of-sale management; almost 4 years of experience in retail. Information on responsibilities: development of sales strategy, implementation of sales campaigns, region and sales staff management (3 out of 4 required); English at B2 level and German at B1 level a | |
| 5 months of experience in customer service in a call centre (out of 6 required); additional not matching experience (cashier); currently studying at university; listed responsibilities such as customer advice, after-sales service, computer skills (3 out of 4 required); English at A1 level (A2 required) b | ||
| Engineer's degree (as required); one year of both work experience and internship as a mechanical engineer (out of 1.5 required). Responsibilities: machine diagnosis, performing initial runs, troubleshooting (3 out of 4 listed in the job description); English at A2 level c | ||
| No fit | 6 months’ experience in the sale of B2B telephone services and 2 years in retail; no duties as required by the job description; English at B2 level and no information on German a | |
| 5 months’ experience as a florist and 10 months experience as a cashier; discontinued education in vocational school; no matching responsibilities and no foreign language information listed in résumé b | ||
| Vocational qualification of construction technician (with no engineer degree); 4 years’ experience as a welder (not matching the job requirement); no matching job responsibilities listed; no English skills information c | ||
| Irrelevant components | Untypical interest | ‘Psychology of drawing’ and ‘composing music’ listed under ‘personal interest section of résumé a |
| ‘Parachuting’, ‘meditation’ and ‘augmented reality’ listed under ‘personal interests’ section of résumé b | ||
| ‘Reading dictionaries’, ‘survival’ and ‘expression of emotion through facial expressions’ listed under ‘personal interests’ section of résumé c | ||
| Work-related interest | ‘Sales management’, ‘personal growth’ and ‘coaching’ listed under ‘personal interests’ section of résumé a | |
| ‘Long-term customer relations’ and ‘interactive marketing’ listed under ‘personal interests’ section of résumé b | ||
| ‘Computer diagnostics and vehicle mechanics’ and ‘HSW manufacturing systems’ listed under personal interests’ section of résumé c | ||
| Typical interest | ‘Music’ and ‘books’ listed under ‘personal interests’ section of résumé a | |
| Sport’ and ‘music’ listed under ‘personal interests’ section of résumé b | ||
| ‘Books and movies’ listed under ‘personal interests’ section of résumé c | ||
| Formal components | Professional format | No typos nor inconsistencies (same font, bullet types, spacing and margins across entire document); candidate’s name written in larger, capitalized font at the top of the document; matching, sea-green lines used to separate résumé sections; thick, sea-green line highlighting the candidate’s name a |
| No typos or inconsistencies (same font, bullet types, spacing and margins across entire document); design includes usage of dark-red and black colour, with thick dark-red line at the top of every page, dark-red font colour for candidate name (and black for last name) and red, capitalized job titles; consistent order of information about work experience; well-designed two-column layout with dates on the left; special signs for bullets; an opening statement with light-grey highlight (matching the red palette) b | ||
| No typos or inconsistencies (same font, bullet types, spacing and margins across entire document); dark-red bold line as a left-side border for candidate’s name; section titles underlined by a long, thick, dark-grey line and supplemented by matching flat icons (dark-red); contact data supplemented by matching gray icons; consistent margin size for section titles, dates and content across entire document; language skills visualized with graphic scale (stars) c | ||
| Standard format | Plain black and white document with no typos or inconsistencies (same font, bullet types, spacing and margins across entire document); no additional graphic design elements (only space for photo); well-formatted (two-column layout with section titles and dates on the left and content on the right; section and sub-section titles and job positions bolded, clean spacing between sections) a | |
| No typos or inconsistencies (same font, bullet types, spacing and margins across entire document); no additional graphic design elements (only space for photo); well-formatted (bolded and capitalized section titles, résumé sections separated by spacing, dates and responsibilities listed in two-column format) b | ||
| Plain black and white document with no typos or inconsistencies (same font, bullet types, spacing and margins across entire document); no additional graphic design elements (only space for photo); well-formatted (section titles consistently highlighted by larger font and underline, dates highlighted, consistent order in job experience description, spacing between sections to increase clarity) c | ||
| Unprofessional format | Inconsistency in the types of font used (use of two different fonts, serif and sans serif); different margin sizes applied before the bullets (between sections); typo in the word internet (‘internet’); no graphical design elements to increase document clarity a | |
| Dark green font colour with two lines in black font; 8 job titles stated and only two of them underlined; only 1 of 4 section titles bolded; semicolon used as a line end indicator only in single section; lack of national language-specific letters (diacritics) in one case and underline character is used before one responsibility description b | ||
| Inconsistent spacing between lines and sections, with job experience description written with no spacing; both comma and full-stop used to indicate line end (at random); typo in university name; different margin sizes c |
Note: Source of example: afrom the bundle of résumés for expert sales position; b from the bundle of résumés for entry-level customer service position; c from bundle of résumés for mid-level engineering position.