Appendix A: Report Methodology
Research Framework and Survey Design
The analysis that forms the basis of this Report is the result of an extensive survey of Chief Human Resources and Chief Strategy Officers of leading global employers, and consists of four interrelated parts, providing a uniquely flexible dataset that can be recombined in various ways to obtain further specific insights into relevant dimensions of interest (see Figure A1).
The aim of our survey was to understand as comprehensively as possible expectations regarding the future of jobs, work and skills by the largest employers in some of the world’s biggest economies, in order to ultimately contribute to a more informed debate on these issues and provide an evidence base and guidance to businesses, governments and civil society organizations such as labour unions and education providers.
As a starting point, we designed a survey targeted at heads of human resources departments, as well as chief strategy officers, as these are likely to hold the most detailed information on corporate policies and strategic workforce planning in light of current industry trends and disruptions. Our research framework has been informed by an extensive literature review on the various dimensions covered by the survey and by continuous consultation with leading experts from academia, international organizations, business and civil society through the World Economic Forum’s Global Agenda Council on the Future of Jobs and Global Agenda Council on Gender Parity, which served as partners and advisory bodies to the study.
The survey collection process was carried out via online questionnaire in the first half of 2015, whereby the World Economic Forum’s Employment and Gender Challenge Initiative teams worked in close collaboration with Global Agenda Council Members, Global Challenge Partners and the Forum’s industry and regional teams to ensure a maximum number of responses from target companies.
The World Economic Forum team, supported by Global Agenda Council Members, used the Hoovers and Bloomberg company databases to identify target companies and compile an extensive database of contact details of human resources department directors. For target companies for which we were not able to obtain contact information, Global Agenda Council Members carried out their own research to complete the database.
Our pool of respondents comprised, as primary selection criterion, the 100 largest global employers in each target industry sector (as classified by the World Economic Forum; see Appendix B, Table B1), supplemented, as a secondary criterion, by the 50 largest national employers in each of our target countries (see target country list in Appendix B, Table B2), if these were not already included as part of the first group.1 This approach was chosen in order to ensure sufficient geographic diversity and coverage to be able to provide balanced country-level as well as industry-level results. Furthermore, a number of leading, fast-growing small and medium-sized companies were identified qualitatively through the World Economic Forum network and included in the survey pool to account for significant future job creators and ‘disruptors. In total, our target survey pool consisted of 2,450 companies.
In various instances the largest employers in a country comprised the local units of well-known multinational enterprises, meaning that our survey approach entailed obtaining results from these companies both at a global, headquarter-level and from one or more of their national operations. When looking at results through a country lens, this approach was considered the most appropriate given the importance of many of these large employers in their respective local labour markets. However, when looking at results through an industry lens, it was important to avoid skewing results by double counting responses from local units of the same company. We addressed this complexity through our survey design, by asking respondents to clearly associate all employment and occupation-related responses to a specific geography.
In total, 366 individual companies responded to our survey over the first half of 2015, providing us with 1,346 unique data points around employment and occupations based in specific geographic locations across these companies’ global operations. As outlined, the structure of our survey means that the sampling unit and base sample—the unique data points that can be used for most of our analysis—commonly refer to these 1,346 occupation-level responses; although in some cases, such as when looking at results through an overall industry lens, it is restricted to the 366 company-level responses to avoid double counting.
In light of the above, companies were required to provide their name as well as the title and contact information of the person filling in the survey. Respondents were assured of strict confidentiality of their data at the level of individual company-level results. In addition, participating companies were given the option to be informed of their individual results relative to the range of responses in their country and/or industry.
Our initial target list of countries comprised the three largest economies, by either GDP or population, in every world region. In addition, through industry networks, the survey was carried out in companies based in other non-targeted countries. To ensure quality of results, only countries with a critical mass of responses have been included in the final Report. Our threshold for inclusion was a minimum of 30 unique data points per country. Countries in our original target list for which we were unable to obtain a representative sample include Argentina, Colombia, Russian Federation, Republic of Korea, Egypt and Nigeria. In addition, due to an insufficient number of individual country-level responses, we have aggregated responses for a number of significant economies into two broader regional groupings: the Association of South East Asian Nations (ASEAN), combining results for Indonesia, Malaysia, Thailand and Singapore; and the Gulf Cooperation Council (GCC), combining results for Kuwait, Qatar, Saudi Arabia and the United Arab Emirates.
The difficulty of obtaining responses from some geographies despite the dedicated work and support of Global Agenda Council Members and the Forum’s industry and regional teams appears to stem from a variety of causes, including sensitivity around the topic, concerns about confidentiality and business culture norms.
There was near gender balance (54% female, 46% male) among the Chief Human Resources Officers and other senior executives who answered the survey.
Interpretation of Results: Sample Coverage and Representativeness
Overall, our survey sample represents more than 13 million employees across nine industry sectors (Financial Services & Investors; Information and Communication Technology; Energy; Basic and Infrastructure; Mobility; Consumer; Healthcare; Media, Entertainment and Information; and Professional Services) in 13 major developed and emerging economies (Australia, Brazil, China, France, Germany, India, Italy, Japan, Mexico, South Africa, Turkey, United Kingdom, United States) and two broader regional groupings, ASEAN (combining results for Indonesia, Malaysia, Singapore, Thailand) and the GCC (combining results for Kuwait, Qatar, Saudi Arabia, and the United Arab Emirates).
In line with the Report’s overall aim of contributing to a more informed conversation on the future of jobs and providing guidance to businesses, governments and civil society organizations, the survey largely consisted of perception-based questions, with response options designed to focus on the key issues outcomes and trends as perceived by decision-makers. In addition there were a number of quantitative baseline questions. That is, our study is designed to reveal what the world’s leading employers think will happen, when they think it will happen, and what they propose to do about it. While, between them, the expectations and corporate practices of the leading employers represented in our sample will significantly shape the future global employment landscape, it should be stated explicitly that our trend data hopes to be roughly right rather than precisely wrong. Similarly, in highlighting companies’ current thinking concerning intended change management and future workforce strategies, the Report aims to point to potential shortcomings and inconsistencies as a basis for further, more informed debate, rather than offering these as ready-made good practice solutions.
In taking this perspective, the Report consciously focuses on actors with a key role in shaping the evolution of the global labour market. While only a minority of the world’s workforce is directly employed by large and emerging multinational employers such as the ones covered by our sample, these companies often act as anchors for smaller local firms, as suppliers within global value chains and as catalysts for the development of local entrepreneurship ecosystems. In addition to their own significant share of employment, workforce-planning decisions by these firms therefore have the potential to transform local labour markets through indirect employment effects that set the pace for changing skills and occupational requirements. Similarly, the inclusion of medium-sized high-growth enterprises in our sample ensures representation of the category of companies generally considered the largest source of new net job creation in most economies.2
Nevertheless, given the Report’s aim of providing guidance and stimulating discussion, it is important to treat with caution the extent to which it is possible to generalize and extrapolate from our findings in a manner that could be considered representative of all trends across an entire industry or country.
Representativeness at the country level
As a first important caveat, our survey only covers wage employment jobs in the formal economy, whereas the majority of the workforce in some developing countries may be based in agriculture or in the informal sector. According to the World Bank, roughly half of the world’s working population of more than 3 billion people are small farmers or self-employed in low productivity activities such as street vending, limiting our findings’ applicability in these contexts.
It should be noted, however, that many informal sector jobs are also subject to dramatic change due to the same drivers of change, trends and disruptions affecting those based in large established firms. For example, the increasing ubiquity of mobile phones in developing countries has led to a mobile payments revolution, whereby a large range of informal sector transactions are now taking place via such platforms. Similarly, several technological trends hold out the promise of integrating an increasing number of informal sector workers into the global labour market in unprecedented ways, including by formally linking them to large established employers.33
A second consideration regarding representativeness of our results at the country level concerns the question of what percentage of the formal sector labour force is employed in large firms such as the ones in our sample. The available data suggests that, in general, fewer people in lower income countries are employed in large firms, limiting the generalizability of findings in these countries.4
In summary, the Report’s survey design seems well-suited to the target countries studied, namely: developed economies and large emerging markets, with certain caveats on generalizability for the latter. It seems less applicable for extrapolating results directly for developing economies not covered by the survey, although we believe our findings may provide valuable information for long-term human capital development strategies in these countries.
More broadly, the Report focuses on two types of job categories in any particular country, rather than on the entirety of the national labour market as a whole: firstly, on mass employment jobs currently held by a large share of the labour force in that country; secondly, on strategic or specialist jobs that are critical to a company’s operations and may link the country to the worldwide network of international production and global value chains. The first category takes its significance from the number of employees directly affected; the second category provides an outlook primarily on high-skilled jobs, often including those with a particular bearing on opportunities and threats for the country’s positioning within the global economy.
Representativeness at the industry level
The main questions regarding representativeness of our findings at the industry level concern the extent to which the large employers in our sample are representative of a typical company in their industry sector as a whole, and, therefore, what percentage of the industry’s workforce is employed in companies such as the ones covered by our survey. Available data shows, for example, that there are systematic differences between larger and smaller firms with regard to the amount of in-house training and skills upgrading they are offering, so our analysis of future workforce strategies will be more applicable to industry sectors with a larger share of employment in large firms.
Economic theory suggests that (in a competitive market, i.e. leaving aside monopolies, cartels and regulatory policies) firm size distribution in an industry is mostly the result of underlying structural factors—for example, some goods and services are naturally better produced in large firms; others, in smaller ones.5
A good approximation of the prevalence of employment in large firms in an industry is the degree of industry concentration, as measured by indicators such as the Herfindahl index6 and the concentration ratio7, which calculates the share in total industry output of the industry’s five largest firms. Total output and gross value added in a number of industries—such as Energy, Mining and Metals, and Information and Communication Technology—are highly concentrated in a handful of large companies with tens of thousands of employees each. These large companies included in our sample account for a very large share of total employment in their industries.
For some other industries, our sample population is more representative of the industry’s leading employers, rather than of the industry’s total workforce as a whole. Overall, because of the differences between large enterprises and small and medium-sized ones, especially when it comes to talent management and HR strategies, it is clear that our findings are mainly applicable to larger firms—the biggest employers in each industry in particular.
A second consideration regarding representativeness of our results at the industry level concerns cross-country differences in public and private ownership. Our survey was designed to be applicable to workforces in both sectors. However, to date we have been able to obtain a critical mass of responses only from state-owned enterprises, not from public sector employers such as government departments. This caveat affected our sample selection particularly in industry sectors such as healthcare and education, which are public in some countries and partly or wholly private in others, since business databases such as Hoovers and Bloomberg do not cover entities such as public hospitals. These are a major source of employment in many countries and are undergoing similar workforce challenges and transformations as private sector companies. Given sufficient interest from these constituents, we envisage a follow-up project to the present Report for public sector employees, covering not only the public sector but also civil society and international organisations as a distinct industry sector.
Finally, a methodologically interesting question is the extent to which the identified trends and disruptions might themselves affect the validity of our conclusions regarding representativeness, for example by changing industry concentrations and structures through drivers of change such as 3D printing and sharing economy platforms.8 In general, while many of these trends point to a rise in importance of freelance work and contractors, evidence suggests that much of this work will continue to be influenced by the workforce practices and sourcing decisions of large, established employers,9 validating the Report’s research framework.
Classifying Occupations: ISCO and O*NET
Given the Report’s major aim of bringing specificity to predictions about the future of jobs at the occupation level and moving beyond broad categorizations, we have based our analysis on a recognized reference system widely used by labour market researchers: a streamlined version of the Occupational Information Network (O*NET), developed by the US Department of Labor in collaboration with its Bureau of Labor Statistics’ Standard Classification of Occupations (SOC).
In its unabridged form, the O*NET-SOC taxonomy includes detailed information on 974 individual occupations in the United States, grouped into approximately 20 broader job families, which are regularly revised and updated for new and emerging occupations to keep up with the changing occupational landscape. The virtually unparalleled level of descriptive detail of O*NET allows us to dive deep into the job profile of individual occupations and to supplement our analysis with a range of job-specific further information, compiled in collaboration with industry experts and HR practitioners. For example, in addition to a complete profile of the skills and abilities currently perceived as required to perform a job successfully, O*NET provides further detailed information such as common qualifying degrees or certifications, typical activities performed on-the-job and physical working conditions—nuances which have been used by researchers such as Frey and Osborne (2013) to predict the extent of upcoming job task automation and which enable us to directly link our findings to these bodies of research for further customized analysis going forward.
O*NET also provides hard data on compensation, absolute employment numbers by occupation and their growth outlook to 2022 for the United States, which we used as one of several reference points to sense-check and calibrate our perception-based results.
In addition, the O*NET-SOC taxonomy can be converted into corresponding occupations under the International Labour Organization’s International Standard Classification of Occupations (ISCO), allowing for internationalization of results.10 We have made use of this possibility, for example, when estimating the absolute number of employees by job family in the Report’s Country Profiles. Note that, due to differences in the two classification systems, an O*NET-ISCO harmonized list of occupations reduces to around 350 from O*NET’s original list of 974 occupations, and it is this streamlined list of occupations that we have used in practice in the survey and Report. Furthermore, respondents to the Future of Jobs Survey had the ability to self-specify additional occupations they considered of particular relevance if they did not find these reflected in pre-given response options.
Finally, one implication of the structure of our survey questionnaire is that we have received an uneven number of responses per occupation—with, on the one hand, a strong convergence of data points around largest occupations in terms of employment and occupations perceived as critical/strategic for particular industries and, on the other hand, a relatively long tail of responses distributed across occupations with a lower number of individual mentions, affecting the reliability and margins of error of any individual predictions for the latter. In general, we address this by making use of O*NET’s two-level structure to report results aggregated at the broader job family level, not at the level of individual occupations.
Estimated employment effects have been converted into compound growth rates for the 2015–2020 period, i.e. the mean growth rate over the specified period of time if employment were to grow or decline at a steady rate. The compound annual growth rate is essentially a representational figure that describes the rate at which a quantity would have grown if it had grown at a constant rate. The simplest limitation of a compound growth rate is that because it calculates the smooth average of growth over a period, it ignores volatility and implies that the growth during that time was steady, which is rarely the case.
Reported employment outlook totals for job families have been weighted for the frequency of underlying occupation-level data points, using the standard weighted arithmetic mean formula:
on the simplifying assumption that more frequently mentioned occupations will also tend to employ a correspondingly larger absolute number of people in the concerned job family.
It is possible to calculate the net absolute number of jobs expected to be created or lost in the job family whenever the underlying quantity of workers employed in the corresponding occupation is known for the country or industry. In practice, we have chosen only do this on an illustrative basis for our sample as a whole in order to keep the focus of our analysis firmly on the relative outlook for particular job types and job families compared to each other over the 2015–2020 period.
The concept of skills is used in many different and sometimes ambiguous ways in different contexts and, unlike for occupations, a widely agreed shared definition or taxonomy of the term at the international level does not exist.11 In a general sense, the term skills is used to refer to the work-related capabilities of people to perform a job successfully. It should be distinguished from the concept of abilities, which refers to more fundamental and enduring attributes of an individual, such as physical or cognitive abilities that are formed over a longer period, often beginning in early childhood education.
In national and international statistics, achieved education levels or qualifications are often used as a proxy for or representation of skills. However, this approach is of limited use for the purposes of our Report and has been at the heart of conflicting perceptions about the current state of skills mismatches and the availability of qualified talent between education providers and employers. The education sector refers mainly to formal qualifications and credentials, whereas actors from the employment side refer to the actual practical capabilities or competences that employees (or new graduates) are able to use.
Given the unprecedented rate of change of both on-the-job skills required to perform in an occupation successfully and of the content or hard knowledge of the core curriculum in many academic fields, we felt that the Report should adopt a taxonomy that would allow for the highest level of nuance, detail and specificity possible in order to adequately capture the pace of skill set disruption. Accordingly, the Report’s analysis of skills focuses on a core set of 35 specific and widely used work-relevant skills and abilities that have been empirically derived from today’s most-frequently cited skills and abilities across all occupations and job families in the O*NET database. Our analysis found that the core skill set of most occupations consisted of a range of combinations of these 35 core skills. In addition, respondents to the Future of Jobs Survey had the possibility to self-specify further occupation-specific skills they considered essential if they did not already find these reflected in the pre-given response options.
Table A1 shows the full list of work-related core skills and abilities that we have adapted from the O*NET database. A number of these skills have been manually combined, renamed or reclassified from their original designation within O*NET for concision and clarity.12
Calculating skills stability
The Report repeatedly uses the concept of skills stability to illustrate the degree to which, by the year 2020, particular occupations and job types are expected to require competence in new skills that have hitherto not been part of that occupation’s core skill set today.
The data points for this calculation are derived from our executive survey, in which respondents were twice asked to specify the top five skills most frequently used by their specified occupation: once for today, once for the year 2020. Respondents could select their answer from the 35 skills shown in Table A1, and also had the option to self-specify skills requirements should they not be satisfied by the response options given. The degree of skills stability refers to the share of skills mentioned both in today’s core skill set and re-mentioned as a core skill for the year 2020. Growing skills demand refers to skills only mentioned as part of occupations’ specified core skill set in 2020. Declining skills demand refers to skills only mentioned as part of today’s core skill set but not re-mentioned for 2020. The various skills stability measures used throughout the Report are the result of aggregating these data points for the unit of analysis in question, such as a job family or industry sector.
As elsewhere, our approach represents a streamlining of the full O*NET taxonomy—which uses a longer list of skills—in order to make it amenable for use in an executive survey. In practice, limiting what we refer to as an occupation’s “core skillset” to the top five skills might mean certain skills that enter or drop out of the top five are merely experiencing a relative shift in importance, rather than being wholly new or ‘never used again thereafter’. However, even such relative shifts still strongly imply substantial ongoing change and transformation of skills requirements, convincing us of the soundness and validity of skills stability as a concept as used in the Report.
- International Standard Classification of Occupations (ISCO), International Labour Organization, http://www.ilo.org/public/english/bureau/stat/isco/.
- The Occupational Information Network (O*NET), US Department of Labor, Employment and Training Administration, https://www.onetonline.org.
- Standard Occupational Classification (SOC), US Department of Labor, Bureau of Labor Statistics, http://www.bls.gov/soc/.
- O*NET-ISCO Conversion Tables, http://www.bls.gov/soc/ISCO_SOC_Crosswalk.xls.
- Cappelli, P. and J. Keller, “Classifying Work in the New Economy”, Academy of Management, vol. 38, no. 4, 2013, pp. 575–596.
- Djordjevic, J., Why Becoming Large Matters: How Scalable, High-Growth Entrepreneurs Can Help Solve the Jobs Crisis, Endeavor Insight and Omidyar Network, 2013.
- European Centre for the Development of Vocational Training (CEDEFOP), Piloting a European Employer Survey on Skill Needs: Illustrative Findings, CEDEFOP Research Paper No. 36, 2013.
- ———, Quantifying Skills Needs in Europe – Occupational Skills Profiles: Methodology and Application, CEDEFOP Research Paper No. 30, 2013.
- European Training Foundation, Anticipating and Matching Skills Demand and Supply: Synthesis of National Reports, 2012.
- Frey, C. and M. Osborne, The Future of Employment: How Susceptible are Jobs to Computerisation?, Oxford Martin School Programme on the Impacts of Future Technology, 2013.
- ILO and GIZ, Is Small Still Beautiful? Literature Review of Recent Empirical Evidence on the Contribution of SMEs to Employment Creation, 2013.
- Kumar, K, R. Rajan and L. Zingales, What Determines Firm Size?, University of Chicago Graduate School of Business, Working Paper No. 496, 2001.
- McKinsey Global Institute, A Labor Market that Works: Connecting Talent with Opportunity in the Digital Age, 2015.
- National Center for O*NET Development, New and Emerging Occupations: Methodology Development Report, 2006.
- Pajarinen, M, P. Rouvinen and A. Ekeland, Computerization and the Future of Jobs in Norway, Discussion Paper, Research Institute of the Finnish Economy and Statistics Norway, 2015.
- UK Office for National Statistics, Concentration ratios for businesses by industry in 2004, 2006, http://www.ons.gov.uk/ons/rel/elmr/economic-trends–discontinued-/no–635–october-2006/concentration-ratios-for-businesses-by-industry-in-2004.pdf.
- World Bank, Small vs. Young Firms across the World Contribution to Employment, Job Creation, and Growth, Policy Research Working Paper 5631, 2011.
- Zervas, G, D. Proserpio and J. Byers, The Rise of the Sharing Economy: Estimating the Impact of Airbnb on the Hotel Industry, Working Paper, Boston University, 2015.