For the last 40 years, the Executive Opinion Survey (the Survey) has been a key ingredient of the Global Competitiveness Report series and other Forum benchmarking activities. It is the longest-running and most extensive survey of its kind and provides a yearly evaluation of critical aspects of competitiveness for which statistical data is missing because it is either impossible or extremely difficult to measure on a global scale. The aim of the Survey is to capture reality as well as possible, and business leaders are arguably the best positioned to assess the business environment in which they operate.
Through the Survey, respondents are asked to evaluate the situation for specific domains at the country level—such as the appetite for entrepreneurial risk, the level of corruption, and the extent of the skills gap. The results complement other statistical data to provide a more complete assessment of the business environment and the drivers of productivity.
The indicators derived from the Survey are used in the calculation of the Global Competitiveness Index 4.0 (GCI 4.0) in addition to a number of other World Economic Forum indexes, such as the Global Gender Gap Index and the Travel & Tourism Competitiveness Index; and several publications, including The Inclusive Growth and Development Report and The Global Risks Report; as well as many Forum initiatives. A truly unique source of data, the Survey has also long been used by numerous international and nongovernmental organizations, think tanks and academia for empirical and policy work.
The survey 2019 in numbers
The 2019 edition of the Survey captured the views of 16,936 business executives in 139 economies between January and April 2019. Following the data editing process described below, a total of 12,987 responses were retained. In an effort to move away from paper surveys, this year more than half of the retained surveys (59.1%) were completed online (see Figure 1). The 2019 edition of the Survey was made available in 41 languages (see Table 1). Three additional countries are surveyed in 2019 compared to the previous edition (Barbados, Gabon, Madagascar) and two countries are not covered this year (Liberia and Sierra Leone).
Figure 1: Descriptive statistics of the Executive Opinion Survey 2019
Source: World Economic Forum, Executive Opinion Survey, 2019 edition. Note: Not all charts are drawn to scale. * Following data treatment. See text for details.
Table 1: Available languages in 2019
Survey structure, administration and methodology
The Survey comprises 78 questions. Most questions ask respondents to evaluate on a scale of 1 (considered among the worst in the world) to 7 (considered among the best in the world) the performance on various topics of the country where the respondent operates. The questions are organized into 10 topical areas: Infrastructure; Technology; Financial Environment; Foreign Trade and Investment; Domestic Competition; Business Operations and Innovation; Security; Governance; Education and Human Capital; and Risks. The 2019 edition of the Survey instrument is available in the Downloads section of the Global Competitiveness Report’s page at http://gcr.weforum.org/.
The administration of the Survey is supervised by the World Economic Forum and conducted at the national level by the Forum’s network of Partner Institutes. Partner Institutes are universities or other research organizations, business associations, competitiveness councils, or in some cases survey companies. These organizations have the private-sector network for reaching out to leading business executives and a firm commitment to improving the competitiveness of their respective economies (for the full list, see the Partners Institutes section of this report).
In administering the Survey and in order to gather the strongest dataset, Partner Institutes are asked to follow detailed sampling guidelines and collect the data in a specific timeframe. The collection process is based on best practices in the field of Survey administration and on discussions with survey experts. It is put in place to ensure that the sample of respondents is the most representative possible and comparable across the globe.
The sampling guidelines specify that the Partner Institutes create a sample frame (Figure 2)—a list of business executives from companies of various sizes and from the various sectors of activity.
The sample frame should reflect the structure of the economy as follows:
- It should be in proportion to the share of GDP by sector: agriculture, manufacturing industry, non-manufacturing industry (mining and quarrying, electricity, gas and water supply, construction), and services.
- It should ensure the representation of both large- (more than 250 employees) and small-sized companies (250 employees or fewer), again reflecting each sector. At least one-third of companies are large and one-third are small, and the remaining one-third are determined by the structure of the economy in proportion to the share of GDP by company size.
- It should ensure that the chosen companies also have a sufficiently wide geographical coverage.
The Partner Institutes are asked to separate the sample frame into two lists: one that includes only large firms, and a second that includes all other firms, retaining sectoral representation in both lists. Partner Institutes then randomly select from each list the firms that will receive the Survey.
The Survey is administered in a variety of formats. The primary method of administration is the online survey tool, but other methods are used: mail-in surveys, face-to-face interviews and telephone interviews.
In addition to administering the Survey, Partner Institutes play an active and essential role in disseminating the findings of The Global Competitiveness Report and other reports published by the World Economic Forum by holding press events and workshops to highlight the results at the national level to the business community, the public sector and other stakeholders.
Figure 2: Sample frame requirements
Data treatment and score computation
This section details the process whereby individual responses are edited and aggregated in order to produce the Survey question scores of each country. These are the results that, together with other indicators obtained from different sources, feed into the GCI 4.0 and other research projects.
Data editing
Prior to aggregation, the respondent-level data is subjected to a careful editing process. The following responses are excluded from the dataset: surveys where the respondent gives the same answer to at least 80% of the questions; surveys with a completion rate inferior to 50%; respondents who are not based in the same country as the Partner Institute; respondents who do not have the required level of seniority; and duplicate surveys—which can occur, for example, when a survey is both completed online and mailed in.
A univariate outlier test is then applied at the country level for each question of each survey. We use the standardized score—or “z-score”—method, which indicates by how many standard deviations any one individual answer deviates from the mean of the country sample. Individual answers with a standardized score greater than 3 are dropped. Additional statistical tests aimed at detecting responses that exhibit too little or too much variance across answers are used to exclude individual responses.
Computation of single-edition country scores
We use a simple average to compute scores at the country level. As the sample frame aims to replicate an economy’s sectoral composition and includes companies of different sizes, the country-level score of each Executive Opinion Survey question is the arithmetic mean of all answers in each country. That is, for a given question, all individual answers carry the same weight.
Formally, the average of a Survey indicator i for country c, denoted qi,c , is computed as follows:
where qi,c,j is the answer to question i in country c from respondent j; and Ni,c is the number of respondents to question i in country c.
Once responses have been aggregated at the country level, a test to detect statistical outliers is run. We leverage the strong relationship between the indicators derived from the Survey and some 50 statistical indicators included in the GCI 4.0: countries doing well on these indicators tend to do well in the Survey. A univariate linear regression is used to predict the expected average score of Survey indicators based on the average performance in the other indicators. Average Survey scores that lie outside the 90% confidence interval around the predicted values are considered “outliers”. The scores of individual Survey indicators are systematically corrected by a factor corresponding to the distance between the observed average Survey score and the predicted Survey average at the limit of the confidence interval.
In addition, an analysis to assess the reliability and consistency of the Survey data over time is carried out. As part of this analysis, an inter-quartile range (IQR) test is performed to identify large swings—positive and negative—between two editions. For each country, we compute the year-on-year difference, d, in the average score of a core set of 53 Survey questions. We then compute the inter-quartile range (i.e. the difference between the 25th percentile and the 75th percentile). Any value d outside the range bounded by the 25th percentile minus 1.5 times the IQR and the 75th percentile plus 1.5 times the IQR is identified as a potential outlier. This test is complemented by a series of additional empirical tests, including an analysis of five-year trends and a comparison of changes in the Survey results with changes in other indicators capturing similar concepts. We interview local experts and consider the latest developments in a country in order to assess the plausibility of the Survey results.
Country score computation
For each country and each Survey question, in the general case, the final country score is a weighted average of the single-edition scores of the two most recent editions of the Survey. The weighted average approach makes results less sensitive to the specific point in time when the Survey is administered. Second, it increases the amount of available information by providing a larger sample size. Additionally, because the Survey is carried out during the first quarter of the year, the average of the responses in the first quarter of 2018 and the first quarter of 2019 better aligns the Survey data with many of the data indicators from sources other than the Survey, which are often annual-averages data.
The weighted scheme used to compute the final country score is composed of two overlapping elements. We place more weight on the year with the larger sample size to attribute equal weight to each response. At the same time, we attribute greater weight to the most recent sample because it contains most up-to-date information. That is, we also “discount the past.” Table 2 reports the exact weights used in the computation of the scores of each country.
Table 2: Executive Opinion Survey: Descriptive statistics and weightings
The country scores thus obtained are then used for the computation of the Global Competitiveness Index 4.0.
Formally, for any given Survey question i, country c’s score,qi,c2018-19, is given by:
where is country c’s score on question i in year t, with t = 2018, 2019, as computed following the approach described in the text; and is the weight applied to country c’s score in year t.
The weights for each year are determined as follows:
and
where is the sample size (i.e. the number of respondents) for country c in year t, with t = 2018, 2019. a is the discount factor that accounts for temporality set at 0.6.
Plugging Equations (2a) and (2b) into (1) and rearranging yields:
In Equation (3), the first component of the weighting scheme is the discounted-past weighted average. The second component is the sample-size weighted average. These two components are given half-weight each. One additional characteristic of this approach is that it prevents a country sample that is much larger in one year from overwhelming the smaller sample from the other year. In the case of Survey questions that were introduced in 2019 for which, by definition, no past data exists, full weight is given to the 2019 score. For newly covered economies, this treatment is applied to all questions. For countries whose 2019 data were discarded, the results from the previous editions of the report are used instead. Box 1 provides an example of country score calculation.
Box 1: Example of score computation
For this example, we compute the score of Argentina on the indicator Diversity of workforce, which is included in the Global Competitiveness Index 4.0 (indicator 12.01). The indicator is derived from the following Survey question: “In your country, to what extent do companies have a diverse workforce (e.g. in terms of ethnicity, religion, sexual orientation, gender)?” (1 = not at all, 7 = to a great extent). Argentina’s score was 4.76 in 2018 and 5.04 in 2019. The weighting scheme described above indicates how the two scores are combined. In Argentina, the size of the sample was 84 in 2018 and 121 in 2019. Using α = 0.6 as discount factor and applying Equations (2a) and (2b) yields weights of 0.405 for 2018 and 0.595 for 2019 (see Table 2). The final country score for this question is therefore:
While numbers are rounded to two decimal places in this example and to one decimal place in result tables, full-precision figures are used in all calculations.