copy the linklink copied! 8. The Survey of Adult Skills (PIAAC) and the measurement of human capital

copy the linklink copied! Defining “human capital”

In considering the value of direct measures of cognitive skills as a measure of human capital, it is important first to define “human capital”. A useful definition is provided by OECD (1998), which defines human capital as “the knowledge, skills, competencies and other attributes embodied in individuals that are relevant to economic activity” (OECD, 1998, p. 9). The dimensions of human capital identified in the OECD definition are described in more detail in Table 8.1, drawing on the descriptions of similar concepts found in the competency literature.1

The components of human capital may be further specified in that knowledge, skills, competencies and attributes may be broadly transferable (or generic) in that they are relevant in a wide variety of situations (e.g. in different occupations and in different firms). Alternatively, they may be transferable to a limited extent or relevant in a limited set of situations (e.g. specific to an occupation or a particular enterprise) or related to a particular domain of knowledge or activity.

copy the linklink copied! Coverage of the dimensions of human capital in the survey of adult skills (PIAAC)

To what extent do educational qualifications and the measures provided by the Survey of Adult Skills cover the various dimensions of human capital as outlined in above?

Table 8.2 locates the skills assessed directly by the Survey of Adult Skills in a matrix defined in one dimension by the components of human capital and in the other by the degree of their transferability.

The direct-assessment component of the Survey of Adult Skills focuses on measuring three cognitive skills (literacy, numeracy and problem solving in technology-rich environments) that are broadly transferable (generic) in nature. As is clear from the way these skills are defined in their frameworks (see Chapter 1), the assessment’s interest in these skills is centred on the application of knowledge and know-how in contexts that are relevant to adults generally. Content knowledge and technical skills represent a secondary focus of the assessment. A relatively limited amount of information is provided concerning respondents’ content knowledge (e.g. knowledge of basic mathematical concepts and operations in the case of numeracy). Some information is also provided regarding the mastery of certain technical skills (e.g. the capacity to use basic computer devices, commands, functions and applications) by the ICT core test and the problem-solving assessment, which assumes a basic level of skills in the use of applications and functionalities, such as e-mail, word processing, and spreadsheets.

Neither inter- and intra-personal skills nor personal attitudes are the object of direct assessment in the Survey of Adult Skills, even if questions are asked about the use of some inter- and intra-personal skills at work. Domain-specific skills (e.g. specific vocational or professional skills, firm-specific skills and knowledge related to particular fields of study) are also outside the scope of the survey, as is the extent to which individuals can act autonomously (competency).

The Survey of Adult Skills’ focus on assessing a small number of broadly transferable cognitive skills reflects both the importance attributed to measuring literacy, numeracy and problem solving in technology-rich environments as key information-processing skills, and the limits on what can be measured in a large-scale, international adult assessment given the current state of measurement science, the need to minimise the burden on respondents, and the amount of resources that can be reasonably be expected to be devoted to this type of exercise.

Direct measurement of inter- and intra-personal skills poses considerable methodological challenges in large-scale, cross-country surveys. These relate to both the definition of constructs and the methods of measurement. For example, what is considered to be the appropriate form of interaction between colleagues and superiors and, therefore, what behaviours define a concept such as “teamwork” are likely to vary between countries, given different cultural expectations and norms. Moreover, it is not obvious that individual survey-based approaches are effective for measuring inter- and intra-personal skills. These may be better assessed through observation or by using the judgements of the subject’s behaviour and interpersonal interactions by colleagues and/or supervisors. In the Adult Literacy and Life Skills Survey (ALL), for example, a framework for measuring teamwork was developed but not implemented as it was judged to be not sufficiently robust for a large-scale, cross-country assessment (see Murray et al., 2005, pp. 229-270). For the moment at least, information on inter- and intra-personal skills must be collected through indirect methods of the type used in the Survey of Adult Skills, whatever their limitations.2

Scales such as the “big Five”, “locus of control” and “grit” exist for measuring personality traits and behavioural dispositions. The “big Five” consists of an inventory of questions relating to five traits considered to represent personality at the broadest level of abstraction (see John and Srivastava, 2001): extraversion, agreeableness, conscientiousness, neuroticism, and openness to experience. “Locus of control” relates to beliefs about the extent to which life’s outcomes are under the subject’s own control as opposed to being determined by factors beyond his/her control. Individuals with an internal locus of control generally believe that life’s outcomes are due to their own efforts, while those with an external locus of control believe that outcomes are mainly due to external factors (Gatz and Karel, 1993). “Grit” relates to “perseverance and passion for long-term goals”, in other words, attributes related to “working strenuously toward challenges, maintaining effort and interest over years despite failure, adversity, and plateaus in progress” (Duckworth et al., 2007).

The “big Five” and “locus of control” inventories have been used to measure non-cognitive and personality traits in large-scale surveys such as the Household, Income and Labour Dynamics in Australia (HILDA) Survey (see HILDA, n.d.) and the German Panel survey (Headey and Holst, 2008). As noted in Chapter 5, both the “big Five” and “grit” scales are being administered as part of the World Bank’s STEP measurement study. Items relating to “locus of control” and “grit” were included in the field-test version of the Survey of Adult Skills background questionnaire. They were, however, dropped for the main study due to evidence of lack of comparability between countries.

There has also been work on assessing vocational, domain-specific skills and knowledge using large-scale survey techniques in an international context (see Baethge and Arends, 2009). The OECD completed a feasibility study on the Assessment of Higher Education Learning Outcomes (AHELO) in 2013, which investigated the feasibility of conducting an international assessment of university students that focuses on discipline-specific skills in economics and engineering as well as a set of generic skills (critical thinking, analytical reasoning, problem solving and written communication). The main issue regarding the measurement of domain-specific skills is less whether they can be validly and reliably measured in a cross-country context – which the AHELO feasibility study demonstrated to be possible – than the practicality and costs of measurement using household-survey methods, given their number and variety.

copy the linklink copied! Educational attainment as a measure of human capital

Educational attainment (or its variants, such as years of schooling) represents the most commonly used summary measure of human capital. This is due to its ready availability (information on educational qualifications is collected in most social surveys), the importance of qualifications as a signal of skills in the labour market, and the fact that educational qualifications provide a considerable amount of information regarding the breadth and depth of the knowledge, skills and competency of the individuals to which they have been awarded.3 The role and importance of formal education and training in the development of individuals’ store of knowledge and skills can hardly be disputed.

A good overview, albeit at a reasonably high level of generality, of the information summarised by the award of different educational qualifications can be gained by examining the descriptors of qualifications offered by national (and cross-national) qualifications frameworks. First, qualifications certify a broad range of learning outcomes. A common “horizontal” classification of the types of learning outcomes that education programmes are expected to impart and that graduates of these programmes are expected to display used in qualifications frameworks is that of “knowledge”, “skills” and “competency” (European Commission, 2008) or some variation of this.4 Second, qualifications offer information on the depth of knowledge and skills that graduates are expected to have acquired. Typically, qualifications frameworks group qualifications in terms of “levels”5 that represent stages in an ordered progression of the complexity and depth of knowledge and skills different educational programmes are intended to impart and that their “graduates” are, therefore, expected to display.

Taking the descriptors used in national and cross-national frameworks (e.g. the European Qualifications Framework) as a guide, educational qualifications can be regarded as offering relatively comprehensive measures of human capital in that they provide information about individuals’ stocks of both broadly transferable and less transferable knowledge, skills and competency (Table 8.3). They also provide information on the complexity and depth of these skills. The extent to which they cover any of the particular cells in the table will depend on the nature of the qualification. For example, vocationally oriented qualifications will certify the existence of skills with limited transferability to a far greater extent than will a general qualification, such as a certificate of senior secondary education.

While representing reasonably comprehensive measures of human capital, educational attainment has some well-documented limitations as a measure of an individual’s level of skills:

• Educational qualifications certify only the knowledge and skills developed through a course of study.6 They, thus, provide information about a subset of the skills of an individual. However, as noted above, this is by no means a negligible component of an individual’s skills, particularly in the case of young adults.

• An educational qualification certifies the achievement of certain learning outcomes at a particular point in time. The currency of the measure will depend on the period of time that has elapsed since the qualification was awarded, and the experience (professional and otherwise) of individuals during this period. Skills can be lost as well as maintained and enhanced over time.

• The quality of education and training offered at different levels of the education and training system can vary considerably between countries and, within countries, over time. Thus, the level of knowledge and skills certified by a qualification of ostensibly the same type and level may vary widely.

copy the linklink copied! Comparing measures of human capital

As can be seen from the above, direct measures of literacy, numeracy and problem solving in technology-rich environments and educational qualifications have different strengths and weaknesses as proxies of human capital or “global skills”. A comparison of four criteria is presented in Table 8.4 below:

• Coverage – the extent to which the measure covers the different dimensions of human capital.

• Context dependence – the extent to which the measure covers skills learned in a particular context, such as an educational institution.

• Currency – the extent to which the measure is “up-to-date” as a measure of skills at the date information is collected.

• Comparability – the extent to which the measure is comparable across countries and across time within countries.

The Survey of Adult Skills’ direct measures provide detailed information about a narrow range of the skills that is highly current, not related to any particular context of acquisition, and is highly comparable within and between countries/economies. Qualifications provide information about most of the dimensions of human capital, but cover only those skills developed through formal education and training, are of varying currency (most current for the young and least current for the old), and are of sometimes dubious comparability.

copy the linklink copied! Empirical evidence

Analysis of data from the Survey of Adult Skills, International Adult Literacy Survey (IALS) and ALL provides some empirical evidence relevant to the question of the value of direct measures of proficiency in information processing skills and educational attainment as indicators of human capital. First, direct measures and educational qualifications do not appear to measure the same underlying traits. While educational attainment and literacy proficiency, for example, are closely correlated, there is considerable variation evident in literacy proficiency among individuals with similar levels of attainment (see Chapter 5 of OECD Skills Outlook [OECD, 2013] and OECD/Statistics Canada, 2000 and 2011). Second, educational attainment and literacy proficiency each have an independent and positive impact on earnings (see Chapter 6 of OECD Skills Outlook [OECD, 2013], OECD/Statistics Canada, 2000, pp. 76-79; OECD/Statistics Canada, 2011).

copy the linklink copied! Enhancing the measurement of human capital

In sum, direct measures of skills are best seen as offering an important complement to the indirect measures of human capital provided by educational attainment rather than as a substitute for them. By providing information both on educational attainment and proficiency in literacy, numeracy and problem solving in technology-rich environments, the Survey of Adult Skills offers greater insight into the human capital endowments of individuals and nations than would otherwise be available. Linked to the fact that it covers more countries/economies than previous adult skills surveys, measures new domains of skills and, in some countries/economies, provides for comparisons with previous surveys, the Survey of Adult Skills should offer a more accurate picture of skills relevant to the labour market and could help to explain differences in earnings and economic growth.