copy the linklink copied! 8. The Survey of Adult Skills (PIAAC) and the measurement of human capital

copy the linklink copied! picture

This chapter briefly discusses the concept of “human capital” and examines the extent to which the Survey of Adult Skills (PIAAC) assesses some of its components. It also compares the strengths and weaknesses of using direct measures of skills, such as those afforded by the Survey of Adult Skills, with those of using educational attainment to assess human capital.

    

Robust, internationally comparable measures of the proficiency of adults in cognitive skills such as literacy, numeracy and problem solving arguably have the potential to provide better proxy measures of human capital than commonly used measures, such as educational attainment or years of schooling, as well as providing important information in themselves. In 1998, a report on the measurement of human capital, Human Capital Investment, prepared by the OECD Centre for Educational Research and Innovation (CERI) concluded that: “To achieve a better understanding and measurement of human capital, it is necessary to develop direct measures of skill, competency and aptitudes, as well as the broad social and economic impact of human capital” (OECD, 1998, p. 81). In line with this conclusion, Hanushek and Woessman, in particular (see, for example, Woessman, 2003; Hanushek and Woessmann, 2009; and Hanushek, and Woessman, 2011), have argued that the results from international assessments of school students, such as PISA and Trends in International Mathematics and Science Study (TIMSS) (and results of adult surveys where they exist), constitute good measures of human capital and have considerable advantages over quantity-based measures, particularly completed years of schooling, at least in growth-accounting studies.

This chapter explores the extent to which it is legitimate to interpret the skills assessed in the Survey of Adult Skills (PIAAC) as (proxy) measures of human capital, the advantages and disadvantages of direct measures of key information-processing skills and measures based on educational qualifications as measures of human capital and, the ways in which these direct measures complement traditional measures to enhance the quality of indicators of human capital.

copy the linklink copied! Defining “human capital”

In considering the value of direct measures of cognitive skills as a measure of human capital, it is important first to define “human capital”. A useful definition is provided by OECD (1998), which defines human capital as “the knowledge, skills, competencies and other attributes embodied in individuals that are relevant to economic activity” (OECD, 1998, p. 9). The dimensions of human capital identified in the OECD definition are described in more detail in Table 8.1, drawing on the descriptions of similar concepts found in the competency literature.1

copy the linklink copied!
Table 8.1. Components of human capital

Component

Description

Knowledge

The body of facts, principles, theories and practices relevant to a field of work or study.

Skills

The ability to apply knowledge and use know-how to complete tasks and solve problems. Skills are commonly further classified into:

  • cognitive skills

  • technical skills

  • interpersonal and intrapersonal skills

  • communication skills.

Competency/Application

The ability to use knowledge and skills appropriately in real-life contexts and situations. Competency is often conceived in terms of capacity to exercise responsibility and act autonomously.

Personal attributes

The personality traits, behavioural dispositions and physical characteristics, such as strength, manual dexterity, height or even personal appearance, which may have a value on the labour market.

The components of human capital may be further specified in that knowledge, skills, competencies and attributes may be broadly transferable (or generic) in that they are relevant in a wide variety of situations (e.g. in different occupations and in different firms). Alternatively, they may be transferable to a limited extent or relevant in a limited set of situations (e.g. specific to an occupation or a particular enterprise) or related to a particular domain of knowledge or activity.

copy the linklink copied! Coverage of the dimensions of human capital in the survey of adult skills (PIAAC)

To what extent do educational qualifications and the measures provided by the Survey of Adult Skills cover the various dimensions of human capital as outlined in above?

Table 8.2 locates the skills assessed directly by the Survey of Adult Skills in a matrix defined in one dimension by the components of human capital and in the other by the degree of their transferability.

copy the linklink copied!
Table 8.2. Coverage of the dimensions of human capital directly assessed in the Survey of Adult Skills (PIAAC)

Broadly transferable

Less transferable

Knowledge

Assessed to a limited extent (literacy and numeracy)

Not assessed

Skills (cognitive)

Assessed (literacy, numeracy and problem solving)

Not assessed

Skills (technical)

Assessed to a limited extent (computer use)

Not assessed

Skills (inter and intra-personal)

Not assessed

Not assessed

Competency/Application

Not assessed

Not assessed

Personal attributes

Not assessed

Not assessed

The direct-assessment component of the Survey of Adult Skills focuses on measuring three cognitive skills (literacy, numeracy and problem solving in technology-rich environments) that are broadly transferable (generic) in nature. As is clear from the way these skills are defined in their frameworks (see Chapter 1), the assessment’s interest in these skills is centred on the application of knowledge and know-how in contexts that are relevant to adults generally. Content knowledge and technical skills represent a secondary focus of the assessment. A relatively limited amount of information is provided concerning respondents’ content knowledge (e.g. knowledge of basic mathematical concepts and operations in the case of numeracy). Some information is also provided regarding the mastery of certain technical skills (e.g. the capacity to use basic computer devices, commands, functions and applications) by the ICT core test and the problem-solving assessment, which assumes a basic level of skills in the use of applications and functionalities, such as e-mail, word processing, and spreadsheets.

Neither inter- and intra-personal skills nor personal attitudes are the object of direct assessment in the Survey of Adult Skills, even if questions are asked about the use of some inter- and intra-personal skills at work. Domain-specific skills (e.g. specific vocational or professional skills, firm-specific skills and knowledge related to particular fields of study) are also outside the scope of the survey, as is the extent to which individuals can act autonomously (competency).

The Survey of Adult Skills’ focus on assessing a small number of broadly transferable cognitive skills reflects both the importance attributed to measuring literacy, numeracy and problem solving in technology-rich environments as key information-processing skills, and the limits on what can be measured in a large-scale, international adult assessment given the current state of measurement science, the need to minimise the burden on respondents, and the amount of resources that can be reasonably be expected to be devoted to this type of exercise.

Direct measurement of inter- and intra-personal skills poses considerable methodological challenges in large-scale, cross-country surveys. These relate to both the definition of constructs and the methods of measurement. For example, what is considered to be the appropriate form of interaction between colleagues and superiors and, therefore, what behaviours define a concept such as “teamwork” are likely to vary between countries, given different cultural expectations and norms. Moreover, it is not obvious that individual survey-based approaches are effective for measuring inter- and intra-personal skills. These may be better assessed through observation or by using the judgements of the subject’s behaviour and interpersonal interactions by colleagues and/or supervisors. In the Adult Literacy and Life Skills Survey (ALL), for example, a framework for measuring teamwork was developed but not implemented as it was judged to be not sufficiently robust for a large-scale, cross-country assessment (see Murray et al., 2005, pp. 229-270). For the moment at least, information on inter- and intra-personal skills must be collected through indirect methods of the type used in the Survey of Adult Skills, whatever their limitations.2

Scales such as the “big Five”, “locus of control” and “grit” exist for measuring personality traits and behavioural dispositions. The “big Five” consists of an inventory of questions relating to five traits considered to represent personality at the broadest level of abstraction (see John and Srivastava, 2001): extraversion, agreeableness, conscientiousness, neuroticism, and openness to experience. “Locus of control” relates to beliefs about the extent to which life’s outcomes are under the subject’s own control as opposed to being determined by factors beyond his/her control. Individuals with an internal locus of control generally believe that life’s outcomes are due to their own efforts, while those with an external locus of control believe that outcomes are mainly due to external factors (Gatz and Karel, 1993). “Grit” relates to “perseverance and passion for long-term goals”, in other words, attributes related to “working strenuously toward challenges, maintaining effort and interest over years despite failure, adversity, and plateaus in progress” (Duckworth et al., 2007).

The “big Five” and “locus of control” inventories have been used to measure non-cognitive and personality traits in large-scale surveys such as the Household, Income and Labour Dynamics in Australia (HILDA) Survey (see HILDA, n.d.) and the German Panel survey (Headey and Holst, 2008). As noted in Chapter 5, both the “big Five” and “grit” scales are being administered as part of the World Bank’s STEP measurement study. Items relating to “locus of control” and “grit” were included in the field-test version of the Survey of Adult Skills background questionnaire. They were, however, dropped for the main study due to evidence of lack of comparability between countries.

There has also been work on assessing vocational, domain-specific skills and knowledge using large-scale survey techniques in an international context (see Baethge and Arends, 2009). The OECD completed a feasibility study on the Assessment of Higher Education Learning Outcomes (AHELO) in 2013, which investigated the feasibility of conducting an international assessment of university students that focuses on discipline-specific skills in economics and engineering as well as a set of generic skills (critical thinking, analytical reasoning, problem solving and written communication). The main issue regarding the measurement of domain-specific skills is less whether they can be validly and reliably measured in a cross-country context – which the AHELO feasibility study demonstrated to be possible – than the practicality and costs of measurement using household-survey methods, given their number and variety.

copy the linklink copied! Educational attainment as a measure of human capital

Educational attainment (or its variants, such as years of schooling) represents the most commonly used summary measure of human capital. This is due to its ready availability (information on educational qualifications is collected in most social surveys), the importance of qualifications as a signal of skills in the labour market, and the fact that educational qualifications provide a considerable amount of information regarding the breadth and depth of the knowledge, skills and competency of the individuals to which they have been awarded.3 The role and importance of formal education and training in the development of individuals’ store of knowledge and skills can hardly be disputed.

A good overview, albeit at a reasonably high level of generality, of the information summarised by the award of different educational qualifications can be gained by examining the descriptors of qualifications offered by national (and cross-national) qualifications frameworks. First, qualifications certify a broad range of learning outcomes. A common “horizontal” classification of the types of learning outcomes that education programmes are expected to impart and that graduates of these programmes are expected to display used in qualifications frameworks is that of “knowledge”, “skills” and “competency” (European Commission, 2008) or some variation of this.4 Second, qualifications offer information on the depth of knowledge and skills that graduates are expected to have acquired. Typically, qualifications frameworks group qualifications in terms of “levels”5 that represent stages in an ordered progression of the complexity and depth of knowledge and skills different educational programmes are intended to impart and that their “graduates” are, therefore, expected to display.

Taking the descriptors used in national and cross-national frameworks (e.g. the European Qualifications Framework) as a guide, educational qualifications can be regarded as offering relatively comprehensive measures of human capital in that they provide information about individuals’ stocks of both broadly transferable and less transferable knowledge, skills and competency (Table 8.3). They also provide information on the complexity and depth of these skills. The extent to which they cover any of the particular cells in the table will depend on the nature of the qualification. For example, vocationally oriented qualifications will certify the existence of skills with limited transferability to a far greater extent than will a general qualification, such as a certificate of senior secondary education.

copy the linklink copied!
Table 8.3. Coverage of the dimensions of human capital by educational qualifications

Broadly transferable

Transferable to a limited extent

Knowledge

low-high

low-high

Skills (cognitive)

low-high

low-high

Skills (technical)

low-high

low-high

Skills (inter and intra-personal)

low-high

low-high

Competency

low-high

low-high

Personal attributes

not covered

not covered

While representing reasonably comprehensive measures of human capital, educational attainment has some well-documented limitations as a measure of an individual’s level of skills:

  • Educational qualifications certify only the knowledge and skills developed through a course of study.6 They, thus, provide information about a subset of the skills of an individual. However, as noted above, this is by no means a negligible component of an individual’s skills, particularly in the case of young adults.

  • An educational qualification certifies the achievement of certain learning outcomes at a particular point in time. The currency of the measure will depend on the period of time that has elapsed since the qualification was awarded, and the experience (professional and otherwise) of individuals during this period. Skills can be lost as well as maintained and enhanced over time.

  • The quality of education and training offered at different levels of the education and training system can vary considerably between countries and, within countries, over time. Thus, the level of knowledge and skills certified by a qualification of ostensibly the same type and level may vary widely.

copy the linklink copied! Comparing measures of human capital

As can be seen from the above, direct measures of literacy, numeracy and problem solving in technology-rich environments and educational qualifications have different strengths and weaknesses as proxies of human capital or “global skills”. A comparison of four criteria is presented in Table 8.4 below:

  • Coverage – the extent to which the measure covers the different dimensions of human capital.

  • Context dependence – the extent to which the measure covers skills learned in a particular context, such as an educational institution.

  • Currency – the extent to which the measure is “up-to-date” as a measure of skills at the date information is collected.

  • Comparability – the extent to which the measure is comparable across countries and across time within countries.

copy the linklink copied!
Table 8.4. Comparison of direct measures from the Survey of Adult Skills (PIAAC) and qualifications on four criteria

Direct assessment (Survey of Adult Skills)

Qualifications

Coverage (content)

Limited (only 3 cognitive skills tested)

Broad

Context dependence

Low

High

Currency

High

Variable (depends on the time elapsed since the respondent’s highest qualification was completed)

Comparability

High

Variable both between and within countries

The Survey of Adult Skills’ direct measures provide detailed information about a narrow range of the skills that is highly current, not related to any particular context of acquisition, and is highly comparable within and between countries/economies. Qualifications provide information about most of the dimensions of human capital, but cover only those skills developed through formal education and training, are of varying currency (most current for the young and least current for the old), and are of sometimes dubious comparability.

copy the linklink copied! Empirical evidence

Analysis of data from the Survey of Adult Skills, International Adult Literacy Survey (IALS) and ALL provides some empirical evidence relevant to the question of the value of direct measures of proficiency in information processing skills and educational attainment as indicators of human capital. First, direct measures and educational qualifications do not appear to measure the same underlying traits. While educational attainment and literacy proficiency, for example, are closely correlated, there is considerable variation evident in literacy proficiency among individuals with similar levels of attainment (see Chapter 5 of OECD Skills Outlook [OECD, 2013] and OECD/Statistics Canada, 2000 and 2011). Second, educational attainment and literacy proficiency each have an independent and positive impact on earnings (see Chapter 6 of OECD Skills Outlook [OECD, 2013], OECD/Statistics Canada, 2000, pp. 76-79; OECD/Statistics Canada, 2011).

copy the linklink copied! Enhancing the measurement of human capital

In sum, direct measures of skills are best seen as offering an important complement to the indirect measures of human capital provided by educational attainment rather than as a substitute for them. By providing information both on educational attainment and proficiency in literacy, numeracy and problem solving in technology-rich environments, the Survey of Adult Skills offers greater insight into the human capital endowments of individuals and nations than would otherwise be available. Linked to the fact that it covers more countries/economies than previous adult skills surveys, measures new domains of skills and, in some countries/economies, provides for comparisons with previous surveys, the Survey of Adult Skills should offer a more accurate picture of skills relevant to the labour market and could help to explain differences in earnings and economic growth.

References

Australian Qualifications Framework Council (2013), Australian Qualifications Framework Second Edition January 2013, https://www.aqf.edu.au/sites/aqf/files/aqf-2nd-edition-january-2013.pdf.

Baethge, M. and L. Arends (2009), Feasibility Study VET-LSA: A Comparative Analysis of Occupational Profiles and VET Programmes in 8 European Countries: International Report, Federal Ministry of Education and Research, Bonn.

Barro, R.J. and J.-W. Lee (2010), “A New Data Set of Educational Attainment in the World, 1950-2010”, NBER Working Paper No. 1590, www.nber.org/papers/w15902.pdf?new_window=1.

Duckworth, A.L., C. Peterson, M.D. Matthews and D.R. Kelly (2007), “Grit: Perseverance and Passion for Long-Term Goals”, Journal of Personality and Social Psychology, Vol. 92, No. 6, pp. 1087-1101.

European Commission (EC) (2008), The European Qualifications Framework for Lifelong Learning (EQF), Office for Official Publications of the European Communities, Luxembourg.

Gatz, M. and J. Karel (1993), “Individual Change in Perceived Control over 20 Years”, International Journal of Behavioral Development, No. 16, pp. 305-322.

Hanushek, E.A. and L. Woessmann (2011), “The Economics of International Differences in Educational Achievement”, in E.A. Hanushek, S. Machin and L. Woessmann (eds), Handbooks in Economics, Vol. 3, The Netherlands, pp. 89-200.

Hanushek, E.A. and L. Woessmann (2009), “Do Better Schools Lead to More Growth? Cognitive Skills, Economic Outcomes, and Causation”, NBER Working Paper, No. 14633, www.nber.org/papers/w14633.pdf?new_window=1.

Headey, B. and E. Holst (eds.) (2008), A Quarter Century of Change: Results from the German Socio-Economic Panel (SOEP), DIW, Berlin.

Household of Income and Labour Dynamics Australia Survey (HILDA) (n.d.), HILDA Website: www.melbourneinstitute.com/hilda/.

John, O. and S. Srivastava (2001), “The Big-Five Trait Taxonomy: History, Measurement, and Theoretical Perspectives”, in L. Pervin, O. John (Eds.), Handbook of Personality: Theory and Research, Chap. 4, Guilford Press, New York, 2nd ed., pp. 102-138.

Murray, S., Y. Clermont and M. Binkley (eds.) (2005), Measuring Adult Literacy and Life Skills: New Frameworks for Assessment, Statistics Canada, Ottawa, Catalogue No. 89-552-MIE, No. 13.

OECD (2013), OECD Skills Outlook 2013: First Results from the Survey of Adult Skills, OECD Publishing, Paris, http://dx.doi.org/10.1787/9789264204256-en.

OECD (1998), Human Capital Investment: An international Comparison, OECD Publishing, Paris, http://dx.doi.org/10.1787/9789264162891-en.

OECD/Statistics Canada (2011), Literacy for Life: Further Results from the Adult Literacy and Life Skills Survey, OECD Publishing, Paris, http://dx.doi.org/10.1787/9789264091269-en.

OECD/Statistics Canada (2000), Literacy in the Information Age: Final Report of the International Adult Literacy Survey, OECD Publishing, Paris, http://dx.doi.org/10.1787/9789264181762-en.

Scottish Credit and Qualifications Framework (SCQF) (n.d.), Scottish Credit and Qualifications Framework Website: https://scqf.org.uk/the-framework/about-the-framework.

UNESCO (2011), Revision of the International Standard Classification of Education (ISCED), Paper 36 C/19, 34th Session of the General Conference, 2011, UNESCO, http://uis.unesco.org/sites/default/files/documents/international-standard-classification-of-education-isced-2011-en.pdf.

Woessmann, L. (2003), “Specifying Human Capital”, Journal of Economic Surveys, Vol. 17, No. 3, pp. 239-270.

Notes

← 1. See the previous chapter for a discussion of the uses of the terms “skill” and “competency”.

← 2. These are well known. First, while it can be inferred from the fact that a person undertakes certain tasks at work that he/she possesses the skills necessary to undertake these tasks to a greater or lesser extent, the level of his/her proficiency in these skills cannot be accurately inferred. Second, the degree of overlap between what people are required to do at work and what they can do is not necessarily particularly high. It is likely that many, if not most, adults possess the skills to effectively perform many tasks that they are not required to undertake at work.

← 3. Barro and Lee (2010) argue that at the macro-level, accurate time series of years of schooling (based on attainment measures) can be developed for most countries and that these provide a reasonable proxy for the stock of human capital in a broad range of countries.

← 4. The Australian Qualifications Framework Council (2013) defines three dimensions of learning outcomes: knowledge, skills and application. The Scottish Credit and Qualifications Framework (SCQF, n.d) defines five classes of learning outcomes: knowledge and understanding (mainly subject-based); practice (applied knowledge and understanding); generic cognitive skills (e.g. evaluation, critical analysis); communication, numeracy and IT skills; and autonomy, accountability and working with others.

← 5. See, for example, the explanation of “levels” in the International Standard Classification of Education (ISCED): “The notion of “levels” of education is represented by an ordered set of categories, intended to group educational programmes in relation to gradations of learning experiences and the knowledge, skills and competencies which each programme is designed to impart” (UNESCO, 2011, p. 10).

← 6. With the exception of awarding qualifications based on the recognition of prior learning (RPL). Qualifications awarded on the basis of RPL represent a minute proportion of the qualifications held by the adult population.

Metadata, Legal and Rights

This document, as well as any data and map included herein, are without prejudice to the status of or sovereignty over any territory, to the delimitation of international frontiers and boundaries and to the name of any territory, city or area. Extracts from publications may be subject to additional disclaimers, which are set out in the complete version of the publication, available at the link provided.

https://doi.org/10.1787/f70238c7-en

© OECD 2019

The use of this work, whether digital or print, is governed by the Terms and Conditions to be found at http://www.oecd.org/termsandconditions.

8. The Survey of Adult Skills (PIAAC) and the measurement of human capital