5. Quality and impact
What distinguishes high-quality career guidance services and how can the current provision of these services in Latin American countries be improved? This chapter analyses survey evidence of adults’ satisfaction with available services as well as the perceived training and employment outcomes of career guidance. It then reviews approaches to raise the quality of services, including using high-quality labour market information, tailoring services to adults’ needs, standardising staff training and qualifications, certifying career guidance providers and monitoring outcomes.
A recent OECD report on adult career guidance systems laid out three components of quality career guidance: high-quality labour market information; tailored to an individual’s needs; and provided by professional career guidance advisors. It also suggested several ways of to promote quality assurance in career guidance by: standardising training and qualifications of career guidance advisors; certifying providers against quality standards; and monitoring outcomes. The main findings from this chapter are:
Overall satisfaction with career guidance services in Latin American countries is high (88%). Furthermore, most users (82%) say that they experienced a positive outcome (e.g. enrolling in a training programme, finding a new job, getting a promotion) in the six months following career guidance. However, only a third of these adults attribute the positive outcomes to the career guidance they received. Adults are much more likely to report positive outcomes if they receive a personalised career development roadmap as an output of career guidance. They are also more likely to report employment outcomes if they receive services face-to-face, and if services are provided through their employer or an employer group. Enrolment in education and training participation is more likely if career services are provided by an education and training institution.
Career guidance advisors in the countries under review rely heavily on online platforms to keep them up-to-date about labour market developments. Mexico is the only country where career guidance advisors in the public employment service receive briefings about labour market developments.
In Latin American countries, as in most OECD countries, the job of ‘career guidance advisor’ is not a regulated profession. Countries in this review have taken steps towards streamlining the qualifications and competencies of career guidance advisors, though these steps have fallen short of developing competence frameworks.
Some 62% of adult users reported receiving a personalised career development roadmap as an output of guidance, despite roadmaps not being a required output of public programmes. This is a higher share than the average across all countries in the survey.
Interviews are the most common method for assessing an adult’s skills in Latin America, as in other countries. Self-reported information is often used in tandem with interviews and tends to focus on qualifications and occupational history. Skills profiling tools, including direct assessment tests, which more objectively assess an individual’s skills against a benchmark, are not used by career guidance advisors in Latin America. Given that they cover both skills acquired formally or informally, such tools may be particularly useful to provide guidance to adults in informal employment and those without formal qualifications.
Quality standards in career guidance are rare in Latin America, with Argentina being the only country that applies a comprehensive quality management system to career guidance provision.
Monitoring of career guidance provided by the public employment service is carried out by government bodies in Chile, Argentina and Mexico. Mexico also monitors the outcomes of educational guidance as part of its monitoring of education services more generally.
As outlined in Chapter 2, the labour market in Latin American countries is characterised by high inequality, a high share of informal employment, and high rates of over-qualification. By helping individuals make more informed employment and training decisions, career guidance can mitigate skills imbalances and potentially help individuals to move into more secure employment. It can help to reduce inequality by making sure all individuals have access to quality labour market and training information and guidance, regardless of their socio-economic situation. These positive outcomes are only possible, however, if services are of high quality.
This chapter reviews survey evidence on the impact of career guidance in Latin America, focusing on reported satisfaction and perceived employment and training outcomes. It compares Argentina, Brazil, Chile and Mexico along three different dimensions of quality career guidance: the use of high-quality labour market information; the implementation of a tailored approach to guidance provision; and the professional requirements for career guidance advisors. It also discusses two types of quality assurance mechanisms: the certification of providers against quality standards; and the monitoring of outcomes.
According to the SCGA, most adults in Latin America report being satisfied with the career guidance they receive. Some 87% of adults in Latin American countries are satisfied with the career guidance they received, which is on par with the average across all countries in the SCGA (Figure 4.1). A similar share of adults say that guidance is well-informed (88%), while a lower share report that it is well-targeted to their individual needs (81%). Perceived satisfaction is highest in Brazil (91%) and Mexico (89%), and just below average in Argentina (86%) and Chile (82%).
While user satisfaction is a helpful indicator of quality, cross-country comparisons should be interpreted with caution. User satisfaction surveys are highly subjective, and there is room for cultural bias. A review of global user satisfaction surveys found that respondents from Latin America are more likely than those elsewhere to choose ratings at the extreme ends of a scale (for a 1-10 scale, this generally means 1-4 or 9-10) (Wilcock, 2021[1]). Given these limitations, user satisfaction rates are likely most informative when looked at as a trend within a particular country.
Users of career guidance in Latin America are more likely to report that they experienced an improvement in employment or training outcomes within six months of receiving the service, compared to adults in other countries. The large majority of adults in Latin American countries (82%) confirmed that they had experienced an employment or training outcome – higher than the average for all countries in the SCGA (70%). The most common outcomes were progressing in one’s current job (37%), enrolling in training or education (26%), or finding a new job (26%). About 12% of adults reported that they had moved from informal to formal employment.
However, while most adults reported an employment or training outcome, a smaller share attributed the change to having received career guidance (Figure 5.2). Only about a third (32%) of adults in Latin America said that career guidance was useful in achieving employment and training outcomes. While this is a higher share compared to the average of all countries in the survey (27%), it suggests there is room for improving the quality of career guidance services.
Table 5.1 summarises results from a regression of the likelihood of achieving employment or training outcomes after receiving career guidance, while controlling for a set of individual, job and firm characteristics. The regression pools data from the four Latin American countries (Argentina, Brazil, Chile, and Mexico). However, coefficients are not significantly different from those obtained when including all countries in the regression, implying that the following results are not unique to Latin America:
Two factors stand out as being highly associated with achieving positive employment outcomes: receiving a personalised career development roadmap (increases the likelihood by 26%), and using services delivered by an employer or employer group (both increase the likelihood by 12%, relative to services delivered by the PES). Having face-to-face interaction with a career guidance advisor is also associated with a higher likelihood of achieving positive employment outcomes (5% higher than remote alternatives).
An adult is more likely to enrol in an education or training programme after receiving career guidance from an education or training provider (18% higher than when provided by the PES), followed by an employer group (10% higher). Receiving a personalised career development roadmap also raises the likelihood of enrolling in an education or training programme by 10%
Note: Average of Argentina, Brazil, Chile and Mexico. Respondents could choose more than one answer. Data refer to the last time the respondent spoke with a career guidance advisor.
Source: Survey of Career Guidance for Adults (SCGA).
A recent OECD report laid out three components of quality career guidance: it relies on high-quality labour market information, is tailored to an individual’s needs, and is provided by professional career guidance advisors (OECD, 2020[2]). The report also suggested several ways of promoting quality assurance in career guidance: by standardising training and qualifications of career guidance advisors, certifying providers against quality standards and monitoring outcomes.
This section compares Argentina, Brazil, Chile and Mexico along these quality dimensions. It draws good practice examples from OECD countries when applicable.
Using high-quality labour market information
According to the SCGA, adults in Latin America were more likely to receive some types of labour market information than others during their interaction with a career guidance advisor (Figure 4.3). Just over half received information about education and training programmes (53%), compared with 48% in all countries covered by the SCGA. This was followed by information about sectors in high or low demand (35% vs. 32% in all countries), job vacancies (30% vs. 35%) and sectors forecasted to be in high or low demand in the future (26%, similar to the overall average). Fewer adults reported receiving information on the quality of training providers (21%) or financial support available for training (12%).
Career guidance advisors need to keep up-to-date about labour market developments to provide quality career guidance. In Chile, for instance, career guidance advisors who are part of the Programa de Intermediación Laboral are expected to consult an online platform which is regularly updated with job vacancies by companies, and with training opportunities by SENCE. Similar websites exist in Brazil and Mexico, providing up to date labour market information. In some cases, training is provided to career guidance advisors. As part of Mexico’s Employment Support Programme, advisors receive training on the behaviour of local and regional labour markets.
Note: Average of Argentina, Brazil, Chile and Mexico. Respondents could choose more than one answer. Data refer to the last time the respondent spoke to a career guidance advisor.
Source: Survey of Career Guidance for Adults (SCGA).
Tailoring career guidance services to adults’ needs
While a high share of adults in Latin American countries report that career guidance is tailored to their needs (81%), a lower share report receiving a personalised career development roadmap (62%). A personalised career development roadmap – otherwise known as an individual training plan or personal action plan – spells out the activities needed to reach one’s career and training objectives. Receiving such an individualised roadmap increases the likelihood of achieving positive employment outcomes after career guidance by 25%, and of enrolling in an education or training programme by 7% (OECD, 2021[3]). It does so by empowering adults to take informed action towards their goals.
Adults in Brazil and Mexico were more likely than those in Argentina and Chile to receive a personalised career development roadmap (Figure 4.4). But roadmaps are not a required output of any of the publicly subsidised career guidance programmes in the countries in this review, while this is the case in some OECD countries. For example, in Australia, adults who participate in the Career Transition Assistance programme must receive a personalised “Career Pathway Plan” that provides tailored information on retraining opportunities in line with labour market needs.
A necessary starting point in providing personalised career and training pathway advice for adults is to conduct a thorough assessment of their skills. As in other countries, interviews are the most common method for assessing an adult’s skills in Latin American countries (Figure 4.5). Some 68% of adults were asked about their skills and experience, and another 21% were asked about their certificates or qualifications. Though common, interviews are subjective methods for assessing someone’s skills and they tend to rely heavily on job history and educational qualifications as a proxy for skills. They are also potentially costly in terms of staff time required to conduct them.
Using tests to assess an individual’s skills can provide an alternative or complement to interviews, though such tests are not widely used in public programmes in Latin American countries. Some 46% of adults who spoke with a career guidance advisor in Latin American countries were asked to complete a test to assess their skills. In Chile, career guidance advisors in the Programa de Intermediación Laboral complement interviews with various self-assessment tests. For the most part, these self-assessment tests try to capture the individual’s personality type and help them to narrow down their interests and preferences.1 In Mexico, adults can complete skills and vocational tests on the public employment support website, Observatorio Laboral.
Skills profiling tools go a step beyond self-assessments. They involve completing a test that can be graded against an answer key, making it more objective. These tools provide a measure of what an individual can actually do, and do not discriminate between skills acquired formally or informally. This makes them particularly effective in supporting adults in informal employment or those without formal qualifications. A handful of public employment services have started using skills profiling tools to support flexible pathways and to redeploy adults from declining to growing jobs and sectors. The PES in Italy and Spain piloted an online version of the OECD’s Survey of Adult Skills to test the literacy, numeracy and digital skills of jobseekers (Education and Skills Online).
Standardising training and qualifications of career guidance advisors
The training, qualifications and competencies of career guidance advisors are key for high-quality service delivery. Defining standardised requirements for this occupation can help to improve career guidance services, and make their quality more coherent across different providers. In Latin American countries, as in most OECD countries, the job of ‘career guidance advisors’ is not a regulated profession. No specialised training course, certificate or license exists for this occupational group. Nevertheless, working as a career guidance advisor in Latin American countries generally requires a certain level of training and qualification, usually a tertiary degree.
As in other countries, the qualifications and competencies of career guidance staff in Latin America differ according to the type of provider. Compared to career guidance professionals working for private providers, counsellors working for public employment services tend to have lower and less specialised qualification requirements, as their services are often broader and not restricted to career guidance. Furthermore, the variance in local governance and institutional capacity of public employment services that is common in Latin American countries is likely to also cause differences in staff qualifications. Career guidance advisors working for private providers or at universities, on the other hand, usually have higher or more specialised qualification requirements, such as tertiary-level degrees in psychology.
If career guidance advisors do not receive specialised training but instead hold general tertiary degrees, the risk is that they might lack crucial skills and knowledge to provide high-quality career guidance. This can also result in a varying standard of practice across providers. To address this, OECD (2004) recommended establishing competency frameworks as a first step to improve and standardise career guidance services, which many countries have since developed (OECD, 2021[3]). Competency frameworks define what career guidance advisors should know and do, and they form a foundation for designing training and qualifications. Examples from other countries show that introducing competency frameworks for career guidance advisors can be an important step towards more coherent and high-quality service provision. Examples from Austria and Canada are provided in Box 4.1.
Latin American countries have taken steps towards streamlining the qualifications and competencies of career guidance advisors, though these steps have fallen short of developing competence frameworks. In Argentina, general efforts to reduce disparities and streamline service provision across public employment offices included the improvement of training provided for counsellors. Mexico has a general competence framework for all providers of public services (atención al público). While this framework applies to career guidance advisors working for the public employment service (SNE – Servicio Nacional de Empleo), it is not a dedicated competence framework for this professional group. In Brazil, there is so far no regulation of the practice of career guidance. Professionals from various fields of knowledge work as career advisors, but most are psychologists. Several initiatives have aimed to regulate and professionalise career guidance occupations, such as a competence framework proposal developed based on the IAEVG (International Association for Educational and Vocational Guidance) model. None of these approaches, however, has been widely implemented.
Continuing professional development helps advisors develop and maintain their skills and knowledge in the face of changes in the labour market and in technology. Refresher training for career guidance advisors can help to uphold the quality of career guidance in the face of constant technological change. The need for career guidance advisors to develop or strengthen digital skills became especially apparent in the COVID-19 pandemic, as career guidance service delivery came to a halt or shifted online.
The public employment service in Austria uses the European Career Guidance Certificate (ECGC), which is based on the MEVOC standards (Quality Manual for Educational and Vocational Counselling). These standards were developed as a project of the European Union under the leadership of Austria (ibw – Institut für Bildungsforschung der Wirtschaft) in co-operation with 19 partner institutions from nine countries. They describe quality standards for educational and vocational counsellors. Based on the MEVOC standards, a competence grid was developed with 35 required competences in the following four areas: education, career, counselling practice, personality and ICT competences. To achieve the ECGC, counsellors can complete an online test and an assessment centre.
The Canadian Standards and Guidelines for Career Development Practitioners (S&G) determine the competences needed to provide high-quality career guidance. It was the first competence framework developed internationally and served as a model for other countries. The S&Gs provide a foundation for designing training, to provide quality assurance to the public, to recognise and validate the skill sets of practitioners in the field, and to create a common voice and vocabulary for career development. Competencies are organised in three areas: core competencies, specialisation competencies and ethical principles. Five provinces in Canada have developed professional certifications linked to the S&Gs (British Colombia, Alberta, Ontario, New Brunswick and Nova Scotia). The competence framework is currently being updated and expanded based on national consultations. A new national professional credential, tied to the new competence framework, is also being developed.
Source: OECD 2020 Policy Questionnaire ‘Career Guidance for Adults’.
Certifying providers against quality standards in service delivery
Quality standards help to promote high-quality service delivery across the country. They can govern several or all aspects of service delivery including professional standards, partnerships, labour market information, client satisfaction, evaluation and leadership (Dodd et al., 2019[4]).
Quality standards for career guidance in Latin American countries tend to focus on public provision of services. In Argentina, for example, a quality management system (QMS) with a customer-based approach for public employment services has been developed by the Argentinian Standards Institute (IRAM) in accordance with ISO 9 000 standards. The Argentinian Ministry of Employment and Social Security (MTEySS) also defined quality criteria, which included: a clear office structure, a marketing strategy, profiling methods, capacity for providing job and vocational counselling. If these criteria are met, a quality management certificate is issued. In 2015, 72 employment offices had been awarded the certificate (ILO, 2015[5]). On a less comprehensive scale, public funding of municipal employment offices (OMIL) in Chile is tied to achieving employment outcomes, as measured by a three-month job placement indicator.
In Argentina, providers that belong to the Network of Continuing Education and Training Institutions (Red de Formación Continua) also have to follow quality requirements introduced by the Ministry of Labour. These institutions can be run by unions, business associations, universities or non-for-profit providers. Many of them have been strengthened as part of a quality certification process, with the aim of aligning training and skills programmes with local labour market demand.
Monitoring outcomes
Monitoring outcomes of career guidance helps providers to evaluate and improve their performance, and holds the system to account. Common types of outcomes measured include user satisfaction, employment, wages, unemployment benefit receipt, and training participation.
In Latin American countries, career guidance programmes offered by the public employment service are monitored and evaluated by government bodies. In Argentina, the quality of public employment programmes is monitored by the Ministry of Labour, Employment and Social Security through its labour market observatories and in support of the quality management system (QMS) discussed above. Argentina’s Office of Studies and Statistics prepares and publishes reports about the effectiveness of employment programmes. Similarly, in Mexico, the National Council for the Evaluation of Social Development Policy (CONEVAL) monitors the outcomes of SNE’s programmes, and publishes data on its website. Indicators focus on the number of jobseekers who receive support. In Chile, the SENCE head office monitors the Regional Directorates, which in turn monitor the OMIL offices. SENCE tracks three main indicators: the number of users of labour intermediation services, labour market placements, and user satisfaction.
In Mexico, monitoring of career guidance outcomes is also carried out as part of the evaluation of education services more generally. A national evaluation of the results of each educational programme and project is carried out in the State Institutes and Delegations (Institutos y Delegaciones Estatales). Results of the evaluations are considered in the process of analysis and approval of the federation’s expenditure budget for the following fiscal year. These evaluations are conducted by academic and research institutions or organisations that specialise in adult education, and that meet minimum requirements set by the ministry.
Brazil stands out among the countries in this review for its lack of systematic monitoring practices for career guidance programmes. Brazil has data in place that could be better exploited to monitor the impacts of guidance. Rich administrative data from the Annual Social Information Report (RAIS) could track inputs and outputs of various policies, including guidance and labour intermediation programmes, especially when combined with other databases such as the General Register of Employed and Unemployed Individuals (CAGED). As argued in other analyses, these data could be used to develop a comprehensive monitoring and evaluation system focused on tracking outcomes such as employability (Joana Silva, 2015[6]).
Generally, satisfaction with currently available career guidance services is high and perceived outcomes are strong, even if most adults do not directly attribute those outcomes to career guidance. Still, there are many ways that the quality of services could be improved. Competence frameworks, which standardise what career guidance advisors should know and do, are not yet in place in Latin America and quality standards around service delivery are rare. The quality of public career guidance services also varies significantly across local employment offices in the countries reviewed.
Strengthen general public service delivery quality standards and include a component that considers career guidance specifically.
Further, countries should initiate national consultations to build a competence framework for career guidance advisors (see Chapter 6).
To be effective, guidance should be personalised. In Latin American countries, many adults are in informal employment and lack formal qualifications. Skills profiling tools can be extremely useful in this context to assess what adults can do, but they are not widely used in Latin America. Such a tool would help the advisor to assess what adults can actually do, and could be used as a starting point for making personalised recommendations about training and job pathways.
References
[4] Dodd, V. et al. (2019), Quality Assurance Standards: A synthesis of quality standards across partner countries, Erasmus, http://guidancequality.eu/quality-assurance-standards-a-synthesis-of-quality-standards-across-partner-countries/.
[5] ILO (2015), Public Employment Services in Latin America: Argentina, ILO Notes, https://www.ilo.org/wcmsp5/groups/public/---ed_emp/---emp_policy/---cepol/documents/publication/wcms_426610.pdf (accessed on 22 January 2021).
[6] Joana Silva, R. (2015), Sustaining Employment and Wage Gains in Brazil - A Skills and Jobs Agenda, The World Bank, https://openknowledge.worldbank.org/bitstream/handle/10986/22545/9781464806445.pdf;sequence=5.
[7] OECD (2021), Career Guidance for Adults in a Changing World of Work, OECD Publishing, Paris, https://www.oecd-ilibrary.org/employment/career-guidance-for-adults-in-a-changing-world-of-work_9a94bfad-en (accessed on 8 January 2021).
[3] OECD (2021), Career Guidance for Adults in a Changing World of Work, Getting Skills Right, OECD Publishing, Paris, https://dx.doi.org/10.1787/9a94bfad-en.
[2] OECD (2020), “Career guidance for adults”, 137th Session of the Employment, Labour and Social Affairs Committee (ELSAC).
[1] Wilcock, C. (2021), Comparing Apples to Pommes: Understanding and Accounting for Cultural Bias in Global B2B Research, https://www.b2binternational.com/publications/understanding-accounting-cultural-bias-global-b2b-research/ (accessed on 1 February 2021).
Note
← 1. In Chile, the Holland Test and Prediger Test are used to narrow down interests and preferences. Other tests assess the individual’s personality type (e.g. DISC test, Catell’s 16 Factor Personality Test, Personality Inventory for Salespeople, MBTI Personality Indicator).