List of contributors
Phillip L. Ackerman (Chapter 8) is Professor of Psychology at the Georgia Institute of Technology. He received his Ph.D. from the University of Illinois, Urbana-Champaign. He has conducted basic and applied research in cognitive psychology, individual differences, psychological testing, and human abilities. He has written extensively on the nature of adolescent and adult learning, skill acquisition, selection, training, abilities, personality, and motivation. He has co-edited three books on individual differences and is the editor of a book on cognitive fatigue. Dr. Ackerman’s main contributions involve integration across multiple fields of psychological inquiry, specifically related to individual differences. Noteworthy contributions include integration of information processing and ability approaches to individual differences in skill learning; of ability and motivational determinants of learning and performance; of ability, personality, and interest traits; and explication of an integrated approach to adolescent and adult intellectual development. He is a Fellow of the American Psychological Association; Human Factors & Ergonomics Society, American Educational Research Association, Psychonomic Society, and he is a Charter Fellow of the Association for Psychological Science.
Guillaume Avrin (Chapter 15) is in charge of the "Evaluation of Artificial Intelligence" department at LNE, French National Laboratory for Metrology and Testing. In this context, Dr. Avrin conducts scientific and technical work on the definition of evaluation protocols, metrics, and test environments (databases, simulators, physical test benches) for intelligent systems. He is an expert on trustworthy AI for various institutional actors (DG CONNECT, COFRAC, HAS, various ministries) and coordinates various consortia and working groups at an international level with the aim of defining and eventually converging on a fundamental and operational metrology for AI (Dr. Avrin coordinates in particular the European consortium METRICS, made up of 17 testing centers for intelligent robots, as well as a working group of about fifteen industrial companies that created the first certification standard for AI). It also participates in standardization committees on AI and robotics (Afnor IA, ISO/IEC JTC1/SC42, CEN-CENELEC JTC21, UNM81, etc.), intervening notably as head of the French delegation to the JTC21.
A Distinguished Professor at UCLA, Eva L. Baker (Chapter 19) researches design and validation of multipurpose training and assessments systems, and is now developing games, simulations, and scenario-based assessments (workforce skills) for the U.S. Navy and PBS (early learning). Her AI studies include benchmarking as well as evaluations of ITSs, games, interventions, and applying AI to assessment. She has served as Co-Chair of the Standards for Educational and Psychological Testing, has been President of both the American and World Educational Research Associations, and is Founding Director of CRESST. A member of the National Academy of Education and a fellow in scholarly associations (e.g., American Psychological Association, American Educational Research Association, American Psychological Society), she has numerous awards in measurement and is widely published.
Abel Baret (Editor, Chapter 1) is a research assistant in the AI and Future of Skills project. He holds a BA degree in economics from the Toulouse School of Economics and a MSc in economics and psychology from Paris 1 Panthéon-Sorbonne and Université de Paris. Before joining the OECD, he was involved as a research assistant in experimental and behaviour economics at the CNRS Maison des Sciences Economiques in Paris.
Lucy Cheke (Chapter 17) is an Assistant Professor at the Department of Psychology, University of Cambridge. She is also the Director of the Kinds of Intelligence Programme at the Leverhulme Centre for the Future of Intelligence, which focuses on artificial intelligence in the context of other – human and nonhuman - cognition. She has worked for many years on assessment of learning and memory in nonhuman animals, children and adults - and more recently AI agents. Much of this work has focused on the comparability and interpretability of cognitive assessments across and within groups/species, with an emphasis on pattern, rather than sum, of performance.
Sylvie Chokron (Chapter 4) is a neuropsychologist and a Senior Researcher at CNRS. She is the head of the I3N (Institut de Neuropsychologie, Neurovision et NeuroCognition) at the Fondation Ophtalmologique Rothschild, Paris, where babies, children, and adults with visual and cognitive deficits are diagnosed and treated. She also heads the Perception, Action and Cognitive Development team (INCC, CNRS and University Paris-Descartes) where she develops fundamental research on visual cognition, attention and spatial representation in typical, atypical and brain-damaged patients as well as clinical applications in the field of visual cognition. Sylvie Chokron is a lecturer in several master programs and has a regular Neuroscience chronicle in the French Newspaper ‘Le Monde’ as well as in ‘Le magazine de la Santé’ (TV show on France 5).
Anthony G. Cohn (Chapter 14) is Professor of Automated Reasoning in the School of Computing, University of Leeds. His current research interests range from theoretical work on spatial calculi (receiving a KR test-of-time classic paper award in 2020) and spatial ontologies, to cognitive vision, modelling spatial information in the hippocampus, and Decision Support Systems, particularly for the built environment, as well as robotics. He is Editor-in-Chief of Spatial Cognition and Computation and was previously Editor-in-chief of the AI journal. He is the recipient of Distinguished Service Awards from IJCAI and AAAI as well as the 2021 Herbert A Simon Cognitive Systems prize. He is a Fellow of the Royal Academy of Engineering, the Alan Turing Institute in the UK, and is also a Fellow of AAAI, AISB, and EurAI. He holds Distinguished Visiting Professor positions at three Chinese Universities.
Matthew Crosby (Chapter 17) is a research scientist at DeepMind and creator of the AnimalAI testbed. He is primarily interested in discovering how to build and understand agents capable of solving the kinds of cognitive tasks that humans, and many animals, find easy so that we can later build *and understand* agents capable of solving the kinds of cognitive tasks we find hard. While working on this problem he has collected a PhD and three Masters across AI, Philosophy, Mathematics and Cognitive Science and hopes to bring ideas from each of the fields together to solve the problem.
Ernest Davis (Chapter 12) is Professor of Computer Science at New York University. He received his B.Sc. in mathematics from MIT and his Ph.D. in computer science from Yale. Davis' research area is the representation of commonsense knowledge in artificial intelligence systems, particularly for spatial and physical reasoning. He is the author of more than ninety scientific papers and four books: "Representing and Acquiring Geographic Knowledge" (1986); "Representations of Commonsense Knowledge" (1990); "Linear Algebra and Probability for Computer Science Applications" (2012); and, with Gary Marcus, "Rebooting AI: Building Artificial Intelligence We Can Trust" (2019). He also has published numerous book reviews and articles for a general readership in The New York Times, the New Yorker, the Times Literary Supplement, WIRED, and elsewhere.
Filip De Fruyt (Chapter 5) is senior full professor of Differential Psychology and Personality Assessment at Ghent University in Belgium. He is also a member of Edulab21, the research branch of the Institute Ayrton Senna in Brazil. He is specialised in assessing and building psychometric models on how individuals differ from each other and how that affects their functioning in daily life and work. He currently holds the Institute Ayrton Senna chair at Ghent University. He is the Past President of the European Association of Personality Psychology (EAPP) and is a Fellow of the Society of Industrial and Organizational Psychology (SIOP). De Fruyt has (co-)authored over 200 research papers in a broad range of leading academic journals that are cited over 20.000 times (Google Scholar, 2021).
Jan Dörendahl (Chapter 7) is a Psychologist and Data Scientist. From 2016 to 2021 he was part of the research group Computer-Based Assessment at the University of Luxembourg. During that time, he investigated the assessment of fundamental motives and goals and obtained his PhD in psychology in 2019. Further, he lectured multivariate statistics and was among the lead item developers for the assessment of adaptive problem solving in the PIAAC 2021 cycle. Since 2021, Dr. Dörendahl is working as a Data Scientist combining machine learning algorithms and internet-of-things technologies into innovative solutions.
David Dorsey (Chapter 10) currently serves as a Vice President at the Human Resources Research Organization (HumRRO). Prior to joining HumRRO, Dr. Dorsey was a senior executive in the U.S. Department of Defense, where he served as the Chief of Organizational Effectiveness and Workforce Research and as a Senior Data Scientist. Prior to his government service, Dr. Dorsey was a Vice President at Personnel Decisions Research Institutes (PDRI). Dr. Dorsey has produced over 70 professional book chapters, articles, and presentations. For his overall contributions to the field, Dr. Dorsey was elected a Fellow by the Society for Industrial and Organizational Psychology. He is the recipient of two major research awards and an award for being a top leader in government. Dr. Dorsey received his PhD in Industrial and Organizational Psychology with a graduate minor in Computer Science from the University of South Florida.
Stuart W. Elliott (Editor, Chapter 20) is a senior analyst at the OECD where he leads the AI and the Future of Skills project. He holds a doctorate degree in economics from the Massachusetts Institute of Technology and a BA in economics from Columbia University. He also received postdoctoral training in cognitive psychology at Carnegie Mellon University. He authored the 2017 CERI report on Computers and the Future of Skill Demand, which provided the groundwork for the design of the AI and the Future of Skills project. He is also a scholar at the National Academies of Sciences, Engineering, and Medicine in the US where he has led studies on educational tests and indicators, assessment of science and 21st century skills, applications of information technology, occupational preparation and certification, and measuring productivity.
Kenneth D. Forbus (Chapter 2) is the Walter P. Murphy Professor of Computer Science and Professor of Education at Northwestern University. His research interests include qualitative reasoning, analogical reasoning and learning, spatial reasoning, sketch understanding, natural language understanding, cognitive architecture, reasoning system design, intelligent educational software, and the use of AI in interactive entertainment. He is a Fellow of the Association for the Advancement of Artificial Intelligence, the Cognitive Science Society, the Association for Computing Machinery, and the American Association for the Advancement of Science. He is the inaugural recipient of the Herbert A. Simon Prize, a recipient of the Humboldt Research Award and served as Chair of the Cognitive Science Society.
Matthew Gill (Editor) is the project assistant for the OECD’s AI and the Future of Skills project. Matthew holds a BA (Hons) in Business Management from Manchester Metropolitan University. He is an administrative professional with over five years of skilled experience, within international and intercultural environments. Before joining the OECD, Matthew worked for the British Embassy, Paris – more specifically within the visas and immigration department.
Art Graesser (Chapter 18) is professor emeritus in the Department of Psychology and the Institute of Intelligent Systems at the University of Memphis, and Honorary Research Fellow at University of Oxford. His research is in discourse processing, cognitive science, and education. He has developed software in learning, language, and discourse technologies, including systems that hold a conversation in natural language with computer agents (AutoTutor) and that analyze text on multiple levels of language and discourse (Coh-Metrix). He served as editor of Discourse Processes and Journal of Educational Psychology, as president of Society for Text and Discourse and International Society for Artificial Intelligence in Education, and on four panels with the National Academy of Sciences and four OECD expert panels on problem solving (PIAAC 2011, 2021; PISA 2012, 2015).
Yvette Graham (Chapter 16) is a Natural Language Processing (NLP) researcher and Assistant Professor in AI at Trinity College Dublin, Ireland. Her work includes development of systems for a wide range of AI/NLP tasks, including Machine Translation, Dialogue Systems, Sentiment Analysis, Video Captioning, and Lifelog Information Retrieval. Besides NLP, Dr. Graham is also widely known for her work on NLP evaluation that has revealed misconceptions and bias in system evaluations and has been adopted by high profile competitions including the Conference on Machine Translation (WMT) and TRECvid video captioning task. She has published more than 70 papers in venues such as EMNLP, ACL and JNLE, and was previously awarded best paper at the Annual Conference for the Association of Computational Linguistics in 2015.
Richard Granger (Chapter 13) received his Bachelor's and Ph.D. from MIT and Yale. He is a full professor at Dartmouth, with joint positions in the Psychological and Brain Science Dept and the Thayer School of Engineering; he directs Dartmouth's Brain Engineering Laboratory (brainengineering.org), with publications and patents ranging from computation and robotics to cognitive and basic neuroscience. He advises multiple technology corporations and government research agencies, is co-inventor of FDA-cleared devices and drugs in clinical trials, and has been the principal architect of a series of advanced computational systems for military, commercial, and medical applications.
Samuel Greiff (Chapter 7) is head of research group, principal investigator, and Full Professor of Educational Assessment and Psychology at University of Luxembourg. He holds a PhD in cognitive and experimental psychology from the University of Heidelberg, Germany. Prof Dr. Greiff has been awarded several research funds by diverse funding organisations such as the German Ministry of Education and Research and the European Union (overall funding approx. 9.3 M €), was fellow in the Luxembourg research programme of excellency, and has published articles in national and international scientific journals and books (>100 contributions in peer-reviewed journals; many of them leading in their field). He has an extensive record of conference contributions and invited talks (>200 talks) and serves as editor for several journals, for instance as editor-in-chief for European Journal of Psychological Assessment, as associate editor for Intelligence and Journal of Educational Psychology. He has been and continues to be involved in the Programme for International Student Assessment (PISA) since the 2012 cycle. He serves also as chair of the problem solving expert group for the 2nd cycle of the Programme for the International Assessment of Adult Competencies (PIAAC). In these positions, he has considerably shaped the understanding of transversal skills across several large-scale assessments.
Marta Halina (Chapter 17) is University Associate Professor in the Department of History and Philosophy of Science at the University of Cambridge. Halina co-founded the Kinds of Intelligence program at the Leverhulme Centre for the Future of Intelligence, which draws on current work in psychology, neurobiology, computer science and philosophy to develop and critically assess notions of intelligence. Halina also co-organises the Animal-AI Testbed, which benchmarks current AI against animal species using a range of established animal cognition tasks. In addition to her philosophical writings on animal minds, artificial intelligence and scientific methods, Halina has designed and implemented experiments for testing the social cognitive abilities of nonhuman primates. Her recent publications include “Replications in Comparative Psychology” (Animal Behavior and Cognition) and “Insightful Artificial Intelligence” (Mind & Language).
José Hernández-Orallo (Chapter 11) is Professor at the Universitat Politècnica de València, Spain and Senior Research Fellow at the Leverhulme Centre for the Future of Intelligence, University of Cambridge, UK. His academic and research activities have spanned several areas of artificial intelligence, with a focus on its capabilities, generality, progress, impact and risks. He has published five books and more than two hundred journal articles and conference papers on these topics. His research in the area of machine intelligence evaluation has been covered by several popular outlets, such as The Economist, New Scientist or Nature. He keeps exploring a more integrated view of the evaluation of natural and artificial intelligence, as vindicated in his book "The Measure of All Minds" (Cambridge University Press, 2017, PROSE Award 2018).
Margarita Kalamova (Editor) is an analyst in the OECD’s AI and the Future of Skills project and a project lead in the Higher Education Policy team of the OECD Directorate of Education and Skills. She holds a doctorate degree in Economics from the Freie Universität Berlin and MSc degrees in Economics and Business. At the OECD, she has participated in and led several projects in the domain of skills and employment, including editions of the OECD Skills Outlook, the Employment Outlook, and country reviews on the labour market relevance and outcomes of higher education. She has research experience also in other policy domains, such as innovation, international trade and investment, energy and environment. Prior to joining the OECD, she was a research fellow at the WZB Berlin Social Science Center.
Patrick Kyllonen (Chapter 3) is Distinguished Presidential Appointee in the R&D Division of Educational Testing Service in Princeton, NJ. Dr. Kyllonen received a B.A. from St. John's University, Ph.D. from Stanford University, and authored Generating Items for Cognitive Tests (with S. Irvine, 2001); Learning and Individual Differences (with P. L. Ackerman & R.D. Roberts, 1999); Extending Intelligence: Enhancement and New Constructs (with R. Roberts and L. Stankov, 2008); and Innovative Assessment of Collaboration (with A. von Davier and M. Zhu, 2017). He is a fellow of American Psychological Association and American Educational Research Association and has coauthored several National Academy of Sciences reports, Education for Life and Work: Developing Transferable Knowledge and Skills in the 21st Century (2012), Measuring Human Capabilities (2015), and Supporting Students’ College Success: The Role of Assessment of Intrapersonal and Interpersonal Competencies (2017).
Harold F. O’Neil Jr. (Chapter 19) is a Professor of Educational Psychology and Technology at the University of Southern California’s Rossier School of Education. His research interests include the effectiveness of intelligent tutoring systems, computer games and simulations, the computer-based teaching and assessment of 21st Century Skills, and the teaching and measurement of affective skills. A prolific writer, Dr. O'Neil has recently co-edited several publications: Theoretical Issues of Using Simulations and Games in Educational Assessment (2022) and Using Cognitive and Affective Metrics in Educational Simulations and Games (2021). He is a Fellow of the American Psychological Association (APA), the American Educational Research Association (AERA), and the Association for Psychological Sciences (APS). In these organisations less than 10% of the membership has Fellow status.
Scott Oppler (Chapter 10) is a Principal Scientist at the Human Resources Research Organization in Alexandria, VA, where he has served as an in-house technical expert since 2019. During the first 18 years of his career, Dr. Oppler worked for the American Institutes for Research (AIR), where he led a variety of applied measurement and evaluation projects. Following his tenure at AIR, Dr. Oppler spent eight years at the Association of American Medical Colleges, where he served as Director of Development & Psychometrics for the Medical College Admissions Test, and four years at the Society for Human Resource Management (SHRM), where he served as Vice President, Exam Development & Research for SHRM’s Human Resource Professional certification program. Dr. Oppler received his Ph.D. in industrial-organizational psychology from the University of Minnesota in 1990 and was granted the status of Fellow in the Society for Industrial and Organizational Psychology in 2013.
Nóra Révai (Editor, Chapter 1) is an analyst in the OECD’s AI and the Future of Skills project and is leading the Strengthening the Impact of Education Research project. In recent years, she played a key role in developing the OECD’s Teacher Knowledge Survey. Her research and policy interests include assessing AI capabilities, knowledge dynamics in policy and practice, and networks and leadership. Before joining the OECD, she was managing EU-funded international projects on school leadership at the Hungarian national agency for European cooperation programmes in education. She had also worked as a secondary school teacher. Nóra holds an MSc in Mathematics and a BA in English Teaching from Eötvös Loránd University, Hungary, and a PhD in Sociology from the University of Strasbourg, France.
Britta Rüschoff (Chapter 9) is a work-and organisational psychologist specialised in vocational education and (early) career development. She received her master’s degree from the Radboud University Nijmegen (the Netherlands) and her PhD from the University of Groningen (the Netherlands). She later worked as a work- and organisational psychologist in the industry, as a research associate to the University of Helsinki (Finland), as well as for the German Federal Institute for Vocational Education and Training (BIBB). She currently holds a professorship for Business Psychology at the FOM University of Applied Sciences for Economics and Management in Germany. Her research primarily focuses on vocational decisions and development, competence development, and early career decisions and transitions.
Mila Staneva (Editor, Chapter 1) is an analyst in the OECD’s AI and the Future of Skills project. Her background is in quantitative social research, specifically in the areas of education and labour markets. She completed a PhD at the Max Planck Institute for Human Development on employment alongside higher education. During this time, she worked as a junior researcher at the Education Department at the German Institute for Economic Research (DIW Berlin), where she assisted her team in policy consulting by preparing evidence-based reports on education topics. After her PhD, Mila worked as a consultant at the Education and Science Department at the VDI/VDE-IT in Berlin. In this role she was involved in several projects focused on analysis and policy advice.
Anita Williams Woolley (Chapter 6) is a Professor of Organizational Behavior and Theory at Carnegie Mellon University's Tepper School of Business. She has a PhD in Organizational Behavior from Harvard University. Prof. Woolley’s research includes seminal work on collective intelligence, which was first published in Science in 2010 and has been featured in over 5000 publications and media outlets since. Her papers have been published in Science, Proceedings of the National Academy of Sciences, Academy of Management Review, Organization Science, and Management Science among others and has been funded by grants from the National Science Foundation, the U.S. Army, and DARPA, as well as private corporations. Currently, Professor Woolley is a Senior Editor at Organization Science and a founding Associate Editor of Collective Intelligence.