Can Machines Learn Creativity Needs? An Approach Based on Matrix Completion

Abstract

Technological progress has been recently associated with a crowding-out of cognitive-skill intensive jobs in favour of jobs requiring soft skills, such as ones related to social intelligence, flexibility and creativity. The nature of soft skills makes them hardly replaceable by machine work and among subsets of soft skills, creativity is one of the hardest to define and codify. Therefore, creativity-intensive occupations have been shielded from automation. Given this framework, our study contributes to a nascent field on interdisciplinary research to predict the impact of artificial intelligence on work activities and future jobs using machine learning. In our work, we focus on creativity, starting from its possible definitions, then we get significant insights on creativity patterns and dynamics in the Italian labour market, using a machine learning approach. We make use of the INAPP-ISTAT Survey on Occupations (ICP), where we identify 25 skills associated with creativity. Then, we apply matrix completion—a machine learning technique which is often used by recommender systems—to predict the average importance levels of various creative skills for each profession, showing its excellent prediction capability for the specific problem. We also find that matrix completion typically underestimates the average importance levels of soft skills associated with creativity, especially in the case of professions belonging to the major group of legislators, senior officials and managers, as well as intellectual professionals. Conversely, overestimates are typically obtained for other professions, which may be associated with a higher risk of being automated.