Contextual Skill Proficiency via Multi-task Learning at LinkedIn

Abstract

The ability to infer an individual's expertise for a given skill has proven to be crucial in creating economic opportunity for every talent of the global workforce. Applications ranging from recommending relevant job opportunities to talents to providing better candidate suggestions to recruiters, all benefit from deep understanding of the skill "proficiency"of the talent pool. LinkedIn's "Skill"profile section can be leveraged in this expert finding task. Whereas it is easy to incentivize members to put skills on their profile, estimating members' expertise is much more challenging for several reasons. First, the collection of ground-truth data at scale can be expensive and challenging. Second, "being proficient at a certain skill"can have very different meaning in different contexts - a professor in machine learning having deep theoretical knowledge might lack the practical skill for implementing a large-scale recommendation system unlike experienced ML practitioners. We present our proposed framework to infer a member's expertise in a certain skill based upon a multi-view, multi-task learning scheme that incorporates signals from multiple contexts. We show the efficacy of the proposed framework with offline evaluation results as well as online A/B testing in multiple products, from finding experts among friends, to recommending jobs to qualified members. We also show that our estimated proficiency can help alleviate the cold-start problem when applied to a new context (i.e., through transfer learning) where only a small amount of labeled data is needed to achieve reasonable performance. Finally, we share the insights that demonstrate the talent market is shocked disproportionately among members with different skill proficiency levels by COVID-19.