Medical concept embedding with time-aware attention

Abstract

Embeddings of medical concepts such as medication, procedure and diagnosis codes in Electronic Medical Records (EMRs) are central to healthcare analytics. Previous work on medical concept embedding takes medical concepts and EMRs as words and documents respectively. Nevertheless, such models miss out the temporal nature of EMR data. On the one hand, two consecutive medical concepts do not indicate they are temporally close, but the correlations between them can be revealed by the time gap. On the other hand, the temporal scopes of medical concepts often vary greatly (e.g., common cold and diabetes). In this paper, we propose to incorporate the temporal information to embed medical codes. Based on the Continuous Bagof-Words model, we employ the attention mechanism to learn a "soft" time-aware co for each medical concept. Experim lic and proprietary datasets through nearest neighbour search tasks demo fectiveness of our model, showing t forms five state-of-the-art baselines.