Differentially private deep learning

Abstract

In recent years, deep learning has rapidly become one of the most successful approaches to machine learning. The essential idea of deep learning is to apply a multiple-layer structure to extract complex features from high-dimensional data and use those features to build models. However, deep learning models are susceptible to several types of attacks. For example, a centralized collection of photos, speech, and video clips from millions of individuals might meet with privacy risks when they are shared with others. Learning models can also disclose sensitive information. To integrate differential privacy to deep learning, we need to consider two challenges: high sensitivity and limited privacy budget. This chapter first presents the traditional Laplace method and illustrates the limitations of the method, and then present Private SGD Method, Deep Private Auto-Encoder Algorithm and Distributed Private SGD. Each of them is focusing on a particular deep learning algorithm and is dealing with those two challenges in different ways. Finally, this chapter shows several popular datasets that can be used in differentially private deep learning.