공유 표현 학습을 통한 프라이버시 보호 예측 분석

Abstract

Recent advances in machine learning and artificial intelligence have evolved from the benefits of the availability of large dataset, increase in hardware's computing power, and many sophisticated algorithms. A single institution may have insufficient amount of data, or biased data, so gathering data from diverse data sources is important to learn reliable predictive models. However, many recent regulations restrict access to and collection of multiple data sources to preserve privacy of data. Accordingly, many studies have used federated learning to learn a global model by sharing intermediate statistics instead of raw data. These studies also have used privacy mechanisms to prevent inference attacks from the statistics, but the mechanisms causes training inefficiency. This thesis aims toefficiently learn representations that can be shared across multiple data sources while preserving personal privacy. The use of representation learning can alleviate privacy concerns and does not needany assumption of data distribution unlike many existing studies. Also, as it's success in many fields, representation learning can also improve predictive performance. When data sources correspond to organizations, each data source can learn representations independently, but they cannot be used together. We propose a method that efficiently harmonizes these representations from different data sources. Since these representations do not contain instance-level private information, the overall process does not require any additional privacy mechanisms. In the case that data sources correspondto separate instances, they need to collaboratively learn representations while preserving their privacy. Federated learning with heavy privacy mechanisms leads to impractical solution in practice. We propose a method that efficiently applies local differential privacy to federated representation learning by concentrating on privacy-sensitive attributes. Proposed methods show predictive accuracy comparable to the ideal performance of global model in two different environments. Both of them results in shared representations, which can be used for predictive tasks on their own or used to build more complex predictive models.