On connecting stochastic gradient MCMC and differential privacy

Bai Li, Changyou Chen, Hao Liu, Lawrence Carin

Research output: Chapter in Book/Report/Conference proceedingConference contribution

15 Scopus citations

Abstract

Concerns related to data security and confidentiality have been raised when applying machine learning to real-world applications. Differential privacy provides a principled and rigorous privacy guarantee for machine learning models. While it is common to inject noise to design a model satisfying a required differential-privacy property, it is generally hard to balance the trade-off between privacy and utility. We show that stochastic gradient Markov chain Monte Carlo (SG-MCMC) - a class of scalable Bayesian posterior sampling algorithms - satisfies strong differential privacy, when carefully chosen stepsizes are employed. We develop theory on the performance of the proposed differentially-private SG-MCMC method. We conduct experiments to support our analysis, and show that a standard SG-MCMC sampler with minor modification can reach state-of-the-art performance in terms of both privacy and utility on Bayesian learning.
Original languageEnglish (US)
Title of host publicationAISTATS 2019 - 22nd International Conference on Artificial Intelligence and Statistics
PublisherPLMR
StatePublished - Jan 1 2020
Externally publishedYes

Fingerprint

Dive into the research topics of 'On connecting stochastic gradient MCMC and differential privacy'. Together they form a unique fingerprint.

Cite this