Machine learning techniques based on neural networks are achieving remarkable
results in a wide variety of domains. Often, the training of models requires
large, representative datasets, which may be crowdsourced and contain sensitive
information. The models should not expose private information in these
datasets. Addressing this goal, we develop new algorithmic techniques for
learning and a refined analysis of privacy costs within the framework of
differential privacy. Our implementation and experiments demonstrate that we
can train deep neural networks with non-convex objectives, under a modest
privacy budget, and at a manageable cost in software complexity, training
efficiency, and model quality.