Subject

Federated learning [1] is a distributed machine learning framework that makes many devices participate in a training of the shared model without transferring their local data. Dataset distillation [2] is used to synthesize a small number of data, which is approximately equal to the original data used for training a model. For example, [2] shows that it is possible to compress 60, 000 MNIST training images into just 10 synthetic distilled images (one per class) and achieve close to the original performance. Recently, [3] presents an Inverting Gradient attack (IGA) algorithm and shows that it is possible to invert/leak training data from the model gradients namely, they reconstruct the input by optimizing dummy data to generate the gradient close to the true gradient.

Kind of work

Within this work, the student will focus on (1) distilling the dataset by IGA. IGA can generate dummy data that is closed to the true training data. However, if IGA reconstructs only one dummy data instead of the number of batch size, we compress a batch size of training data into only one data. (2) training a model by the distilled data via non-gradient or -parameters transition. Each client distillates their own dataset locally and upload it to the server side to implement a central training. (3) comparison of the performance with other federated learning optimization methods.

Framework of the Thesis

[1] H. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-Efficient Learning of Deep Networks from Decentralized Data,” Feb. 2016, doi: 10.48550/arxiv.1602.05629.
[2] T. Wang, J.-Y. Zhu, A. Torralba, and A. A. Efros, “Dataset Distillation,” Nov. 2018, doi: 10.48550/arxiv.1811.10959.
[3] J. Geiping, H. Bauermeister, H. Dröge, and M. Moeller, “Inverting gradients - How easy is it to break privacy in federated learning?,” in Advances in Neural Information Processing Systems, 2020, vol. 2020-December.

Number of Students

1 - 2

Expected Student Profile

Proven programming experience (e.g., Python) and background in machine learning.