TY - GEN
T1 - FL-PyTorch
T2 - 2nd ACM International Workshop on Distributed Machine Learning, DistributedML 2021, co-located with the 17th International Conference on emerging Networking EXperiments and Technologies, CoNEXT 2021
AU - Burlachenko, Konstantin
AU - Horváth, Samuel
AU - Richtárik, Peter
N1 - Publisher Copyright:
© 2021 Owner/Author.
PY - 2021/12/7
Y1 - 2021/12/7
N2 - Federated Learning (FL) has emerged as a promising technique for edge devices to collaboratively learn a shared machine learning model while keeping training data locally on the device, thereby removing the need to store and access the full data in the cloud. However, FL is difficult to implement, test and deploy in practice considering heterogeneity in common edge device settings, making it fundamentally hard for researchers to efficiently prototype and test their optimization algorithms. In this work, our aim is to alleviate this problem by introducing FL-PyTorch : a suite of open-source software written in python that builds on top of one the most popular research Deep Learning (DL) framework PyTorch. We built FL-PyTorch as a research simulator for FL to enable fast development, prototyping and experimenting with new and existing FL optimization algorithms. Our system supports abstractions that provide researchers with a sufficient level of flexibility to experiment with existing and novel approaches to advance the state-of-the-art. Furthermore, FL-PyTorch is a simple to use console system, allows to run several clients simultaneously using local CPUs or GPU(s), and even remote compute devices without the need for any distributed implementation provided by the user. FL-PyTorch also offers a Graphical User Interface. For new methods, researchers only provide the centralized implementation of their algorithm. To showcase the possibilities and usefulness of our system, we experiment with several well-known state-of-the-art FL algorithms and a few of the most common FL datasets.
AB - Federated Learning (FL) has emerged as a promising technique for edge devices to collaboratively learn a shared machine learning model while keeping training data locally on the device, thereby removing the need to store and access the full data in the cloud. However, FL is difficult to implement, test and deploy in practice considering heterogeneity in common edge device settings, making it fundamentally hard for researchers to efficiently prototype and test their optimization algorithms. In this work, our aim is to alleviate this problem by introducing FL-PyTorch : a suite of open-source software written in python that builds on top of one the most popular research Deep Learning (DL) framework PyTorch. We built FL-PyTorch as a research simulator for FL to enable fast development, prototyping and experimenting with new and existing FL optimization algorithms. Our system supports abstractions that provide researchers with a sufficient level of flexibility to experiment with existing and novel approaches to advance the state-of-the-art. Furthermore, FL-PyTorch is a simple to use console system, allows to run several clients simultaneously using local CPUs or GPU(s), and even remote compute devices without the need for any distributed implementation provided by the user. FL-PyTorch also offers a Graphical User Interface. For new methods, researchers only provide the centralized implementation of their algorithm. To showcase the possibilities and usefulness of our system, we experiment with several well-known state-of-the-art FL algorithms and a few of the most common FL datasets.
KW - federated learning
KW - optimization
KW - simulator
UR - http://www.scopus.com/inward/record.url?scp=85121605779&partnerID=8YFLogxK
U2 - 10.1145/3488659.3493775
DO - 10.1145/3488659.3493775
M3 - Conference contribution
AN - SCOPUS:85121605779
T3 - DistributedML 2021 - Proceedings of the 2nd ACM International Workshop on Distributed Machine Learning, Part of CoNEXT 2021
SP - 1
EP - 7
BT - DistributedML 2021 - Proceedings of the 2nd ACM International Workshop on Distributed Machine Learning, Part of CoNEXT 2021
PB - Association for Computing Machinery, Inc
Y2 - 7 December 2021
ER -