FedGS: A Federated Group Synchronization Framework Implemented by LEAF-MX.

Related tags

Deep Learningfederated-learning
Overview

FedGS: Data Heterogeneity-Robust Federated Learning via Group Client Selection in Industrial IoT

Preparation

  • For instructions on generating data, please go to the folder of the corresponding dataset. For FEMNIST, please refer to femnist.

  • NVIDIA-Docker is required.

  • NVIDIA CUDA version 10.1 and higher is required.

How to run FedGS

Build a docker image

Enter the scripts folder and build a docker image named fedgs.

sudo docker build -f build-env.dockerfile -t fedgs .

Modify /home/lizh/fedgs to your actual project path in scripts/run.sh. Then run scripts/run.sh, which will create a container named fedgs.0 if CONTAINER_RANK is set to 0 and starts the task.

chmod a+x run.sh && ./run.sh

The output logs and models will be stored in a logs folder created automatically. For example, outputs of the FEMNIST task with container rank 0 will be stored in logs/femnist/0/.

Hyperparameters

We categorize hyperparameters into default settings and custom settings, and we will introduce them separately.

Default Hyperparameters

These hyperparameters are included in utils/args.py. We list them in the table below (except for custom hyperparameters), but in general, we do not need to pay attention to them.

Variable Name Default Value Optional Values Description
--seed 0 integer Seed for client selection and batch splitting.
--metrics-name "metrics" string Name for metrics file.
--metrics-dir "metrics" string Folder name for metrics files.
--log-dir "logs" string Folder name for log files.
--use-val-set None None Set this option to use the validation set, otherwise the test set is used. (NOT TESTED)

Custom Hyperparameters

These hyperparameters are included in scripts/run.sh. We list them below.

Environment Variable Default Value Description
CONTAINER_RANK 0 This identify the container (e.g., fedgs.0) and log files (e.g., logs/femnist/0/output.0).
BATCH_SIZE 32 Number of training samples in each batch.
LEARNING_RATE 0.01 Learning rate for local optimizers.
NUM_GROUPS 10 Number of groups.
CLIENTS_PER_GROUP 10 Number of clients selected in each group.
SAMPLER gbp-cs Sampler to be used, can be random, brute, bayesian, probability, ga and gbp-cs.
NUM_SYNCS 50 Number of internal synchronizations in each round.
NUM_ROUNDS 500 Total rounds of external synchronizations.
DATASET femnist Dataset to be used, only FEMNIST is supported currently.
MODEL cnn Neural network model to be used.
EVAL_EVERY 1 Interval rounds for model evaluation.
NUM_GPU_AVAILABLE 2 Number of GPUs available.
NUM_GPU_BEGIN 0 Index of the first available GPU.
IMAGE_NAME fedgs Experimental image to be used.

NOTE: If you wish to specify a GPU device (e.g., GPU0), please set NUM_GPU_AVAILABLE=1 and NUM_GPU_BEGIN=0.

NOTE: This script will mount project files /home/lizh/fedgs from the host into the container /root, so please check carefully whether your file path is correct.

Visualization

The visualizer metrics/visualize.py reads metrics logs (e.g., metrics/metrics_stat_0.csv and metrics/metrics_sys_0.csv) and draws curves of accuracy, loss and so on.

Reference

  • This demo is implemented on LEAF-MX, which is a MXNET implementation of the well-known federated learning framework LEAF.

  • Li, Zonghang, Yihong He, Hongfang Yu, et al. "Data Heterogeneity-Robust Federated Learning via Group Client Selection in Industrial IoT." Submitted to IEEE Internet of Things Journal, (2021).

  • If you get trouble using this repository, please kindly contact us. Our email: [email protected]

Owner
Lizonghang
Intelligent Communication System, Distributed Machine Learning, Federated Learning
Lizonghang
GitHub Repository
SSD-based Object Detection in PyTorch

SSD-based Object Detection in PyTorch 서강대학교 현대모비스 SW 프로그램에서 진행한 인공지능 프로젝트입니다. Jetson nano를 이용해 pre-trained network를 fine tuning시켜 차량 및 신호등 인식을 구현하였습니다

Haneul Kim 1 Nov 16, 2021
Reproduce ResNet-v2(Identity Mappings in Deep Residual Networks) with MXNet

Reproduce ResNet-v2 using MXNet Requirements Install MXNet on a machine with CUDA GPU, and it's better also installed with cuDNN v5 Please fix the ran

Wei Wu 531 Dec 04, 2022
Irrigation controller for Home Assistant

Irrigation Unlimited This integration is for irrigation systems large and small. It can offer some complex arrangements without large and messy script

Robert Cook 176 Jan 02, 2023
meProp: Sparsified Back Propagation for Accelerated Deep Learning

meProp The codes were used for the paper meProp: Sparsified Back Propagation for Accelerated Deep Learning with Reduced Overfitting (ICML 2017) [pdf]

LancoPKU 107 Nov 18, 2022
Code accompanying "Evolving spiking neuron cellular automata and networks to emulate in vitro neuronal activity," accepted to IEEE SSCI ICES 2021

Evolving-spiking-neuron-cellular-automata-and-networks-to-emulate-in-vitro-neuronal-activity Code accompanying "Evolving spiking neuron cellular autom

SOCRATES: Self-Organizing Computational substRATES 2 Dec 02, 2022
Instance-level Image Retrieval using Reranking Transformers

Instance-level Image Retrieval using Reranking Transformers Fuwen Tan, Jiangbo Yuan, Vicente Ordonez, ICCV 2021. Abstract Instance-level image retriev

UVA Computer Vision 87 Jan 03, 2023
TuckER: Tensor Factorization for Knowledge Graph Completion

TuckER: Tensor Factorization for Knowledge Graph Completion This codebase contains PyTorch implementation of the paper: TuckER: Tensor Factorization f

Ivana Balazevic 296 Dec 06, 2022
Hierarchical Time Series Forecasting with a familiar API

scikit-hts Hierarchical Time Series with a familiar API. This is the result from not having found any good implementations of HTS on-line, and my work

Carlo Mazzaferro 204 Dec 17, 2022
A deep learning tabular classification architecture inspired by TabTransformer with integrated gated multilayer perceptron.

The GatedTabTransformer. A deep learning tabular classification architecture inspired by TabTransformer with integrated gated multilayer perceptron. C

Radi Cho 60 Dec 15, 2022
From a body shape, infer the anatomic skeleton.

OSSO: Obtaining Skeletal Shape from Outside (CVPR 2022) This repository contains the official implementation of the skeleton inference from: OSSO: Obt

Marilyn Keller 166 Dec 28, 2022
HCQ: Hybrid Contrastive Quantization for Efficient Cross-View Video Retrieval

HCQ: Hybrid Contrastive Quantization for Efficient Cross-View Video Retrieval [toc] 1. Introduction This repository provides the code for our paper at

13 Dec 08, 2022
[NeurIPS 2020] Official Implementation: "SMYRF: Efficient Attention using Asymmetric Clustering".

SMYRF: Efficient attention using asymmetric clustering Get started: Abstract We propose a novel type of balanced clustering algorithm to approximate a

Giannis Daras 46 Dec 22, 2022
MGFN: Multi-Graph Fusion Networks for Urban Region Embedding was accepted by IJCAI-2022.

Multi-Graph Fusion Networks for Urban Region Embedding (IJCAI-22) This is the implementation of Multi-Graph Fusion Networks for Urban Region Embedding

202 Nov 18, 2022
A Semantic Segmentation Network for Urban-Scale Building Footprint Extraction Using RGB Satellite Imagery

A Semantic Segmentation Network for Urban-Scale Building Footprint Extraction Using RGB Satellite Imagery This repository is the official implementati

Aatif Jiwani 42 Dec 08, 2022
Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics

Dataset Cartography Code for the paper Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics at EMNLP 2020. This repository cont

AI2 125 Dec 22, 2022
VOneNet: CNNs with a Primary Visual Cortex Front-End

VOneNet: CNNs with a Primary Visual Cortex Front-End A family of biologically-inspired Convolutional Neural Networks (CNNs). VOneNets have the followi

The DiCarlo Lab at MIT 99 Dec 22, 2022
PyTorch implementation of "LayoutTransformer: Layout Generation and Completion with Self-attention"

PyTorch implementation of "LayoutTransformer: Layout Generation and Completion with Self-attention" to appear in ICCV 2021

Kamal Gupta 75 Dec 23, 2022
Code for Paper: Self-supervised Learning of Motion Capture

Self-supervised Learning of Motion Capture This is code for the paper: Hsiao-Yu Fish Tung, Hsiao-Wei Tung, Ersin Yumer, Katerina Fragkiadaki, Self-sup

Hsiao-Yu Fish Tung 87 Jul 25, 2022
RLDS stands for Reinforcement Learning Datasets

RLDS RLDS stands for Reinforcement Learning Datasets and it is an ecosystem of tools to store, retrieve and manipulate episodic data in the context of

Google Research 135 Jan 01, 2023
Multi-task Learning of Order-Consistent Causal Graphs (NeuRIPs 2021)

Multi-task Learning of Order-Consistent Causal Graphs (NeuRIPs 2021) Authors: Xinshi Chen, Haoran Sun, Caleb Ellington, Eric Xing, Le Song Link to pap

Xinshi Chen 2 Dec 20, 2021