Personalized Federated Learning with Moreau Envelopes

Paper title
Personalized Federated Learning with Moreau Envelopes

Journal Publishing
NeurIPS 2020

Code
https://github.com/CharlieDinh/pFedMe
torch

Problem solved ：
statistical diversity differences among clients,which restricts the global model from delivering good performance on client‘s task.

The method adopted ：
using Moreau envelopes as clients’ regularized loss functions

contributions：
① decouple decoupling the process of optimizing personalized models from learning the global model. （updates the global model similarly to the standard FL algorithm,yet parallelly optimizes the personalized models with low complexity ）
② facilitate the convergence analysis of pFedMe, which characterizes both client-sampling and client-drift errors, and sublinear speedup of order 2/3.
③ outperforms the vanilla algorithms in terms of convergence rate and local accuracy.

experiment ：
dataset: MNIST, Synthetic
methods: FedAvg, Per-FedAvg, pFedMe-GM/PM

Related Work:
FL：one-shot FL（global model to learn in one single round of communcation), statistical diversity,preserving privacy,quantization methods（address the limitations on communications in a FL network),multiple local optimization rounds
Several directions of personalization ：
mixing model,contextualization Contextualization ,meta-learning,multi-task learning

pFedMe thought ：

pFedMe Than Per-FedAvg The advantages of ：

1. Per-FedAvg Only one step of gradient updating is needed to get the personalized model ,pFedMe You can update any number of times .
   First, while Per-FedAvg is optimized for one-step
   gradient update for its personalized model, pFedMe is agnostic to the inner optimizer, which means
   (3) can be solved using any iterative approach with multi-step updates.

2. Per-FedAvg Only optimize $f_i$ The first order approximation of ,pFedMe Minimize directly (3) Medium $f_i$
   

3. Per-FedAvg（ Or other based on MAML Methods ） All need calculation Hession matrix ,pFedMe Just calculate the first derivative of the gradient .

Knowledge point ：
Purpose of local model ： And FedAvg similar , Build a global model by reducing the number of communication rounds between the client and the server .