FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learning (NeurIPS 2025)

Zhiqin Yang, Yonggang Zhang, Chenxin Li, Yiu-ming Cheung, Bo Han, Yixuan Yuan

Keywords: Goal-Path Synergy, Statistical Rectification, Data Heterogeneity, Federated Learning

Abstract: Federated Learning (FL) confronts a significant challenge known as data heterogeneity, which impairs model performance and convergence. Existing methods have made notable progress in addressing this issue. However, improving performance in certain heterogeneity scenarios remains an overlooked question: How robust are these methods to deploy under diverse heterogeneity scenarios? To answer this, we conduct comprehensive evaluations across varied heterogeneity scenarios, showing that most existing methods exhibit limited robustness. Meanwhile, insights from these experiments highlight that sharing statistical information can mitigate heterogeneity by enabling clients to update with a global perspective. Motivated by this, we propose FedGPS (Federated Goal-Path Synergy), a novel framework that seamlessly integrates statistical distribution and gradient information from others. Specifically, FedGPS statically modifies each client’s learning objective to implicitly model the global data distribution using surrogate information, while dynamically adjusting local update directions with gradient information from other clients at each round. Extensive experiments show that FedGPS outperforms state-of-the-art methods across diverse heterogeneity scenarios, validating its effectiveness and robustness.

The Table of Contents

• 😬 Dependencies and installation
• 🥳 How to run:
• 🌹 Experimental results
• 😃 Citation
• 📕 License
• 🥰 Acknowledgement
• ☎️ Contact

😉 If FedGPS is helpful to you, please star this repo. Thanks! 🤗

😬 Dependencies and installation

Before running or modifying the code, you need to:

• Make sure Anaconda or Miniconda is installed.

• Clone this repo to your machine.

  # git clone this repository
  git clone https://github.com/visitworld123/FedGPS
  cd FedGPS
  
  # create a new Anaconda env 
  conda create -n FedGPS python=3.8 -y
  conda activate FedGPS  
  

• required packages in requirements.txt

  # install python dependencies
  pip install -r requirements.txt
  

🥳 How to run:

python main.py --algorithms fedgps --datasets CIFAR10 --optim SGD 

In config.yaml, you can change the hyper-parameters and some settings. In our experiments, we use wandb as the default tool to record the running metrics like Acc and Loss. If you don't want to use wandb, you can set record = False. And if you want to use tensorboard to record, you can set record_tool = 'tensorboard.

Please refer to the ./conf/config.yaml file for the list of parameters. We conduct hyper-parameters management by hydra, you can refer to this document to modify different settings and algorithms.

🌹 Experimental results

😃 Citation

If our work is useful for your research, please consider citing:

@inproceedings{
    yang2023fedfed,
    title={FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learning},
    author={Yang, Zhiqin and Zhang, Yonggang and Li, Chenxin and Cheung, Yiu-ming and Han, Bo and Yuan, Yixuan},
    booktitle={Thirty-Ninth Conference on Neural Information Processing Systems},
    year={2025}
}

📕 License

This project is licensed under MIT. Redistribution and use should follow this license.

🥰 Acknowledgement

This project is partly based on VHL and FedLESAM.

This Readme follows the FedFed style.

☎️ Contact

If you have any questions, please feel free to reach me out at [email protected].

Star History