Our Federated Split Learning (FSL) algorithm cuts down on communication overheads in traditional Split Learning methods by directly estimating server-returned gradients at each client using auxiliary models. The auxiliary models are much smaller versions of the server model which are explicitly trained to estimate the gradients that the server model would return for the client's local input.
The algorithm is summarized in the following schematic:
The project requirements can be simply installed using the environment config
file conda_env.yaml as follows:
conda env create -f conda_env.yamlThis project is powered by Hydra, which allows
hierarchical configurations and easy running of multiple ML experiments.
The config files for hydra are located in the folder
hydra_config.
There is a high degree of customizability here; datasets, models and FL algorithms can be plugged in using configs.
To run FSL-SAGE with defaults from hydra_config, you can
simply run
python main.pyTo choose a specific model or algorithm, the Hydra command-line override functionality can be used
python main.py model=resnet18 algorithm=cse_fslWe also support multiruns in parallel using the hydra-joblib-launcher Thus, it is possible to run multiple experiments for different combinations of hyperparams, models, datasets or algorithms.
python main.py -m model=resnet18,simple_conv algorithm=fed_avg,sl_single_server,sl_multi_server,cse_fsl,fsl_sageThe above would create parallel jobs that would run main.py on all combinations of specified options