Nature-Inspired Algorithm-driven Convolutional Autoencoder Architecture search: Empowered by High-Performance Computing (HPC)
The proposed method NiaNetCAE attempts to pick hyperparameters and Convolutional autoencoder architecture that will result in a successful encoding and decoding (minimal difference between input and output). NiaNetCAE uses the collection of algorithms available in the library NiaPy to navigate efficiently in waste search-space.
- Construct novel CONV AE's architecture using nature-inspired algorithms.
- Selects the best hyperparameters for the given dataset.
- It can be utilized for any kind of dataset, which has 3D images values.
- Applied for depth estimation
Our example of the search space for the depth estimation problem:
- Search space is defined by:
- Architecture problem:
- x: layer step (difference between input and output dimension)
- y: Number of layers (architecture depth)
- Hyperparameter problem:
- z: activation function
- w: optimizer algorithm
- Architecture problem:
- Total solutions:
- x: 304
- y: 304
- z: 8
- w: 6
- x * y * z * w = 4.435.968 unique solutions
- NiaNetCAE can find the best solution by leveraging the power of nature-inspired algorithms.
Installing NiaNetCAE with pip3:
TODO: Publish it to PyPi
pip3 install nianetcaeThe purpose of this paper is to get an understanding of the NiaNetCAE approach.
TODO - Future Journal: NiaNetCAE for depth estimation
Usage examples can be found here. Currently, there is an example for finding the appropriate Convolutional Autoencoder architectures for depth estimation on NYU Depth Dataset V2.
- Replace the dataset in data folder.
- Modify the parameters in main_config.py
- Adjust the dataloader logic in dataloaders folder.
- Specify the search space in conv_ae.py from your problem domain.
- Redesign the fitness function in cae_architecture_search.py based on your optimization.
Once the dataset is changed, dataloaders needs to be modified to be able for forwarding new shape of data to models.
Set the boundaries of your search space with conv_ae.py.
The following dimensions can be modified:
- x: layer step
- y: number of layers,
- z: activation function
- w: optimizer algorithm.
You can run the NiaNetCAE script once your setup is complete.
docker build --tag spartan300/nianet:cae .
docker run \
--name=nianet-cae \
-it \
-v $(pwd)/logs:/app/nianetcae/logs \
-v $(pwd)/data:/app/data \
-v $(pwd)/configs:/app/configs \
-w="/app" \
--shm-size 8G \
--gpus all spartan300/nianet:cae \
python main.py
Running NiaNetCAE script with Poetry help:
- Run the installation via
poetry install - Then run the task with
poetry run poe autoinstall-torch-cuda
- First build an image with docker (above example)
- Docker push to Docker Hub:
docker push username/nianet:cae - SSH into a HPC Cluster via your access credentials
- Create the following nianetcae.sh script:
cat > nianetcae.sh
#!/bin/bash
## Running code on SLURM cluster
##https://pytorch-lightning.readthedocs.io/en/stable/clouds/cluster_advanced.html
#SBATCH -J nianet-pso
#SBATCH -o nianet-pso-%j.out
#SBATCH -e nianet-pso-%j.err
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --partition=gpu
#SBATCH --mem-per-gpu=8GB # memory per GPU
#SBATCH --gres=gpu:1
#SBATCH --time=72:00:00
singularity exec -e \
--pwd /app \
-B $(pwd)/logs:/app/logs,$(pwd)/data:/app/data,$(pwd)/configs:/app/configs \
--nv docker://spartan300/nianet:cae \
python main.py -alg particle_swarm
- Make script executable:
chmod +x nianetcae.sh - Make sure that you have the following folders in your current directory: logs, data, configs
- Set folder permissions to 777:
chmod -R 777 logs data configs - Submit your script to a job scheduler:
SBATCH nianetcae.sh
-
NiaNetCAE was developed under the supervision of prof. dr. Domenec Puig at University Rovira i Virgili.
-
Together with dr. Saddam Abdulwahab at University Rovira i Virgili
-
This code is a fork of NiaPy. I am grateful that the authors chose to open-source their work for future use.
This package is distributed under the MIT License. This license can be found online at http://www.opensource.org/licenses/MIT.
This framework is provided as-is, and there are no guarantees that it fits your purposes or that it is bug-free. Use it at your own risk!