Provision 3 lxc nodes k8s cluster on Proxmox using ansible.
- proxmox host
- NVidia GPU configured on proxmox
reference on how to install nvidia-driver on proxmox
- Ansible ready machine example - setting ansible
-
provision 3 lxc containers -
provision_3_lxc_ct.yml -
( Optional ) mount shared disks -
mount_shared_disks.yml
For simple home lab setup : map shared directory lab into all lxc containers
root@pve:/mnt/pve/wd80/private# tree -L 2
.
└── lab --> lxc:/lab
├── dataset
├── k8s-n00
├── k8s-n01
├── k8s-n02
├── k8s-n03
└── shareexample - storage on proxmox
-
fine tuning lxc configuration for k8s - `prepare_k8s_lxc.yml'
-
install docker -
install_docker_inside_lxc.yml -
install nvidia drivers -
install_nvidia_driver_gpu_nodes.yml( Note! : the same version of the driver should be installed on proxmox host )
-
passthrough gpu -
gpu_passthrough_lxc.yml -
install nvidia-docker -
install_nvidia_docker_gpu_nodes.yml -
setup k8s master node -
setup_k8s_master.yml -
setup k8s worker nodes -
setup_k8s_workers.yml -
setup k8s plugins (flannel network, helm, nvidia gpu plugin ) -
setup_k8s_plugins.yml
$ cd playbooks
$ ansible-playbook -e @vars.yml -i inventory.ini ansible_build_k8s.yml$ cd playbooks
$ ansible-playbook -e @vars.yml -i inventory.ini ansible_purge_k8s.ymlproxmox pct command pct manual
docker in lxc docker in lxc
Bootstrap-kubernetes-with-LXC https://github.com/zimmertr/Bootstrap-Kubernetes-with-LXC
Kubernetes-nvidia-ansible-playbook https://github.com/mrdeanplumbley/kubernetes-nvidia-ansible-playbook
NVIDIA-docker https://github.com/NVIDIA/nvidia-docker#quickstart