🚧Work in Progress🚧: This project is currently under development and is not yet complete. Features, configurations, and documentation may change frequently as work continues. Feedback, and suggestions are welcome, but please be aware that the repository may not yet be fully functional or stable.
This repository offers an easy-to-use and customizable solution for setting up a Kubernetes clusters in a Proxmox VE home lab environment.
Key Features:
- Automated Provisioning: Streamlines cluster setup by leveraging OpenTofu to automate the deployment and configuration of Talos Linux nodes.
- Advanced Network Security and Observability: Utilizes Cilium to enforce fine-grained network policies and provide robust observability within the Kubernetes cluster.
- Secure Secret Management: Protects sensitive Kubernetes secrets with Sealed Secrets, ensuring secure encryption and storage in Git.
- GitOps-Driven Workflow: Implements ArgoCD for automated continuous delivery, maintaining alignment between application deployments and the desired state defined in Git repositories.
- Infrastructure as Code (IaC): Adheres to IaC best practices, ensuring infrastructure configurations are reproducible, version-controlled, and easy to manage.
- Hardened Kubernetes Environment: Delivers a secure and reliable cluster through the integration of GitOps practices, automated provisioning, Cilium network policies, Kyverno Policies and Sealed Secrets.
Here's an improved version of your Getting Started section and the Folder Structure Overview. The rewrite aims to make it more actionable, readable, and concrete. I’ve also updated or removed vague sections and added clarity to the GitOps deployment part.
This project provides a streamlined and reproducible way to deploy Talos Kubernetes clusters on Proxmox VE using OpenTofu and GitOps with ArgoCD.
Make sure the following tools are installed:
- Proxmox VE server(s)
- OpenTofu CLI [
brew install opentofu] - kubectl CLI [
brew install kubernetes-cli] - k9s [
brew install k9s] (optional but recommended) - ArgoCD CLI [
brew install argocd] (optional, for GitOps workflows)
-
Clone the Repository
git clone https://github.com/erwinkersten/homelab.git cd homelab/infrastructure/kubernetes -
Prepare Environment Configuration
Copy the example configuration and edit to match your setup:
cp environment.prod.tfvars.example environment.prod.tfvars # Or for development: cp environment.prod.tfvars.example environment.dev.tfvarsOpen the file in your editor and customize values like Proxmox IPs, storage settings, and VM specs.
-
Deploy the Kubernetes Cluster
Run the deploy script with your environment name:
./tofu-deploy.sh prod # or for dev: ./tofu-deploy.sh devThis will provision Talos-based K8s nodes on your Proxmox server.
-
Bootstrap Kubernetes with Core services
⚠️ Temporary manual step: After the cluster is created, manually bootstrap it with the core Kubernetes configuration and essential operators to prepare the environment.cd kubernetes/core ./deploy-core.sh cd ../..
-
Bootstrap GitOps with ArgoCD
⚠️ Temporary manual step: After bootstrapping the cluster, you can bootstrap the GitOps setup using ArgoCD’s App-of-Apps pattern.kubectl apply -k kubernetes/argo-apps/
This will install ArgoCD and trigger the deployment of your infrastructure and app resources defined in Git.
A quick guide to what's where in the Kubernetes deployment setup:
kubernetes/
├── core/ # Talos bootstrapping components (CRs, secrets, etc.)
├── argo-apps/ # ArgoCD App-of-Apps configuration
├── infra/ # Cluster-wide infrastructure (e.g. ingress, certs)
└── apps/ # Application workloads and services
Contains the base components needed to initialize the Talos cluster.
Manual deployment (if needed):
./deploy-core.shDefines the ArgoCD App-of-Apps hierarchy. This is the entry point for GitOps bootstrapping.
Deploy with:
kubectl apply -k kubernetes/argo-apps/This directory contains all the infrastructure resources that are deployed using ArgoCD. These resources include networking, storage, and other infrastructure components necessary for the applications to run.
This directory contains all the application resources that are deployed using ArgoCD. Each application has its own directory with the necessary manifests and configurations to deploy and manage the application in the Kubernetes cluster.
- Proxmox VE: https://www.proxmox.com/en/
- Talos Linux https://www.talos.dev/
- OpenTofu: https://opentofu.org/
- Kubernetes: https://kubernetes.io/
- ArgoCD: https://argo-cd.readthedocs.io/
- Cilium: https://cilium.io/
- CertManager https://cert-manager.io/
- Sealed Secrets https://github.com/bitnami-labs/sealed-secrets