Skip to content

k8ssandra/k8ssandra-terraform

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

K8ssandra Terraform

This repo contains a Terraform modules for creating Kubernetes clusters on Google Cloud Platform (GCP), Amazon Web Services(AWS), Microsoft Azure, Tanzu.

What's in this repo

  • gcp: Google cloud terraform module to create kubernetes cluster using Google Kubernetes Engine(GKE).
    • env This folder contains modules, version, variables, output files. you will run all the terraform command from this folder.
    • modules This folder contains the main implementation code for this Module, broken down into multiple standalone submodules.
      • gcs: Google Cloud Storage bucket module.
      • vpc: Google Compute Network module.
      • gke: Google Kubernetes Engine module.
      • iam: Identity Access Management modules.
  • aws: Amazon Web Services terraform module to create kubernetes cluster using Elastic Kubernetes Service(EKS).
  • azure: Azure terraform module to create kubernetes cluster using Azure Kubernetes Service(AKS).
  • tanzu: Tanzu terraform module to create kubernetes cluster using Elastic Kubernetes Service.
  • test: Automated tests for the files in this project repository.

What is Kubernetes?

Kubernetes is an open source container management system for deploying, scaling, and managing containerized applications. Kubernetes is built by Google based on their internal proprietary container management systems (Borg and Omega). Kubernetes provides a cloud agnostic platform to deploy your containerized applications with built in support for common operational tasks such as replication, autoscaling, self-healing, and rolling deployments.

What is Managed Kubernetes services?

Managed Kubernetes is when third-party providers take over responsibility for some or all of the work necessary for the successful set-up and operation of K8s. Depending on the vendor, “managed” can refer to anything from dedicated support, to hosting with pre-configured environments, to full hosting and operation. We will be using GKE, AKS, EKS, Tanzu.

What is Terraform?

Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. Terraform can manage existing and popular service providers as well as custom in-house solutions. We will be using terraform version 0.14 to provision out infrastructure.

What is a Terraform module?

A Terraform Module is a canonical, reusable, best-practices definition for how to run a single piece of infrastructure, such as a database or server cluster. Each Module is written using a combination of Terraform and scripts (mostly bash) and include automated tests, documentation, and examples. It is maintained both by the open source community and companies that provide commercial support.

Instead of figuring out the details of how to run a piece of infrastructure from scratch, you can reuse existing code that has been proven in production. And instead of maintaining all that infrastructure code yourself, you can leverage the work of the Module to pick up infrastructure improvements through a version number bump.

Prerequisites

At a minimum 61 GiB of memory, 8 vCPUs virtual machines are needed to run k8ssandra. Minimum recommendation for volumes is 1.5 - 2 TB, but that's all set up through the persistent volume requests.

Resource Naming Conventions

  • Naming Conventions: All the resources will be created with the prefix of environment-project_name.

    • eg: environment="development" and Project_name="k8ssandra" resource_name "development-k8ssandra-gke-cluster"
  • Naming Limitation: Every cloud provider have limitations on the resource names, they will only allow resource names up to some characters long.

    • eg: If we pass environment=production the project_name=K8ssandra-terraform-project-resources-for-multiple-cloud-providers your resource will create as resource_name =production-K8ssandra-terraform-project-resources-for-multiple-cloud-providers-gke-cluster
    • In the above example the resource name exceeds more than 63 characters long. It is an invalid resource name, these will error out when you run Terraform plan or Terraform validate commands. These limitations are hard limitations which can not be changed by your cloud provider. make sure you followed naming standards while creating your resources. It is a good practice maintain limits on length of resource names.

    • refer the following documentation

Project directory Structure

k8ssandra-terraform/
|   aws/
|   gcp/
|    ├──modules/
|    |  ├──gcs
|    |     ├── main.tf 
|    |     └── variables.tf 
|    |     └── outputs.tf 
|    |     └── README.md 
|    |  ├──vpc
|    |     ├── main.tf 
|    |     └── variables.tf 
|    |     └── outputs.tf 
|    |     └── README.md 
|    |  ├──iam
|    |     ├── main.tf 
|    |     └── variables.tf 
|    |     └── outputs.tf 
|    |     └── README.md
|    |  ├──gke
|    |     ├── main.tf 
|    |     └── variables.tf 
|    |     └── outputs.tf 
|    |     └── README.md
|    └──README.md
|    └──gitignore
|    ├──env
|       ├── dev.tf
|         ../modules/vpc
|         ../modules/iam
|         ../modules/gke_cluster
|       ├── version.tf 
|       └── backend.tf 
|       └── variables.tf 
|       └── outputs.tf
|       └── README.md
|    ├──scripts
|  azure/
|  tanzu/
|  test/
|  LICENSE
|  Makefile
|  README.md

How to use Makefile

  • Terraform scripts are in different folders choose your cloud provider to create resources in. list of available providers "aws, gcp, azure "
    • make help To list out the available options to use.
    • make init "provider=<REPLACEME>" Initialize and configure Backend.
    • make plan "provider=<REPLACEME>" Plan all Terraform resources.
    • make apply "provider=<REPLACEME>" Create or update Terraform resources.
    • make destroy "provider=<REPLACEME>" Destroy all Terraform resources.
    • make lint Check syntax of all scripts.
    • make getpods Get running pods IPs and Namespaces run this command after apply

Create GKE resources

  • Testing this project Locally gcp

  • Set up environment on your machine before running the make commands. use the following links to setup your machine.

  • How to create GKE cluster resources by using the make command Before using the make commands export the following terraform environment variables(TFVARS) for terraform to create the resources.

export TF_VAR_environment=<ENVIRONMENT_REPLACEME>
ex:- export TF_VAR_environment=dev

export TF_VAR_name=<CLUSTERNAME_REPLACEME>
ex:- export TF_VAR_name=k8ssandra

export TF_VAR_project_id=<PROJECTID_REPLACEME>
ex:- export TF_VAR_project_id=k8ssandra-testing

export TF_VAR_region=<REGION_REPLACEME>
ex:- export TF_VAR_region=us-central-1
#To list out the available options to use.
make help

important: Before running the following command, we need to Export the environment variables as show above.

# Initialize and configure Backend.
make init "provider=gcp"
# Plan all GCP resources.
make plan "provider=gcp"

This command will create a Kubernetes cluster and deploy k8ssandra on the cluster.

# Create or update GCP resources
# This command takes some time to execute. 
make apply "provider=gcp"
# Destroy all GCP resources

make destroy "provider=gcp"

Create EKS resources

  • Testing this project Locally AWS EKS

  • Set up environment on your machine before running the make commands. use the following links to setup your machine.

  • How to create AWS EKS cluster resources by using the make command Before using the make commands export the following terraform environment variables(TFVARS) for terraform to create the resources.

# Environment name, eg. "dev"
# bash, zsh
export TF_VAR_environment=dev

#fish
set -x TF_VAR_environment dev

# Kubernetes cluster name, eg. "k8ssandra"
# bash, zsh
export TF_VAR_name=k8ssandra

# Resource Owner name, eg. "k8ssandra"
# bash, zsh
export TF_VAR_resource_owner=k8ssandra

#fish
set -x TF_VAR_resource_owner k8ssandra

#fish
set -x TF_VAR_name k8ssandra

# AWS region name, eg. "us-east-1" 
# bash, zsh
export TF_VAR_region=us-east-1

#fish
set -x TF_VAR_region us-east-1
#To list out the available options to use.
make help

important: Before running the following command, we need to Export the environment variables as show above.

# Initialize and configure Backend.
make init "provider=aws"
# Plan all GCP resources.
make plan "provider=aws"

This command will create a Kubernetes cluster and deploy k8ssandra on the cluster.

# Create or update AWS resources
# This command takes some time to execute. 
make apply "provider=aws"
# Destroy all AWS resources created 

make destroy "provider=aws"

Create AKS resources

  • Testing this project Locally azure

  • Set up environment on your machine before running the make commands. use the following links to setup your machine.

  • How to create AKS cluster resources by using make command Before using the make commands export the following terraform environment variables(TFVARS) for terraform to create the resources.

export TF_VAR_environment=<ENVIRONMENT_REPLACEME>
ex:- export TF_VAR_environment=dev

export TF_VAR_name=<CLUSTERNAME_REPLACEME>
ex:- export TF_VAR_name=k8ssandra

export TF_VAR_region=<REGION_REPLACEME>
ex:- export TF_VAR_region=eastus
#To list out the available options to use.
make help

important: Before running the following command, we need to Export the environment variables as show above.

# Initialize and configure Backend.
make init "provider=azure"
# Plan all Azure resources.
make plan "provider=azure"

This command will create a Kubernetes cluster and deploy k8ssandra on the cluster.

# Create or update Azure resources
# This command will take some time to execute. 
make apply "provider=azure"
# Destroy all Azure resources created with terraform.

make destroy "provider=azure"

Troubleshooting

  • The create script fails with a Permission denied when running Terraform - The credentials that Terraform is using do not provide the necessary permissions to create resources in the selected projects. Ensure that the account listed in gcloud config list has necessary permissions to create resources. If it does, regenerate the application default credentials using gcloud auth application-default login.

  • Terraform timeouts - Sometimes resources may take longer than usual to create and Terraform will timeout. The solution is to just run make create again. Terraform should pick up where it left off.

  • Terraform state lock - Sometime if two are more people working on the same Terraform state file a lock will be placed on your remote Terraform state file, to unlock the state run the following command terraform force-unlock <LOCK_ID>.

  • Terraform Incomplete resource deletion - If you created some resources manually on the cloud console and attach those resources to the resources created by the Terraform, terraform destroy or make destroy commands will fail. To resolve those errors you will have to login into the cloud console, delete those resource manually and run make destroy or terraform destory.

Relevant Material