DevOps Project-26: Automating AWS Infrastructure Deployment Using Terraform and GitLab CI/CD

Before starting, ensure you have a basic understanding of:

• Basic Terraform Knowledge

• Understanding of CI/CD

• GitLab CI Knowledge

📝 Prerequisites

1. AWS Account Creation

   • Check out the official site to create an AWS account here.

2. GitLab Account

   • Login to GitLab.

   • Sign in via GitHub/Gmail.

   • Verify email and phone.

   • Fill up the questionnaires.

   • Provide group name & project name as per your choice.

3. Terraform Installed

   • Check out the official website to install Terraform here.

     

4. AWS CLI Installed

   • Navigate to the IAM dashboard on AWS, then select "Users."

   • Enter the username and proceed to the next step.

     

   • Assign permissions by attaching policies directly, opting for "Administrator access," and then create the user.

     

     

   • Locate "Create access key" in user settings, and choose the command line interface (CLI) option to generate an access key.

     

   • View or download the access key and secret access key either from the console or via CSV download.

     

   sudo apt install unzip  
   curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"  
   unzip awscliv2.zip  
   sudo ./aws/install  
   aws configure (input created access key id and secret access key)  
   cat ~/.aws/config  
   cat ~/.aws/credentials  
   aws iam list-users (to list all IAM users in an AWS account)
   

5. Code Editor (VS Code)

   • Download it from here.

📂 Project Structure

The project is divided into two parts:

1. Manual Setup: Write Terraform code, run Terraform commands, and create infrastructure manually.

   

2. Automation: Create a CI/CD pipeline script on GitLab to automate Terraform resource creation.

   

Part 1: Manual Setup

1. Create a new folder named “cicdtf” and open it in VS Code to start writing the code.

   

2. Write Terraform code in the “cicdtf” folder:

   

   • Create a file called provider.tf to define a provider.

   • Deploy a VPC, a security group, a subnet, and an EC2 instance.

Folder Structure

1. VPC Module (vpc folder)

• Files:

  • main.tf: Defines resources like VPC, subnets, and security groups.

  • variables.tf: Declares input variables for customization.

  • outputs.tf: Specifies outputs like VPC ID, subnet IDs, etc.

2. EC2 Module (web folder)

• Files:

  • main.tf: Configures EC2 instance details, including AMI, instance type, and security groups.

  • variables.tf: Defines variables needed for EC2 instance customization.

  • outputs.tf: Outputs instance details like public IP, instance ID, etc.

    

• main.tf for VPC Module

resource "aws_vpc" "main" {
  cidr_block = "10.0.0.0/16"
  enable_dns_support = true
  enable_dns_hostnames = true
}

resource "aws_subnet" "main" {
  vpc_id = aws_vpc.main.id
  cidr_block = "10.0.1.0/24"
  map_public_ip_on_launch = true
  availability_zone = "us-east-1a"
}

resource "aws_security_group" "main" {
  vpc_id = aws_vpc.main.id
  ingress {
    from_port = 22
    to_port = 22
    protocol = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  egress {
    from_port = 0
    to_port = 0
    protocol = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

• Define outputs.tf in the VPC module:

  output "pb_sn" {
    value = aws_subnet.main.id
  }
  
  output "sg" {
    value = aws_security_group.main.id
  }
  

• Define variables.tf in the EC2 module:

  variable "subnet_id" {}
  variable "security_group_id" {}
  

3. Initialize and Validate Terraform:

terraform init
terraform validate
terraform plan
terraform apply -auto-approve

4. Backend Configuration:

• Set up a backend using S3 and DynamoDB.

# backend.tf
terraform {
  backend "s3" {
    bucket = "my-terraform-state"
    key    = "terraform/state"
    region = "us-east-1"
    dynamodb_table = "terraform-lock"
  }
}

5. Push Code to GitLab:

• Initialize the GitLab repository and create a .gitignore file.

• Create a branch named "dev" and push the code.

git remote add origin https://gitlab.com/your-repo.git
git checkout -b dev
git add .
git commit -m "initial commit"
git push -u origin dev

Part 2: CI/CD Pipeline

1. Create a GitLab CI/CD pipeline:

   

• Write a .gitlab-ci.yml file to automate Terraform commands.

  

• Store access keys and secret access keys in GitLab CI/CD variables.

  

  

# .gitlab-ci.yml
image: hashicorp/terraform:latest

variables:
  TF_LOG: DEBUG
  TF_IN_AUTOMATION: true

cache:
  paths:
    - .terraform/

stages:
  - validate
  - plan
  - apply
  - destroy

validate:
  script:
    - terraform init
    - terraform validate

plan:
  script:
    - terraform plan -out=planfile
  artifacts:
    paths:
      - planfile

apply:
  script:
    - terraform apply "planfile"
  when: manual

destroy:
  script:
    - terraform destroy -auto-approve
  when: manual

2. Logs and Execution:

   

• Validate stage: terraform init and terraform validate

• Plan stage: terraform plan

  

  

• Apply stage: terraform apply

  

  

3. Destroy stage: terraform destroy

   

   

   

🌟 Final Notes

• The pipeline performs the following steps:

  • Initializes Terraform with the specified backend configuration.

  • Applies the Terraform plan to create infrastructure resources (VPC, Subnet, Security Group, and EC2 instance).

  • Saves .terraform directory to cache for future use.

  • Cleans up the environment after the job is completed.

🛠️ Author & Community

This project is crafted by Harshhaa 💡.
I’d love to hear your feedback! Feel free to share your thoughts.

📧 Connect with me:

• GitHub: @NotHarshhaa
• Blog: ProDevOpsGuy
• Telegram Community: Join Here

⭐ Support the Project

If you found this helpful, consider starring ⭐ the repository and sharing it with your network! 🚀

📢 Stay Connected