🔒 Secure | ⚙️ Modular | ☁️ AWS-Powered

In this blog, I’ll show you how I deployed a 3-tier application on AWS using a custom VPC. This architecture includes:

• Nginx as a Reverse Proxy (Web Layer)

• Apache Tomcat as the Application Server

• MySQL as the Database Server

🧱 What is 3-Tier Architecture?

A 3-tier architecture separates the app into:

1. Web Tier (Nginx) – Handles incoming HTTP requests

2. App Tier (Tomcat) – Runs backend application logic

3. DB Tier (MySQL) – Stores application data

🔧 Tech Stack & AWS Services Used

🗺️ High-Level Architecture Diagram

This setup includes:

• 1 Public Subnet for Nginx

• 2 Private Subnets: App Tier (Tomcat) and DB Tier (MySQL)

• Security Groups with limited, directional access

• NAT Gateway for outbound internet access from private subnets

🪜 Step-by-Step Implementation

✅ Step 1: Create VPC

• CIDR Block: 10.1.0.0/16

✅ Step 2: Create Subnets

• 10.1.1.0/24 – Public (Web: Nginx)

• 10.1.2.0/24 – Private (App: Tomcat)

• 10.1.3.0/24 – Private (DB: MySQL)

✅ Step 3: Setup Internet Gateway + NAT

• IGW for public subnet (Nginx)

• NAT Gateway for public subnets (Web)

✅ Step 4: Configure Route Tables

• Public Route Table: 0.0.0.0/0 → IGW

• Private Route Table: 0.0.0.0/0 → NAT

✅ A. Public Route Table Configuration

• I created a Route Table named web-rt.

• I associated the public subnet (used for Nginx) with this web-rt.

• Then, I edited the route in web-rt:

  • Destination: 0.0.0.0/0 (this allows internet traffic)

  • Target: Internet Gateway (attached to the VPC)

• This allows public instances like Nginx server to access the internet directly.

🔒 B. Private Route Table Configuration

• I created another Route Table named private-rt.

• I associated both private subnets (one for Tomcat app and one for MySQL DB) with this private-rt.

• Then, I edited the route in private-rt:

  • Destination: 0.0.0.0/0

  • Target: NAT Gateway (deployed in the public subnet)

• This setup allows private instances to access the internet only for updates (e.g., apt install), without being exposed to incoming public traffic.

✅ Step 5: Launch EC2 Instances

🌍 Web Tier (Nginx)

• EC2 in public subnet

• Installed Nginx

• Acts as Reverse Proxy forwarding to Tomcat

• SG allows ports: 80 (HTTP) and 8080 (proxy)

🛡 Bastion Host

• EC2 in public subnet for SSH access to private EC2s

• SG allows port: 22

⚙ App Tier (Tomcat)

• EC2 in private subnet

• Installed Apache Tomcat

• SG allows traffic only from Nginx EC2 (Web SG)

💾 DB Tier (MySQL)

• EC2 in private subnet

• Installed MySQL, secured

• SG allows port 3306 only from App Server

🔐 Step 3: Private Server Access via Public EC2 (Jump Server Method)

Since App and DB servers are in private subnets, I used the Web Server (Public EC2) as a jump host to access them.

Steps Followed:

1. SSH into Public Web Server using .pem file:

     ssh -i "my-key.pem" ubuntu@<public_ip>
   

2. Created a new file using:

     vim jump.pem
   

3. Pasted the private key of the internal servers (App/DB) in jump.pem.

4. Changed permission:

     chmod 400 jump.pem
   

5. Then, from the public server, logged in to private server:

     ssh -i jump.pem ubuntu@10.1.2.97  # App Server
     ssh -i jump.pem ubuntu@10.1.3.105 # DB Server
   

✅ This way, I securely accessed private servers using the public EC2 as a jump point.

🌐 Nginx Installation on Web EC2 Instance (Public Subnet)

After launching the Web EC2 instance (in the public subnet), I connected to it using SSH with the default Ubuntu user. Then I installed and configured Nginx as follows:

🔧 Steps Performed:

1. SSH into EC2 Instance:

     ssh -i "keypair.pem" ubuntu@<Public-IP>
   

2. Update the System Packages:

     sudo apt update -y
   

3. Install Nginx Web Server:

     sudo apt install nginx -y
   

4. Start the Nginx Service:

     sudo systemctl start nginx
   

5. Enable Nginx to Start on Boot:

     sudo systemctl enable nginx
   

🚀 Tomcat Installation on App EC2 Instance (Private Subnet)

On the App Server EC2 instance (launched in the private subnet), I installed and started Apache Tomcat to host Java-based web applications.

Since Tomcat requires Java, I installed JDK first, then downloaded and configured Tomcat.

🔧 Steps Performed:

1. Update System Packages:

     sudo apt update -y
   

2. Install Java Development Kit (Required for Tomcat):

     sudo apt install default-jdk -y
   

3. Download the Latest Tomcat (Version 11.0.9):

     wget https://downloads.apache.org/tomcat/tomcat-11/v11.0.9/bin/apache-tomcat-11.0.9.tar.gz.asc
   

4. Extract the Downloaded Archive:

     tar -xvzf apache-tomcat-11.0.9.tar.gz
   

5. Start the Tomcat Server:

     ls
     cd apache-tomcat-11.0.9.tar.gz
     ls
     cd bin
     ./startup.sh
   

🛢️ MySQL Installation on DB EC2 Instance (Private Subnet)

On the Database Server EC2 instance (placed in the private subnet), I installed MySQL Server to manage the backend database of the application securely.

🔧 Steps Performed:

1. Update System Packages:

     sudo apt update -y
   

2. Install MySQL Server:

     sudo apt install mysql-server -y
   

3. Start and Enable MySQL Service:

     sudo systemctl start mysql
     sudo systemctl enable mysql
   

⚙️ MySQL Configuration on DB EC2 (Private Subnet)

After installing MySQL on the DB Server (private subnet), I performed additional configuration to allow internal app server access by setting the bind-address to the server’s private IP.

🔐 Step 1: Login to MySQL as Root User

To securely access MySQL, I logged in using the root user:

sudo mysql -u root -p

(You’ll be prompted to enter the root password set during secure installation.)

🛠️ Step 2: Edit MySQL Configuration File

I modified the MySQL bind-address to allow access from the app server (within the VPC):

sudo vim /etc/mysql/mysql.conf.d/mysqld.cnf

Inside this file, I located the following line:

bind-address = 127.0.0.1

And changed it to my DB EC2’s private IP, for example:

bind-address = 10.0.2.15

✅ This step ensures MySQL accepts connections only from internal sources (e.g., the app server), not from the public internet — keeping the database secure.

🔄 Step 3: Restart MySQL to Apply Changes

sudo systemctl restart mysql

🔗 Network Connectivity Testing (Ping & Telnet)

To ensure all the instances in my 3-tier architecture (Web, App, and DB) are properly connected and communicating within the VPC, I performed two essential network checks:

✅ 1. Ping Test (Initial Connectivity Verification)

Before running any application-level commands, I verified the basic connectivity between all EC2 instances using ping.

• From Web (10.1.1.44):

  • Ping to App server (10.1.2.97)

  • Ping to DB server (10.1.3.105)

• From App (10.1.2.97):

  • Ping to Web server (10.1.1.44)

  • Ping to DB server (10.1.3.105)

• From DB (10.1.3.105):

  • Ping to Web server (10.1.1.44)

  • Ping to App server (10.1.2.97)

📝 All ping tests were successful, confirming that the subnet routing and security group rules were correctly configured for basic communication.

✅ Verifying Internal Connectivity using Telnet in 3-Tier Architecture

To ensure all components in the 3-Tier Architecture (Web, App, and DB) can communicate with each other, we use the telnet command to test port-level connectivity.

Below is how each instance should verify connection with others using Telnet:

🔹 From Web Server (Public Subnet - IP: 10.1.1.44)

• Check connectivity to App Server (Tomcat):

      telnet 10.1.2.97 8080
  

• Check connectivity to Database Server (MySQL):

      telnet 10.1.3.105 3306
  

🔹 From App Server (Private Subnet - IP: 10.1.2.97)

• Check connectivity to Web Server:

      telnet 10.1.1.44 22
  

• Check connectivity to Database Server:

      telnet 10.1.3.105 3306
  

🔹 From DB Server (Private Subnet - IP: 10.1.3.105)

• Check connectivity to Web Server:

      telnet 10.1.1.44 22
  

• Check connectivity to App Server:

      telnet 10.1.2.97 8080
  

✅ If Telnet successfully connects (i.e., blank screen or "Connected"), it confirms that the port is open and reachable from that instance.

❌ If Telnet fails (i.e., "Connection refused" or "Unable to connect"), check Security Groups, NACLs, and Routing Tables.

📝 Conclusion

This project demonstrates a successful deployment of a secure and scalable 3-tier architecture on AWS using Nginx (Web), Tomcat (App), and MySQL (DB).
It follows best practices for network isolation, access control, and modular application deployment in a cloud environment.

Today’s challenge was about User Management in Linux.
I had to create a temporary user named anita on App Server 1 in the Stratos Datacenter. The special condition was that this user should expire automatically on 2023-12-07.

This ensures security and smooth access management, especially when developers or external people need access for a limited time.

Steps I Followed

1. Logged in to App Server 1

First, I connected to the jump host and from there accessed App Server 1. and password : Ir0nM@n .

#ssh user@appserver1
ssh tony@stapp01.stratos.xfusioncorp.com

2. Created User with Expiry Date

To create a user with expiry, Linux provides the -e flag in the useradd command.

sudo useradd -e 2023-12-07 anita

Here:

• -e 2023-12-07 → Sets the expiry date.

• anita → Username in lowercase as per standard protocol.

3. Verified the User Expiry

I used the chage command to confirm if the expiry was set correctly:

sudo chage -l anita

Output showed:

Account expires : Dec 07, 2023

✅ Perfect!

Learnings from Day 2

• Learned how to create Linux users with expiry dates.

• Understood the importance of temporary access in real-world DevOps & system administration.

• Practiced using useradd and chage commands.

🔮 What’s Next?

Tomorrow Day 3: Secure Root SSH Access

Stay tuned for the 100 Days of DevOps Challenge.

Task Description

At xFusionCorp Industries, the system admin team required me to:

• Create a user named ammar

• Assign the user a non-interactive shell

• Perform this on App Server 3

Steps & Commands

1. Connect to App Server 3 and add password : BigGr33n

    ssh banner@stapp03.stratos.xfusioncorp.com
   

2. Create the user with a non-interactive shell

    sudo useradd -s /sbin/nologin ammar
   

   Here:

   • useradd → command to create a new user

   • -s /sbin/nologin → prevents the user from logging in interactively

3. Verify the user shell

    grep ammar /etc/passwd
   

   Output should look like this:

    ammar:x:1001:1001::/home/ammar:/sbin/nologin
   

Challenges Faced

• Remembering the difference between /sbin/nologin and /bin/false.

• Making sure I was connected to the correct server (App Server 3) before running the command.

Lessons Learned

• User management is one of the most important Linux admin skills for DevOps.

• Using a non-interactive shell ensures security for accounts that exist only for processes, not real users.

• Always double-check the server name before making changes in multi-server environments.

Conclusion

Day 1 taught me how even simple tasks like user creation play a huge role in system security and infrastructure management. This was a strong starting point for my DevOps journey.

👉 Next up: Day 2 – Temporary User Setup with Expiry

1. Prerequisites

• AWS Account with S3 bucket

• AWS CLI installed & configured

• Ubuntu/Linux environment

• Basic shell scripting knowledge

• cronjob (for real-time backup)

2. Project Requirements (as per assignment PDF)

• Backup a local directory to S3

• Incremental backups

• Automate backups (scheduled or real-time)

• Logging

• Restore process

• Database backup

• Optional encryption before upload

3. Launch an Instance:

Setting up AWS CLI

sudo apt update
sudo apt install unzip -y
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install
aws configure

4. Creating S3 Bucket

5. Setting up the Environment

I started by creating a folder called mybackup where all the files to be backed up would be stored:

mkdir mybackup

echo "task1" > vishal.txt
echo "task2" > vishal1.txt
touch vishal2

For testing, I created a couple of files:

6. Creating the Backup Script

I wrote a shell script named backup-to-s3.sh to upload the contents of mybackup to my S3 bucket.

#!/bin/bash

FOLDER_PATH="/home/ubuntu/mybackup"
BUCKET_NAME="my-backup-bucket-vishal"
LOG_FILE="/home/ubuntu/backup-log.txt"
DATE=$(date +"%Y-%m-%d %H:%M:%S")

echo "[$DATE] Backup started..." | tee -a "$LOG_FILE"

aws s3 sync "$FOLDER_PATH" "s3://$BUCKET_NAME/" \
    --exact-timestamps \
    --storage-class STANDARD_IA \
    >> "$LOG_FILE" 2>&1

I made it executable:

chmod +x /home/ubuntu/backup-to-s3.sh

7. 

8. Automating the Backup

Manual execution wasn’t enough. I needed automation.
I used cron to schedule the backup every 2 minutes:

crontab -e

Added the line: (for testing)

*/1 * * * /home/ubuntu/backup-to-s3.sh

Now, the script runs automatically every 1 minutes and backs up any new or modified files.

9. Restore Script

I also wrote a restore script to pull files from S3 back to my local folder:

#!/bin/bash

RESTORE_PATH="/home/ubuntu/mybackup"
BUCKET_NAME="my-backup-bucket-vishal"
LOG_FILE="/home/ubuntu/restore-log.txt"
DATE=$(date +"%Y-%m-%d %H:%M:%S")

echo "[$DATE] Restore started..." | tee -a "$LOG_FILE"

aws s3 sync "s3://$BUCKET_NAME/" "$RESTORE_PATH" \
    --exact-timestamps \
    >> "$LOG_FILE" 2>&1

The files vg.py, vishal.txt, and vishal2 in the mybackup directory are deleted, and then they are restored from the my-backup-bucket-vishal S3 bucket back into the mybackup directory.

10. Results

• Incremental backups: Only changed or new files are uploaded.

• Automation: Cron ensures backups run on schedule.

• Logging: Both backup and restore processes record their activities.

• Restore process: I can get all files back from S3 whenever needed.

CLICK HERE FOR GITHUB REPOSITORY

Steps:-

Step 1 — Launch an Ubuntu(22.04) T2 Large Instance

Step 2 — Install Jenkins, Docker and Trivy. Create a Sonarqube Container using Docker.

Step 3 — Create a TMDB API Key.

Step 4 — Install Prometheus and Grafana On the new Server.

Step 5 — Install the Prometheus Plugin and Integrate it with the Prometheus server.

Step 6 — Email Integration With Jenkins and Plugin setup.

Step 7 — Install Plugins like JDK, Sonarqube Scanner, Nodejs, and OWASP Dependency Check.

Step 8 — Create a Pipeline Project in Jenkins using a Declarative Pipeline

Step 9 — Install OWASP Dependency Check Plugins

Step 10 — Docker Image Build and Push

Step 11 — Deploy the image using Docker

Step 12 — Kubernetes master and slave setup on Ubuntu (20.04)

Step 13 — Access the Netflix app on the Browser.

Step 14 — Terminate the AWS EC2 Instances.

Now, let’s get started and dig deeper into each of these steps:-

STEP 1 : Launch an Ubuntu(22.04) T2 Large Instance

Launch an AWS T2 Large Instance. Use the image as Ubuntu. You can create a new key pair or use an existing one. Enable HTTP and HTTPS settings in the Security Group and open all ports (not best case to open all ports but just for learning purposes it’s okay).

Step 2 — Install Jenkins, Docker and Trivy

2A — To Install Jenkins

Connect to your console, and enter these commands to Install Jenkins

vi jenkins.sh #make sure run in Root (or) add at userdata while ec2 launch

#!/bin/bash
sudo apt update -y
#sudo apt upgrade -y
wget -O - https://packages.adoptium.net/artifactory/api/gpg/key/public | tee /etc/apt/keyrings/adoptium.asc
echo "deb [signed-by=/etc/apt/keyrings/adoptium.asc] https://packages.adoptium.net/artifactory/deb $(awk -F= '/^VERSION_CODENAME/{print$2}' /etc/os-release) main" | tee /etc/apt/sources.list.d/adoptium.list
sudo apt update -y
sudo apt install temurin-17-jdk -y
/usr/bin/java --version
curl -fsSL https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key | sudo tee \
                  /usr/share/keyrings/jenkins-keyring.asc > /dev/null
echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] \
                  https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
                              /etc/apt/sources.list.d/jenkins.list > /dev/null
sudo apt-get update -y
sudo apt-get install jenkins -y
sudo systemctl start jenkins
sudo systemctl status jenkins

sudo chmod 777 jenkins.sh
./jenkins.sh    # this will installl jenkins

Once Jenkins is installed, you will need to go to your AWS EC2 Security Group and open Inbound Port 8080, since Jenkins works on Port 8080.

Now, grab your Public IP Address

<EC2 Public IP Address:8080>
sudo cat /var/lib/jenkins/secrets/initialAdminPassword

Unlock Jenkins using an administrative password and install the suggested plugins.

Jenkins will now get installed and install all the libraries.

Create a user click on save and continue.

Jenkins Getting Started Screen.

2B — Install Docker

sudo apt-get update
sudo apt-get install docker.io -y
sudo usermod -aG docker $USER   #my case is ubuntu
newgrp docker
sudo chmod 777 /var/run/docker.sock

After the docker installation, we create a sonarqube container (Remember to add 9000 ports in the security group).

docker run -d --name sonar -p 9000:9000 sonarqube:lts-community

Now our sonarqube is up and running

Enter username and password, click on login and change password

username admin
password admin

Update New password, This is Sonar Dashboard.

2C — Install Trivy

vi trivy.sh

sudo apt-get install wget apt-transport-https gnupg lsb-release -y
wget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | gpg --dearmor | sudo tee /usr/share/keyrings/trivy.gpg > /dev/null
echo "deb [signed-by=/usr/share/keyrings/trivy.gpg] https://aquasecurity.github.io/trivy-repo/deb $(lsb_release -sc) main" | sudo tee -a /etc/apt/sources.list.d/trivy.list
sudo apt-get update
sudo apt-get install trivy -y

Step 3: Create a TMDB API Key

Next, we will create a TMDB API key

Open a new tab in the Browser and search for TMDB

Click on the first result, you will see this page

Click on the Login on the top right. You will get this page.

You need to create an account here. click on click here. I have account that’s why i added my details there.

once you create an account you will see this page.

Let’s create an API key, By clicking on your profile and clicking settings.

Now click on API from the left side panel.

Now click on create

Click on Developer

Now you have to accept the terms and conditions.

Provide basic details

Click on submit and you will get your API key.

Step 4 — Install Prometheus and Grafana On the new Server

First of all, let’s create a dedicated Linux user sometimes called a system account for Prometheus. Having individual users for each service serves two main purposes:

It is a security measure to reduce the impact in case of an incident with the service.

It simplifies administration as it becomes easier to track down what resources belong to which service.

To create a system user or system account, run the following command:

sudo useradd \
    --system \
    --no-create-home \
    --shell /bin/false prometheus

–system – Will create a system account.
–no-create-home – We don’t need a home directory for Prometheus or any other system accounts in our case.
–shell /bin/false – It prevents logging in as a Prometheus user.
Prometheus – Will create a Prometheus user and a group with the same name.

Let’s check the latest version of Prometheus from the download page.

You can use the curl or wget command to download Prometheus.

wget https://github.com/prometheus/prometheus/releases/download/v2.47.1/prometheus-2.47.1.linux-amd64.tar.gz

Then, we need to extract all Prometheus files from the archive.

tar -xvf prometheus-2.47.1.linux-amd64.tar.gz

Usually, you would have a disk mounted to the data directory. For this tutorial, I will simply create a /data directory. Also, you need a folder for Prometheus configuration files.

sudo mkdir -p /data /etc/prometheus

Now, let’s change the directory to Prometheus and move some files.

cd prometheus-2.47.1.linux-amd64/

First of all, let’s move the Prometheus binary and a promtool to the /usr/local/bin/. promtool is used to check configuration files and Prometheus rules.

sudo mv prometheus promtool /usr/local/bin/

Optionally, we can move console libraries to the Prometheus configuration directory. Console templates allow for the creation of arbitrary consoles using the Go templating language. You don’t need to worry about it if you’re just getting started.

sudo mv consoles/ console_libraries/ /etc/prometheus/

Finally, let’s move the example of the main Prometheus configuration file.

sudo mv prometheus.yml /etc/prometheus/prometheus.yml

To avoid permission issues, you need to set the correct ownership for the /etc/prometheus/ and data directory.

sudo chown -R prometheus:prometheus /etc/prometheus/ /data/

You can delete the archive and a Prometheus folder when you are done.

cd
rm -rf prometheus-2.47.1.linux-amd64.tar.gz

Verify that you can execute the Prometheus binary by running the following command:

prometheus --version

To get more information and configuration options, run Prometheus Help.

prometheus --help

We’re going to use some of these options in the service definition.

We’re going to use Systemd, which is a system and service manager for Linux operating systems. For that, we need to create a Systemd unit configuration file.

sudo vim /etc/systemd/system/prometheus.service

Prometheus.service

[Unit]
Description=Prometheus
Wants=network-online.target
After=network-online.target
StartLimitIntervalSec=500
StartLimitBurst=5
[Service]
User=prometheus
Group=prometheus
Type=simple
Restart=on-failure
RestartSec=5s
ExecStart=/usr/local/bin/prometheus \
  --config.file=/etc/prometheus/prometheus.yml \
  --storage.tsdb.path=/data \
  --web.console.templates=/etc/prometheus/consoles \
  --web.console.libraries=/etc/prometheus/console_libraries \
  --web.listen-address=0.0.0.0:9090 \
  --web.enable-lifecycle
[Install]
WantedBy=multi-user.target

Let’s go over a few of the most important options related to Systemd and Prometheus. Restart – Configures whether the service shall be restarted when the service process exits, is killed, or a timeout is reached.
RestartSec – Configures the time to sleep before restarting a service.
User and Group – Are Linux user and a group to start a Prometheus process.
–config.file=/etc/prometheus/prometheus.yml – Path to the main Prometheus configuration file.
–storage.tsdb.path=/data – Location to store Prometheus data.
–web.listen-address=0.0.0.0:9090 – Configure to listen on all network interfaces. In some situations, you may have a proxy such as nginx to redirect requests to Prometheus. In that case, you would configure Prometheus to listen only on localhost.
–web.enable-lifecycle — Allows to manage Prometheus, for example, to reload configuration without restarting the service.

To automatically start the Prometheus after reboot, run enable.

sudo systemctl enable prometheus

Then just start the Prometheus.

sudo systemctl start prometheus

To check the status of Prometheus run the following command:

sudo systemctl status prometheus

Suppose you encounter any issues with Prometheus or are unable to start it. The easiest way to find the problem is to use the journalctl command and search for errors.

journalctl -u prometheus -f --no-pager

Now we can try to access it via the browser. I’m going to be using the IP address of the Ubuntu server. You need to append port 9090 to the IP.

<public-ip:9090>

If you go to targets, you should see only one – Prometheus target. It scrapes itself every 15 seconds by default.

Install Node Exporter on Ubuntu 22.04

Next, we’re going to set up and configure Node Exporter to collect Linux system metrics like CPU load and disk I/O. Node Exporter will expose these as Prometheus-style metrics. Since the installation process is very similar, I’m not going to cover as deep as Prometheus.

First, let’s create a system user for Node Exporter by running the following command:

sudo useradd \
    --system \
    --no-create-home \
    --shell /bin/false node_exporter

You can download Node Exporter from the same page.

Use the wget command to download the binary.

wget https://github.com/prometheus/node_exporter/releases/download/v1.6.1/node_exporter-1.6.1.linux-amd64.tar.gz

Extract the node exporter from the archive.

tar -xvf node_exporter-1.6.1.linux-amd64.tar.gz

Move binary to the /usr/local/bin.

sudo mv \
  node_exporter-1.6.1.linux-amd64/node_exporter \
  /usr/local/bin/

Clean up, and delete node_exporter archive and a folder.

rm -rf node_exporter*

Verify that you can run the binary.

node_exporter --version

Node Exporter has a lot of plugins that we can enable. If you run Node Exporter help you will get all the options.

node_exporter --help

–collector.logind We’re going to enable the login controller, just for the demo.

Next, create a similar systemd unit file.

sudo vim /etc/systemd/system/node_exporter.service

node_exporter.service

[Unit]
Description=Node Exporter
Wants=network-online.target
After=network-online.target
StartLimitIntervalSec=500
StartLimitBurst=5
[Service]
User=node_exporter
Group=node_exporter
Type=simple
Restart=on-failure
RestartSec=5s
ExecStart=/usr/local/bin/node_exporter \
    --collector.logind
[Install]
WantedBy=multi-user.target

Replace Prometheus user and group to node_exporter, and update the ExecStart command.

To automatically start the Node Exporter after reboot, enable the service.

sudo systemctl enable node_exporter

Then start the Node Exporter.

sudo systemctl start node_exporter

Check the status of Node Exporter with the following command:

sudo systemctl status node_exporter

If you have any issues, check logs with journalctl

journalctl -u node_exporter -f --no-pager

At this point, we have only a single target in our Prometheus. There are many different service discovery mechanisms built into Prometheus. For example, Prometheus can dynamically discover targets in AWS, GCP, and other clouds based on the labels. In the following tutorials, I’ll give you a few examples of deploying Prometheus in a cloud-specific environment. For this tutorial, let’s keep it simple and keep adding static targets. Also, I have a lesson on how to deploy and manage Prometheus in the Kubernetes cluster.

To create a static target, you need to add job_name with static_configs.

sudo vim /etc/prometheus/prometheus.yml

prometheus.yml

- job_name: node_export
    static_configs:
      - targets: ["localhost:9100"]

By default, Node Exporter will be exposed on port 9100.

Since we enabled lifecycle management via API calls, we can reload the Prometheus config without restarting the service and causing downtime.

Before, restarting check if the config is valid.

promtool check config /etc/prometheus/prometheus.yml

Then, you can use a POST request to reload the config.

curl -X POST http://localhost:9090/-/reload

Check the targets section

http://<ip>:9090/targets

Install Grafana on Ubuntu 22.04

To visualize metrics we can use Grafana. There are many different data sources that Grafana supports, one of them is Prometheus.

First, let’s make sure that all the dependencies are installed.

sudo apt-get install -y apt-transport-https software-properties-common

Next, add the GPG key.

wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -

Add this repository for stable releases.

echo "deb https://packages.grafana.com/oss/deb stable main" | sudo tee -a /etc/apt/sources.list.d/grafana.list

After you add the repository, update and install Garafana.

sudo apt-get update

sudo apt-get -y install grafana

To automatically start the Grafana after reboot, enable the service.

sudo systemctl enable grafana-server

Then start the Grafana.

sudo systemctl start grafana-server

To check the status of Grafana, run the following command:

sudo systemctl status grafana-server

Go to http://<ip>:3000 and log in to the Grafana using default credentials. The username is admin, and the password is admin as well.

username admin
password admin

When you log in for the first time, you get the option to change the password.

To visualize metrics, you need to add a data source first.

Click Add data source and select Prometheus.

For the URL, enter localhost:9090 and click Save and test. You can see Data source is working.

<public-ip:9090>

Click on Save and Test.

Let’s add Dashboard for a better view

Click on Import Dashboard paste this code 1860 and click on load

Select the Datasource and click on Import

You will see this output

Step 5 — Install the Prometheus Plugin and Integrate it with the Prometheus server

Let’s Monitor JENKINS SYSTEM

Need Jenkins up and running machine

Goto Manage Jenkins –> Plugins –> Available Plugins

Search for Prometheus and install it

Once that is done you will Prometheus is set to /Prometheus path in system configurations

Nothing to change click on apply and save

To create a static target, you need to add job_name with static_configs. go to Prometheus server

sudo vim /etc/prometheus/prometheus.yml

Paste below code

- job_name: 'jenkins'
    metrics_path: '/prometheus'
    static_configs:
      - targets: ['<jenkins-ip>:8080']

Before, restarting check if the config is valid.

promtool check config /etc/prometheus/prometheus.yml

Then, you can use a POST request to reload the config.

curl -X POST http://localhost:9090/-/reload

Check the targets section

http://<ip>:9090/targets

You will see Jenkins is added to it

Let’s add Dashboard for a better view in Grafana

Click On Dashboard –> + symbol –> Import Dashboard

Use Id 9964 and click on load

Select the data source and click on Import

Now you will see the Detailed overview of Jenkins

Step 6 — Email Integration With Jenkins and Plugin Setup

Install Email Extension Plugin in Jenkins

Go to your Gmail and click on your profile

Then click on Manage Your Google Account –> click on the security tab on the left side panel you will get this page(provide mail password).

2-step verification should be enabled.

Search for the app in the search bar you will get app passwords like the below image

Click on other and provide your name and click on Generate and copy the password

In the new update, you will get a password like this

Once the plugin is installed in Jenkins, click on manage Jenkins –> configure system there under the E-mail Notification section configure the details as shown in the below image

Click on Apply and save.

Click on Manage Jenkins–> credentials and add your mail username and generated password

This is to just verify the mail configuration

Now under the Extended E-mail Notification section configure the details as shown in the below images

Click on Apply and save.

post {
     always {
        emailext attachLog: true,
            subject: "'${currentBuild.result}'",
            body: "Project: ${env.JOB_NAME}<br/>" +
                "Build Number: ${env.BUILD_NUMBER}<br/>" +
                "URL: ${env.BUILD_URL}<br/>",
            to: 'postbox.aj99@gmail.com',  #change Your mail
            attachmentsPattern: 'trivyfs.txt,trivyimage.txt'
        }
    }

Next, we will log in to Jenkins and start to configure our Pipeline in Jenkins

Step 7 — Install Plugins like JDK, Sonarqube Scanner, NodeJs, OWASP Dependency Check

7A — Install Plugin

Goto Manage Jenkins →Plugins → Available Plugins →

Install below plugins

1 → Eclipse Temurin Installer (Install without restart)

2 → SonarQube Scanner (Install without restart)

3 → NodeJs Plugin (Install Without restart)

7B — Configure Java and Nodejs in Global Tool Configuration

Goto Manage Jenkins → Tools → Install JDK(17) and NodeJs(16)→ Click on Apply and Save

7C — Create a Job

create a job as Netflix Name, select pipeline and click on ok.

Step 8 — Configure Sonar Server in Manage Jenkins

Grab the Public IP Address of your EC2 Instance, Sonarqube works on Port 9000, so <Public IP>:9000. Goto your Sonarqube Server. Click on Administration → Security → Users → Click on Tokens and Update Token → Give it a name → and click on Generate Token

click on update Token

Create a token with a name and generate

copy Token

Goto Jenkins Dashboard → Manage Jenkins → Credentials → Add Secret Text. It should look like this

You will this page once you click on create

Now, go to Dashboard → Manage Jenkins → System and Add like the below image.

Click on Apply and Save

The Configure System option is used in Jenkins to configure different server

Global Tool Configuration is used to configure different tools that we install using Plugins

We will install a sonar scanner in the tools.

In the Sonarqube Dashboard add a quality gate also

Administration–> Configuration–>Webhooks

Click on Create

Add details

#in url section of quality gate
<http://jenkins-public-ip:8080>/sonarqube-webhook/

Let’s go to our Pipeline and add the script in our Pipeline Script.

pipeline{
    agent any
    tools{
        jdk 'jdk17'
        nodejs 'node16'
    }
    environment {
        SCANNER_HOME=tool 'sonar-scanner'
    }
    stages {
        stage('clean workspace'){
            steps{
                cleanWs()
            }
        }
        stage('Checkout from Git'){
            steps{
                git branch: 'main', url: 'https://github.com/gurjar-vishal/Netflix-clone.git'
            }
        }
        stage("Sonarqube Analysis "){
            steps{
                withSonarQubeEnv('sonar-server') {
                    sh ''' $SCANNER_HOME/bin/sonar-scanner -Dsonar.projectName=Netflix \
                    -Dsonar.projectKey=Netflix '''
                }
            }
        }
        stage("quality gate"){
           steps {
                script {
                    waitForQualityGate abortPipeline: false, credentialsId: 'Sonar-token'
                }
            }
        }
        stage('Install Dependencies') {
            steps {
                sh "npm install"
            }
        }
    }
    post {
     always {
        emailext attachLog: true,
            subject: "'${currentBuild.result}'",
            body: "Project: ${env.JOB_NAME}<br/>" +
                "Build Number: ${env.BUILD_NUMBER}<br/>" +
                "URL: ${env.BUILD_URL}<br/>",
            to: 'vgahir099@gmail.com',
            attachmentsPattern: 'trivyfs.txt,trivyimage.txt'
        }
    }
}

Click on Build now, you will see the stage view like this

To see the report, you can go to Sonarqube Server and go to Projects.

You can see the report has been generated and the status shows as passed. You can see that there are 3.2k lines it scanned. To see a detailed report, you can go to issues.

Step 9 — Install OWASP Dependency Check Plugins

GotoDashboard → Manage Jenkins → Plugins → OWASP Dependency-Check. Click on it and install it without restart.

First, we configured the Plugin and next, we had to configure the Tool

Goto Dashboard → Manage Jenkins → Tools →

Click on Apply and Save here.

Now go configure → Pipeline and add this stage to your pipeline and build.

stage('OWASP FS SCAN') {
            steps {
                dependencyCheck additionalArguments: '--scan ./ --disableYarnAudit --disableNodeAudit', odcInstallation: 'DP-Check'
                dependencyCheckPublisher pattern: '**/dependency-check-report.xml'
            }
        }
        stage('TRIVY FS SCAN') {
            steps {
                sh "trivy fs . > trivyfs.txt"
            }
        }

The stage view would look like this,

You will see that in status, a graph will also be generated and Vulnerabilities.

Step 10 — Docker Image Build and Push

We need to install the Docker tool in our system, Goto Dashboard → Manage Plugins → Available plugins → Search for Docker and install these plugins

Docker

Docker Commons

Docker Pipeline

Docker API

docker-build-step

and click on install without restart

Now, goto Dashboard → Manage Jenkins → Tools →

Add DockerHub Username and Password under Global Credentials

Add this stage to Pipeline Script

stage("Docker Build & Push"){
            steps{
                script{
                   withDockerRegistry(credentialsId: 'docker', toolName: 'docker'){
                       sh "docker build --build-arg TMDB_V3_API_KEY=6af817c812a4a8481f914c627b9ba292 -t netflix ."
                       sh "docker tag netflix vgahir/netflix:latest "
                       sh "docker push vgahir/netflix:latest "
                    }
                }
            }
        }
        stage("TRIVY"){
            steps{
                sh "trivy image vgahir/netflix:latest > trivyimage.txt"
            }
        }

You will see the output below, with a dependency trend.

When you log in to Dockerhub, you will see a new image is created

Now Run the container to see if the game coming up or not by adding the below stage

stage('Deploy to container'){
            steps{
                sh 'docker run -d --name netflix -p 8081:80 vgahir/netflix:latest'
            }
        }

stage view

<Jenkins-public-ip:8081>

You will get this output

Step 11 — Kuberenetes Setup

Connect your machines to Putty or Mobaxtreme

Take-Two Ubuntu 20.04 instances one for k8s master and the other one for worker.

Install Kubectl on Jenkins machine also.

Kubectl is to be installed on Jenkins also

sudo apt update
sudo apt install curl
curl -LO https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
kubectl version --client

Part 1 ———-Master Node————

sudo hostnamectl set-hostname K8s-Master

———-Worker Node————

sudo hostnamectl set-hostname K8s-Worker

Part 2 ————Both Master & Node ————

sudo apt-get update
sudo apt-get install -y docker.io
sudo usermod –aG docker Ubuntu
newgrp docker
sudo chmod 777 /var/run/docker.sock

Part 3 ————— Master —————

sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl gpg
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.29/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.29/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

———-Worker Node————

sudo kubeadm init --pod-network-cidr=10.244.0.0/16
# in case your in root exit from it and run below commands
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

Copy the config file to Jenkins master or the local file manager and save it

copy it and save it in documents or another folder save it as secret-file.txt

Note: create a secret-file.txt in your file explorer save the config in it and use this at the kubernetes credential section.

Install Kubernetes Plugin, Once it’s installed successfully

goto manage Jenkins –> manage credentials –> Click on Jenkins global –> add credentials

Install Node_exporter on both master and worker

Let’s add Node_exporter on Master and Worker to monitor the metrics

First, let’s create a system user for Node Exporter by running the following command:

sudo useradd \
    --system \
    --no-create-home \
    --shell /bin/false node_exporter

You can download Node Exporter from the same page.

Use the wget command to download the binary.

wget https://github.com/prometheus/node_exporter/releases/download/v1.6.1/node_exporter-1.6.1.linux-amd64.tar.gz

Extract the node exporter from the archive.

tar -xvf node_exporter-1.6.1.linux-amd64.tar.gz

Move binary to the /usr/local/bin.

sudo mv \
  node_exporter-1.6.1.linux-amd64/node_exporter \
  /usr/local/bin/

Clean up, and delete node_exporter archive and a folder.

rm -rf node_exporter*

Verify that you can run the binary.

node_exporter --version

Node Exporter has a lot of plugins that we can enable. If you run Node Exporter help you will get all the options.

node_exporter --help

–collector.logind We’re going to enable the login controller, just for the demo.

Next, create a similar systemd unit file.

sudo vim /etc/systemd/system/node_exporter.service

node_exporter.service

[Unit]
Description=Node Exporter
Wants=network-online.target
After=network-online.target
StartLimitIntervalSec=500
StartLimitBurst=5
[Service]
User=node_exporter
Group=node_exporter
Type=simple
Restart=on-failure
RestartSec=5s
ExecStart=/usr/local/bin/node_exporter \
    --collector.logind
[Install]
WantedBy=multi-user.target

Replace Prometheus user and group to node_exporter, and update the ExecStart command.

To automatically start the Node Exporter after reboot, enable the service.

sudo systemctl enable node_exporter

Then start the Node Exporter.

sudo systemctl start node_exporter

Check the status of Node Exporter with the following command:

sudo systemctl status node_exporter

If you have any issues, check logs with journalctl

journalctl -u node_exporter -f --no-pager

At this point, we have only a single target in our Prometheus. There are many different service discovery mechanisms built into Prometheus. For example, Prometheus can dynamically discover targets in AWS, GCP, and other clouds based on the labels. In the following tutorials, I’ll give you a few examples of deploying Prometheus in a cloud-specific environment. For this tutorial, let’s keep it simple and keep adding static targets. Also, I have a lesson on how to deploy and manage Prometheus in the Kubernetes cluster.

To create a static target, you need to add job_name with static_configs. Go to Prometheus server

sudo vim /etc/prometheus/prometheus.yml

prometheus.yml

- job_name: node_export_masterk8s
    static_configs:
      - targets: ["<master-ip>:9100"]
  - job_name: node_export_workerk8s
    static_configs:
      - targets: ["<worker-ip>:9100"]

By default, Node Exporter will be exposed on port 9100.

Since we enabled lifecycle management via API calls, we can reload the Prometheus config without restarting the service and causing downtime.

Before, restarting check if the config is valid.

promtool check config /etc/prometheus/prometheus.yml

Then, you can use a POST request to reload the config.

curl -X POST http://localhost:9090/-/reload

Check the targets section

http://<ip>:9090/targets

final step to deploy on the Kubernetes cluster

stage('Deploy to kubernets'){
            steps{
                script{
                    dir('Kubernetes') {
                        withKubeConfig(caCertificate: '', clusterName: '', contextName: '', credentialsId: 'k8s', namespace: '', restrictKubeConfigAccess: false, serverUrl: '') {
                                sh 'kubectl apply -f deployment.yml'
                                sh 'kubectl apply -f service.yml'
                        }
                    }
                }
            }
        }

stage view

In the Kubernetes cluster(master) give this command

kubectl get all 
kubectl get svc

STEP 12:Access from a Web browser with

<public-ip-of-slave:service port>

output:

Monitoring

Step 13: Terminate instances.

Complete Pipeline

pipeline{
    agent any
    tools{
        jdk 'jdk17'
        nodejs 'node16'
    }
    environment {
        SCANNER_HOME=tool 'sonar-scanner'
    }
    stages {
        stage('clean workspace'){
            steps{
                cleanWs()
            }
        }
        stage('Checkout from Git'){
            steps{
                git branch: 'main', url: 'https://github.com/gurjar-vishal/Netflix-clone.git'
            }
        }
        stage("Sonarqube Analysis "){
            steps{
                withSonarQubeEnv('sonar-server') {
                    sh ''' $SCANNER_HOME/bin/sonar-scanner -Dsonar.projectName=Netflix \
                    -Dsonar.projectKey=Netflix '''
                }
            }
        }
        stage("quality gate"){
           steps {
                script {
                    waitForQualityGate abortPipeline: false, credentialsId: 'Sonar-token' 
                }
            } 
        }
        stage('Install Dependencies') {
            steps {
                sh "npm install"
            }
        }
        stage('OWASP FS SCAN') {
            steps {
                dependencyCheck additionalArguments: '--scan ./ --disableYarnAudit --disableNodeAudit', odcInstallation: 'DP-Check'
                dependencyCheckPublisher pattern: '**/dependency-check-report.xml'
            }
        }
        stage('TRIVY FS SCAN') {
            steps {
                sh "trivy fs . > trivyfs.txt"
            }
        }
        stage("Docker Build & Push"){
            steps{
                script{
                   withDockerRegistry(credentialsId: 'docker', toolName: 'docker'){   
                       sh "docker build --build-arg TMDB_V3_API_KEY=6af817c812a4a8481f914c627b9ba292 -t netflix ."
                       sh "docker tag netflix vgahir/netflix:latest "
                       sh "docker push vgahir/netflix:latest "
                    }
                }
            }
        }
        stage("TRIVY"){
            steps{
                sh "trivy image vgahir/netflix:latest > trivyimage.txt" 
            }
        }
        stage('Deploy to container'){
            steps{
                sh 'docker run -d --name netflix -p 8081:80 vgahir/netflix:latest'
            }
        }
        stage('Deploy to kubernets'){
            steps{
                script{
                    dir('Kubernetes') {
                        withKubeConfig(caCertificate: '', clusterName: '', contextName: '', credentialsId: 'k8s', namespace: '', restrictKubeConfigAccess: false, serverUrl: '') {
                                sh 'kubectl apply -f deployment.yml'
                                sh 'kubectl apply -f service.yml'
                        }   
                    }
                }
            }
        }

    }
    post {
     always {
        emailext attachLog: true,
            subject: "'${currentBuild.result}'",
            body: "Project: ${env.JOB_NAME}<br/>" +
                "Build Number: ${env.BUILD_NUMBER}<br/>" +
                "URL: ${env.BUILD_URL}<br/>",
            to: 'vgahir099@gmail.com',
            attachmentsPattern: 'trivyfs.txt,trivyimage.txt'
        }
    }
}

Hope you found this helpful. Do connect/ follow for more such content.

~GURJAR VISHAL

Instance Instance OS: Amazon Linux

Task Objective: Generate a new RSA SSH key pair inside the EC2 instance

Step-by-Step Commands:

1. SSH into the EC2 instance

ssh -i your-key.pem ec2-user@<ec2-public-ip>

2. Verify you are inside the EC2

 whoami

Output: ec2-user

3. Generate a new SSH key pair

ssh-keygen -t rsa -b 4096 -C "new-key-inside-ec2"

When prompted:

- Press Enter to accept default file location - Press Enter twice for no pass

Sample Output:

❖ Generating public/private rsa key pair.
❖ Enter file in which to save the key (/home/ec2-user/.ssh/id_rsa): [Press Enter]
❖ Enter passphrase (empty for no passphrase): [Press Enter]
❖ Enter same passphrase again: [Press Enter]
❖ Your identification has been saved in /home/ec2-user/.ssh/id_rsa.
❖ Your public key has been saved in /home/ec2-user/.ssh/id_rsa.pub.

4. List and verify SSH key files

ls -l ~/.ssh/

Expected Output: -rw------- 1 ec2-user ec2-user 3243 Jun 26 09:15 id_rsa

-rw-r--r-- 1 ec2-user ec2-user 743 Jun 26 09:15 id_rsa.pub

5. (Optional) View the Public Key

 cat ~/.ssh/id_rsa.pub

Task Status: Completed Successfully

New SSH key pair generated at:

Private key: /home/ec2-user/.ssh/id_rsa

Public key: /home/ec2-user/.ssh/id_rsa.pub

STEP1: CREATE IAM USER IN AWS

Go to Aws console and login with your credentials

IN Search bar TYPE IAM

This is IAM Dashboard

Click on Users and click on Create User

When creating an IAM user in AWS, generate an access key for programmatic access. This key includes an Access Key ID and a Secret Access Key, which should be stored securely.

STEP2: Create EC2 Instance

1. Sign in to AWS Console: Log in to your AWS Management Console.

2. Navigate to EC2 Dashboard: Go to the EC2 Dashboard by selecting “Services” in the top menu and then choosing “EC2” under the Compute section.

3. Launch Instance: Click on the “Launch Instance” button to start the instance creation process.

4. Choose an Amazon Machine Image (AMI): Select an appropriate AMI for your instance. For example, you can choose Ubuntu image.

5. Choose an Instance Type: In the “Choose Instance Type” step, select t2.medium as your instance type. Proceed by clicking “Next: Configure Instance Details.”

6. Configure Instance Details:

   • For “Number of Instances,” set it to 1 (unless you need multiple instances).

   • Configure additional settings like network, subnets, IAM role, etc., if necessary.

   • For “Storage,” click “Add New Volume” and set the size to 8GB (or modify the existing storage to 16GB).

   • Click “Next: Add Tags” when you’re done.

7. Add Tags (Optional): Add any desired tags to your instance. This step is optional, but it helps in organizing instances.

8. Configure Security Group:

   • Choose an existing security group or create a new one.

   • Ensure the security group has the necessary inbound/outbound rules to allow access as required.

9. Review and Launch: Review the configuration details. Ensure everything is set as desired.

10. Select Key Pair:

    • Select “Choose an existing key pair” and choose the key pair from the dropdown.

    • Acknowledge that you have access to the selected private key file.

    • Click “Launch Instances” to create the instance.

11. Access the EC2 Instance: Once the instance is launched, you can access it using the key pair and the instance’s public IP or DNS.

Ensure you have necessary permissions and follow best practices while configuring security groups and key pairs to maintain security for your EC2 instance.

Step3: Connect to Instance and Install Required Packages

Eksctl

sudo apt update
curl --silent --location "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp
sudo mv /tmp/eksctl /usr/local/bin
eksctl version

Kubectl

curl -O https://s3.us-west-2.amazonaws.com/amazon-eks/1.28.3/2023-11-14/bin/linux/amd64/kubectl
chmod +x ./kubectl
mkdir -p $HOME/bin && cp ./kubectl $HOME/bin/kubectl && export PATH=$HOME/bin:$PATH
kubectl version --client

Aws CLI

sudo apt install unzip -y
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install

Helm

curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
chmod 700 get_helm.sh
./get_helm.sh

STEP4: EKS Setup

Aws configure ( Use us-east-1 region please )

aws configure

Lets clone GitHub repo

git clone https://github.com/gurjar-vishal/3Tier-Robot-shop.git
cd 3Tier-Robot-shop

Create cluster

eksctl create cluster --name demo-cluster-three-tier-1 --region us-east-1

Now Setup

Commands to configure IAM OIDC provider

USE CLUSTER NAME demo-cluster-three-tier-1

export cluster_name=<CLUSTER-NAME>

The command “export cluster_name=” is used in a computer’s command-line interface to create a named storage space (variable) that holds a specific value. It’s like giving a name to something so you can use it later. In this case, it’s creating a storage space called “cluster_name” and putting a value in it, which represents the name of a cluster. This helps remember and use the cluster’s name in other commands or programs without typing it repeatedly.

oidc_id=$(aws eks describe-cluster --name $cluster_name --query "cluster.identity.oidc.issuer" --output text | cut -d '/' -f 5)

This command uses the AWS CLI (Command Line Interface) to extract a specific piece of information about an Amazon EKS (Elastic Kubernetes Service) cluster

Check if there is an IAM OIDC provider configured already

aws iam list-open-id-connect-providers | grep $oidc_id | cut -d "/" -f4

This command utilizes the AWS CLI (Command Line Interface) to list OpenID Connect (OIDC) providers in your AWS Identity and Access Management (IAM) and extract specific information

eksctl utils associate-iam-oidc-provider --cluster $cluster_name --approve

EKSCTL command used to associate the IAM OIDC provider with an Amazon EKS (Elastic Kubernetes Service) cluster.

Setup alb add on

Download IAM policy

curl -O https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.5.4/docs/install/iam_policy.json

Create IAM Policy

aws iam create-policy \
    --policy-name AWSLoadBalancerControllerIAMPolicy \
    --policy-document file://iam_policy.json

create IAM role

Please Add cluster name and Aws account ID

eksctl create iamserviceaccount \
  --cluster=<your-cluster-name> \
  --namespace=kube-system \
  --name=aws-load-balancer-controller \
  --role-name AmazonEKSLoadBalancerControllerRole \
  --attach-policy-arn=arn:aws:iam::<your-aws-account-id>:policy/AWSLoadBalancerControllerIAMPolicy \
  --approve

To get aws account id

Go to aws console and click on Right side profile name and copy it

Deploy ALB controller

Add helm repo

helm repo add eks https://aws.github.io/eks-charts

Update the repo

helm repo update eks

Install

please update VPC_ID in this command

go to eks and copy vpc id

helm install aws-load-balancer-controller eks/aws-load-balancer-controller -n kube-system --set clusterName=demo-cluster-three-tier-1 --set serviceAccount.create=false --set serviceAccount.name=aws-load-balancer-controller --set region=us-east-1 --set vpcId=<vpc-id>

Verify that the deployments are running

kubectl get deployment -n kube-system aws-load-balancer-controller

EBS CSI Plugin configuration

The Amazon EBS CSI plugin requires IAM permissions to make calls to AWS APIs on your behalf.

Create an IAM role and attach a policy. AWS maintains an AWS managed policy or you can create your own custom policy. You can create an IAM role and attach the AWS managed policy with the following command. Replace my-cluster with the name of your cluster. The command deploys an AWS CloudFormation stack that creates an IAM role and attaches the IAM policy to it.

Please add Cluster name

eksctl create iamserviceaccount \
    --name ebs-csi-controller-sa \
    --namespace kube-system \
    --cluster <YOUR-CLUSTER-NAME> \
    --role-name AmazonEKS_EBS_CSI_DriverRole \
    --role-only \
    --attach-policy-arn arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy \
    --approve

Run the following command. Replace with the name of your cluster, with your account ID.

eksctl create addon --name aws-ebs-csi-driver --cluster <YOUR-CLUSTER-NAME> --service-account-role-arn arn:aws:iam::<AWS-ACCOUNT-ID>:role/AmazonEKS_EBS_CSI_DriverRole --force

Now Go inside the helm and create a namespace

cd helm
kubectl create ns robot-shop

Now

helm install robot-shop --namespace robot-shop .

Now check pods

kubectl get pods -n robot-shop

Check service

kubectl get svc -n robot-shop

Now Apply ingress

kubectl apply -f ingress.yaml

Now go to AWS CONSOLE
search for Ec2 and Go to load balancers

COPY DNS

Open a new tab and paste

STEP5: DELETE CLUSTER

JUST PROVIDE THIS COMMAND

eksctl delete cluster --name demo-cluster-three-tier-1 --region us-east-1

In conclusion, our journey through the deployment and configuration of Stan’s Robot Shop, a versatile microservices application, has been an enlightening exploration into the world of containerized applications, orchestration, and monitoring.

Throughout this guide, we’ve covered a range of essential steps, from deploying the application using Docker Compose to associating IAM OIDC providers with Amazon EKS clusters, unlocking the potential for secure access to AWS resources through Kubernetes service accounts.

Stan’s Robot Shop serves not only as a sandbox for experimenting with diverse technologies like NodeJS, Java, Python, and more but also as a practical learning ground for understanding orchestration tools like Kubernetes and monitoring solutions like Instant.

As you continue to delve into the intricacies of microservices architectures, container orchestration, and monitoring practices, remember that Stan’s Robot Shop is an ideal starting point—a playground where you can further explore, test, and refine your skills in a safe and controlled environment.

We hope this guide has provided valuable insights and practical guidance, empowering you to take your knowledge and understanding of containerized applications and Kubernetes to the next level.

Hope you found this helpful. Do connect/ follow for more such content.

~GURJAR VISHAL