Continuous Integration Pipeline
Objective:
In this document, I want to present my learning through a demo project, where I applied
my learning by creating a CI pipeline using Jenkins that build, test and push application
image to AWS ECR.
Requirements/Tech stack:
1. AWS EC2, ECR
2. Jenkins
3. Git
4. Maven
5. Snyk
6. Docker
Planned Pipeline structure:
Implementation:
To start with we need EC2 instance to set up and host Jenkins. For that we create an EC2
instance of following configuration:
Instance Name: Jenkins-server
AMI Type: Ubuntu Server 24.04 LTS (HVM),EBS General Purpose (SSD) Volume
Type.
Instance Type: [Link] (2vCPU, 4GiB Memory)
Key Pair: Generate a new key pair
Security group: Create new SG with following rules:
o Inbound rules:
Type: ssh
Protocol: TCP
Port: 22
Source: [Link]/0 or “My IP”
o Outbound Rule:
Type: All traffic
Protocol: all
Port: all
Destination: [Link]/0
EBS volume: 8GiB (gp3)
User Data:
#!/bin/bash
sudo apt update –y
sudo wget -O /etc/apt/keyrings/[Link] \
[Link]
.key
echo "deb [signed-by=/etc/apt/keyrings/jenkins-
[Link]]" \
[Link] binary/ |
sudo tee \
/etc/apt/[Link].d/[Link] > /dev/null
sudo apt-get update
sudo apt install openjdk-17-jdk
sudo apt-get install jenkins -y
sudo systemctl enable jenkins
sudo systemctl start jenkins
sudo apt install maven -y
sudo apt install git -y
Now we can access Jenkins server on the browser using Public ip:
[Link]
After that we will proceed with installation of recommended plugins and setting up
login credentials to access Jenkins dashboard. In addition to recommended plugins, we
will add some other plugins according to our project needs, such as:
1. Pipeline maven integration
2. Snyk security
3. Docker pipeline
4. Cloudbees Docker build and publish
5. AWS Credentials
6. AWS SDK: all
7. ECR
In Jenkins, go to Manage Jenkins > Tools and make the following changes:
For Snyk to work we need to provide snyk api token, which Jenkins shall use to
interact with snyk server. We can get the token from snyk’s account setting. Place
the token inside Manage Jenkins > Credentials > System > Global credentials >
Snyk token
Now we have setup for stages that will help maven to build and test the source
code, and snyk to perform code scanning, we will proceed to docker and ecr setup.
For Docker, First we need to install docker engine inside Jenkins server.
So SSH into Jenkins ec2 instance :
ssh -I <path to key-pair file> ubuntu@<public ip>
Then execute the following:
# Add Docker's official GPG key:
sudo apt-get update
Now we
sudo needinstall
apt-get to addca-certificates
Jenkins user curl
to docker group to enable Jenkins to execute
docker
sudo commands.
install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL-i
sudo [Link] -o
usermod -aG docker jenkins
/etc/apt/keyrings/[Link]
sudo chmod a+r /etc/apt/keyrings/[Link]
# Add the repository to Apt sources:
echo \
"deb [arch=$(dpkg --print-architecture)
signed-by=/etc/apt/keyrings/[Link]]
[Link] \
$(. /etc/os-release && echo "${UBUNTU_CODENAME:-
$VERSION_CODENAME}") stable" | \
Now let’s move to setup ecr. First of all, we need to create an IAM user in AWS, as
“jenkins” to assign policies to access ECR. We can do the same in the IAM section:
Don’t forget to download the access keys. We will upload them to Jenkins
credentials (Manage Jenkins > Credentials > System > Global credentials > Aws
credentials):
Now we proceed creating registry in ECR:
With all that in place, we are ready to push our Jenkins file to git repo and fetch
that from Jenkins:
pipeline{
agent any
tools{
maven 'MAVEN3.8.7'
}
environment{
PROJECT_DIR = 'demo'
registryUrl = '[Link]
[Link]'
imageName = '[Link]-
[Link]/demo-app-ci'
registryCredentials = 'ecr:us-east-1:aws-
jenkins-creds'
}
stages{
stage("Fetch Code from Repo"){
steps{
git branch: 'ci-pipeline', url:
'[Link]
}
}
stage("Unit test"){
steps{
dir("${env.PROJECT_DIR}"){
sh 'mvn clean test'
}
}
}
stage("Build project"){
steps{
dir("${env.PROJECT_DIR}"){
sh 'mvn clean install -DskipTests'
}
}
}
stage("Snyk scan"){
steps{
dir("${env.PROJECT_DIR}"){
// make mvnw file executable for snyk
scan
echo "running snyk test"
sh 'chmod +x ./mvnw'
snykSecurity(
snykInstallation: 'snyk@latest',
snykTokenId: 'snyk-api-token',
failOnIssues: false,
monitorProjectOnBuild: true
)
}
}
}
stage("Build image"){
steps{
dir("${env.PROJECT_DIR}"){
script{
dockerImage = [Link]("$
{[Link]}" + ":$BUILD_NUMBER")
}
}
}
}
stage("Push image to registry"){
steps{
dir("${env.PROJECT_DIR}"){
script{
[Link]("$
{[Link]}", "${[Link]}"){
[Link]("$BUILD_NUMBER")
[Link]("latest")
}
}
}
}
}
stage("clean workspace"){
steps{
sh 'docker rmi -f $(docker images -a
-q)'
}
}
}
}
Next, we hit the “Build now” button and let Jenkins do the rest.
We can find the snyk scan report in the Build history page:
Note:
Since monitorProjectOnBuild: true is set, by which snyk takes a
snapshot of the vulnerabilities and on subsequent builds, snyk will monitor the
vulnerabilities.
We can see the image being uploaded to ECR registry.
