Module 4

Continuous integration

Continuous integration is a DevOps software development practice where developers regularly

merge their code changes into a central repository, after which automated builds and tests are run.

Continuous integration most often refers to the build or integration stage of the software release
process and entails both an automation component (e.g. a CI or build service) and a cultural
component (e.g. learning to integrate frequently).

The key goals of continuous integration are to find and address bugs quicker, improve software
quality, and reduce the time it takes to validate and release new software updates.

There could be scenarios when developers in a team, work in isolation for an extended period and
only merge their changes to the master branch once their work is completed.

This not only makes the merging of code very difficult, prone to conflicts, and time-consuming but
also results in bugs accumulating for a long time which are only identified in later stages of
development.

These factors make it harder to deliver updates to customers quickly.

With Continuous Integration, developers frequently commit to a shared common repository using a
version control system such as Git.

A continuous integration pipeline can automatically run builds, store the artefacts, run unit tests, and
even conduct code reviews using tools like Sonar.

We can configure the CI pipeline to be triggered every time there is a commit/merge in the
codebase.

Why is Continuous Integration Needed?

In the past, developers on a team might work in isolation for an extended period and only merge
their changes to the master branch once their work is completed.

This made merging code changes difficult and time-consuming and also resulted in bugs
accumulating for a long time without correction.

These factors made it harder to deliver updates to customers quickly.

How does Continuous Integration Work?

With continuous integration, developers frequently commit to a shared repository using a version
control system such as Git.

Before each commit, developers may choose to run local unit tests on their code as an extra
verification layer before integrating.

A continuous integration service automatically builds and runs unit tests on the new code changes to
immediately surface any errors.

Continuous integration refers to the build and unit testing stages of the software release process.
Every committed revision triggers an automated build and test.
CI Workflow

The workflow from developers checking in their code to its automated build, test, and final
notification of the build status.

Once the developer commits their code to a version control system like Git, it triggers the CI pipeline
which fetches the changes and runs automated build and unit tests. Based on the status of the step,
the server then notifies the concerned developer whether the integration of the new code to the
existing code base was a success or a failure.

This helps in finding and addressing the bugs much more quickly, makes the team more productive
by freeing the developers from manual tasks, and helps teams deliver updates to their customers
more frequently. It has been found that integrating the entire development cycle can reduce the
developer’s time by ~25 – 30%.

Continuous Integration Benefits

Improve Developer Productivity

Continuous integration helps your team be more productive by freeing developers from manual tasks
and encouraging behaviours that help reduce the number of errors and bugs released to customers.

Find and Address Bugs Quickly

With more frequent testing, your team can discover and address bugs earlier before they grow into
larger problems later.

Deliver Updates Faster

Continuous integration helps your team deliver customer updates faster and more frequently.
Continuous Integration (CI) Pipeline Cycle

In a Continuous Integration (CI) pipeline, key stages include code integration, build (compiling the
code), test (ensuring code quality through various tests), and code analysis (identifying potential
issues early), all contributing to a faster and more reliable software development process.

1. Code Integration:

In the code integration phase of Continuous Integration (CI), developers frequently merge their code
changes into a shared repository, where automated builds and tests are then executed to ensure the
integrity and quality of the software.

Key aspects of the code integration phase:

 Purpose:

To enable developers to integrate their code changes frequently and reliably into a shared repository,
facilitating collaboration and minimizing integration issues.

 Process:

 Developers write and test their code locally.

 They then commit their code changes to a central repository (using version control
systems like Git).

 A CI server (like Jenkins, GitLab CI or GitHub Actions) automatically detects these

changes and triggers a series of automated tests.

 These tests verify that the new code does not introduce bugs or break existing
functionality.
  If the tests pass, the code is integrated into the main codebase; otherwise,
developers are notified of any errors to fix.

 Tools:

 Version Control Systems (VCS): Git, SVN, etc., are used to manage and track changes
to the codebase.

 CI Server/Pipeline: Tools like Jenkins, GitLab CI or GitHub Actions automate the

build, testing, and deployment processes.

 Benefits:

 Early bug detection: Identifying and fixing issues early in the development process
reduces the risk of costly rework later.

 Faster feedback: Developers receive immediate feedback on the success or failure of

their code changes.

 Improved code quality: Automated testing and frequent integration help ensure the
reliability and quality of the software.

 Simplified collaboration: Frequent integration makes it easier for developers to work

together on a codebase.

 Faster time to market: CI/CD pipelines enable a much shorter time-to-market for
new product features, resulting in happier customers and reducing the burden on
development.

2. Build Stage

In a Continuous Integration (CI) workflow, the "build phase" or "build stage" involves compiling,
testing, and packaging the software code, ensuring a consistent and reliable build for deployment.

The build stage is the first step in the CI pipeline, where the source code is compiled into executable
artifacts. This process involves transforming the code written by developers into a format that can be
run on a machine. During this stage, any syntax errors or compilation issues are identified early,
allowing developers to address them promptly. A successful build is crucial as it serves as the
foundation for subsequent stages in the CI pipeline.

Here's a more detailed breakdown:

 Purpose:

The build stage transforms human-readable source code into executable or deployable artifacts,
preparing it for testing and deployment.

 Key Activities:

 Compiling: Converts code written in a programming language (e.g., Python, Java,

JavaScript) into machine-readable instructions.

 Testing: Runs automated tests to ensure the code functions correctly and meets
quality standards, with build verification tests (BVTs) often occurring immediately
after the build.
  Packaging: Creates a deployable package of the application (e.g., a .jar file, a Docker
image, a .exe file), that can be distributed.

 Dependency Management: Downloads and installs any necessary tools, libraries,

and dependencies to create the build environment.

 Benefits:

 Early Issue Detection: Catches integration problems and potential bugs early in the
development process, reducing the likelihood of issues later in the pipeline.

 Consistency: Ensures that the software is built consistently, making it easier to test
and deploy reliably.

 Examples:

Tools like Jenkins, GitLab CI, and GitHub Actions can automate the build process in CI/CD
pipelines. Semaphore CI/CD platform also has a build stage in their CI/CD pipeline.

3. Test Stage

Following the build stage, the test stage is where the quality of the code is ensured through various
automated tests. These tests can include unit tests, integration tests, and end-to-end tests, among
others. The goal of this stage is to verify that the code behaves as expected and meets the defined
requirements. By running tests automatically after each build, teams can quickly identify and fix
bugs, leading to higher code quality and reduced time spent on manual testing.
In a Continuous Integration (CI) pipeline, the test phase involves running automated tests to verify
the quality and functionality of the code changes before they are integrated into the main codebase,
ensuring a higher quality software release.

 Purpose:

This stage ensures the code is functional and meets quality standards by running various tests.

 Test Types:

Includes unit tests, integration tests, and potentially automated UI tests, depending on the
application's needs.

 Automated Tools:

Tools such as Jenkins, CircleCI, and Semaphore are often used to automate these tests and provide
feedback to developers quickly.
  Unit Tests: Unit tests form the foundation of any testing strategy. They are typically written
using the Test-Driven Development (TDD) approach and run during the build process. These
tests focus on individual code units to ensure they function as expected.

 Integration Tests: Every time a commit is created and deployed to the development
environment; integration tests are conducted. New modules and changes are tested to
ensure they work well together in these tests. Integration tests may even be carried out in a
dedicated environment in some organizations.

 Regression Tests: A regression test is run to ensure that the new changes do not affect the
existing code. As a result, it is easier to ensure that the day’s work is up to standard and that
any necessary changes have been made.

 Performance and Load Tests: A series of tests are run before releasing code to the
production environment to ensure that it is responsive and stable. Testing is conducted on
the UAT/Pre-Production environment after the code has been deployed.

 End-to-end Tests: End-to-end tests examine the entire application. While they can be run
through the GUI, they don’t have to; an API call can also exercise multiple parts of the
system. It is recommended to have fewer of these tests, since they are more fragile and take
longer to run.

4. Code Analysis Stage

The code analysis stage involves examining the code for potential issues, such as code smells,
security vulnerabilities, and adherence to coding standards. This analysis can be performed using
static code analysis tools that scan the codebase without executing it. By identifying potential
problems early in the development process, teams can make informed decisions about code
improvements and maintain a clean codebase. This proactive approach helps prevent technical debt
and enhances the overall maintainability of the software.

In a Continuous Integration (CI) process, the code analysis phase uses static analysis tools to inspect
the code for quality issues, potential problems, and adherence to coding standards, ensuring a
maintainable and reliable codebase.

Here's a more detailed breakdown:

 What is Code Analysis in CI?

 Purpose: This phase aims to identify potential problems in the code before they lead
to errors, bugs, or security vulnerabilities during runtime or deployment.

 Method: Static analysis, also known as code analysis, involves examining the code
without actually running it, which means that it can provide an understanding of the
code structure and ensure that the code adheres to industry standards.

 Tools: Various static analysis tools are used, such as linters, static code analyzers, and
security scanners.

 Benefits of Code Analysis in CI:

  Improved Code Quality: Early identification of issues leads to better code
maintainability and readability.

 Reduced Bugs: Finding potential problems early in the development cycle reduces
the risk of introducing bugs and issues into the application.

 Enhanced Security: Security scanners help detect potential vulnerabilities in the

code, improving the overall security of the application.

 Improved Developer Productivity: Faster feedback on code quality enables

developers to fix issues quickly, increasing productivity.

 Faster Feedback Loops: Integrating code analysis into the CI pipeline provides
developers with immediate feedback on code quality and potential problems.

 Typical Tools:

 Linters (e.g., ESLint, Prettier)

 Static Code Analyzers (e.g., SonarLint, FindBugs)

 Security Scanners (e.g., Semgrep, Checkmarx)

 CodeQL

Code quality checks and static analysis

Integrating code quality checks and static analysis into your Continuous Integration (CI)
pipeline automates identifying and resolving issues early in the development cycle, leading to higher
quality, more secure, and maintainable software.

Here's a breakdown of why and how:

Why Integrate Code Quality Checks and Static Analysis?

 Early Issue Detection:

Static analysis tools examine code without running it, identifying potential bugs, vulnerabilities, and
coding style issues before they cause problems during runtime.

 Improved Code Quality:

By enforcing coding standards and best practices, these checks ensure a consistent and maintainable
codebase.

 Enhanced Security:

Static analysis tools can detect security vulnerabilities, helping to prevent potential exploits and data
breaches.

 Automated Feedback:

Integrating these tools into the CI pipeline provides immediate feedback to developers, allowing
them to address issues quickly.

 Reduced Technical Debt:

By identifying and resolving issues early, you can reduce the amount of technical debt that
accumulates in your codebase.

Why Is Static Code Analysis Essential for CI?

Static code analysis tools find code defects by examining the code without executing the program. As
such, static code analysis has become an essential part of CI because:

1. It is very efficient in identifying hundreds of types of defects such as concurrency, data flow,
dynamic memory, and numerical defects.

2. It finds defects early in the development cycle, that is, as soon as code is written or modified.

3. It verifies compliance with coding standards such as MISRA C®, MISRA C++, JSF++, and
custom naming conventions.
 4. It finds security vulnerabilities and checks for adherence with security standards such as
CERT® C, CERT C++, ISO 17961, and MISRA C:2012 Amendment 1.

5. By using formal methods, it can prove the absence of overflow, divide-by-zero, out-of-bounds
array access, and other run-time errors.

What is Static Code Analysis

In simple words, static analysis is a DevOps practice that enables developers to automatically identify
defects in the embedded codebase of a software, such as security threats, possible bugs, design
errors, code replication, etc.

Whereas a typical code review cycle takes around 1-2 hours, implementing source code analysis in it
is an efficient way to review the code as frequently as possible. With the help of Static Application
Security Testing (SAST) tools, you can pick up on all those issues, even the most discreet
vulnerabilities that could easily get away from reviewers. This results in much faster review cycles
and more reliable codebases.

Moreover, you can analyze the codebase without even executing the program. By building an
abstract syntax tree, you can review the source code, or in some cases, even the object code.

Developers thus frequently integrate SAST tooling in the codebase's build task. Most programming
errors fall into a set of categories which can be defined by certain programming patterns. And static
analysis picks out these patterns to identify defects.

How Does Static Analysis Work?

As long as it is automatic, static analysis is a rather straightforward process. In traditional software

development, static analysis typically comes before testing. However, it takes place in
the integration phases of DevOps software development.

Let us explain:

Developers use static code analysis tools after writing the codebase to determine whether the code
complies with the desired quality and standard. However, sometimes it is crucial to review and
remove any false positives that may have been flagged by the SAST scan.

Once you dismiss the false positives, you can start fixing every obvious bug, usually starting with the
most complex ones, and finally proceed to execute the code for testing.

Here is a simple example of how a static analyzer finds code errors:

Collection<Integer> myNumbers = buildNumbers();

for (Integer myNumber : myNumbers) {
[Link](myNumber);
}

The error in this snippet is a NullPointerException that sneaked past reviewers. The BuildNumbers()
function may return a Nullable object in the first snippet, so while using it in the for loop, it could
cause the software to throw an unchecked exception.

However, with the help of code analysis tools, developers can quickly identify and resolve such issues
through the embedded rules within the code.
Collection<Integer> myNumbers = buildNumbers();
if (myNumbers != null) {
for (Integer myNumber : myNumbers) {
[Link](myNumber);
}
}

As you can see, the second code shows us how to deal with this situation, which reviewers and
programmers often fail to notice. It is therefore essential to have an SASR testing tool integrated into
your Continuous Integration and Continuous Delivery pipeline to identify these vulnerabilities.

Different Types of Static Analysis Methodologies

There are quite a few methods for static code review, such as:

1. Failure/Fault Analysis

In this static code analysis technique, developers check failures (incorrect component behavior) and
faults (invalid components) in the codebase. With the transformation description of the input-
output, you can pinpoint the errors. The model design specification is verified to identify issues in
particular scenarios.

2. Data Analysis

The data analysis method of static analysis helps developers implement correct operations on
different data objects and properly use the defined data. There are two distinct ways to run this
static code review technique: data flow analysis to assess the context and definition of variables and
data dependency to analyze the synchronization accuracy in different processors.

3. Control Analysis

As the name suggests, control analysis is about reviewing control flows in the calling structure and
state transition, such as a function, subroutine, method, or process. You can analyze the sequential
order of control transfers and create a graph of the model consisting of model junctions and
conditional branches represented by nodes.

4. Interface Analysis

The process of user interface analysis involves checking the UI of the program and integrated security
measures so that users don’t face any errors while navigating the software. Static analysis tools of
this kind ensure the interfaces are properly integrated with the overall program to provide the best
user experience.

Key Benefits of Static Analysis

Here comes the good part: SAST security testing offers several advantages, especially when
compared to traditional software testing methods. For example:

1. Improved Productivity
2. Reduced Technical Debt
3. High-Quality Code
4. Strong Data Security
5. Faster Release Cycle
1. Improved Productivity

Developers perform at their peak when given the freedom to solely work on the project at hand. If
they have to go back and work on past projects every time there’s a bug, it sidetracks them from the
current task. As a result, it decreases their productivity and delays the release cycle.

With source code analysis tools, development teams can run automation testing on the embedded
codebase, and concentrate on writing new lines of code.

2. Reduced Technical Debt

For your developers, having a lot of technical debt means more effort. They cannot have the time to
handle new issues since they will be too busy fixing up old projects. This delays your Software
Development life Cycle, and your company will ultimately incur more resources.

Using a source code scanner dramatically cuts down the technical debt your development team has
to deal with by detecting possible threats and vulnerabilities as early as possible,

3. High-Quality Code

One of the greatest benefits of SAST software is that it increases the quality of your code. Developers
can be confident in the code they write with regular testing and health checks on the codebase.

Moreover, customers will see you as the market leader if your products comply with global coding
standards and your releases are consistently of a high quality.

4. Strong Data Security

Cybercriminals are constantly searching for bottlenecks in your system that they may exploit. These
fraudsters may be able to gain access to your IT ecosystem through a backdoor created by an
incorrect software upgrade with vulnerabilities.

Static Application Security Testing tools saves you from such catastrophes by creating robust
codebase structures and optimizing your data security measures in general.

6. Faster Release Cycles

Development teams can automate the most time-consuming operations in a DevOps CI/CD
pipeline and consistently create quality products with secure code review tools.

By automating code reviews, you can give your team members more time to work on other critical
tasks like moving a project down the pipeline, working on the upcoming release, delivering products
at a faster rate, and getting a competitive edge in the market.

Static Analysis vs Dynamic Analysis: What’s the Difference?

Although both static and dynamic analysis complement each other to a large extent, there's some
basic differences between these two.

Static code analysis focuses on the source code to identify problems with the logic and programming.
On the other hand, testing potential code execution paths is a part of dynamic code analysis, which
involves running code and analyzing the results.
When developers test the code, they engage in dynamic analysis, although in the most basic form.
Programmers carry out static analysis as they evaluate the code. Whichever tools they use,
programmers and developers conduct analysis that ultimately leads to improved code.

Teams should optimize both static and dynamic code because using either one alone is not the best
choice. Static and dynamic analysis cannot be alternatives to development teams. Instead, think of
them as interdependent and complementary.

Top Static Code Analysis Tools

Now that we are well aware of the definition and benefits of static code analysis, let’s look at some
of the best SAST tools available in the market:

1. SonarQube
2. Veracode
3. DeepSource
4. Codacy
5. Reshift

1. SonarQube

SonarQube is the most popular static SAST application that you can use to help development teams
during code reviews while also continuously checking the security and quality of your codebases. In
addition, SonarQube code analysis offers code-review tools, a CI server, IDE and Jenkins services, and
much more to help you actively make the necessary corrections.

Top Features

 Multi-language support
 Quality code release
 Identification of tricky issues
 Security analysis
 Easy maintainability

2. Veracode

Veracode is one of the most well-known code analysis tools focusing solely on security
vulnerabilities. The service offered by this tool comprises policy scans, pipeline scans, and IDE scans,
in addition to code checks across the Continuous Integration and Continuous Deployment pipeline
to look for security flaws. As part of the system, it crafts a code evaluation report for audit.

Top Features

 Contextual platform data

 Optimal pipeline outcomes
 Continuous scanning
 Comprehensive DevSecOps
 Cloud-native SaaS architecture

3. DeepSource
Throughout the code review process, DeepSource enables you to automatically detect and correct
errors in your code. You can integrate it with your Bitbucket, GitLab or GitHub account. This static
analyzer searches for performance glitches, potential bug sources, anti-patterns, and accordingly
raise issues. Additionally, DeepSource generates and keeps track of metrics like resource usage,
documentation coverage, dependency count, etc.

Top Features

 Bug tracking and dashboard software

 Support for parallel execution
 Quality check of pull requests
 Single file configuration
 Code formatters for commits

4. Codacy

Here's another great tool for SAST testing that enables development teams to increase the quality of
their code in every commit. and reduce technical debt. In addition to saving time during code
reviews, you may enforce your standards for code quality and security measures.

Top Features

 Quality evolution tracking

 Code standardization
 Configuration file support
 Cluster installation
 Self-hosting capabilities

5. Reshift

Reshift is a SaaS-based software platform that empowers developers to continually deliver reliable
deliverables without hindering the CI/CD solution. It helps software firms meet regulatory and
compliance protocols by lowering the cost and time associated with discovering and fixing
vulnerabilities, assessing potential data breaches, and identifying possible risks.

Top Features

 Penetration testing as a service

 Security scanning
 Developer training
 Integrated management portal
 Thread modeling