dotexe3301

    e2e devsecops implementation with infra

    12 may, 2026

    Project Motivation

    I started creating this project in Mid of April 2026 to integrate all the tools & concept in one instead of creating multiple ones to make a proper system by documenting each and every possible thing.

    Another motivation was to make something to include in resume to showcase my skills and knowledge. Although this project doesn't include IaC at all, for which I planned to make differnet project.

    Project Infra/ Flow

    • This is the current infra on which this project is deployed:

      Project Infra

    • This is the CI/CD flow to deploy a service on infra:

      Project Flow

    Application Architecture

    Online Boutique is a cloud-first microservices demo application. The application is a web-based e-commerce app where users can browse items, add them to the cart, and purchase them. This application works on any Kubernetes cluster.

    It is composed of 11 microservices written in different languages that talk to each other over gRPC.

    Architecture of microservices

    Find Protocol Buffers Descriptions at the ./protos directory.

    Service Language Description
    frontend Go Exposes an HTTP server to serve the website. Does not require signup/login and generates session IDs for all users automatically.
    cartservice C# Stores the items in the user's shopping cart in Redis and retrieves it.
    productcatalogservice Go Provides the list of products from a JSON file and ability to search products and get individual products.
    currencyservice Node.js Converts one money amount to another currency. Uses real values fetched from European Central Bank. It's the highest QPS service.
    paymentservice Node.js Charges the given credit card info (mock) with the given amount and returns a transaction ID.
    shippingservice Go Gives shipping cost estimates based on the shopping cart. Ships items to the given address (mock)
    emailservice Python Sends users an order confirmation email (mock).
    checkoutservice Go Retrieves user cart, prepares order and orchestrates the payment, shipping and the email notification.
    recommendationservice Python Recommends other products based on what's given in the cart.
    adservice Java Provides text ads based on given context words.
    loadgenerator Python/Locust Continuously sends requests imitating realistic user shopping flows to the frontend.

    Screenshots

    Home Page Checkout Screen
    Screenshot of store homepage Screenshot of checkout screen

    Tooling Specs

    Tool Name Version Located at
    Ubuntu Host22.04.15 LTS-
    containerd1.7.14Cluster Nodes
    runc1.1.12Cluster Nodes
    CNI Plugin1.5.0Cluster Nodes
    kubelet1.33.0Cluster Nodes
    kubeadm1.33.0Cluster Nodes
    kubectl1.33.0Cluster Nodes
    Calico CNI3.26.0kube-system namespace
    Metrics Server0.8.1kube-system namespace
    Helm4.1.3Master Node 1 VM
    HAProxy2.8.16Load Balancer VM
    Azure DevOps Agent4.271.0Ubuntu Host
    Gitleaks8.30.1Ubuntu Host
    Trivy0.68.2Ubuntu Host
    Vault2.0.0Ubuntu Host
    Docker29.4.1Ubuntu Host
    Git2.34.1Ubuntu Host
    External Secrets Operator0.15.0external-secrets namespace
    Loki2.6.1monitoring namespace
    Grafana10.3.3monitoring namespace
    Prometheus3.11.3-distrolessmonitoring namespace
    ArgoCD9.5.11argocd namespace
    Sonarqube25.11.0Azure Cloud VM
    Azure Container RegistryBasic TierAzure cloud
    MendBolt22.12.10Azure DevOps Extension
    SARIF SAST Scans Tab0.5.11Azure DevOps Extension

    Setup Documentation

    1. Multi-Control Plane Cluster Setup
    2. Ubuntu Host Machine Setup
    3. Helm Charts
    4. Azure Cloud: Sonarqube on VM and ACR
    5. Azure DevOps Setup

    CI/CD Workflow

    • It starts with developer writing the code in their local environment; when it's time to make a commit, a git prehook is run that will do linting of source code and run some local test cases.
    • After a commit is created and pushed to the azure repos with developer own branch dev/dev_name. A PR is created from it to the protected dev branch for review.
    • When code is merged into dev, the CI/CD pipeline whose flow looks like this:

    CI

    Stage 1:

    1. Code is checked out on the pipeline agent directory at /home/shivesh/myagent/_work/2/s/.
    2. Create a directory that will store the reports files generated throughout the stage run.
    3. Perform SCA (Software Composition Analysis) Scan using Mend Bolt (Whitesource) tool. It's result are uploaded onto Azure Devops pipeline run tab automatically.

      Mendbolt SCA

    4. Run test cases and generate code coverage report file. Publish test results and coverage report to azure devops.
    5. Prepare Sonarqube with the configuration for a service source code path and settings.
    6. Perform Sonarqube Code scanning.

      Sonarqube Scan

    7. Publish Sonarqube Quality Gate Result, if failed it will fail the entire pipeline.

      Sonarqube Quality Gate

    8. Publish Gitleaks report to azure devops.

    Stage 2:

    1. Code is checked out on the pipeline agent directory at /home/shivesh/myagent/_work/2/s/.
    2. Create a directory that will store the reports files generated throughout the stage run.
    3. Build the Docker image targeting the service's Dockerfile using SERVICE_SOURCE_PATH variable.
    4. Perform trivy image scan to check for any vulnerabilities and generate its report.
    5. Push the image to the Azure Container Registry with Build.BuildId as image tag for uniqueness.

      ACR repos

    6. Upload Trivy report to azure devops and publish trivy report to be used by SARIF SAST Scan extension to display the report on azure devops pipeline run tab.

      Trivy Report

    Stage 3:

    1. Finally Update the newly built image tag into the gitops repo containing service's manifest file.

      Tag Update

    CD

    After successfully finishing the CI, CD will look like this:

    1. As the image is get updated in the gitops repo with new tag. The ArgoCD reconciliation loop will see the drift between application state, and will go OutOfSync.

      argocd outofsync

    2. If the sync policy is set to automated, argocd will apply new changes to the application or manually sync the application for newly built image.

      argocd apps

    3. For successful and Healthy state of argocd application, a mail is received via argocd-notifications:

      argocd mail

    After Full Workflow

    The application can be seen by forwading frontend service by running kubectl port-forward service/frontend 8081:80 -n online-boutique --address 0.0.0.0 :

    frontend service

    cart service

    confim

    Monitoring with Grafana Dashboard

    Loki for Logs:

    loki logs

    Prometheus for metrics:

    prom metrics

    Some wrongdoings/ Future scope work

    In this project, you may came across some things that look wrong, missing or the way they are configured. For that the explanation is I made this project to showcase integration and working of all the concepts and tools only. I am aware of all the wrong's I intentionally did, here are following:

    • Vault server is in dev mode and runs on 0.0.0.0 to make it reachable to KVM.
    • No Build pipeline for PRs.
    • Full Fledged RBAC missing in kubeadm cluster.
    • Metric server in kube-system namespace running with insecure-tls flag.
    • Unoptimized Dockerfiles. (to catch things in trivy scan for reports).
    • No unit test cases, the source code were missing those so the task is commented out in CI pipelines.
    • No persistance in monitoring stack's pods data using PV and PVC.
    • ArgoCD Image Updater implementation.
    • Gateway API and DNS implementation to expose application.

    © 2026 shiveshvijay.me