What I’ve Learned Building a CI/CD Cloud Pipeline from Scratch

#aws #terraform #devops #cicd

When I first began my transition from a Full Stack Developer to focusing heavily on Cloud & DevOps, the learning curve felt like an absolute mountain.

Between managing the infrastructure for thousands of daily orders and squeezing in certification study time during my 3.5-hour daily commute on the Mumbai local trains, my overarching goal was simple: stop doing things manually.

Over the last few months, I have documented every step of this journey right here on Atul Codes. From writing my first Dockerfile to provisioning cloud infrastructure with code, it has been an incredible phase of growth.

Here is a look back at the biggest lessons I’ve learned while building a modern CI/CD cloud pipeline from scratch.

Phase 1: Escaping "It Works on My Machine"

The first major bottleneck I had to solve was environment inconsistency. Realizing that manual server configurations were slowing down our development cycles pushed me straight into containerization.

Learning Docker: I started by figuring out How to Dockerize a Node.js App. Packaging the runtime and the code together changed how I viewed deployments entirely.
The Power of Automation: But building containers manually wasn't enough. The real "aha!" moment came when I learned to Automate Docker Builds with GitHub Actions, turning code commits into instant, deployable images.
Embracing the Terminal: Of course, containerization comes with its own headaches. I spent hours debugging until I learned How to Fix the "Port Already in Use" Error, and I quickly realized that mastering the 5 Linux Bash Commands Every DevOps Engineer Uses Daily was non-negotiable for troubleshooting headless servers.

Phase 2: Killing "ClickOps" with Infrastructure as Code

Once the applications were containerized, I needed a place to run them. The AWS Management Console is great for learning, but clicking through UI menus to spin up servers is dangerous for production.

Entering the Terraform Ecosystem: I forced myself to learn HashiCorp Configuration Language (HCL). Writing the code to Provision an AWS EC2 Instance with Terraform was a massive milestone. It took away the fear of breaking the cloud because everything was version-controlled.
Building a Second Brain: As I studied deeper for my Terraform Associate certification, the syntax became overwhelming. To save time, I compiled my Ultimate Terraform Cheatsheet for DevOps Engineers—which has become my most used daily reference.
Optimizing the Workflow: To write all this HCL effectively, I had to upgrade my environment. Finding the Top 5 VS Code Extensions for DevOps and Cloud Engineering completely streamlined my development process.

Phase 3: The Reality of Production & Security

Building infrastructure is fun, but securing it is what actually keeps you employed.

Securing the Pipeline: Early on, I realized the immense danger of leaked keys. I immediately audited my workflows and documented How to Stop Hardcoding AWS Credentials using environment variables and AWS CLI profiles.
Protecting the Wallet: The cloud is cheap until you make a typo. Setting up AWS Billing Alerts gave me the peace of mind to experiment with new architectures without the fear of a $1,000 surprise bill.
Exploring the Future: With security locked down, I started exploring beyond traditional servers, successfully Deploying My First Serverless Function on AWS Lambda.

What’s Next for Atul Codes?

Writing these articles over the last quarter hasn't just been about sharing knowledge—it has been the ultimate forcing function for my own upskilling. When you have to explain a complex topic clearly to someone else, you uncover the gaps in your own understanding.

So, what is the next big goal?

I am currently diving deep into AWS ECS Fargate for scalable container orchestration, and preparing to knock out my HashiCorp Terraform Associate certification. You can expect a deep-dive series on both of those topics very soon.

Thank you to everyone who has read, commented, and shared my posts so far. Let’s keep building!