CI/CD Fundamentals: A Practical Introduction

"But it works on my machine!?"

If you have ever been working with software development, you probably have heard (or said) this phrase. It's often followed by hours of debugging, comparing environments, and questioning life choices. This is exactly the kind of problem that CI/CD was created to solve.

What is CI/CD, Really?

CI/CD isn't just another tech buzzword – it's a set of practices that helps teams to deliver better software more reliably. CI/CD combines two key practices: Continuous Integration, which automatically tests and validates code changes, and Continuous Delivery/Deployment, which automates the preparation and deployment of code to production. Think of it as your development team's assembly line, quality control, and delivery service all in one (automated) system.

Let's break CI/CD down:

Continuous Integration (CI) is like having a quality control inspector, checking every single change you make to your code. Every time you push new code, it may automatically:

• Builds your application to make sure everything compiles correctly

• Runs your test suite to catch any bugs

• Performs code quality checks to maintain standards

• Validates that your changes play nicely with everyone else's work

• Checks dependencies for security vulnerabilities

• Ensures code style consistency

Continuous Delivery/Deployment (CD) is your automated delivery pipeline. Once your code passes all checks, it:

• Packages your application consistently

• Deploys to staging environments for final verification

• Either automatically deploys to production (Continuous Deployment) or prepares everything for a one-click deploy (Continuous Delivery)

When Should You Use CI/CD?

While CI/CD might sound like a no-brainer, it is important to match your tools to your needs. Here's when you should definitely consider (not) implementing CI/CD:

Perfect For:

• Data Pipelines: When you need reliable and regular data processing with consistent environments

• APIs and Services: Where uptime, reliability, and rapid deployment of fixes are crucial

• Team Projects: Multiple developers working on the same codebase with frequent integrations

• Client-Facing Applications: When deployment mistakes are costly and rollbacks need to be quick

• Regulated Environments: Where audit trails and process documentation are required

• ML Operations: When model training and deployment need to be reproducible

• Microservices: Where multiple services need coordinated deployment

Maybe Skip It For:

• One-off analysis projects with no reuse planned

• Quick proof-of-concept work that won't go to production

• Small personal projects where setup time outweighs benefits

• Static websites or simple applications with minimal dependencies

• Projects with very infrequent updates

Essential Setup: Your CI/CD Foundation

Version Control: The Foundation

Think of version control as the foundation of your house – building it solid makes that everything on top can be stable. Here's what you need:

1. Git Basics:

   • Protected main/master branch

   • Clear branch naming conventions

   • Pull request workflow

   • Code review process

2. Security Measures:

   • Branch protection rules

   • Required status checks

   • Signed commits for audit trails

Basic Pipeline Architecture

Your first CI/CD pipeline can start simple and grow with your needs. Here's what the first steps of the journey typically look like:

CI (Continuous Integration) Pipeline:

Code Push → Build → Test

Key CI components:

• Environment setup (e.g. Python, virtual environment)

• Dependency management

• Automated testing

• Basic error reporting

CD (Continuous Delivery) Pipeline:

Approved Changes → Staging Deploy

Key CD components:

• Staging environment configuration

• Basic deployment process

• Simple deployment verification

Levelling up your pipeline:

Ready to spice it up already? The next steps for your CI/CD pipeline might look like this:

Adding CI capabilities:

Code Push → Build → Test → Quality Checks → Security Scan

Adding CD capabilities:

Approved Changes → Staging Deploy → Tests → Production Deploy

Remember: start with the essential components and gradually add more sophisticated features as your team/project matures. We'll cover how to implement these steps (+ more additional capabilities) in our next post about building robust CI/CD pipelines. Keep this in mind when implementing your first CI/CD pipeline:

1. Start Small Begin with just continuous integration. Set up automated builds and basic tests on pull requests. This alone will catch many common issues early.

2. Add Quality Gates Once basic CI is working, add code quality checks and security scans. These prevent technical debt from accumulating.

3. Automate Deployment Start with automated deployment to a staging environment. Once comfortable, consider automation for production.

4. Monitor and Iterate Watch how your pipeline performs and adjust based on team feedback and needs.

Getting Started: Your First Pipeline

Let's look at a practical example using GitHub Actions, one of the most popular CI/CD tools. Here's a basic workflow file (.github/workflows/staging_deploy.yml) that you might use for a Python project:

# Basic CI/CD workflow for Python projects with uv
name: CI/CD Pipeline

# Workflow triggers on push and pull request for the main branch
on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
    - name: Check out repository
      uses: actions/checkout@v4

    - name: Install uv
      uses: astral-sh/setup-uv@v5
      with:
        # Installs the latest version of uv (>=0.4.0) and Python v3.12
        version: ">=0.4.0"
        python-version: "3.12"
    
    - name: Install the project / dependencies
	  # Make sure to include pytest in the requirements
      run: uv sync

    - name: Run tests
      run: uv run pytest
	
	deploy:
		needs: test
		runs-on: ubuntu-latest
		environment: staging
		if: github.ref == 'refs/heads/main'
		
		steps:
	  - uses: actions/checkout@v4
    
    - name: Install uv
      uses: astral-sh/setup-uv@v5
      with:
        version: ">=0.4.0"
        python-version: '3.12'
        
    - name: Install the project / dependencies
      run: uv sync
      
    - name: Deploy to staging
      run: echo "Deploying to staging environment..."
        # Add your deployment commands here
        
        # Example:
        # python deploy.py --environment staging

This workflow:

• Triggers on pushes to main and pull requests

• Sets up a Python environment

• Installs project dependencies using uv (Also see Nathan's blogpost on using uv)

• Runs tests with pytest

• Deploys to staging when code passes tests and is pushed/merged to main

Conclusion

CI/CD isn't just about automating deployments – it's about building confidence in your software delivery process. Start small, focus on the fundamentals, and gradually expand as your team's needs grow. Remember, the goal is to make development more efficient and reliable, not to add unnecessary complexity.

In our next post on CI/CD, we'll dive deeper into best practices and automation strategies for building robust CI/CD pipelines. Until then, start thinking about how these fundamentals could improve your team's development process.

Want to get started? Here's your homework:

1. Audit your current deployment process

2. Identify manual steps that could be automated

3. Set up a basic Git workflow with protected branches

4. Research CI/CD tools that match your stack

Remember: perfect is the enemy of good. Start small, but start today.