Building Robust CI/CD Pipelines with GitHub Actions

Introduction

GitHub Actions provides a powerful platform for automating your software development workflows. This guide covers building production-ready CI/CD pipelines for modern applications.

Basic Workflow Structure

Workflow Fundamentals

name: CI/CD Pipeline

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]
  workflow_dispatch:

env:
  DOTNET_VERSION: '8.0.x'
  NODE_VERSION: '20.x'

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Setup .NET
        uses: actions/setup-dotnet@v3
        with:
          dotnet-version: ${{ env.DOTNET_VERSION }}
      - name: Build
        run: dotnet build --configuration Release

Multi-Stage Pipeline

Complete .NET Application Pipeline

name: .NET Application Pipeline

on:
  push:
    branches: [main, develop]
  pull_request:
    types: [opened, synchronize, reopened]

jobs:
  build-and-test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        dotnet: ['6.0.x', '7.0.x', '8.0.x']

    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Setup .NET
        uses: actions/setup-dotnet@v3
        with:
          dotnet-version: ${{ matrix.dotnet }}

      - name: Cache NuGet packages
        uses: actions/cache@v3
        with:
          path: ~/.nuget/packages
          key: ${{ runner.os }}-nuget-${{ hashFiles('**/packages.lock.json') }}
          restore-keys: |
            ${{ runner.os }}-nuget-

      - name: Restore dependencies
        run: dotnet restore

      - name: Build
        run: dotnet build --no-restore --configuration Release

      - name: Test
        run: dotnet test --no-build --configuration Release --verbosity normal --collect:"XPlat Code Coverage"

      - name: Generate Code Coverage Report
        uses: codecov/codecov-action@v3
        with:
          file: ./coverage.cobertura.xml
          flags: unittests
          name: codecov-${{ matrix.dotnet }}

  security-scan:
    runs-on: ubuntu-latest
    needs: build-and-test

    steps:
      - uses: actions/checkout@v4

      - name: Run Trivy vulnerability scanner
        uses: aquasecurity/trivy-action@master
        with:
          scan-type: 'fs'
          scan-ref: '.'
          format: 'sarif'
          output: 'trivy-results.sarif'

      - name: Upload Trivy results to GitHub Security
        uses: github/codeql-action/upload-sarif@v2
        with:
          sarif_file: 'trivy-results.sarif'

      - name: SonarCloud Scan
        uses: SonarSource/sonarcloud-github-action@master
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}

  docker-build:
    runs-on: ubuntu-latest
    needs: [build-and-test, security-scan]
    if: github.event_name == 'push'

    steps:
      - uses: actions/checkout@v4

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Log in to Docker Hub
        uses: docker/login-action@v3
        with:
          username: ${{ secrets.DOCKER_USERNAME }}
          password: ${{ secrets.DOCKER_PASSWORD }}

      - name: Extract metadata
        id: meta
        uses: docker/metadata-action@v5
        with:
          images: myapp/api
          tags: |
            type=ref,event=branch
            type=ref,event=pr
            type=semver,pattern={{version}}
            type=semver,pattern={{major}}.{{minor}}
            type=sha

      - name: Build and push Docker image
        uses: docker/build-push-action@v5
        with:
          context: .
          platforms: linux/amd64,linux/arm64
          push: true
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
          cache-from: type=gha
          cache-to: type=gha,mode=max

Advanced Deployment Strategies

Blue-Green Deployment

name: Blue-Green Deployment

on:
  workflow_dispatch:
    inputs:
      environment:
        description: 'Environment to deploy to'
categories:
  - devops
draft: false        required: true
        type: choice
        options:
          - staging
          - production

jobs:
  deploy:
    runs-on: ubuntu-latest
    environment: ${{ github.event.inputs.environment }}

    steps:
      - uses: actions/checkout@v4

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: us-east-1

      - name: Deploy to Blue Environment
        id: deploy-blue
        run: |
          # Deploy to blue environment
          aws ecs update-service \
            --cluster my-cluster \
            --service my-app-blue \
            --force-new-deployment

          # Wait for deployment to complete
          aws ecs wait services-stable \
            --cluster my-cluster \
            --services my-app-blue

      - name: Run Smoke Tests
        run: |
          # Run smoke tests against blue environment
          npm run test:smoke -- --url https://blue.myapp.com

      - name: Switch Traffic to Blue
        if: success()
        run: |
          # Update ALB target group
          aws elbv2 modify-listener \
            --listener-arn ${{ secrets.ALB_LISTENER_ARN }} \
            --default-actions Type=forward,TargetGroupArn=${{ secrets.BLUE_TARGET_GROUP_ARN }}

      - name: Verify Deployment
        run: |
          # Health check on production URL
          curl -f https://myapp.com/health || exit 1

      - name: Rollback on Failure
        if: failure()
        run: |
          # Switch traffic back to green
          aws elbv2 modify-listener \
            --listener-arn ${{ secrets.ALB_LISTENER_ARN }} \
            --default-actions Type=forward,TargetGroupArn=${{ secrets.GREEN_TARGET_GROUP_ARN }}

Canary Deployment

name: Canary Deployment

jobs:
  canary-deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Deploy Canary Version
        run: |
          kubectl set image deployment/myapp \
            myapp=myapp:${{ github.sha }} \
            --namespace=production

      - name: Scale Canary
        run: |
          # Start with 10% traffic
          kubectl scale deployment/myapp-canary --replicas=1 --namespace=production
          kubectl scale deployment/myapp-stable --replicas=9 --namespace=production

      - name: Monitor Metrics
        run: |
          # Monitor error rates for 10 minutes
          for i in {1..10}; do
            ERROR_RATE=$(curl -s http://prometheus:9090/api/v1/query \
              --data-urlencode 'query=rate(http_requests_total{status=~"5.."}[5m])' \
              | jq '.data.result[0].value[1]')

            if (( $(echo "$ERROR_RATE > 0.01" | bc -l) )); then
              echo "Error rate too high: $ERROR_RATE"
              exit 1
            fi

            sleep 60
          done

      - name: Promote Canary
        if: success()
        run: |
          kubectl scale deployment/myapp-canary --replicas=10 --namespace=production
          kubectl scale deployment/myapp-stable --replicas=0 --namespace=production

Environment Management

Matrix Strategy with Environments

name: Multi-Environment Deployment

on:
  push:
    branches: [main]

jobs:
  deploy:
    strategy:
      matrix:
        environment: [dev, staging, prod]
        include:
          - environment: dev
            url: https://dev.myapp.com
            approval: false
          - environment: staging
            url: https://staging.myapp.com
            approval: false
          - environment: prod
            url: https://myapp.com
            approval: true

    runs-on: ubuntu-latest
    environment:
      name: ${{ matrix.environment }}
      url: ${{ matrix.url }}

    steps:
      - uses: actions/checkout@v4

      - name: Deploy to ${{ matrix.environment }}
        run: |
          echo "Deploying to ${{ matrix.environment }}"
          # Deployment logic here

      - name: Require Approval for Production
        if: matrix.approval == true
        uses: trstringer/manual-approval@v1
        with:
          secret: ${{ github.TOKEN }}
          approvers: senior-devops-team

Secrets and Configuration Management

Using GitHub Secrets and Variables

name: Secure Configuration

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Setup Environment Variables
        run: |
          echo "API_URL=${{ vars.API_URL }}" >> $GITHUB_ENV
          echo "DB_HOST=${{ secrets.DB_HOST }}" >> $GITHUB_ENV

      - name: Generate Config File
        run: |
          cat > config.json <<EOF
          {
            "api": {
              "url": "${{ vars.API_URL }}",
              "key": "${{ secrets.API_KEY }}"
            },
            "database": {
              "host": "${{ secrets.DB_HOST }}",
              "password": "${{ secrets.DB_PASSWORD }}"
            }
          }
          EOF

      - name: Encrypt Sensitive Files
        run: |
          gpg --quiet --batch --yes --decrypt --passphrase="${{ secrets.GPG_PASSPHRASE }}" \
            --output config.decrypted.json config.json.gpg

Caching Strategies

Advanced Caching

name: Optimized Build with Caching

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Cache Docker layers
        uses: actions/cache@v3
        with:
          path: /tmp/.buildx-cache
          key: ${{ runner.os }}-buildx-${{ github.sha }}
          restore-keys: |
            ${{ runner.os }}-buildx-

      - name: Cache node modules
        uses: actions/cache@v3
        with:
          path: ~/.npm
          key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
          restore-keys: |
            ${{ runner.os }}-node-

      - name: Cache .NET packages
        uses: actions/cache@v3
        with:
          path: ~/.nuget/packages
          key: ${{ runner.os }}-nuget-${{ hashFiles('**/*.csproj') }}
          restore-keys: |
            ${{ runner.os }}-nuget-

      - name: Cache build outputs
        uses: actions/cache@v3
        with:
          path: |
            **/bin
            **/obj
          key: ${{ runner.os }}-build-${{ github.sha }}
          restore-keys: |
            ${{ runner.os }}-build-

Monitoring and Notifications

Comprehensive Notifications

name: Build with Notifications

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Build Application
        id: build
        run: |
          dotnet build
          echo "status=success" >> $GITHUB_OUTPUT

      - name: Slack Notification
        if: always()
        uses: slackapi/slack-github-action@v1
        with:
          payload: |
            {
              "text": "Build ${{ steps.build.outputs.status }}",
              "blocks": [
                {
                  "type": "section",
                  "text": {
                    "type": "mrkdwn",
                    "text": "*Build Status:* ${{ job.status }}\n*Branch:* ${{ github.ref }}\n*Commit:* ${{ github.sha }}"
                  }
                }
              ]
            }
        env:
          SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK }}

      - name: Create GitHub Issue on Failure
        if: failure()
        uses: actions/github-script@v6
        with:
          script: |
            await github.rest.issues.create({
              owner: context.repo.owner,
              repo: context.repo.repo,
              title: `Build Failed: ${context.sha}`,
              body: `Build failed on ${context.ref}\n\nCommit: ${context.sha}\nWorkflow: ${context.workflow}`,
              labels: ['bug', 'ci-failure']
            })

Best Practices

Reusable Workflows

# .github/workflows/reusable-test.yml
name: Reusable Test Workflow

on:
  workflow_call:
    inputs:
      dotnet-version:
        required: false
        type: string
        default: '8.0.x'
    secrets:
      sonar-token:
        required: true

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Setup .NET
        uses: actions/setup-dotnet@v3
        with:
          dotnet-version: ${{ inputs.dotnet-version }}
      - name: Test
        run: dotnet test

# Using reusable workflow
name: Main Pipeline

jobs:
  test:
    uses: ./.github/workflows/reusable-test.yml
    with:
      dotnet-version: '8.0.x'
    secrets:
      sonar-token: ${{ secrets.SONAR_TOKEN }}

Conclusion

GitHub Actions provides a flexible and powerful platform for implementing CI/CD pipelines. By leveraging advanced features like matrix builds, reusable workflows, and sophisticated deployment strategies, you can create robust automation that scales with your application’s needs.

Comments