CodcompassCodcompass
← Back to Blog
DevOps2026-05-14·82 min read

Three post-deploy checks I run after every Cloudflare Pages build

By MORINAGA

Static Site Deployment Validation: A Targeted Verification Framework for CDN-Hosted SSGs

Current Situation Analysis

Static site generators (SSGs) like Astro, Next.js, and Hugo fundamentally shift application complexity from runtime execution to build-time compilation. When artifacts are pushed to a CDN like Cloudflare Pages, the traditional post-deployment verification playbook breaks down. Uptime pings, API health checks, and end-to-end user flow tests are designed for dynamic servers with active processes, databases, and stateful sessions. They are misaligned with the actual failure surface of pre-rendered HTML, CSS, and static JSON bundles.

This mismatch is frequently overlooked because CI/CD pipelines prioritize build success, linting, and test coverage. Once the build exits with code 0, teams assume production readiness. However, static deployments introduce a distinct set of silent failure modes:

  1. CDN Routing Conflicts: Files like _redirects or netlify.toml can rewrite crawler paths without affecting browser navigation. A misconfigured rule may serve a 301/302 to search bots while appearing normal in developer tools.
  2. Indexing Window Desynchronization: Search engine submission APIs (IndexNow, Bing Webmaster, Yandex) require live, publicly accessible URLs. Submitting during the build phase or before CDN propagation completes results in rejected payloads or delayed indexing.
  3. Layout & Performance Regressions: In SSG environments, visual regressions rarely stem from runtime errors. They originate from CSS injection, ad slot components, or framework updates (e.g., Tailwind v4 config changes) that alter paint behavior, triggering Cumulative Layout Shift (CLS) spikes.

Data from production deployments across multiple Astro 5 SSG projects demonstrates that these failure modes account for the majority of post-launch incidents. Traditional monitoring catches server outages; it does not catch crawler blocking, indexing delays, or silent asset layout shifts. A targeted verification strategy must align with the static deployment lifecycle: artifact integrity, CDN routing correctness, and search engine visibility.

WOW Moment: Key Findings

The following comparison illustrates why traditional runtime monitoring fails to address SSG-specific failure modes, and why a targeted validation approach yields higher signal-to-noise ratios.

Approach Detection Latency False Positive Rate Infrastructure Overhead Relevance to Static Failure Surface
Traditional Uptime/E2E Monitoring 5–15 minutes High (CDN caching, static routing) High (browser instances, persistent agents) Low (misses crawler blocks, indexing delays, layout shifts)
SSG-Targeted Validation 1–3 minutes Low (direct HTTP status, API responses, Lighthouse CI) Minimal (lightweight scripts, cron jobs) High (covers _redirects misconfigs, IndexNow windows, CLS regressions)

This finding matters because it shifts validation from "is the infrastructure responding?" to "are the actual consumers of static content (crawlers, search indices, and render engines) receiving correct payloads?" By decoupling verification from runtime assumptions, teams can catch silent deployment failures before they impact organic traffic, search rankings, or user experience metrics.

Core Solution

The validation framework consists of three targeted checks, each addressing a specific static deployment failure mode. The implementation uses lightweight Node.js/TypeScript scripts and GitHub Actions, designed for idempotency and CDN-aware execution.

Step 1: Sitemap Integrity & Reachability Verification

Static sites rely on sitemap-index.xml and sub-sitemaps (sitemap-0.xml, etc.) for search engine discovery. A common failure mode occurs when CDN routing rules inadvertently rewrite or block these paths. The verification must:

  • Confirm HTTP 200 status without following redirects
  • Validate sub-sitemap URL count against a known threshold
  • Fail fast if the threshold drops, indicating a broken data pipeline

Implementation:

// scripts/validate-sitemap.ts
import { execSync } from 'child_process';
import type { SiteConfig } from './types';

const SITES: SiteConfig[] = [
  { domain: 'aiappdex.com', minUrls: 1000 },
  { domain: 'findindiegame.com', minUrls: 150 },
  { domain: 'ossfind.com', minUrls: 150 }
];

async function verifySitemapReachability() {
  const results: string[] = [];

  for (const site of SITES) {
    const indexUrl = `https://${site.domain}/sitemap-index.xml`;
    const subUrl = `https://${site.domain}/sitemap-0.xml`;

    // Check index reachability without following redirects
    const indexStatus = execSync(
      `curl -s -o /dev/null -w "%{http_code}" "${indexUrl}"`,
      { encoding: 'utf-8' }
    ).trim();

    if (indexStatus !== '200') {
      results.push(`FAIL: ${site.domain} sitemap-index.xml returned ${indexStatus}`);
      continue;
    }

    // Validate sub-sitemap URL count
    const urlCount = execSync(
      `curl -s "${subUrl}" | grep -o '<url>' | wc -l`,
      { encoding: 'utf-8' }
    ).trim();

    const count = parseInt(urlCount, 10);
    if (count < site.minUrls) {
      results.push(`FAIL: ${site.domain} sitemap-0.xml has ${count} URLs (threshold: ${site.minUrls})`);
    } else {
      results.push(`OK: ${site.domain} sitemap valid (${count} URLs)`);
    }
  }

  console.log(results.join('\n'));
  if (results.some(r => r.startsWith('FAIL'))) {
    process.exit(1);
  }
}

verifySitemapReachability();

Architecture Rationale:

  • curl is invoked without the -L flag. Following redirects masks _redirects misconfigurations that silently rewrite crawler paths.
  • URL count validation catches silent ETL or data pipeline breaks. If the build succeeds but the source data shrinks, the sitemap reflects the regression immediately.
  • The script exits with code 1 on failure, enabling CI/CD integration without custom error handling.

Step 2: Post-Deploy IndexNow Batch Submission

IndexNow enables direct URL submission to Bing, Yandex, Naver, and Seznam. The API requires live URLs and site-specific verification keys. Submitting during the build phase fails because CDN propagation takes 2–3 minutes, and verification files (/<key>.txt) may not be publicly accessible yet.

Implementation:

// scripts/submit-indexnow.ts
import { readFileSync } from 'fs';
import { XMLParser } from 'fast-xml-parser';
import type { IndexNowPayload, SiteEndpoint } from './types';

const ENDPOINTS: SiteEndpoint[] = [
  { domain: 'aiappdex.com', key: 'a1b2c3d4e5f6', endpoint: 'https://www.bing.com/indexnow' },
  { domain: 'findindiegame.com', key: 'f7g8h9i0j1k2', endpoint: 'https://www.bing.com/indexnow' },
  { domain: 'ossfind.com', key: 'l3m4n5o6p7q8', endpoint: 'https://www.bing.com/indexnow' }
];

async function extractUrls(domain: string): Promise<string[]> {
  const response = await fetch(`https://${domain}/sitemap-index.xml`);
  const xml = await response.text();
  const parser = new XMLParser();
  const parsed = parser.parse(xml);
  const sitemaps = parsed.sitemapindex.sitemap.map((s: any) => s.loc);

  const allUrls: string[] = [];
  for (const sitemapUrl of sitemaps) {
    const subResponse = await fetch(sitemapUrl);
    const subXml = await subResponse.text();
    const subParsed = parser.parse(subXml);
    const urls = subParsed.urlset.url.map((u: any) => u.loc);
    allUrls.push(...urls);
  }
  return allUrls;
}

async function runIndexNowSubmission() {
  for (const site of ENDPOINTS) {
    const urls = await extractUrls(site.domain);
    const payload: IndexNowPayload = {
      host: site.domain,
      key: site.key,
      urlList: urls
    };

    const res = await fetch(site.endpoint, {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify(payload)
    });

    if (res.status === 200) {
      console.log(`${site.domain}: submitted ${urls.length} URLs → 200 OK`);
    } else {
      console.error(`${site.domain}: IndexNow failed with ${res.status}`);
    }
  }
}

runIndexNowSubmission();

Architecture Rationale:

  • Execution is decoupled from the build pipeline. A workflow_dispatch trigger ensures submission occurs only after Cloudflare Pages finishes deploying.
  • fast-xml-parser handles sitemap traversal efficiently without loading full DOM trees.
  • Direct API submission bypasses search console UIs, enabling automation and audit trails.

Step 3: Weekly Lighthouse Regression Tracking

Performance and layout stability in SSG environments are deterministic. Regressions typically stem from CSS framework updates, ad component injections, or asset loading changes. Daily checks are wasteful for static sites; weekly spot-checks provide trend data without CI overhead.

Implementation (GitHub Actions Matrix):

# .github/workflows/lighthouse-weekly.yml
name: Weekly Lighthouse Spot-Check
on:
  schedule:
    - cron: '30 4 * * 1' # Monday 04:30 UTC
  workflow_dispatch:

jobs:
  audit:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        target:
          - domain: aiappdex.com
            path: /models/timm-vit-base-patch16-clip-224-openai/
          - domain: findindiegame.com
            path: /games/dredge-1562430/
          - domain: ossfind.com
            path: /alternatives/ghost/
    steps:
      - uses: actions/checkout@v4
      - name: Run Lighthouse CI
        uses: treosh/lighthouse-ci-action@v11
        with:
          urls: |
            https://${{ matrix.target.domain }}${{ matrix.target.path }}
            https://${{ matrix.target.domain }}/
          uploadArtifacts: true
          temporaryPublicStorage: true
          config: |
            {
              "extends": "lighthouse:default",
              "settings": {
                "onlyCategories": ["performance", "accessibility", "best-practices"],
                "formFactor": "desktop"
              }
            }

Architecture Rationale:

  • Matrix strategy covers both homepage and deep-link paths, catching route-specific CSS/JS injection issues.
  • temporaryPublicStorage: true enables historical diffing without persistent infrastructure.
  • No hard failure thresholds. Lighthouse serves as a trend monitor, not a deployment gate, preventing disproportionate blocks for minor score fluctuations.

Pitfall Guide

Pitfall Explanation Fix
Following redirects in health checks Using curl -L or fetch with auto-redirect masks _redirects misconfigurations that block crawlers while appearing normal in browsers. Always disable redirect following (-L flag omitted, redirect: 'manual' in fetch). Validate raw HTTP status codes.
Submitting to IndexNow during build CDN propagation takes 2–3 minutes. Verification files and live URLs aren't accessible yet, causing 403/404 responses from search APIs. Decouple submission from build. Use workflow_dispatch or a post-deploy webhook to trigger after Cloudflare confirms deployment.
Hard-blocking deploys on Lighthouse scores Static sites rarely experience runtime crashes. Blocking deployments for a 94→88 score drop wastes engineering time and delays fixes. Treat Lighthouse as a trend monitor. Set alert thresholds, not gate thresholds. Investigate regressions, don't block merges.
Ignoring sitemap URL count thresholds A successful build doesn't guarantee data pipeline integrity. If source data shrinks, the sitemap reflects the regression silently. Define minimum URL counts per domain. Fail the check if counts drop, indicating ETL or CMS sync failures.
Over-engineering with E2E tests for pre-rendered HTML Playwright/Cypress tests validate runtime behavior. SSGs ship static HTML; E2E tests add CI minutes without catching CDN routing or indexing issues. Replace E2E with targeted HTTP status checks, sitemap validation, and Lighthouse CI. Reserve E2E for dynamic routes or client-side apps.
Misplacing IndexNow verification files CDN routing rules or _redirects can intercept /<key>.txt requests, causing IndexNow to reject submissions. Verify verification files are served as static assets, not rewritten. Test with curl -I https://domain.com/<key>.txt post-deploy.
Running performance checks on every commit Lighthouse CI takes 3–4 minutes per URL. Running it on every PR or commit wastes CI minutes and generates noise for static sites. Schedule weekly cron jobs or trigger manually after major CSS/framework updates. Use lightweight bundle analysis for PRs instead.

Production Bundle

Action Checklist

  • Verify sitemap-index.xml returns 200 without following redirects
  • Validate sitemap-0.xml URL count against domain-specific thresholds
  • Confirm IndexNow verification files are publicly accessible post-deploy
  • Trigger IndexNow submission only after CDN propagation completes
  • Configure Lighthouse CI matrix with homepage + deep-link paths
  • Set Lighthouse as a trend monitor, not a deployment gate
  • Audit _redirects rules for unintended crawler path rewrites
  • Document minimum URL thresholds and IndexNow keys in environment config

Decision Matrix

Scenario Recommended Approach Why Cost Impact
Low-traffic SSG (Astro/Hugo) Targeted validation (sitemap + IndexNow + weekly Lighthouse) Matches static failure surface; avoids runtime monitoring overhead Near-zero (lightweight scripts, free CI minutes)
High-traffic dynamic app Full E2E + uptime monitoring + APM Requires runtime health, API latency, and user flow validation High (browser instances, monitoring agents, APM licenses)
Hybrid SSG with client-side routes Targeted validation + selective E2E for dynamic paths Covers static integrity while validating client-side behavior Moderate (split CI strategy, targeted test suites)
Pre-revenue / experimental sites Sitemap reachability + IndexNow only Focuses on search visibility; defers performance gating Minimal (cron jobs, manual triggers)

Configuration Template

# .github/workflows/post-deploy-validation.yml
name: Post-Deploy Static Validation
on:
  workflow_dispatch:
    inputs:
      environment:
        description: 'Target environment'
        required: true
        default: 'production'

jobs:
  validate-sitemap:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm ci
      - run: npx tsx scripts/validate-sitemap.ts

  submit-indexnow:
    needs: validate-sitemap
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm ci
      - run: npx tsx scripts/submit-indexnow.ts
        env:
          INDEXNOW_KEYS: ${{ secrets.INDEXNOW_KEYS }}

Quick Start Guide

  1. Initialize Scripts: Create scripts/validate-sitemap.ts and scripts/submit-indexnow.ts using the templates above. Install dependencies: npm i fast-xml-parser tsx.
  2. Configure Domains & Thresholds: Update the SITES and ENDPOINTS arrays with your domains, minimum URL counts, and IndexNow keys. Store keys in GitHub Secrets.
  3. Add Workflow Dispatch: Create .github/workflows/post-deploy-validation.yml and enable manual triggers. Run it after Cloudflare Pages confirms deployment.
  4. Schedule Lighthouse CI: Add the weekly Lighthouse workflow. Verify temporaryPublicStorage outputs are accessible for historical diffing.
  5. Audit Redirects: Review _redirects or CDN routing rules. Ensure crawler paths (/sitemap-index.xml, /<key>.txt) are not rewritten or blocked.

This framework replaces guesswork with deterministic validation. By aligning checks with the actual failure modes of static deployments, teams catch silent regressions before they impact search visibility, user experience, or infrastructure costs.