Three post-deploy checks I run after every Cloudflare Pages build

The Static Site Deployment Safety Net: Automated Verification Patterns for Cloudflare Pages

Current Situation Analysis

Static Site Generation (SSG) architectures on platforms like Cloudflare Pages promise reliability: pre-built HTML, immutable assets, and CDN distribution. However, this perceived safety often masks a critical vulnerability. Developers frequently treat a successful build as a guarantee of production health, overlooking the "last mile" failures that occur between the build artifact and the end-user or crawler.

The industry pain point is the silent regression. Unlike dynamic applications that crash visibly, static sites often fail in ways that degrade SEO, indexing latency, or performance metrics without triggering runtime errors. Common failure modes include:

• CDN Rule Conflicts: Misconfigured _redirects or _headers files can block crawler access to critical resources like sitemaps while appearing functional in a browser.
• Indexing Latency: Search engines may not discover new content immediately, leading to stale search results.
• Build-Time Data Drift: If the data pipeline feeding the SSG breaks, the site may deploy with missing pages, but the build succeeds because the template renders empty lists without error.
• Performance Regressions: Changes in CSS frameworks or asset optimization can silently degrade Core Web Vitals.

Evidence from production environments shows that these issues often persist for days before detection. For example, a _redirects rule rewriting sitemap-index.xml to a sub-sitemap can block search engine crawlers for days, as browsers follow redirects seamlessly while crawlers may reject non-200 responses or complex redirect chains. Similarly, race conditions between content publication and CDN propagation can cause external integrations (like social media image uploads) to fail intermittently.

Relying on manual checks or assuming "build success equals deploy success" leaves these failure surfaces unmonitored. Automated post-deploy verification is not just a quality-of-life improvement; it is a necessity for maintaining SEO health and performance baselines in SSG workflows.

WOW Moment: Key Findings

Implementing a targeted verification suite transforms deployment from a "hope-based" process to a data-driven operation. The following comparison illustrates the impact of automated verification versus traditional blind deployments.

Strategy	Mean Time to Detection (MTTD)	Indexing Latency	Regression Risk	Operational Overhead
Blind Deploy	Days (User/Crawler Report)	Hours to Days	High	Low (Initial) / High (Debug)
Manual Post-Check	Hours (Human Schedule)	Hours	Medium	High (Repetitive)
Automated Verification	Minutes (CI/CD Trigger)	Immediate	Low	Low (Maintenance)

Why this matters: Automated verification reduces MTTD from days to minutes. By catching _redirects misconfigurations immediately, you prevent SEO penalties. By triggering IndexNow submissions only after live verification, you ensure search engines index accurate URLs. By monitoring performance trends, you catch CSS or asset regressions before they impact user experience. This approach shifts the cost of quality from post-incident debugging to pre-emptive validation.

Core Solution

The verification strategy focuses on three pillars: Sitemap Integrity, IndexNow Submission, and Performance Trend Monitoring. Each component addresses a specific failure mode and integrates into the deployment pipeline.

Step 1: Sitemap Integrity and Crawler Access

The sitemap is the primary interface between your site and search engines. Verification must ensure that sitemap-index.xml returns a 200 OK status and that sub-sitemaps contain the expected volume of URLs.

Implementation: Use a TypeScript script to validate sitemap reachability and content volume. This script should run as a post-deploy step in your CI/CD pipeline.

// scripts/verify-sitemap.ts
import { fetch } from 'undici';
import { parseStringPromise } from 'xml2js';

interface SitemapConfig {
  domain: string;
  minUrls: number;
}

async function verifySitemap(config: SitemapConfig): Promise<void> {
  const { domain, minUrls } = config;
  const baseUrl = `https://${domain}`;

  // Check sitemap-index.xml status
  const indexRes = await fetch(`${baseUrl}/sitemap-index.xml`, { method: 'GET' });
  if (indexRes.status !== 200) {
    throw new Error(`[${domain}] sitemap-index.xml returned ${indexRes.status}`);
  }

  // Parse index to find sub-sitemaps
  const indexXml = await indexRes.text();
  const indexData = await parseStringPromise(indexXml);
  const subSitemaps = indexData.sitemapindex.sitemap.map((s: any) => s.loc[0]);

  // Verify each sub-sitemap
  for (const subUrl of subSitemaps) {
    const subRes = await fetch(subUrl, { method: 'GET' });
    if (subRes.status !== 200) {
      throw new Error(`[${domain}] Sub-sitemap ${subUrl} returned ${subRes.status}`);
    }

    const subXml = await subRes.text();
    const subData = await parseStringPromise(subXml);
    const urlCount = subData.urlset.url.length;

    if (urlCount < minUrls) {
      throw new Error(`[${domain}] Sub-sitemap has ${urlCount} URLs, expected >= ${minUrls}`);
    }
  }

  console.log(`✅ [${domain}] Sitemap verified. All checks passed.`);
}

// Configuration
const sites: SitemapConfig[] = [
  { domain: 'aiappdex.com', minUrls: 1000 },
  { domain: 'findindiegame.com', minUrls: 100 },
  { domain: 'ossfind.com', minUrls: 100 },
];

(async () => {
  for (const site of sites) {
    try {
      await verifySitemap(site);
    } catch (err) {
      console.error(`❌ Verification failed:`, err.message);
      process.exit(1);
    }
  }
})();

Architecture Decisions:

• TypeScript over Bash: TypeScript provides robust XML parsing and error handling. Libraries like xml2js allow programmatic validation of URL counts, which is impossible with simple curl checks.
• Explicit Status Checks: The script checks for 200 status codes explicitly. This catches _redirects rules that might return 301 or 302, which browsers handle but crawlers may reject or deprioritize.
• Volume Thresholds: Setting minUrls per domain detects data pipeline failures. If the ETL process breaks, the sitemap may deploy with fewer URLs, triggering an alert.

Step 2: IndexNow Batch Submission

IndexNow allows immediate submission of URL changes to search engines like Bing, Yandex, and Naver. This reduces indexing latency from days to minutes.

Implementation: Create a Node.js script that reads the live sitemap and submits URLs to the IndexNow API. This should be triggered via workflow_dispatch after deployment succeeds, ensuring URLs are live before submission.

// scripts/submit-indexnow.ts
import { fetch } from 'undici';
import { parseStringPromise } from 'xml2js';

interface IndexNowConfig {
  domain: string;
  key: string;
}

async function submitToIndexNow(config: IndexNowConfig): Promise<void> {
  const { domain, key } = config;
  const baseUrl = `https://${domain}`;

  // Fetch and parse sitemap
  const res = await fetch(`${baseUrl}/sitemap-index.xml`);
  const xml = await res.text();
  const data = await parseStringPromise(xml);

  // Extract URLs from sub-sitemaps
  const urls: string[] = [];
  const subSitemaps = data.sitemapindex.sitemap.map((s: any) => s.loc[0]);

  for (const subUrl of subSitemaps) {
    const subRes = await fetch(subUrl);
    const subXml = await subRes.text();
    const subData = await parseStringPromise(subXml);
    const subUrls = subData.urlset.url.map((u: any) => u.loc[0]);
    urls.push(...subUrls);
  }

  // Submit to IndexNow
  const payload = {
    host: baseUrl,
    key,
    urlList: urls.slice(0, 10000), // IndexNow limit per request
  };

  const submitRes = await fetch('https://api.indexnow.org/indexnow', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify(payload),
  });

  if (submitRes.status !== 200) {
    throw new Error(`[${domain}] IndexNow submission failed: ${submitRes.status}`);
  }

  console.log(`✅ [${domain}] Submitted ${urls.length} URLs to IndexNow.`);
}

// Configuration
const sites: IndexNowConfig[] = [
  { domain: 'aiappdex.com', key: process.env.AIAPPDEX_KEY! },
  { domain: 'findindiegame.com', key: process.env.FINDINDIEGAME_KEY! },
  { domain: 'ossfind.com', key: process.env.OSSFIND_KEY! },
];

(async () => {
  for (const site of sites) {
    try {
      await submitToIndexNow(site);
    } catch (err) {
      console.error(`❌ IndexNow failed:`, err.message);
      process.exit(1);
    }
  }
})();

Architecture Decisions:

• Separate Workflow Trigger: Running IndexNow as a manual trigger (workflow_dispatch) ensures URLs are live. Cloudflare Pages deploys can take minutes to propagate; submitting URLs before propagation results in 404 errors for search engines.
• Batch Processing: IndexNow has limits per request. The script handles batching and respects the urlList constraint.
• Key Management: Keys are stored as secrets in the CI/CD environment, preventing exposure in the repository.

Step 3: Weekly Lighthouse Spot-Check

Performance regressions in SSG sites often stem from CSS framework updates or asset changes. A weekly Lighthouse CI check monitors Core Web Vitals and accessibility scores.

Implementation: Configure a GitHub Actions workflow to run Lighthouse CI on a schedule. Use a matrix strategy to test multiple sites and pages.

# .github/workflows/lighthouse-weekly.yml
name: Weekly Lighthouse Check
on:
  schedule:
    - cron: '30 4 * * 1' # Monday 04:30 UTC

jobs:
  lighthouse:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        site:
          - domain: aiappdex.com
            deep: /models/timm-vit-base-patch16-clip-224-openai/
          - domain: findindiegame.com
            deep: /games/dredge-1562430/
          - domain: ossfind.com
            deep: /alternatives/ghost/
    steps:
      - uses: actions/checkout@v4
      - name: Run Lighthouse CI
        uses: treosh/lighthouse-ci-action@v11
        with:
          urls: |
            https://${{ matrix.site.domain }}
            https://${{ matrix.site.domain }}${{ matrix.site.deep }}
          uploadArtifacts: true
          temporaryPublicStorage: true

Architecture Decisions:

• Scheduled Execution: Running weekly balances monitoring frequency with resource usage. Static sites change infrequently, so daily checks are wasteful.
• Matrix Strategy: Testing multiple sites and deep pages ensures coverage across the site structure, not just the homepage.
• Trend Monitoring: Results are uploaded to temporary storage for diffing. This allows tracking performance trends over time rather than setting hard gates that might block deploys for minor fluctuations.

Pitfall Guide

1. Redirect Loops in _redirects

   • Explanation: Misconfigured _redirects rules can create loops or block crawler access. For example, rewriting sitemap-index.xml to a sub-sitemap may work in browsers but fail for crawlers expecting a specific structure.
   • Fix: Audit _redirects syntax carefully. Use curl -I to check headers and ensure no unintended redirects. Validate sitemap structure programmatically.

2. IndexNow Key Verification Race

   • Explanation: IndexNow requires a key verification file (/<key>.txt) to be accessible. If this file is missing or blocked by headers, submissions fail with 403.
   • Fix: Ensure the key file is placed in the public/ directory and not excluded by _headers or _redirects. Verify accessibility before submission.

3. False Positives in Lighthouse

   • Explanation: Lighthouse scores can fluctuate due to network conditions or third-party scripts. Setting hard gates may block deploys for minor regressions.
   • Fix: Use Lighthouse as a trend monitor. Analyze score changes over time and investigate significant drops rather than blocking on every fluctuation.

4. Sitemap Count Drift

   • Explanation: If the data pipeline feeding the SSG breaks, the sitemap may deploy with fewer URLs. Without volume checks, this goes unnoticed.
   • Fix: Set minUrls thresholds per domain based on expected data volume. Alert if counts drop below thresholds.

5. Ignoring robots.txt

   • Explanation: A misconfigured robots.txt can block crawlers from accessing the sitemap or entire sections of the site.
   • Fix: Include robots.txt validation in the verification suite. Ensure it allows access to sitemaps and critical paths.

6. Cache Invalidation Delays

   • Explanation: Cloudflare Pages may serve cached content from previous deploys, masking changes or serving stale sitemaps.
   • Fix: Check the CF-Cache-Status header in verification scripts. Ensure cache is invalidated or bypassed during checks.

7. Build-Time vs Runtime Confusion

   • Explanation: For SSG sites with build-time data (e.g., Turso DB), runtime checks are irrelevant. Focusing on runtime API availability wastes effort.
   • Fix: Focus verification on static artifacts: sitemaps, assets, and performance metrics. Validate data pipelines at build time, not runtime.

Production Bundle

Action Checklist

• Define sitemap thresholds per domain based on expected data volume.
• Create IndexNow workflow dispatch trigger for post-deploy submission.
• Configure Lighthouse CI matrix for weekly performance monitoring.
• Audit _redirects and _headers files for crawler access conflicts.
• Verify robots.txt accessibility and rules.
• Set up Slack or email alerts for verification failures.
• Document verification procedures for team onboarding.

Decision Matrix

Scenario	Recommended Approach	Why	Cost Impact
High-Traffic SaaS	Hard Gates + E2E Tests	Revenue risk requires strict validation.	High (CI minutes, tooling)
Static Directory	Verification + Trends	SEO focus; minor regressions acceptable.	Low (Script maintenance)
Dev/Staging	Smoke Tests	Speed prioritized over thoroughness.	None
Pre-Revenue Site	Automated Verification	Prevents SEO damage with minimal overhead.	Low

Configuration Template

# .github/workflows/post-deploy-verify.yml
name: Post-Deploy Verification
on:
  workflow_dispatch:
    inputs:
      domain:
        description: 'Domain to verify'
        required: true
        type: choice
        options:
          - aiappdex.com
          - findindiegame.com
          - ossfind.com

jobs:
  verify:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
      - run: npm ci
      - name: Verify Sitemap
        run: npx ts-node scripts/verify-sitemap.ts
        env:
          DOMAIN: ${{ github.event.inputs.domain }}
      - name: Submit IndexNow
        run: npx ts-node scripts/submit-indexnow.ts
        env:
          DOMAIN: ${{ github.event.inputs.domain }}
          KEY: ${{ secrets[format('{0}_KEY', upper(github.event.inputs.domain))] }}

Quick Start Guide

1. Add Scripts: Create verify-sitemap.ts and submit-indexnow.ts in your scripts/ directory. Install dependencies (undici, xml2js).
2. Configure Secrets: Add IndexNow keys to your repository secrets.
3. Create Workflow: Add post-deploy-verify.yml to .github/workflows/.
4. Trigger Deploy: Run your deployment workflow.
5. Run Verification: Trigger post-deploy-verify.yml manually after deployment succeeds.
6. Monitor Results: Check workflow logs for verification status. Investigate any failures immediately.