Three post-deploy checks I run after every Cloudflare Pages build

Current Situation Analysis

Static site generation has fundamentally changed how we ship web applications. When using frameworks like Astro on edge CDNs such as Cloudflare Pages, the build step compiles HTML, CSS, and JSON at compile time. This eliminates runtime server overhead, but it introduces a distinct operational blind spot: build success does not guarantee external service readiness.

Most engineering teams treat CI/CD pipelines as complete once npm run build exits with code 0. This assumption breaks down in production for three reasons:

1. CDN Propagation Lag: Cloudflare Pages distributes assets across edge nodes asynchronously. External services querying your site during this window may receive stale responses, 404s, or misrouted traffic.
2. Routing Configuration Drift: Files like _redirects or headers are deployed alongside assets but are evaluated at the edge. A single misplaced rule can silently rewrite critical paths (e.g., sitemaps or verification files) without triggering a build error.
3. External Service Synchronization: Search engines, crawlers, and performance monitors operate on their own schedules. If your deployment pipeline doesn't explicitly notify them after propagation completes, indexing latency increases and performance baselines drift unnoticed.

The industry overlooks this because traditional testing focuses on unit coverage, linting, and build integrity. Post-deploy validation is frequently treated as manual or deferred until user reports surface. In reality, static CDN deployments have a narrow but critical failure surface: routing rules, asset reachability, and external API synchronization. Addressing these requires a lightweight, targeted verification layer that runs after the edge finishes distributing your build.

WOW Moment: Key Findings

Shifting from build-only validation to a post-deploy verification pipeline dramatically improves external service reliability without adding CI friction. The table below contrasts a traditional compilation-focused pipeline against a targeted post-deploy verification approach across three operational dimensions.

Approach	Time to Detect External Failures	False Positive Rate	Search Indexing Latency
Build-Only CI Pipeline	24–72 hours (relies on crawler discovery)	Low (catches syntax/build errors only)	High (depends on organic crawl frequency)
Post-Deploy Verification	<5 minutes (immediate edge validation)	Medium-High (requires careful threshold tuning)	Low (explicit IndexNow/submit triggers)

Why this matters: The verification pipeline decouples internal build health from external service readiness. By validating sitemap reachability, triggering IndexNow submissions, and sampling performance metrics after deployment, you convert silent CDN routing issues and indexing delays into actionable alerts. This approach is particularly valuable for static sites where runtime infrastructure is minimal, but external discoverability directly impacts traffic and SEO health.

Core Solution

The verification pipeline consists of three independent checks, each targeting a specific failure mode. They are designed to run sequentially after Cloudflare Pages confirms deployment, using a combination of HTTP validation, API submission, and scheduled performance sampling.

Step 1: Sitemap Integrity & Reachability Validation

Sitemaps are the primary signal for crawler discovery. A misconfigured _redirects file or missing asset can silently break indexation. The validation script performs two checks: HTTP status verification and content threshold validation.

// scripts/validate-sitemap.ts
import { fetch } from 'undici';
import { parseStringPromise } from 'xml2js';

interface SitemapConfig {
  domain: string;
  minUrlCount: number;
}

const TARGETS: SitemapConfig[] = [
  { domain: 'aiappdex.com', minUrlCount: 1000 },
  { domain: 'findindiegame.com', minUrlCount: 100 },
  { domain: 'ossfind.com', minUrlCount: 100 }
];

async function validateSitemap(config: SitemapConfig): Promise<void> {
  const indexUrl = `https://${config.domain}/sitemap-index.xml`;
  const subUrl = `https://${config.domain}/sitemap-0.xml`;

  // Check index reachability without following redirects
  const indexRes = await fetch(indexUrl, { redirect: 'manual' });
  if (indexRes.status !== 200) {
    throw new Error(`[${config.domain}] sitemap-index.xml returned ${indexRes.status}`);
  }

  // Validate sub-sitemap content and URL count
  const subRes = await fetch(subUrl);
  if (!subRes.ok) {
    throw new Error(`[${config.domain}] sitemap-0.xml unreachable: ${subRes.status}`);
  }

  const xmlText = await subRes.text();
  const parsed = await parseStringPromise(xmlText);
  const urlCount = parsed.urlset?.url?.length ?? 0;

  if (urlCount < config.minUrlCount) {
    throw new Error(
      `[${config.domain}] URL count ${urlCount} below threshold ${config.minUrlCount}. ETL pipeline may be stale.`
    );
  }

  console.log(`[${config.domain}] ✓ Sitemap valid (${urlCount} URLs)`);
}

async function main() {
  const results = await Promise.allSettled(TARGETS.map(validateSitemap));
  const failures = results.filter(r => r.status === 'rejected');
  
  if (failures.length > 0) {
    console.error('Sitemap validation failed:', failures.map(f => (f as PromiseRejectedResult).reason.message));
    process.exit(1);
  }
}

main().catch(err => {
  console.error('Validation runner crashed:', err);
  process.exit(2);
});

Architecture Rationale:

• Using redirect: 'manual' prevents silent redirect chains from masking routing misconfigurations. Crawlers and search consoles respect explicit 200 responses; redirects can delay or drop indexation.
• Parsing the sub-sitemap (sitemap-0.xml) validates that the underlying data pipeline actually generated content. A 200 on an empty sitemap is functionally identical to a 404 for SEO purposes.
• Promise.allSettled ensures all domains are evaluated even if one fails, providing a complete failure report instead of short-circuiting.

Step 2: IndexNow Batch Submission

IndexNow allows immediate URL submission to Bing, Yandex, Naver, and Seznam. The API requires live URLs and a verified key file. Submitting during the deployment window causes race conditions; the script must run after edge propagation completes.

// scripts/submit-indexnow.ts
import { fetch } from 'undici';
import { parseStringPromise } from 'xml2js';

const INDEXNOW_ENDPOINT = 'https://api.indexnow.org/IndexNow';
const API_KEYS: Record<string, string> = {
  'aiappdex.com': process.env.INDEXNOW_KEY_AIAPPDEX!,
  'findindiegame.com': process.env.INDEXNOW_KEY_FINDINDIE!,
  'ossfind.com': process.env.INDEXNOW_KEY_OSSFIND!
};

async function extractUrls(domain: string): Promise<string[]> {
  const res = await fetch(`https://${domain}/sitemap-0.xml`);
  if (!res.ok) throw new Error(`Failed to fetch sitemap for ${domain}`);
  
  const xml = await res.text();
  const parsed = await parseStringPromise(xml);
  return parsed.urlset.url.map((u: any) => u.loc[0]);
}

async function submitBatch(domain: string, urls: string[]): Promise<void> {
  const key = API_KEYS[domain];
  if (!key) throw new Error(`Missing IndexNow key for ${domain}`);

  const payload = {
    host: domain,
    key,
    keyLocation: `https://${domain}/${key}.txt`,
    urlList: urls
  };

  const res = await fetch(INDEXNOW_ENDPOINT, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify(payload)
  });

  if (res.status === 403) {
    throw new Error(`[${domain}] IndexNow 403: Verify ${key}.txt is deployed and accessible.`);
  }
  if (!res.ok) {
    throw new Error(`[${domain}] IndexNow submission failed: ${res.status}`);
  }

  console.log(`[${domain}] ✓ Submitted ${urls.length} URLs to IndexNow`);
}

async function main() {
  const domains = Object.keys(API_KEYS);
  for (const domain of domains) {
    try {
      const urls = await extractUrls(domain);
      await submitBatch(domain, urls);
    } catch (err) {
      console.error(`[${domain}] IndexNow failed:`, (err as Error).message);
      process.exit(1);
    }
  }
}

main();

Architecture Rationale:

• Sequential domain processing prevents API rate limiting and isolates failures per site.
• The keyLocation field must point to a publicly accessible file. If Cloudflare's _redirects or headers file blocks .txt assets, IndexNow returns 403. Validating this immediately after deploy catches configuration drift before indexing delays compound.
• Running this via workflow_dispatch or a post-deploy hook ensures URLs are actually live. Submitting during the build phase targets pre-deployment URLs, which IndexNow rejects or queues indefinitely.

Step 3: Scheduled Performance & Accessibility Sampling

Static sites rarely change at runtime, but dependency updates, CSS framework upgrades, or ad slot injections can degrade Core Web Vitals. Running Lighthouse on every deploy is wasteful; a weekly cron job provides trend data without blocking releases.

The workflow targets one homepage and one deep-link page per domain. It monitors Performance (<80), Cumulative Layout Shift (>0.1), and accessibility regressions. Results are stored for historical diffing rather than used as hard gates.

Architecture Rationale:

• Astro SSG with zero client-side JavaScript should maintain stable performance baselines. Deviations usually indicate asset bloat, unoptimized images, or layout shifts from dynamic ad components.
• Treating Lighthouse as a trend monitor rather than a deployment gate prevents false-positive blocks. A score drop from 94 to 88 rarely impacts user experience but would halt a strict CI pipeline.
• Sampling deep pages catches routing-specific issues that homepages miss, such as unoptimized media queries or component-level layout shifts.

Pitfall Guide

Pitfall	Explanation	Fix
Ignoring CDN Propagation Timing	Submitting to IndexNow or checking sitemaps before Cloudflare finishes edge distribution results in 404s or stale responses.	Add a 60–90 second delay after deployment success, or trigger verification via workflow_dispatch only after the deploy job completes.
Using _redirects as a Band-Aid	Redirect rules can silently rewrite critical paths like sitemaps or verification files, breaking crawler discovery without build errors.	Validate _redirects syntax in CI using a linter. Prefer framework-level routing or headers files for edge configuration.
Treating Lighthouse as a Hard Gate	Blocking deploys for minor score fluctuations creates CI friction and encourages score manipulation rather than real improvements.	Use soft thresholds, track trends over time, and only block if regression exceeds a defined delta (e.g., >5 points drop).
IndexNow Key Verification Gaps	Forgetting to deploy /<key>.txt or placing it in a directory blocked by _redirects causes 403 responses.	Include the key file in your static assets folder. Add a pre-submit check that fetches the key URL and asserts 200.
Confusing Build-Time vs Runtime Data	Checking database availability or API endpoints for SSG sites is unnecessary; data is baked at compile time.	Scope runtime checks to dynamic routes only. For static sites, validate build output integrity instead.
XML Parsing Fragility	Assuming sitemap structure never changes leads to broken validators when frameworks update namespace formats.	Use robust XML parsers with namespace handling. Validate against the official sitemap schema and handle missing nodes gracefully.
Over-Parallelizing External API Calls	Submitting hundreds of URLs to IndexNow simultaneously can trigger rate limits or IP throttling.	Batch URLs per domain, process sequentially, and implement exponential backoff for 429 responses.

Production Bundle

Action Checklist

• Verify sitemap reachability: Assert HTTP 200 on sitemap-index.xml without following redirects
• Validate sitemap content: Parse sitemap-0.xml and enforce minimum URL count thresholds
• Confirm IndexNow key deployment: Fetch /<key>.txt and verify public accessibility before submission
• Submit URLs sequentially: Process one domain at a time to avoid API rate limits
• Schedule Lighthouse sampling: Run weekly on representative pages, not on every deploy
• Store performance baselines: Upload Lighthouse reports to persistent storage for historical diffing
• Isolate verification from build: Run post-deploy checks in a separate workflow or job to prevent CI blocking

Decision Matrix

Scenario	Recommended Approach	Why	Cost Impact
Static SSG site (Astro, Next.js static)	Post-deploy verification + weekly Lighthouse	Runtime is pre-built; focus on routing, indexing, and asset integrity	Low (minimal CI minutes, no server costs)
Dynamic SSR/ISR site	Build validation + runtime health checks + per-deploy Lighthouse	Server routes and API calls require live endpoint verification	Medium (higher CI usage, potential monitoring costs)
High-traffic e-commerce	Full E2E suite + synthetic monitoring + strict Lighthouse gates	User conversion directly tied to performance and routing accuracy	High (tooling, monitoring, longer CI pipelines)
Pre-revenue / low-traffic directory	Lightweight post-deploy checks + trend monitoring	SEO and indexing matter more than strict performance gates	Minimal (free tier CI, manual triggers acceptable)

Configuration Template

# .github/workflows/post-deploy-verify.yml
name: Post-Deploy Verification
on:
  workflow_dispatch:
    inputs:
      environment:
        description: 'Target environment'
        required: true
        default: 'production'
        type: choice
        options:
          - production

jobs:
  validate-sitemap:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm ci
      - run: npx tsx scripts/validate-sitemap.ts
        env:
          NODE_ENV: production

  submit-indexnow:
    needs: validate-sitemap
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm ci
      - run: npx tsx scripts/submit-indexnow.ts
        env:
          INDEXNOW_KEY_AIAPPDEX: ${{ secrets.INDEXNOW_KEY_AIAPPDEX }}
          INDEXNOW_KEY_FINDINDIE: ${{ secrets.INDEXNOW_KEY_FINDINDIE }}
          INDEXNOW_KEY_OSSFIND: ${{ secrets.INDEXNOW_KEY_OSSFIND }}

  lighthouse-spotcheck:
    if: github.event.schedule == 'weekly'
    runs-on: ubuntu-latest
    strategy:
      matrix:
        site:
          - domain: aiappdex.com
            sample: /models/timm-vit-base-patch16-clip-224-openai/
          - domain: findindiegame.com
            sample: /games/dredge-1562430/
          - domain: ossfind.com
            sample: /alternatives/ghost/
    steps:
      - uses: actions/checkout@v4
      - uses: treosh/lighthouse-ci-action@v11
        with:
          urls: |
            https://${{ matrix.site.domain }}
            https://${{ matrix.site.domain }}${{ matrix.site.sample }}
          uploadArtifacts: true
          temporaryPublicStorage: true
          configPath: .lighthouserc.json

Quick Start Guide

1. Install dependencies: Add undici, xml2js, and tsx to your project. These provide fast HTTP fetching, robust XML parsing, and TypeScript execution without compilation overhead.
2. Create verification scripts: Copy the sitemap validation and IndexNow submission modules into a scripts/ directory. Replace domain arrays and API keys with your environment variables.
3. Configure GitHub Actions: Add the workflow template to .github/workflows/. Set workflow_dispatch as the trigger to run manually after Cloudflare Pages confirms deployment.
4. Store secrets securely: Add your IndexNow API keys to your repository or organization secrets. Never hardcode keys in source control.
5. Schedule Lighthouse sampling: Use a separate cron workflow or modify the trigger to run weekly. Configure .lighthouserc.json to set soft thresholds and enable artifact storage for trend analysis.

This pipeline replaces guesswork with deterministic validation. By targeting the exact failure modes that occur after static deployments—routing drift, indexing delays, and performance regression—you maintain external service health without sacrificing deployment velocity.