Three post-deploy checks I run after every Cloudflare Pages build
Current Situation Analysis
Static site generation has fundamentally changed how we ship web applications. When using frameworks like Astro on edge CDNs such as Cloudflare Pages, the build step compiles HTML, CSS, and JSON at compile time. This eliminates runtime server overhead, but it introduces a distinct operational blind spot: build success does not guarantee external service readiness.
Most engineering teams treat CI/CD pipelines as complete once npm run build exits with code 0. This assumption breaks down in production for three reasons:
- CDN Propagation Lag: Cloudflare Pages distributes assets across edge nodes asynchronously. External services querying your site during this window may receive stale responses, 404s, or misrouted traffic.
- Routing Configuration Drift: Files like
_redirectsorheadersare deployed alongside assets but are evaluated at the edge. A single misplaced rule can silently rewrite critical paths (e.g., sitemaps or verification files) without triggering a build error. - External Service Synchronization: Search engines, crawlers, and performance monitors operate on their own schedules. If your deployment pipeline doesn't explicitly notify them after propagation completes, indexing latency increases and performance baselines drift unnoticed.
The industry overlooks this because traditional testing focuses on unit coverage, linting, and build integrity. Post-deploy validation is frequently treated as manual or deferred until user reports surface. In reality, static CDN deployments have a narrow but critical failure surface: routing rules, asset reachability, and external API synchronization. Addressing these requires a lightweight, targeted verification layer that runs after the edge finishes distributing your build.
WOW Moment: Key Findings
Shifting from build-only validation to a post-deploy verification pipeline dramatically improves external service reliability without adding CI friction. The table below contrasts a traditional compilation-focused pipeline against a targeted post-deploy verification approach across three operational dimensions.
| Approach | Time to Detect External Failures | False Positive Rate | Search Indexing Latency |
|---|---|---|---|
| Build-Only CI Pipeline | 24â72 hours (relies on crawler discovery) | Low (catches syntax/build errors only) | High (depends on organic crawl frequency) |
| Post-Deploy Verification | <5 minutes (immediate edge validation) | Medium-High (requires careful threshold tuning) | Low (explicit IndexNow/submit triggers) |
Why this matters: The verification pipeline decouples internal build health from external service readiness. By validating sitemap reachability, triggering IndexNow submissions, and sampling performance metrics after deployment, you convert silent CDN routing issues and indexing delays into actionable alerts. This approach is particularly valuable for static sites where runtime infrastructure is minimal, but external discoverability directly impacts traffic and SEO health.
Core Solution
The verification pipeline consists of three independent checks, each targeting a specific failure mode. They are designed to run sequentially after Cloudflare Pages confirms deployment, using a combination of HTTP validation, API submission, and scheduled performance sampling.
Step 1: Sitemap Integrity & Reachability Validation
Sitemaps are the primary signal for crawler discovery. A misconfigured _redirects file or missing asset can silently break indexation. The validation script performs two checks: HTTP status verification and content threshold validation.
// scripts/validate-sitemap.ts
import { fetch } from 'undici';
import { parseStringPromise } from 'xml2js';
interface SitemapConfig {
domain: string;
minUrlCount: number;
}
const TARGETS: SitemapConfig[] = [
{ domain: 'aiappdex.com', minUrlCount: 1000 },
{ domain: 'findindiegame.com', minUrlCount: 100 },
{ domain: 'ossfind.com', minUrlCount: 100 }
];
async function validateSitemap(config: SitemapConfig): Promise<void> {
const indexUrl = `https://${config.domain}/sitemap-index.xml`;
const subUrl = `https://${config.domain}/sitemap-0.xml`;
// Check index reachability without following redirects
const indexRes = await fetch(indexUrl, { redirect: 'manual' });
if (indexRes.status !== 200) {
throw new Error(`[${config.domain}] sitemap-index.xml returned ${indexRes.status}`);
}
// Validate sub-sitemap content and URL count
const subRes = await fetch(subUrl);
if (!subRes.ok) {
throw new Error(`[${config.domain}] sitemap-0.xml unreachable: ${subRes.status}`);
}
const xmlText = await subRes.text();
const parsed = await parseStringPromise(xmlText);
const urlCount = parsed.urlset?.url?.length ?? 0;
if (urlCount < config.minUrlCount) {
throw new Error(
`[${config.domain}] URL count ${urlCount} below threshold ${config.minUrlCount}. ETL pipeline may be stale.`
);
}
console.log(`[${config.domain}] â Sitemap valid (${urlCount} URLs)`);
}
async function main() {
const results = await Promise.allSettled(TARGETS.map(validateSitemap));
const failures = results.filter(r => r.status === 'rejected');
if (failures.length > 0) {
console.error('Sitemap validation failed:', failures.map(f => (f as PromiseRejectedResult).reason.message));
process.exit(1);
}
}
main().catch(err => {
console.error('Validation runner crashed:', err);
process.exit(2);
});
Architecture Rationale:
- Using
redirect: 'manual'prevents silent redirect chains from masking routing misconfigurations. Crawlers and search consoles respect explicit 200 responses; redirects can delay or drop indexation. - Parsing the sub-sitemap (
sitemap-0.xml) validates that the underlying data pipeline actually generated content. A 200 on an empty sitemap is functionally identical to a 404 for SEO purposes. Promise.allSettledensures all domains are evaluated even if one fails, providing a complete failure report instead of short-circuiting.
Step 2: IndexNow Batch Submission
IndexNow allows immediate URL submission to Bing, Yandex, Naver, and Seznam. The API requires live URLs and a verified key file. Submitting during the deployment window causes race conditions; the script must run after edge propagation completes.
// scripts/submit-indexnow.ts
import { fetch } from 'undici';
import { parseStringPromise } from 'xml2js';
const INDEXNOW_ENDPOINT = 'https://api.indexnow.org/IndexNow';
const API_KEYS: Record<string, string> = {
'aiappdex.com': process.env.INDEXNOW_KEY_AIAPPDEX!,
'findindiegame.com': process.env.INDEXNOW_KEY_FINDINDIE!,
'ossfind.com': process.env.INDEXNOW_KEY_OSSFIND!
};
async function extractUrls(domain: string): Promise<string[]> {
const res = await fetch(`https://${domain}/sitemap-0.xml`);
if (!res.ok) throw new Error(`Failed to fetch sitemap for ${domain}`);
const xml = await res.text();
const parsed = await parseStringPromise(xml);
return parsed.urlset.url.map((u: any) => u.loc[0]);
}
async function submitBatch(domain: string, urls: string[]): Promise<void> {
const key = API_KEYS[domain];
if (!key) throw new Error(`Missing IndexNow key for ${domain}`);
const payload = {
host: domain,
key,
keyLocation: `https://${domain}/${key}.txt`,
urlList: urls
};
const res = await fetch(INDEXNOW_ENDPOINT, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(payload)
});
if (res.status === 403) {
throw new Error(`[${domain}] IndexNow 403: Verify ${key}.txt is deployed and accessible.`);
}
if (!res.ok) {
throw new Error(`[${domain}] IndexNow submission failed: ${res.status}`);
}
console.log(`[${domain}] â Submitted ${urls.length} URLs to IndexNow`);
}
async function main() {
const domains = Object.keys(API_KEYS);
for (const domain of domains) {
try {
const urls = await extractUrls(domain);
await submitBatch(domain, urls);
} catch (err) {
console.error(`[${domain}] IndexNow failed:`, (err as Error).message);
process.exit(1);
}
}
}
main();
Architecture Rationale:
- Sequential domain processing prevents API rate limiting and isolates failures per site.
- The
keyLocationfield must point to a publicly accessible file. If Cloudflare's_redirectsorheadersfile blocks.txtassets, IndexNow returns 403. Validating this immediately after deploy catches configuration drift before indexing delays compound. - Running this via
workflow_dispatchor a post-deploy hook ensures URLs are actually live. Submitting during the build phase targets pre-deployment URLs, which IndexNow rejects or queues indefinitely.
Step 3: Scheduled Performance & Accessibility Sampling
Static sites rarely change at runtime, but dependency updates, CSS framework upgrades, or ad slot injections can degrade Core Web Vitals. Running Lighthouse on every deploy is wasteful; a weekly cron job provides trend data without blocking releases.
The workflow targets one homepage and one deep-link page per domain. It monitors Performance (<80), Cumulative Layout Shift (>0.1), and accessibility regressions. Results are stored for historical diffing rather than used as hard gates.
Architecture Rationale:
- Astro SSG with zero client-side JavaScript should maintain stable performance baselines. Deviations usually indicate asset bloat, unoptimized images, or layout shifts from dynamic ad components.
- Treating Lighthouse as a trend monitor rather than a deployment gate prevents false-positive blocks. A score drop from 94 to 88 rarely impacts user experience but would halt a strict CI pipeline.
- Sampling deep pages catches routing-specific issues that homepages miss, such as unoptimized media queries or component-level layout shifts.
Pitfall Guide
| Pitfall | Explanation | Fix |
|---|---|---|
| Ignoring CDN Propagation Timing | Submitting to IndexNow or checking sitemaps before Cloudflare finishes edge distribution results in 404s or stale responses. | Add a 60â90 second delay after deployment success, or trigger verification via workflow_dispatch only after the deploy job completes. |
Using _redirects as a Band-Aid |
Redirect rules can silently rewrite critical paths like sitemaps or verification files, breaking crawler discovery without build errors. | Validate _redirects syntax in CI using a linter. Prefer framework-level routing or headers files for edge configuration. |
| Treating Lighthouse as a Hard Gate | Blocking deploys for minor score fluctuations creates CI friction and encourages score manipulation rather than real improvements. | Use soft thresholds, track trends over time, and only block if regression exceeds a defined delta (e.g., >5 points drop). |
| IndexNow Key Verification Gaps | Forgetting to deploy /<key>.txt or placing it in a directory blocked by _redirects causes 403 responses. |
Include the key file in your static assets folder. Add a pre-submit check that fetches the key URL and asserts 200. |
| Confusing Build-Time vs Runtime Data | Checking database availability or API endpoints for SSG sites is unnecessary; data is baked at compile time. | Scope runtime checks to dynamic routes only. For static sites, validate build output integrity instead. |
| XML Parsing Fragility | Assuming sitemap structure never changes leads to broken validators when frameworks update namespace formats. | Use robust XML parsers with namespace handling. Validate against the official sitemap schema and handle missing nodes gracefully. |
| Over-Parallelizing External API Calls | Submitting hundreds of URLs to IndexNow simultaneously can trigger rate limits or IP throttling. | Batch URLs per domain, process sequentially, and implement exponential backoff for 429 responses. |
Production Bundle
Action Checklist
- Verify sitemap reachability: Assert HTTP 200 on
sitemap-index.xmlwithout following redirects - Validate sitemap content: Parse
sitemap-0.xmland enforce minimum URL count thresholds - Confirm IndexNow key deployment: Fetch
/<key>.txtand verify public accessibility before submission - Submit URLs sequentially: Process one domain at a time to avoid API rate limits
- Schedule Lighthouse sampling: Run weekly on representative pages, not on every deploy
- Store performance baselines: Upload Lighthouse reports to persistent storage for historical diffing
- Isolate verification from build: Run post-deploy checks in a separate workflow or job to prevent CI blocking
Decision Matrix
| Scenario | Recommended Approach | Why | Cost Impact |
|---|---|---|---|
| Static SSG site (Astro, Next.js static) | Post-deploy verification + weekly Lighthouse | Runtime is pre-built; focus on routing, indexing, and asset integrity | Low (minimal CI minutes, no server costs) |
| Dynamic SSR/ISR site | Build validation + runtime health checks + per-deploy Lighthouse | Server routes and API calls require live endpoint verification | Medium (higher CI usage, potential monitoring costs) |
| High-traffic e-commerce | Full E2E suite + synthetic monitoring + strict Lighthouse gates | User conversion directly tied to performance and routing accuracy | High (tooling, monitoring, longer CI pipelines) |
| Pre-revenue / low-traffic directory | Lightweight post-deploy checks + trend monitoring | SEO and indexing matter more than strict performance gates | Minimal (free tier CI, manual triggers acceptable) |
Configuration Template
# .github/workflows/post-deploy-verify.yml
name: Post-Deploy Verification
on:
workflow_dispatch:
inputs:
environment:
description: 'Target environment'
required: true
default: 'production'
type: choice
options:
- production
jobs:
validate-sitemap:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
- run: npm ci
- run: npx tsx scripts/validate-sitemap.ts
env:
NODE_ENV: production
submit-indexnow:
needs: validate-sitemap
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
- run: npm ci
- run: npx tsx scripts/submit-indexnow.ts
env:
INDEXNOW_KEY_AIAPPDEX: ${{ secrets.INDEXNOW_KEY_AIAPPDEX }}
INDEXNOW_KEY_FINDINDIE: ${{ secrets.INDEXNOW_KEY_FINDINDIE }}
INDEXNOW_KEY_OSSFIND: ${{ secrets.INDEXNOW_KEY_OSSFIND }}
lighthouse-spotcheck:
if: github.event.schedule == 'weekly'
runs-on: ubuntu-latest
strategy:
matrix:
site:
- domain: aiappdex.com
sample: /models/timm-vit-base-patch16-clip-224-openai/
- domain: findindiegame.com
sample: /games/dredge-1562430/
- domain: ossfind.com
sample: /alternatives/ghost/
steps:
- uses: actions/checkout@v4
- uses: treosh/lighthouse-ci-action@v11
with:
urls: |
https://${{ matrix.site.domain }}
https://${{ matrix.site.domain }}${{ matrix.site.sample }}
uploadArtifacts: true
temporaryPublicStorage: true
configPath: .lighthouserc.json
Quick Start Guide
- Install dependencies: Add
undici,xml2js, andtsxto your project. These provide fast HTTP fetching, robust XML parsing, and TypeScript execution without compilation overhead. - Create verification scripts: Copy the sitemap validation and IndexNow submission modules into a
scripts/directory. Replace domain arrays and API keys with your environment variables. - Configure GitHub Actions: Add the workflow template to
.github/workflows/. Setworkflow_dispatchas the trigger to run manually after Cloudflare Pages confirms deployment. - Store secrets securely: Add your IndexNow API keys to your repository or organization secrets. Never hardcode keys in source control.
- Schedule Lighthouse sampling: Use a separate cron workflow or modify the trigger to run weekly. Configure
.lighthouserc.jsonto set soft thresholds and enable artifact storage for trend analysis.
This pipeline replaces guesswork with deterministic validation. By targeting the exact failure modes that occur after static deploymentsârouting drift, indexing delays, and performance regressionâyou maintain external service health without sacrificing deployment velocity.
Mid-Year Sale â Unlock Full Article
Base plan from just $4.99/mo or $49/yr
Sign in to read the full article and unlock all tutorials.
Sign In / Register â Start Free Trial7-day free trial · Cancel anytime · 30-day money-back
