.metadata.distributions(): dist_info = dist._path.parent / f"{dist.metadata['Name']}-{dist.version}.dist-info" if not dist_info.exists(): continue

    record_file = dist_info / "RECORD"
    if record_file.exists():
        for line in record_file.read_text().splitlines():
            file_path = line.split(",")[0]
            if any(keyword in file_path.lower() for keyword in ["setup", "install", "hook", "post"]):
                full_path = env_root / file_path
                if full_path.exists():
                    content_hash = hashlib.sha256(full_path.read_bytes()).hexdigest()
                    findings.append({
                        "package": dist.metadata["Name"],
                        "file": str(full_path),
                        "sha256": content_hash
                    })
return findings

if name == "main": import sys env_path = pathlib.Path(sys.argv[1]) if len(sys.argv) > 1 else pathlib.Path(sys.prefix) results = scan_distribution_hooks(env_path) for r in results: print(f"[ALERT] {r['package']} -> {r['file']} ({r['sha256'][:12]}...)")

**Why this choice:** Direct inspection of `.dist-info/RECORD` files bypasses `pip` abstraction layers and reveals hidden execution scripts. Hashing the files allows you to cross-reference against known-safe baselines or threat intelligence feeds.

### Step 3: Enforce Egress Filtering for Credential Exfiltration

Even if a payload executes, it cannot exfiltrate data if network egress is restricted. ML training hosts should operate under zero-trust networking principles.

**Architecture Decision:** Deploy an egress proxy or network policy that whitelists only required endpoints (PyPI, cloud storage, model registries). Block all outbound HTTPS to unknown domains. Log and alert on any connection attempts to non-whitelisted IPs.

**Why this choice:** Credential harvesting relies on outbound HTTPS calls. Egress filtering neutralizes the exfiltration vector regardless of payload sophistication. It also provides network-level telemetry for detecting compromised hosts.

## Pitfall Guide

### 1. Assuming Ephemeral Environments Are Low-Risk
**Explanation:** Training pods are spun up and torn down frequently, leading teams to skip security hardening. In reality, these hosts inherit IAM roles, data lake credentials, and artifact store permissions that persist beyond the pod lifecycle.
**Fix:** Treat every training environment as a production host. Apply least-privilege IAM policies, rotate credentials on pod creation, and enforce network segmentation.

### 2. Relying Solely on Import-Time SCA Tools
**Explanation:** Tools that scan `import` statements or dependency graphs miss code executed during `pip install`. Install-time hooks run before any application logic, rendering import-time scanners blind to the initial compromise.
**Fix:** Integrate install-time execution monitoring into CI/CD. Use sandboxed dependency resolution steps that log or block `setup.py`/build hook execution.

### 3. Ignoring Namespace-Adjacent Typosquats
**Explanation:** PyPI allows packages like `pytorch-lightning-gpu` or `lightning-utilities` to coexist with official releases. Engineers often install these from community tutorials without verifying the publisher.
**Fix:** Maintain an allowlist of approved package names and publishers. Use private package proxies (e.g., Artifactory, Nexus) that cache and verify only approved distributions.

### 4. Rotating Tokens Without Revoking Active Sessions
**Explanation:** After detecting credential exposure, teams often rotate API keys but forget to invalidate existing sessions or refresh tokens. Attackers retain access until sessions naturally expire.
**Fix:** Implement token rotation with immediate session revocation. Use short-lived credentials (e.g., AWS STS, OIDC tokens) that automatically expire and cannot be reused.

### 5. Skipping Egress Network Policies
**Explanation:** Training hosts often have unrestricted outbound internet access for convenience. This allows malicious payloads to exfiltrate data to arbitrary endpoints without detection.
**Fix:** Deploy egress filtering at the cluster or pod level. Whitelist only essential endpoints (PyPI, cloud storage, model registries). Block and log all other outbound traffic.

### 6. Trusting `latest` Tags in Container Builds
**Explanation:** Dockerfiles that use `RUN pip install pytorch-lightning` without version pinning silently pull the newest distribution on every build. A single poisoned release compromises all downstream images.
**Fix:** Pin exact versions and hashes in Dockerfiles. Use multi-stage builds to separate dependency resolution from runtime, and scan base images before deployment.

### 7. Overlooking `setup.py` vs `pyproject.toml` Execution Differences
**Explanation:** Modern Python packaging uses `pyproject.toml`, but many packages still rely on `setup.py` for build hooks. Security scanners often treat them identically, missing execution differences in isolation and privilege escalation.
**Fix:** Audit build system declarations. Prefer `pyproject.toml` with isolated builds (`--no-build-isolation` disabled). Monitor both legacy and modern build hooks during dependency resolution.

## Production Bundle

### Action Checklist
- [ ] Audit all ML training environments for `lightning-*` namespace packages using distribution metadata scanning
- [ ] Migrate from version pinning to hash-verified dependency locking using `pip-compile --generate-hashes`
- [ ] Integrate install-time execution monitoring into CI/CD pipelines to log or block `setup.py`/build hooks
- [ ] Deploy egress network policies that whitelist only PyPI, cloud storage, and model registry endpoints
- [ ] Rotate all exposed credentials (AWS, W&B, HF) and enforce short-lived token issuance
- [ ] Replace `latest` tags in Dockerfiles with exact version and hash pins
- [ ] Implement private package proxy caching to intercept and verify PyPI distributions before deployment
- [ ] Schedule quarterly supply chain audits using automated SCA tools with install-time execution coverage

### Decision Matrix

| Scenario | Recommended Approach | Why | Cost Impact |
|----------|---------------------|-----|-------------|
| Small team, rapid prototyping | Version pinning + basic SCA | Low overhead, catches known vulnerabilities quickly | Low |
| Production ML pipeline, cloud GPUs | Hash verification + egress filtering | Cryptographic integrity prevents PyPI tampering; network policies block exfiltration | Medium |
| Enterprise ML platform, multi-tenant | Private package proxy + install-time monitoring | Centralized distribution control; execution sandboxing prevents hook exploitation | High |
| Regulated industry (healthcare/finance) | All of the above + SBOM generation + audit logging | Compliance requires full supply chain traceability and cryptographic verification | High |

### Configuration Template

**requirements.in**

pytorch-lightning==2.2.1 torch==2.1.0 transformers==4.35.0

**Generate Locked Requirements**
```bash
pip install pip-tools
pip-compile --generate-hashes --output-file requirements-locked.txt requirements.in

Install with Hash Verification

pip install --require-hashes -r requirements-locked.txt

GitHub Actions CI Snippet

name: ML Dependency Security Scan
on: [push, pull_request]
jobs:
  dependency-audit:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'
      - name: Install pip-tools
        run: pip install pip-tools safety
      - name: Verify Hashes
        run: pip install --require-hashes -r requirements-locked.txt
      - name: Run SCA Scan
        run: safety check -r requirements-locked.txt --json
      - name: Upload SBOM
        uses: actions/upload-artifact@v4
        with:
          name: dependency-sbom
          path: safety-report.json

Quick Start Guide

1. Inventory Dependencies: Run pip list --format=freeze > requirements.in in your training environment to capture current direct dependencies.
2. Generate Hash-Locked File: Execute pip-compile --generate-hashes --output-file requirements-locked.txt requirements.in to resolve transitive dependencies and embed SHA-256 digests.
3. Validate Installation: Run pip install --require-hashes -r requirements-locked.txt in a clean virtual environment. The installer will reject any package with a mismatched digest.
4. Scan for Execution Hooks: Use the audit_site_packages.py script against your environment path to identify any setup.py or post-install artifacts. Cross-reference hashes against known-safe baselines.
5. Enforce Egress Policies: Configure your cluster network policies or cloud security groups to allow outbound HTTPS only to pypi.org, *.cloudprovider.com, and your model registry. Block all other destinations and enable logging.