66. K-Means Clustering: Find Groups Without Labels

from sklearn.preprocessing import StandardScaler
import numpy as np

# Simulate raw feature matrix with disparate scales
raw_observations = np.array([
    [25, 50000, 2],   # Age, Income, Visits
    [45, 120000, 15],
    [30, 60000, 5],
    [55, 150000, 20]
])

scaler = StandardScaler()
normalized_matrix = scaler.fit_transform(raw_observations)

# Verification: Mean ~0, Std ~1
assert np.allclose(normalized_matrix.mean(axis=0), 0, atol=1e-7)
assert np.allclose(normalized_matrix.std(axis=0), 1, atol=1e-7)

from sklearn.cluster import KMeans as SklearnKMeans
from sklearn.metrics import silhouette_score

class PartitionEngine:
    """
    Encapsulates K-Means logic with explicit convergence tracking.
    """
    def __init__(self, n_partitions: int, max_cycles: int = 300, tolerance: float = 1e-4):
        self.n_partitions = n_partitions
        self.max_cycles = max_cycles
        self.tolerance = tolerance
        self.centroids = None
        self.assignments = None
        self.inertia = None

    def fit(self, data: np.ndarray) -> 'PartitionEngine':
        # Initialize using sklearn's robust K-Means++ logic
        model = SklearnKMeans(
            n_clusters=self.n_partitions,
            init='k-means++',
            n_init=10,
            max_iter=self.max_cycles,
            random_state=42
        )
        model.fit(data)
        
        self.centroids = model.cluster_centers_
        self.assignments = model.labels_
        self.inertia = model.inertia_
        self.n_cycles = model.n_iter_
        
        return self

    def predict(self, data: np.ndarray) -> np.ndarray:
        if self.centroids is None:
            raise ValueError("Model not fitted. Call fit() first.")
        
        # Compute distances to centroids
        distances = np.linalg.norm(
            data[:, np.newaxis, :] - self.centroids[np.newaxis, :, :], 
            axis=2
        )
        return np.argmin(distances, axis=1)

# Execution
engine = PartitionEngine(n_partitions=3)
engine.fit(normalized_matrix)

print(f"Converged in {engine.n_cycles} cycles.")
print(f"Final Inertia: {engine.inertia:.2f}")

K`) requires balancing compactness against interpretability. Relying on a single metric is risky. The production standard is to cross-validate using both Inertia and Silhouette Score.

def evaluate_partition_range(data: np.ndarray, k_min: int, k_max: int):
    """
    Scans K range and returns metrics for decision making.
    """
    results = []
    
    for k in range(k_min, k_max + 1):
        model = SklearnKMeans(
            n_clusters=k, 
            init='k-means++', 
            n_init=10, 
            random_state=42
        )
        labels = model.fit_predict(data)
        
        results.append({
            'k': k,
            'inertia': model.inertia_,
            'silhouette': silhouette_score(data, labels)
        })
        
    return results

# Analysis
metrics = evaluate_partition_range(normalized_matrix, k_min=2, k_max=6)

for entry in metrics:
    print(f"K={entry['k']} | Inertia={entry['inertia']:.2f} | Silhouette={entry['silhouette']:.3f}")

Interpretation Strategy:

• Inertia Curve: Look for the "elbow" where the rate of decrease sharply declines.
• Silhouette Curve: Identify peaks where clusters are well-separated.
• Consensus: Prefer K values where both metrics indicate stability. If they conflict, prioritize Silhouette for separation quality, but validate with domain knowledge.

4. Business Segmentation Application

Clustering gains value when groups map to actionable business entities. The following pattern profiles clusters to generate insights.

import pandas as pd

# Synthetic customer dataset
customer_data = pd.DataFrame({
    'recency_days': [5, 120, 15, 300, 10, 45],
    'frequency_orders': [12, 1, 8, 0, 15, 6],
    'monetary_value': [450.0, 20.0, 300.0, 0.0, 600.0, 150.0]
})

# Pipeline: Scale -> Cluster -> Profile
scaler_cust = StandardScaler()
X_cust = scaler_cust.fit_transform(customer_data)

cluster_model = SklearnKMeans(n_clusters=3, init='k-means++', n_init=10, random_state=42)
customer_data['segment_id'] = cluster_model.fit_predict(X_cust)

# Generate segment profiles
profiles = customer_data.groupby('segment_id').agg({
    'recency_days': 'mean',
    'frequency_orders': 'mean',
    'monetary_value': 'mean'
}).round(2)

print("Segment Profiles:")
print(profiles)

Output Interpretation:

• Segment 0: Low recency, high frequency. Active High-Value Users.
• Segment 1: High recency, low frequency. At-Risk Churn Candidates.
• Segment 2: Moderate recency, moderate frequency. Steady Regulars.

This profiling transforms mathematical assignments into marketing strategies without manual labeling.

Pitfall Guide

Production clustering fails when assumptions are violated. The following pitfalls represent common failure modes and their remediations.

1. Scale Dominance

   • Explanation: Features with larger ranges (e.g., income vs. age) disproportionately influence distance calculations, causing clusters to form along high-variance axes while ignoring others.
   • Fix: Always apply StandardScaler or MinMaxScaler. Verify feature variances post-scaling.

2. Outlier Sensitivity

   • Explanation: Centroids are arithmetic means. Extreme outliers pull centroids away from the dense core of the cluster, distorting boundaries and inflating inertia.
   • Fix: Detect and cap outliers using IQR or Z-score methods before clustering. Alternatively, use K-Medians for robustness against outliers.

3. Non-Spherical Geometry

   • Explanation: K-Means assumes convex, spherical clusters. It fails on crescent shapes, concentric rings, or elongated manifolds, often splitting a single logical group into multiple artificial clusters.
   • Fix: Visualize data in 2D/3D projections. If geometry is non-spherical, switch to DBSCAN or Spectral Clustering.

4. Curse of Dimensionality

   • Explanation: In high-dimensional spaces, Euclidean distances converge to similar values, making proximity meaningless. Clusters become indistinguishable.
   • Fix: Apply dimensionality reduction (PCA, UMAP) before clustering. Retain components that explain >80% variance.

5. Categorical Data Misuse

   • Explanation: K-Means requires numeric input. Encoding categories as integers introduces false ordinal relationships (e.g., Category 3 is "greater" than Category 1), corrupting distance metrics.
   • Fix: Use One-Hot Encoding for nominal data, or employ K-Prototypes for mixed data types. Embeddings are preferred for high-cardinality categories.

6. Single-Run Variance

   • Explanation: Running K-Means once with n_init=1 risks converging to a local minimum. Results may vary significantly between runs, causing instability in production dashboards.
   • Fix: Set n_init >= 10. This runs the algorithm multiple times and selects the best result, ensuring reproducibility.

7. Blind K Selection

   • Explanation: Choosing K based on intuition rather than data structure leads to over-segmentation (noise) or under-segmentation (loss of signal).
   • Fix: Use the Elbow and Silhouette methods. Validate the chosen K by checking if segments are distinct in business terms, not just metrics.

Production Bundle

Action Checklist

• Scale Features: Apply StandardScaler to all numeric inputs. Verify mean=0, std=1.
• Handle Outliers: Trim or winsorize extreme values to prevent centroid drift.
• Validate K: Run Elbow and Silhouette analysis. Select K where metrics agree.
• Configure Init: Use init='k-means++' and n_init=10 for stability.
• Profile Segments: Generate mean/median statistics per cluster to interpret business meaning.
• Check Geometry: Visualize clusters. If non-spherical, consider alternative algorithms.
• Monitor Drift: Track cluster distribution shifts over time. Retrain if segments degrade.

Decision Matrix

Scenario	Recommended Approach	Why	Cost Impact
Spherical, Scaled Data	K-Means (n_init=10)	Optimal speed and compactness for convex clusters.	Low (Fast convergence)
Non-Spherical / Density Varying	DBSCAN	Captures arbitrary shapes and handles noise points.	Medium (Parameter tuning required)
High-Dimensional Data	PCA + K-Means	Reduces dimensionality to restore distance validity.	Medium (Compute overhead for PCA)
Mixed Numeric/Categorical	K-Prototypes	Handles both data types without encoding artifacts.	High (Slower convergence)
Outlier-Heavy Data	K-Medians	Robust to outliers; centroids are medians, not means.	Medium (Higher compute per iteration)

Configuration Template

Use this configuration class to standardize clustering experiments across teams.

from dataclasses import dataclass
from typing import Optional

@dataclass
class ClusteringConfig:
    """
    Standardized configuration for K-Means production runs.
    """
    n_clusters: int
    init_method: str = 'k-means++'
    n_init: int = 10
    max_iterations: int = 300
    tolerance: float = 1e-4
    random_seed: int = 42
    scaling: bool = True
    outlier_threshold: Optional[float] = None

    def validate(self):
        if self.n_clusters < 2:
            raise ValueError("n_clusters must be >= 2")
        if self.init_method not in ['k-means++', 'random']:
            raise ValueError("Invalid init_method")
        if self.n_init < 1:
            raise ValueError("n_init must be >= 1")

# Example Usage
config = ClusteringConfig(
    n_clusters=5,
    scaling=True,
    outlier_threshold=3.0  # Z-score threshold
)
config.validate()

Quick Start Guide

1. Prepare Data: Load dataset and apply StandardScaler.
2. Scan K: Iterate K from 2 to 10. Compute Inertia and Silhouette for each.
3. Select K: Choose K at the elbow of Inertia and peak of Silhouette.
4. Fit Model: Run KMeans(n_clusters=K, init='k-means++', n_init=10).
5. Deploy: Assign labels to production data. Monitor segment stability weekly.