Feature Scaling: Putting Numbers on Equal Footing

Learn why features measured on different scales are normalized before clustering, and how SurgeFlow robust-scales each one by its median and spread.

Category: Machine Learning

In plain English

Imagine sorting people into groups using two facts: their height in centimetres (around 170) and their age in years (around 40). If you just add the raw numbers, height swamps age simply because its numbers are bigger - even though age might matter just as much. Feature scaling fixes this by rescaling every number so they all speak the same 'size of language' before the computer compares them. It is a fairness step: it stops one column from dominating only because its units happen to produce large values.

What it measures

A 'feature' is one column of data describing a stock - for example a 0-to-1 ratio, or a market cap of $2,000,000,000 (market cap = share price times number of shares). Many models measure how 'far apart' two stocks are by combining their columns, so a column with huge raw values dominates that distance. Scaling rewrites each column relative to its own typical value and its own spread, so a 'big move' in one column is comparable to a 'big move' in another. - Raw: stock A market cap = 2,000,000,000; ratio = 0.30 - Raw: stock B market cap = 2,000,001,000; ratio = 0.90 - Unscaled distance is dominated by the tiny 1,000 cap gap, ignoring the large ratio gap - After scaling, the 0.30-vs-0.90 ratio difference becomes visible and counts

How SurgeFlow uses it

This step underpins the ML 'market map' tab (/ml), which groups US stocks into clusters and flags unusual ones. Before any grouping, SurgeFlow robust-scales each feature: it subtracts the column's median (the middle value) and divides by its IQR (the inter-quartile range - the spread of the middle half of the data). Robust scaling is chosen because median and IQR are resistant to outliers and to sparse, patchy columns, unlike the average and standard deviation which a few extreme values can distort. The scaled features then feed PCA (a step that compresses many columns into a few summary directions) and the clustering that produces the map. - Center: subtract the column's median (typical value) - Spread: divide by the IQR (middle-half range) - Outlier-resistant, so a single freak value does not warp the column - Done per feature, before PCA and clustering

How to read it on the site

You do not click a 'scaling' button - it happens silently inside the pipeline behind the ML 'market map' tab (/ml). What you actually see is its result: stocks placed into clusters and anomaly flags where every input column was given a fair voice. The practical takeaway is that the map's groupings reflect a balanced view across all features, not just whichever column had the biggest raw numbers. If you ever wonder why two giant-cap stocks can still land in different clusters, scaling is the reason: differences in smaller-numbered columns are no longer drowned out.

What it cannot tell you

Scaling only makes columns comparable in size - it does not decide which columns matter, prove anything is important, or predict returns. A near-constant feature (one that is almost the same value for every stock) is dropped before scaling, because dividing by an IQR near zero would blow that tiny column up into noise that hijacks the model; removing it loses no real information since it barely varied anyway. And remember the bigger limit: the ML map is descriptive, it shows structure in the data, not buy or sell signals. SurgeFlow is research and education, never investment advice. - Scaling balances size, it does not rank importance - Near-constant columns are dropped (near-zero IQR would explode into noise) - Equal footing is not the same as equal relevance - The clusters describe structure - they are not forecasts or advice