PCA: Compressing Many Numbers Into a Few

Learn how Principal Component Analysis squeezes many overlapping market variables into a few "components" — and why those are not the same as economic factors.

Category: Machine Learning

In plain English

Imagine you measured 30 things about every stock — its price moves, trading volume, size, and so on. Many of these numbers move together, so a lot of the information is repeated. Principal Component Analysis (PCA) is a math trick that squeezes all those overlapping numbers down into just a handful of new summary numbers, called components, while throwing away as little real information as possible. Think of it like summarizing a long book into a few sentences that still capture most of the plot. Robust PCA is a variant that splits the data into a smooth common-structure part shared by most stocks plus a separate sparse part holding the few big outliers, so a handful of extreme names cannot bend the summary directions.

What it measures

PCA measures variation — how spread out the data is — and finds the directions where the data varies the most. The first component captures the single biggest pattern of variation, the second captures the next biggest (that the first missed), and so on. "Keeping 90% of the variance" means we keep enough components to preserve 90% of the total spread and drop the rest as noise. - Tiny example: say height and arm-span are almost always close (tall people have long arms). - Instead of storing both numbers, PCA makes one component, "overall body size," that captures ~95% of the differences between people. - You went from 2 numbers down to 1, losing very little — that is dimensionality reduction. - A loading is how much each original feature contributes to a component (the recipe of the direction); a score (or component value) is where a given stock lands along it — the two are easy to mix up.

How SurgeFlow uses it

PCA underpins the ML "market map" tab. SurgeFlow takes many correlated features per stock, runs PCA, and keeps enough components to retain about 90% of the variance — a compact, less-noisy version of the data. It then groups stocks (clustering) inside that compressed space, and separately uses reconstruction error — how badly a stock is described by the kept components — as one anomaly signal: a stock that does not fit the common pattern stands out. SurgeFlow's anomaly score combines this reconstruction error with each stock's distance to its cluster centre. (This is conceptually related to the academic "Robust PCA" technique, which splits data into shared low-rank structure plus sparse outliers, but SurgeFlow uses ordinary PCA, not Robust PCA.) - Compress: many features down to a few components (~90% variance kept). - Group: cluster stocks in the smaller space. - Flag outliers: large reconstruction error = one (of several) anomaly hints.

How to read it on the site

Open the ML tab (the market map at /ml). You'll see stocks grouped into clusters and some names flagged as anomalies. Read a cluster as "these stocks behaved in a structurally similar way over the period," and read an anomaly flag as "this name didn't fit the usual shared pattern — worth a closer human look." The components themselves are summary axes, not prices or scores you act on. - Clusters group stocks by shared structure, not by quality or expected return. - An anomaly flag is a prompt to investigate, never an automatic buy or sell. - Labels and groupings can shift as new data arrives.

What it cannot tell you

This is the most important section. PCA components are NOT automatically economic factors — a component is just a direction of maximum variation, so calling component 1 "value" or "momentum" is a human guess that may be wrong. PCA describes the structure in the data; it does not predict future returns or give buy/sell signals, and a clean clustering tells you nothing about which stocks will go up. Results also depend on how the data was scaled and which features were included, and outliers can distort the components unless handled carefully. - Components are mathematical, not real-world causes — correlation is not causation. - Keeping 90% of variance is a choice, not a law; a different cutoff changes the picture. - Reconstruction error flags "unusual," not "bad" or "opportunity." - Descriptive only — this is research and education, not investment advice.