The Axiom of Linearity: Why Civilization Keeps Trying to Solve the Same Equation, and the Beautiful, Brutal Ways We've Learned to Cheat at It

Audio article

Uses the speech voice supplied by your browser or device.

Matrix-like grids separated into simpler component structures

The Quiet Tyranny of a Single Equation

I want to tell you about the most important equation you have never thought about, or perhaps—if you are the sort of person who wears hoodies to formal dinners and thinks eigenvalues are conversation starters—the equation you think about far too often, obsessively, perhaps even pathologically, at three in the morning while staring at a ceiling that offers no answers.

It is not $E = mc^2$ , that photogenic celebrity of physics, nor is it Euler’s identity, that precocious child prodigy of mathematics that every pop-science writer trots out like a party trick to prove that math can be beautiful.

No.

It is $Ax = b$ .

Three characters. A matrix, a vector of unknowns, and a vector of outcomes. It looks, to the uninitiated, like a typo in a Scrabble game. But this—this is the equation that holds up bridges, predicts weather, trains artificial intelligences to hallucinate convincingly, and allows your GPS to tell you, with unnerving confidence, that you are definitely, absolutely, probably in a parking lot in New Jersey when you are quite clearly standing in downtown Mumbai.

I am not exaggerating. Not much, anyway.

The universe, it turns out, is not linear. Not even close. It is a writhing, chaotic, nonlinear mess of partial differential equations, quantum entanglement, turbulent fluids, and human emotions. But here is the dirty secret of modern civilization, the one they do not put on the brochure: we have gotten extraordinarily good at pretending everything is linear, solving that linear pretend-version, and then hoping—often with statistically justified optimism—that the answer is close enough to not kill anyone.

And the way we solve this pretense, at the scale of billions of variables, is through a family of techniques called matrix decompositions, which are essentially elaborate mathematical games of “let’s take this complicated thing apart, solve the easy pieces, and put it back together before anyone notices.”

This is their story. And ours.

Who: The Strange Fellowship of People Who Thought This Was a Good Idea

To understand matrix decompositions, you must first understand the kind of people who invent them, which is to say: you must understand a particular strain of human being who looks at a towering edifice of numbers and feels not dread, but a kind of illicit excitement, the same emotion a safecracker feels when confronted with an especially ornate vault.

These people are not normal. They cannot be.

There is Carl Friedrich Gauss, the nineteenth-century German polymath who was so precocious that he allegedly corrected his father’s accounting errors at age three, which must have made for absolutely thrilling dinner table conversation. Gauss did not invent Gaussian elimination—mathematicians are terrible at naming things after the right people, a tradition that continues to this day with the “Arabic” numerals, which are actually Indian—but he systematized it, polished it, and applied it with the relentless efficiency of a man who calculated asteroid orbits in his spare time for relaxation.

There is André-Louis Cholesky, a French military officer and geodesist who, in the muddy trenches of World War I, between artillery barrages and the general unpleasantness of early twentieth-century warfare, developed a method for solving symmetric positive-definite systems of equations that is now used in portfolio optimization, finite element analysis, and machine learning. He died in battle in 1918, killed not by mathematics but by shrapnel, leaving behind a decomposition so elegant that it feels almost indecent to use it for something as mundane as training a neural network to recognize cats.

There is Alston Householder, who looked at the problem of numerical stability—computers rounding off tiny numbers and thereby turning accurate solutions into expensive garbage—and decided the answer was to reflect vectors across hyperplanes, a technique that sounds like something from a psychedelic philosophy seminar but actually revolutionized scientific computing in the 1950s.

And there is Gene Golub, the Stanford professor who, with William Kahan, turned the Singular Value Decomposition from an obscure theoretical curiosity into the computational workhorse of the modern era, enabling everything from Google’s search rankings to the Netflix Prize to the compression of your JPEG vacation photos. Golub was known for his warmth, his wit, and his habit of telling students that the SVD was “the Swiss Army knife of numerical linear algebra,” which undersells it considerably; the Swiss Army knife does not, to my knowledge, underlie the entire field of latent semantic analysis.

These people—and dozens more, whose names are half-forgotten even by specialists, buried in the footnotes of papers with titles like “A Stable Method for the Inversion of Matrices of the Form I + uv^T”—form a kind of invisible pantheon. They are the architects of the infrastructure that makes modernity possible, and they are almost entirely unknown outside of a small, fiercely devoted cult of practitioners who gather at conferences in hotels with inadequate coffee.

What: The Equation That Ate the World

So what, exactly, is $Ax = b$ ?

At its most pedestrian, it is a system of linear equations. The matrix $A$ contains coefficients. The vector $x$ contains the things we want to know. The vector $b$ contains the things we already know, or think we know, or have measured with instruments that cost millions of dollars and are probably slightly miscalibrated.

If $A$ is a 2x2 matrix, this is high school algebra. You can solve it with substitution, elimination, or by staring at it hard enough while muttering under your breath. But when $A$ is a million-by-million matrix—which is not uncommon in weather modeling, structural engineering, or training large language models—staring does not work. Muttering does not work. Even substitution begins to look like a war crime against computational efficiency.

The matrix $A$ might represent the stiffness of every beam in a skyscraper, or the correlations between every word in the English language, or the interactions between every atom in a protein folding simulation. The vector $b$ might represent wind loads, or document frequencies, or electromagnetic forces. And $x$ —poor, hunted $x$ —is the answer we seek: the stresses in each beam, the semantic meaning of each word, the final folded configuration of the protein.

But here is the profound, almost spiritual insight that took mathematicians centuries to fully articulate: solving $Ax = b$ is not just about finding numbers. It is about understanding structure. It is about asking what $A$ is, fundamentally, beneath its grid of numbers, and whether that hidden structure can be exploited to make the impossible merely expensive.

This is where decompositions enter, stage left, carrying slide rules and bad news about computational complexity.

When: From Ancient Ledgers to the Digital Deluge

The urge to solve systems of linear equations is ancient, almost primordial. The Chinese text The Nine Chapters on the Mathematical Art, compiled around the second century CE but drawing on older material, contains a method for solving linear systems that is recognizably Gaussian elimination, complete with the use of matrix-like arrays of numbers and the elimination of variables by subtracting multiples of one equation from another.

Think about that. While the Roman Empire was collapsing into the messy, violent chaos of late antiquity, Chinese mathematicians were already performing operations on arrays of numbers that would not be called “matrices” for another seventeen centuries, because the word “matrix”—meaning womb, or source—was not appropriated by James Joseph Sylvester until 1850, and even then he used it primarily because he liked the biological metaphor of numbers breeding other numbers, which tells you something about Victorian mathematicians that you might prefer not to know.

For millennia, solving linear systems was a manual, agonizing process. Astronomers needed it to compute planetary orbits from observational data. Surveyors needed it to triangulate land boundaries. Navigators needed it to determine their position from the stars. But the systems were small—three equations, maybe four or five if someone was feeling particularly masochistic—and the calculations were done by hand, by candlelight, by people who died young of eye strain and probably scurvy.

The nineteenth century changed everything, though not immediately. Gauss’s method of least squares—fitting a line to noisy data by minimizing the sum of squared errors—required solving the so-called normal equations, which are always of the form $A^T A x = A^T b$ . This was the first widespread, industrial-scale application of linear system solving, and it arose from the desperate need of astronomers to process the torrents of data pouring in from newly built observatories. The universe, it turned out, was not merely expanding; it was generating data faster than humans could reduce it.

Then came the twentieth century, and with it, the computer. Not the computer you are reading this on, which is a miracle of miniaturization, but the computer as a job title: rooms full of human calculators, mostly women, who were employed by the military, by government bureaus, and by engineering firms to solve systems of equations using mechanical calculators that clicked and whirred with the urgency of a thousand typewriters. During the Manhattan Project, during the Apollo program, during the design of the first jet airliners, these human computers solved linear systems of staggering size—staggering, that is, for the era—because there was no other way to model a nuclear explosion or a lunar trajectory.

And then, in the 1940s and 1950s, the electronic computer arrived, and the problem inverted. Suddenly, we could solve systems with hundreds, then thousands, then millions of variables. But speed brought new problems. Rounding errors—tiny inaccuracies introduced when a computer stores a number with finite precision—accumulated like snowdrifts, burying the true answer under an avalanche of numerical garbage. A method that worked beautifully on paper could fail catastrophically on a von Neumann architecture machine. The matrix decompositions we use today were not invented merely to solve systems faster; they were invented to solve them correctly, to wrest accurate answers from machines that were fundamentally, structurally incapable of representing most real numbers exactly.

This is the timeline: from Chinese counting rods to human computers in pencil skirts to supercomputers humming in refrigerated rooms to the GPU in your phone, which is currently using matrix operations to render this text and track your finger on the screen. The problem has not changed. Only the scale, and the desperation, have grown.

Where: The Ubiquity That Borders on the Absurd

If you are looking for $Ax = b$ , you can find it almost anywhere you care to look, which is either deeply comforting or vaguely unsettling, depending on your temperament and whether you have had coffee.

It is in the finite element analysis that ensures the wing of the airplane you will eventually board does not snap off during turbulence, a possibility that haunts the more anxious among us during takeoff. Engineers discretize the continuous equations of elasticity—those beautiful, terrible partial differential equations that describe how solids deform—into enormous matrices, and then they solve $Ax = b$ to find the stresses and strains at thousands of points across the wing’s surface.

It is in the PageRank algorithm that once made Google possible, where the entire web is represented as a gigantic sparse matrix—billions of pages, billions of links—and the steady-state probabilities are found by solving a linear system that is, in essence, $Ax = b$ dressed up in the formal language of Markov chains and eigenvectors.

It is in the training of neural networks, where backpropagation computes gradients through matrix multiplications, where each layer’s weights are updated via linear algebra, and where the entire edifice of deep learning rests, somewhat precariously, on the assumption that we can multiply and decompose matrices fast enough to make training a model with a trillion parameters economically viable, which is a sentence that would have made a 1960s computer scientist laugh until they wept.

It is in computed tomography, where the Radon transform is inverted via linear algebra to reconstruct three-dimensional images of your bones and organs from a series of two-dimensional X-ray slices. When you lie in an MRI machine, you are, in a very real sense, inside a giant $Ax = b$ problem, and the radiologist is looking at the solution.

It is in quantum chemistry, where the Schrödinger equation—linear, despite quantum mechanics’ reputation for weirdness—is discretized and solved to predict molecular properties. It is in economics, where input-output models trace the flow of goods through industries. It is in signal processing, where Fourier transforms decompose waveforms into frequencies via matrix operations. It is in your camera, removing blur; in your phone, correcting GPS errors; in your weather app, lying to you about tomorrow’s forecast with mathematical sophistication.

The equation is everywhere because linearity is the first approximation of everything. When a system is too complex to solve directly, we linearize it—take a Taylor expansion, keep the first-order terms, cross our fingers, and solve $Ax = b$ . The universe is nonlinear, yes, but locally, momentarily, in the small hours of the night when we need an answer by morning, it is linear enough.

Why: The Seduction of the Straight Line

Why does linearity dominate? Why this equation, and not some other?

The answer is part physics, part psychology, and part desperate pragmatism.

Physically, many systems are approximately linear near equilibrium. A spring obeys Hooke’s law—force proportional to displacement—until you stretch it too far and it either breaks or enters a nonlinear regime where the metal crystals begin to rearrange themselves in ways that engineers describe with technical terms like “plastic deformation” and laypeople describe with technical terms like “oh no.” But near equilibrium, linearity reigns. The same is true of electrical circuits (Ohm’s law), heat conduction (Fourier’s law), and fluid flow at low Reynolds numbers (Stokes flow). Nature, it seems, is lazy. It prefers linearity when it can get away with it.

Mathematically, linearity is the only thing we can reliably solve at scale. Nonlinear equations are the wilderness—trackless, full of strange attractors and bifurcations and solutions that blow up to infinity because you changed a parameter by 0.001. Linear equations are the garden. We know where the paths go. We have tools: superposition, eigenvalues, matrix decompositions, the entire apparatus of linear algebra that took two centuries to build.

And psychologically—though no one likes to admit this—linearity is comforting. A linear system has one solution, or infinitely many, or none. It does not surprise you. It does not hide chaotic behavior in innocuous-looking equations. It is the mathematical equivalent of a well-lit street: you can see where you are going, even if where you are going is not exactly where you need to be.

But the deepest reason is this: nonlinear problems are solved by iteration, and iteration is a sequence of linear problems. Newton’s method for finding roots? Solve a linear system at each step. The training of neural networks via gradient descent? A sequence of linear approximations to a nonlinear loss landscape. Weather models? Linearize, solve, advance timestep, repeat. We are trapped in a loop of linearity, and we have become so good at it that we have forgotten, most of the time, that we are trapped at all.

How: The Art of Mathematical Disassembly

So we have $Ax = b$ , and $A$ is huge, and we need to solve it quickly, accurately, and without using more memory than the machine possesses, which is a constraint that sounds trivial until you are trying to factor a matrix that, if printed, would require a stack of paper taller than the Eiffel Tower.

The naive approach is to compute the inverse: $x = A^{-1}b$ . This is what undergraduate linear algebra courses teach you, because undergraduate linear algebra courses are, in many ways, elaborate hazing rituals designed to make you believe that computing inverses is a good idea. It is not. Computing the inverse of a large matrix is computationally expensive, numerically unstable, and conceptually vulgar. It is the mathematical equivalent of using a sledgehammer to open a walnut, except the sledgehammer costs ten million dollars and the walnut is actually a Fabergé egg.

The sophisticated approach—the approach that built modern engineering, that makes AI possible, that separates the professionals from the tourists—is to decompose $A$ into simpler pieces.

Think of it this way: if $A$ is a complicated machine, a decomposition is taking it apart, cleaning the pieces, understanding what each one does, and reassembling it. But because this is mathematics, the pieces are not gears and springs. They are triangular matrices, orthogonal matrices, diagonal matrices, permutation matrices—shapes so geometrically simple that solving systems involving them is trivial, or nearly so.

The central insight, the one that makes all of this work, is that matrix multiplication is associative but not commutative, which means we can group operations strategically, and that certain matrix structures—upper triangular, lower triangular, diagonal, orthogonal—have properties that make them computationally tractable in ways that general matrices are not. A triangular system can be solved by back-substitution or forward-substitution, which is essentially a mechanical process of peeling away variables one by one, like the layers of an onion, except less tear-inducing and more algorithmic.

So the game becomes: can we write $A$ as a product of simple matrices? And if so, which simple matrices, and in what order, and at what cost?

This is the domain of matrix decompositions, and it is where the stories live.

Which: The Decompositions, Their Personalities, and Their Histories

Each matrix decomposition is a character in this drama, with its own origin story, its own strengths, its own tragic flaws, and its own preferred domain of application. To know them is to understand not just how we solve $Ax = b$ , but how we have learned to think about structure itself.

LU Decomposition: The Workhorse with a Thousand Names

The LU decomposition factors a matrix $A$ into the product of a lower triangular matrix $L$ and an upper triangular matrix $U$ , sometimes with a permutation matrix $P$ thrown in to handle row exchanges, because not every matrix is well-behaved enough to decompose without a little bureaucratic shuffling of rows.

$A = LU$ (or $PA = LU$ ).

Once you have this factorization, solving $Ax = b$ becomes a two-step dance: first solve $Ly = b$ for $y$ by forward substitution, then solve $Ux = y$ for $x$ by back substitution. Both steps are $O(n^2)$ , while the factorization itself is $O(n^3)$ . This means that if you need to solve $Ax = b$ for many different right-hand sides $b$ —which is common in engineering, where you might test many load configurations on the same structure—you pay the expensive factorization cost once, and then each subsequent solve is fast.

The history is ancient, as I mentioned, but the modern understanding emerged in the 1940s with the development of electronic computers. Alan Turing, in a 1948 paper that is less famous than his work on computability but arguably more practical, analyzed the rounding errors in Gaussian elimination and showed that, with partial pivoting—strategically swapping rows to keep the numbers from growing too large—the method was numerically stable for most practical matrices.

The LU decomposition is the pickup truck of numerical linear algebra. It is not elegant. It is not exciting at parties. But it hauls loads, it starts in cold weather, and it is what MATLAB uses when you type the backslash operator A\b, which is arguably the most powerful single keystroke in computational mathematics.

Cholesky Decomposition: The Elegant Soldier

If $A$ is symmetric and positive definite—a condition that sounds technical but essentially means the matrix describes a bowl-shaped energy landscape where every direction goes uphill, guaranteeing a unique minimum—then something miraculous happens: $A$ can be factored as $LL^T$ , where $L$ is lower triangular with positive diagonal entries.

This is the Cholesky decomposition, and it is roughly twice as fast as LU, because symmetry allows you to compute only half the matrix. It requires no pivoting. It is numerically stable. It is, in the words of Nicholas Higham, the leading authority on numerical linear algebra, “the method of choice for solving symmetric positive definite systems.”

And it was invented by a man who died in the trenches of World War I.

André-Louis Cholesky was a French artillery officer and geodesist who developed his method around 1905 while working on the adjustment of geodetic networks—essentially, using least squares to reconcile inconsistent survey measurements into a coherent map. His method was published posthumously by a colleague in 1924, and it languished in relative obscurity until the 1940s, when it was rediscovered and recognized for its computational efficiency.

There is something almost unbearably poignant about this: a man who spent his life measuring the earth, killed by the war that tore it apart, leaving behind a mathematical tool that now underlies portfolio optimization, finite element methods, and the training of Gaussian process models. The Cholesky decomposition does not know its own history. It simply works, silently, in the background of a thousand MATLAB scripts, a memorial in matrix form.

QR Decomposition: The Geometry of Orthogonality

The QR decomposition factors $A$ into the product of an orthogonal matrix $Q$ —meaning $Q^T Q = I$ , its columns are perpendicular unit vectors, preserving lengths and angles—and an upper triangular matrix $R$ .

$A = QR$ .

This decomposition is the geometric heart of linear algebra. Orthogonal matrices represent rotations and reflections, the rigid motions of Euclidean space. To decompose $A$ into $QR$ is to say: any linear transformation can be seen as a rotation/reflection followed by a stretching along coordinate axes. This is the essence of the QR factorization, and it is beautiful.

But beauty is not its only virtue. The QR decomposition is numerically stable in ways that LU is not, because orthogonal matrices do not amplify rounding errors. If you multiply by an orthogonal matrix, the length of your vector stays the same. This is not true for general triangular matrices, which can stretch and compress vectors in ways that magnify tiny errors into gross inaccuracies.

The history of QR is a small saga in itself. The Gram-Schmidt process, a method for orthogonalizing vectors named after Jørgen Pedersen Gram and Erhard Schmidt, provides one route to QR, but it is numerically unstable in its classical form—tiny rounding errors accumulate and the resulting $Q$ matrix gradually ceases to be orthogonal, like a machine slowly going out of alignment.

Alston Householder, in the 1950s, introduced a better method using Householder reflections: instead of gradually orthogonalizing vectors one by one, you reflect the entire matrix across hyperplanes, zeroing out subdiagonal entries in systematic, stable sweeps. It is like cleaning a window with a single, decisive wipe rather than a series of tentative dabs.

James Wallace Givens, around the same time, proposed using plane rotations—Givens rotations—to zero out individual entries. This is slower for dense matrices but advantageous for sparse matrices or structured problems, where you only want to touch certain entries.

Today, QR is the backbone of least squares problems (solving $Ax \approx b$ when there is no exact solution), eigenvalue algorithms (the QR algorithm, which iteratively refines approximations to eigenvalues), and countless signal processing applications. When your phone does face recognition, when GPS satellites correct their orbits, when statisticians fit regression models, QR is there, quietly ensuring that the geometry of the problem is respected.

Eigendecomposition: The Spectral Revelation

The eigendecomposition factors a diagonalizable matrix $A$ as $A = V \Lambda V^{-1}$ , where $\Lambda$ is a diagonal matrix of eigenvalues and $V$ is a matrix whose columns are the corresponding eigenvectors.

This is not just a factorization. It is a revelation.

An eigenvector of $A$ is a direction that is only scaled, not rotated, by the transformation $A$ . The eigenvalue is the scaling factor. To eigendecompose $A$ is to find its natural axes, the coordinate system in which it acts as a simple diagonal matrix—just stretching space along certain directions, leaving others untouched.

The spectral theorem, which guarantees that real symmetric matrices have real eigenvalues and orthogonal eigenvectors, was developed in the nineteenth century by Cauchy, Jacobi, Weierstrass, and others, in a slow accumulation of insights that took decades to cohere into the elegant form we know today. The word “spectrum” comes from David Hilbert, who borrowed it from physics because the eigenvalues of an operator correspond to the frequencies of vibration of a system—literally, the colors of light emitted by an atom, the acoustic tones of a drum, the resonant frequencies of a bridge.

The eigendecomposition tells you whether a system is stable (all eigenvalues negative, perturbations decay) or unstable (any eigenvalue positive, perturbations grow). It tells you the principal modes of vibration of a structure. It underlies Principal Component Analysis (PCA), where the eigenvectors of the covariance matrix point in the directions of maximum variance in high-dimensional data.

But not all matrices are diagonalizable. This is the tragedy at the heart of the eigendecomposition: some matrices are defective, lacking a full set of eigenvectors, and for these the Jordan normal form— $A = V J V^{-1}$ , where $J$ is nearly diagonal but with ones on the superdiagonal—provides a consolation prize, though it is numerically delicate and rarely computed in practice for large matrices.

The QR algorithm, mentioned earlier, is the standard method for computing eigenvalues. It is one of the great algorithms of the twentieth century: by repeatedly factoring a matrix into QR and then reversing the product— $A_{k+1} = R_k Q_k$ —the sequence converges to a matrix whose eigenvalues are visible on the diagonal. It is elegant, robust, and, like many great ideas, seems almost obvious in retrospect, though it took years of work by John Francis and Vera Kublanovskaya in the early 1960s to fully develop.

Singular Value Decomposition: The Ultimate Decomposition

If the eigendecomposition is a revelation, the Singular Value Decomposition (SVD) is a transcendence.

For any matrix $A$ —any matrix at all, rectangular or square, rank-deficient or full-rank, symmetric or wildly asymmetric—the SVD guarantees a factorization:

$A = U \Sigma V^T$

where $U$ and $V$ are orthogonal matrices and $\Sigma$ is a diagonal matrix with non-negative entries called singular values, arranged in descending order.

The columns of $U$ are the left singular vectors. The columns of $V$ are the right singular vectors. The singular values are the scaling factors, but unlike eigenvalues, they are always real and non-negative, and they exist for every matrix.

This is the Swiss Army knife, the universal adapter, the decomposition that works when all others fail. It was developed in the 1870s by Eugenio Beltrami and Camille Jordan, refined by James Joseph Sylvester, and later by Erhard Schmidt and Hermann Weyl, but its computational importance was not fully realized until the 1960s, when Gene Golub and William Kahan developed the Golub-Kahan bidiagonalization method, making the SVD practical for large matrices.

The SVD solves least squares problems. It computes the pseudoinverse, which gives the minimum-norm solution even when $A$ is singular or rectangular. It provides the optimal low-rank approximation of a matrix, a result known as the Eckart-Young-Mirsky theorem: if you want to approximate $A$ by a matrix of rank $k$ , the best approximation in the Frobenius norm is obtained by keeping the $k$ largest singular values and setting the rest to zero.

This theorem is the mathematical foundation of data compression, image processing, and latent semantic analysis. When you compress an image using PCA, you are using the SVD. When Google finds similar documents, it is using the SVD. When Netflix predicts what you want to watch, it is using matrix factorization methods that are spiritual descendants of the SVD.

The SVD is also deeply geometric. It says that any linear transformation can be understood as a rotation, followed by a scaling along coordinate axes, followed by another rotation. This is the singular value decomposition as a statement about the geometry of linear maps: every map is, at its core, a stretching of space along perpendicular directions, viewed in the right coordinate systems.

I cannot overstate how important this is. In a world of messy, incomplete, noisy data— which is to say, the actual world we inhabit—the SVD provides a way to separate signal from noise, structure from randomness, the meaningful from the incidental. The singular values tell you the “energy” or importance of each dimension. The small singular values correspond to noise, to redundancy, to the fine-grained texture of the data that you can often discard without losing the essential picture.

When Carl Sagan said we are made of starstuff, he was being poetic. When a data scientist says the world is made of singular values, they are being almost literally true, at least for the linear approximations that underlie modern science.

Schur Decomposition: The Safe Middle Ground

The Schur decomposition, developed by Issai Schur in 1909, factors any square matrix as $A = Q U Q^T$ , where $Q$ is orthogonal and $U$ is upper triangular. Unlike the eigendecomposition, it always exists, even for non-diagonalizable matrices. The eigenvalues appear on the diagonal of $U$ , but the matrix is not fully diagonalized.

This decomposition is the workhorse behind many eigenvalue algorithms, including the QR algorithm itself. It is numerically stable, always computable, and provides a triangular canonical form that is almost as good as diagonal for many purposes. Schur, a German mathematician who made profound contributions to representation theory and number theory before being dismissed from his professorship by the Nazis in 1935, gave us a tool that balances theoretical generality with computational practicality.

Jordan Normal Form: The Beautiful Impossibility

The Jordan decomposition, $A = V J V^{-1}$ , where $J$ is block-diagonal with Jordan blocks, is the most refined decomposition in theory and the least used in practice. It exists for every matrix over an algebraically closed field, but it is numerically unstable: tiny perturbations to $A$ can change the Jordan structure completely. It is a monument to mathematical beauty and a warning about the gap between theory and computation.

Camille Jordan, the French mathematician who introduced it, was a giant of nineteenth-century mathematics, and his normal form is taught in every linear algebra course as the culmination of the theory of canonical forms. But if you try to compute it numerically for a large matrix, you will likely encounter grief, despair, and the sudden realization that your matrix is not quite as well-behaved as you hoped. It lives in the realm of exact arithmetic, of symbolic computation, of mathematical ideals. The real world, with its rounding errors and noisy measurements, prefers the Schur form, or the SVD, or the humble LU.

The Bigger Picture: What We Are Really Doing When We Factor Matrices

I have told you about equations and decompositions, about soldiers and geodesists and hyperplane reflections, about the ancient Chinese and the modern GPU. But I want to step back, now, and ask what all of this means, not just for mathematics, but for how we understand the world and our place in it.

When we decompose a matrix, we are doing something profoundly human. We are taking the complex and rendering it simple. We are finding the hidden structure beneath the surface noise. We are saying: this tangled web of numbers, this incomprehensible grid of ten billion coefficients, is not arbitrary. It has axes. It has scales. It has directions that matter and directions that do not. It can be rotated into clarity.

This is what the SVD does for data, what the eigendecomposition does for physics, what the Cholesky decomposition does for engineering. It is an act of interpretation, of making meaning from mess. The matrix $A$ is the world as we find it: complicated, interconnected, overwhelming. The decomposition is the world as we understand it: structured, hierarchical, amenable to analysis.

And $Ax = b$ —that humble equation, that workhorse of civilization—is the form that every question takes when we strip away the noise and look for the underlying relationship. Who is involved? The variables $x$ . What is happening? The transformation $A$ . What do we observe? The outcomes $b$ . The equation is a question posed in the language of algebra, and the decompositions are the methods we have developed to answer it without losing our minds or our precision.

But there is a deeper pattern here, one that connects mathematics to the broader human project of managing complexity. We live, now, in an age of data deluge, where the matrices are not thousands but trillions of entries, where training a language model requires solving optimization problems that are, at their core, sequences of linear systems so large that no human can comprehend their scale. And we are solving them, imperfectly, approximately, but usefully, because of the accumulated ingenuity of two centuries of decomposition theory.

The matrix decompositions are not just algorithms. They are ways of seeing. To perform an LU decomposition is to see a matrix as an elimination process, a systematic removal of complexity. To perform a QR decomposition is to see it geometrically, as rotations and stretches. To perform an SVD is to see its essential dimensions, the signal amid the noise. Each decomposition is a lens, and together they form an optics of understanding that allows us to peer into problems that would otherwise be opaque.

And yet, for all our sophistication, we are still solving $Ax = b$ . The equation has not changed since the Nine Chapters. The problems have grown, the machines have accelerated, the applications have multiplied beyond any single person’s comprehension, but at the bottom of it all, we are still looking for $x$ , still factoring $A$ , still hoping that the linear approximation holds long enough to get an answer that matters.

This is both humbling and exhilarating. Humbling, because it suggests that the deepest problems of science and engineering reduce, again and again, to the same ancient form. Exhilarating, because it means that the tools we build—each new decomposition, each faster algorithm, each GPU kernel optimized for sparse triangular solves—are not just technical achievements. They are extensions of human cognition, prosthetics for the imagination, allowing us to hold problems in our minds that would otherwise be too vast to contemplate.

The next time you look at a weather forecast, or board an airplane, or ask a chatbot a question, remember that somewhere, deep in the stack of abstractions, a computer is factoring a matrix. It is performing a Cholesky decomposition on a covariance matrix, or an SVD on a term-document matrix, or a QR factorization inside an eigenvalue solver. It is doing what mathematicians have done for centuries, only faster, and at a scale that would have seemed like magic to Cholesky in his trench, or to Gauss at his desk, or to the unknown Chinese scribe arranging counting rods in the second century.

We are, all of us, living inside $Ax = b$ . The decompositions are how we breathe in there.

And the remarkable thing—the thing that keeps me up at night, in the best possible way—is that we are still finding new ones. New randomized algorithms that approximate the SVD in sublinear time. New tensor decompositions that extend these ideas beyond matrices to higher-dimensional arrays. New ways of exploiting sparsity, structure, and symmetry that we did not know existed thirty years ago.

The equation is old. But our relationship to it is constantly renewed, like a conversation with an old friend who never runs out of things to say.

So here is the thought I want to leave you with, the one that lingers after the algorithms and the history have faded: every matrix decomposition is, at its heart, an act of faith. Faith that the world has structure. Faith that complexity can be untangled. Faith that beneath the noise, there is signal, and that with the right rotation, the right reflection, the right triangular factor, we can make it visible.

We factor matrices because we believe, against all evidence of chaos and entropy and the second law of thermodynamics, that things can be understood. And so far, that faith has been rewarded, again and again, with bridges that stand, medicines that work, and artificial minds that can, however imperfectly, mimic our own.

That is the real decomposition: not $A$ into $LU$ or $QR$ or $U \Sigma V^T$ , but the world into pattern and noise, and our stubborn, magnificent refusal to accept the noise as the final word.

Search & shortcuts

Quick destinations