Linear algebra: a reading path from intuition to real fluency
This curriculum builds a deep, intuition-first understanding of linear algebra across three tightly sequenced stages. Starting from geometric and visual foundations, it moves through rigorous matrix theory and transformations, and culminates in the abstract structures and applied techniques that power modern data science and machine learning.
Geometric Intuition & Core Vocabulary
IntermediateBuild a strong visual and conceptual foundation — understanding vectors, linear transformations, and matrix operations as geometric objects before formalizing them algebraically.
A clean, proof-driven introduction that prioritizes vector spaces and linear maps over determinants, building the right mental model from the start. Its intermediate-friendly prose makes abstract ideas feel inevitable rather than arbitrary.
Strang's applied, example-rich style perfectly complements Axler's abstraction — covering row reduction, matrix factorizations, and real-world interpretations that anchor the theory in concrete computation.
Transformations, Decompositions & Deeper Structure
IntermediateMaster the mechanics and meaning of matrix transformations, orthogonality, determinants, and the key decompositions (LU, QR, SVD) that are the workhorses of applied mathematics.
▸ Study plan for this stage
Pace: 8–10 weeks, ~40–50 pages/day (mix of reading and worked examples)
- Matrix transformations as linear maps between vector spaces and their geometric interpretation
- Orthogonality, orthonormal bases, and the Gram-Schmidt process for constructing them
- Determinants: definition, properties, and geometric meaning as signed volume scaling
- LU decomposition: Gaussian elimination as matrix factorization and its computational role
- QR decomposition: orthogonalization and its stability advantages over LU
- Singular Value Decomposition (SVD): structure, interpretation, and universal applicability to any matrix
- Eigenvalues and eigenvectors: spectral properties and diagonalization of symmetric matrices
- Matrix norms, conditioning, and numerical stability in practical computations
- How does a matrix represent a linear transformation, and what do its columns tell you about where basis vectors are mapped?
- What is the geometric meaning of the determinant, and how does it relate to invertibility?
- Why is QR decomposition numerically more stable than LU decomposition, and when would you use each?
- What does the Singular Value Decomposition reveal about any matrix, and why is it considered the most fundamental decomposition?
- How do eigenvalues and eigenvectors characterize the behavior of a matrix, and what special properties do symmetric matrices have?
- What role does orthogonality play in numerical stability and in simplifying matrix computations?
- How would you use SVD to solve least-squares problems or perform data compression?
- What is the relationship between matrix norms, conditioning, and the reliability of numerical solutions?
- Work through Strang's examples of 2×2 and 3×3 matrix transformations by hand, sketching how they map unit vectors and the unit square/cube
- Compute LU decompositions (with and without pivoting) for 4–5 matrices of varying sizes and verify by multiplying back
- Apply Gram-Schmidt orthogonalization to 3–4 sets of linearly independent vectors and verify orthonormality
- Compute QR decompositions using both Gram-Schmidt and Householder reflections; compare numerical accuracy on an ill-conditioned matrix
- Calculate determinants using cofactor expansion, row reduction, and the product of eigenvalues; verify they match
- Compute the SVD of 3–4 matrices (including rectangular ones) by hand or with computational tools, and interpret the singular values and vectors
- Solve a least-squares problem using normal equations, QR decomposition, and SVD; compare results and stability
- Analyze the conditioning of a matrix using norms and the condition number; predict how errors propagate in solving Ax=b
Next up: This stage equips you with the decompositions and structural insights—LU, QR, SVD, eigenvalue analysis—that are the foundation for advanced topics like iterative solvers, optimization algorithms, and applications to data science and scientific computing.
Strang's most celebrated textbook, organized around four fundamental subspaces and culminating in SVD — the ideal bridge between mechanical fluency and deeper structural insight.
Elevates matrix theory to a rigorous, comprehensive level — covering norms, canonical forms, and spectral theory — providing the analytical depth needed before tackling eigenvalue-heavy applications.
Eigenvalues, Vector Spaces & Machine Learning Applications
ExpertAchieve mastery of eigenvalue theory, abstract vector spaces, and the linear algebraic machinery behind PCA, neural networks, and optimization in data science.
▸ Study plan for this stage
Pace: 12–14 weeks, ~40–50 pages/day (Shilov: 4–5 weeks; Trefethen: 3–4 weeks; Deisenroth: 4–5 weeks)
- Eigenvalues, eigenvectors, and the characteristic polynomial as tools for understanding linear transformations
- Diagonalization and Jordan normal form for analyzing matrix structure and behavior
- Abstract vector spaces, subspaces, and linear independence as foundational structures beyond ℝⁿ
- Inner product spaces, orthogonality, and the Gram-Schmidt process for constructing orthonormal bases
- Singular Value Decomposition (SVD) as a unifying framework for rank, approximation, and data compression
- Numerical stability and conditioning: why theory differs from computation in practice
- Principal Component Analysis (PCA) as eigenvalue decomposition applied to covariance matrices
- Linear algebraic foundations of neural networks: weight matrices, activation functions, and backpropagation as chain rule on computational graphs
- What is the relationship between eigenvalues, eigenvectors, and the diagonalization of a matrix, and when does diagonalization fail?
- How do you compute eigenvalues and eigenvectors, and what does the Jordan normal form tell you about matrices that cannot be diagonalized?
- What is a vector space, and how do concepts like basis, dimension, and linear independence generalize beyond ℝⁿ?
- How does the Singular Value Decomposition relate to eigenvalue decomposition, and why is SVD more numerically stable?
- Explain how PCA uses eigenvalue decomposition to perform dimensionality reduction, and what are its limitations?
- How do numerical errors accumulate in eigenvalue and linear system computations, and what is matrix conditioning?
- How are weight matrices in neural networks related to linear transformations, and how does backpropagation use the chain rule on linear algebra?
- What is the role of orthogonality and orthonormal bases in both theoretical linear algebra and practical machine learning?
- Compute eigenvalues and eigenvectors by hand for 2×2 and 3×3 matrices; verify diagonalization A = PDP⁻¹
- Implement eigenvalue decomposition in Python/NumPy and compare your results to numpy.linalg.eig(); test on symmetric vs. non-symmetric matrices
- Construct a Jordan normal form for a non-diagonalizable matrix and verify its properties
- Prove that eigenvectors corresponding to distinct eigenvalues are linearly independent; verify computationally
- Apply Gram-Schmidt orthogonalization to a set of linearly independent vectors and verify orthonormality
- Compute the SVD of a rectangular matrix by hand (for small examples) and via numpy.linalg.svd(); reconstruct the matrix from rank-k approximations
- Implement PCA from scratch using eigenvalue decomposition of the covariance matrix on a real dataset (e.g., Iris); compare to sklearn.decomposition.PCA
- Analyze the condition number of a matrix using numpy.linalg.cond(); solve a linear system with high condition number and observe numerical instability
Next up: This stage equips you with the deep linear algebraic foundations—eigenvalues, vector spaces, SVD, and numerical stability—that underpin advanced topics like spectral methods, convex optimization, manifold learning, and the theoretical analysis of deep learning architectures.
A rigorous Soviet-tradition text that treats determinants, eigenspaces, and canonical forms with full mathematical precision — solidifying the abstract vector space perspective needed for advanced work.
Transitions from pure theory to computational practice, covering how eigenvalue algorithms, SVD, and least-squares methods actually work in software — essential for data science and ML practitioners.
Synthesizes linear algebra, probability, and optimization through the lens of machine learning, showing exactly how vectors, matrices, and eigendecompositions underpin PCA, regression, and neural networks.
Discussion
Keep reading
Paths that share books, cover the same subject, or open a related topic.
