Markov Steady State
Compute the steady-state probability vector of a row-stochastic matrix via power iteration.
Overview
The Markov Steady State tool takes a row-stochastic transition matrix and returns the stationary probability vector — the long-run distribution that doesn't change when you apply the matrix one more time. It uses power iteration, so even moderately large matrices converge quickly.
It is useful for operations researchers modelling queueing systems, marketers analysing customer state transitions (active → churned → reactivated), web search students playing with PageRank and probability students learning ergodic theory.
How it works
A row-stochastic matrix P has non-negative entries with each row summing to 1. The stationary vector π satisfies π * P = π and Σ π_i = 1. Power iteration starts with any probability vector v_0 and repeatedly multiplies: v_{k+1} = v_k * P. Under mild conditions (irreducibility and aperiodicity), v_k converges to π.
Convergence is monitored by comparing successive iterates; the tool stops when their L1 difference falls below a tolerance or after a maximum iteration count. If the chain is reducible or periodic the result may depend on the starting vector, which the tool flags.
Examples
P = [[0.7, 0.3], [0.4, 0.6]]
→ steady state ≈ (0.571, 0.429)
P = [[0.5, 0.5, 0], [0.2, 0.6, 0.2], [0, 0.3, 0.7]]
→ steady state ≈ (0.182, 0.455, 0.364)
P = [[0, 1], [1, 0]] (periodic)
→ doesn't converge from arbitrary start
FAQ
What does row-stochastic mean?
Each row sums to 1 and entries are non-negative. Row i of P is the distribution over next states given you're currently in state i.
Is the steady state unique?
Yes for an irreducible aperiodic chain. If the chain has multiple recurrent classes you may get different limits from different starting vectors.
Can I use column-stochastic matrices?
Convert by transposing — the column-stochastic convention satisfies P * π = π instead.
Why power iteration instead of solving the eigenvalue problem?
It is simple, memory-light and well-suited to sparse matrices. For small dense matrices an eigendecomposition is equally good.
What's the link to PageRank?
PageRank is the stationary distribution of a random surfer Markov chain on the web graph, with a damping factor added to ensure irreducibility.