Markov chain calculator
Type the transition matrix of a Markov chain with up to 10 states and a starting distribution. The calculator gives the distribution after n steps, the long-run stationary distribution in exact fractions, the mean return time of each state and any absorbing states.
Runs in your browser: nothing you enter is sent anywhere.
How to use it
- Type the matrix row by row, for example
0.9 0.1; 0.5 0.5.
- Enter the starting distribution and the number of steps.
- Read the distribution after n steps and, below, the stationary distribution.
How it works
Matrix entries are read as exact fractions and each row is checked to add up to exactly 1. The n-step distribution comes from repeated vector–matrix multiplication. The stationary distribution solves πP = π together with Σπ = 1 by Gaussian elimination over the rational numbers; if the system has no single solution, the chain has several closed classes and the tool says so.
Worked example
Weather as a chain: sunny stays sunny with 0.9 and turns rainy with 0.1; rainy turns sunny with 0.5 and stays rainy with 0.5. Starting on a sunny day, the chance of sun two days later is 0.86. In the long run π_sunny · 0.1 = π_rainy · 0.5, so π = [5/6, 1/6]: about 83 % of days are sunny.
Tips
- Write 1/3 rather than 0.333 so a row adds up to exactly 1.
- States are numbered in the order of the rows: the first row is state 1.
Frequently asked questions
How is the distribution after n steps calculated?
The starting row vector is multiplied by the transition matrix n times: πₙ = π₀ · Pⁿ. With P = [0.9 0.1; 0.5 0.5] and a start in state 1, one step gives [0.9, 0.1] and two steps give [0.9·0.9 + 0.1·0.5, …] = [0.86, 0.14].
What is the stationary distribution?
The distribution π that does not change from one step to the next: π · P = π, with the entries adding up to 1. The calculator solves these linear equations exactly. For the matrix above, π = [5/6, 1/6].
What does the mean return time mean?
If the chain is in state i, it takes on average 1 / πᵢ steps to come back to it. With π = [5/6, 1/6], state 1 recurs every 1.2 steps on average and state 2 every 6 steps.
What is an absorbing state?
A state you can never leave: its row has 1 on the diagonal. In a chain with one absorbing state that can be reached from everywhere, the stationary distribution puts all the probability there.
Does the chain always settle to the stationary distribution?
Not always. A periodic chain, such as one that swaps between two states every step, keeps oscillating even though a stationary distribution exists. Compare the n-step result for large n with π to see.