Markov Chain Calculator

Markov chains model systems where the next state depends only on the current state.

Use the result as a learning aid. For classwork, still show the formula and intermediate reasoning so the final answer is not just a black-box number.

A finite Markov chain models a system whose next-state probabilities depend on its current state. With a row-vector convention, the distribution update is πₙ₊₁=πₙP, where P is the transition matrix. Repeating the update n times gives πₙ=π₀Pⁿ.

For a valid probability transition matrix, every entry should be between 0 and 1 and each row should sum to 1. The starting distribution should also contain nonnegative probabilities summing to 1. Under those conditions, matrix multiplication redistributes probability without changing the total probability mass.

For a two-state matrix P=[[0.8,0.2],[0.3,0.7]], the first row says that a system currently in state 1 stays there with probability 0.8 and moves to state 2 with probability 0.2. The second row describes transitions from state 2.

A steady-state distribution, when it exists and is reached, is unchanged by another transition. Whether convergence occurs depends on the structure of the chain; not every transition matrix converges from every starting state.

Check probability conservation after every update. For a valid stochastic matrix and a normalized starting distribution, the entries of πₙ should remain nonnegative and sum to approximately 1. Small floating-point deviations are normal; large deviations usually indicate an invalid matrix or a mismatched convention.

Long-run behavior depends on communicating classes and periodicity. An irreducible, aperiodic finite chain has a unique stationary distribution that attracts every starting distribution. Reducible chains can have multiple stationary distributions, while periodic chains can oscillate forever. Those structural facts explain why repeatedly multiplying by P may converge smoothly in one example but alternate in another.