# Markov Chains – Tough Given transition matrix: \[ P = \begin{pmatrix} & 1 & \\ \frac{1}{2} & & \frac{1}{2} \\ & 1 & \end{pmatrix} \] Let the states be labeled as 1, 2, 3. --- ## 1. **Is the chain irreducible?** A Markov chain is **irreducible** if it is possible to get from any state to any other state (not necessarily in one step). ### **Step-by-Step Analysis** - **From state 1:** - Can go to 2 in 1 step. - From 2, can go to 1 or 3 in 1 step. - **From state 2:** - Can go to 1 or 3 in 1 step. - From 1, can go to 2. - From 3, can go to 2. - **From state 3:** - Can go to 2 in 1 step. - From 2, can go to 1 or 3. #### **Summary:** - 1 → 2 (1 step), 2 → 3 (1 step), 3 → 2 (1 step), 2 → 1 (1 step), so all states communicate. **Conclusion:** **Yes, the chain is irreducible.** --- ## 2. **Find the stationary distribution, if it exists.** Let the stationary distribution be \(\pi = (\pi_1, \pi_2, \pi_3)\), where \(\pi P = \pi\) and \(\pi_1+\pi_2+\pi_3=1\). ### **System of Equations** \[ \pi_1 = \cdot\pi_1 + \frac{1}{2}\pi_2 + \cdot\pi_3 = \frac{1}{2}\pi_2 \] \[ \pi_2 = 1\cdot\pi_1 + \cdot\pi_2 + 1\cdot\pi_3 = \pi_1 + \pi_3 \] \[ \pi_3 = \cdot\pi_1 + \frac{1}{2}\pi_2 + \cdot\pi_3 = \frac{1}{2}\pi_2 \] Also, \(\pi_1 + \pi_2 + \pi_3 = 1\). #### Substitute \(\pi_1 = \pi_3 = x\), and \(\pi_2 = y\): From above: - \(\pi_1 = \pi_3 = \frac{1}{2}\pi_2\) Let \(x = \frac{1}{2}y \implies y = 2x\). Now, normalization: \[ \pi_1 + \pi_2 + \pi_3 = x + y + x = 2x + y = 2x + 2x = 4x = 1 \implies x = \frac{1}{4} \] \[ y = 2x = \frac{1}{2} \] So, \[ \boxed{\pi = \left( \frac{1}{4}, \frac{1}{2}, \frac{1}{4} \right)} \] --- ## 3. **Is the chain ergodic?** A chain is **ergodic** if it is irreducible and aperiodic. - We already showed it is irreducible. - **Aperiodicity:** - State 1 can only return to itself in even steps (since from 1 → 2 → 1, takes 2 steps). - Similarly, state 3. - State 2 can return to itself in 2 steps (2 → 1 → 2 or 2 → 3 → 2), but not in 1 step. All states have period 2 (must return in even steps). **Conclusion:** **The chain is NOT ergodic, because it is not aperiodic (period = 2).** --- ## **Summary Table** | Part | Answer | |------|--------| | 1 | Yes, irreducible | | 2 | Stationary distribution: \(\left(\frac{1}{4}, \frac{1}{2}, \frac{1}{4}\right)\) | | 3 | Not ergodic (period = 2) |

# Markov Chains – Tough Given transition matrix: \[ P = \begin{pmatrix} 0 & 1 & 0 \\ \frac{1}{2} & 0 & \frac{1}{2} \\ 0 & 1 & 0 \end{pmatrix} \] Let the states be labeled as 1, 2, and 3. --- ## 1. **Is the chain irreducible?** A Markov chain is **irreducible** if every state can be reached from any other state. ### **Step-by-Step Analysis** - **From state 1:** - Can transition to state 2 in one step. - **From state 2:** - Can transition to state 1 or state 3 in one step. - **From state 3:** - Can transition to state 2 in one step. ### **Communication Check** - 1 can reach 2. - 2 can reach 1 and 3. - 3 can reach 2, which can reach 1. **Conclusion:** **Yes, the chain is irreducible.** --- ## 2. **Find the stationary distribution, if it exists.** Let the stationary distribution be \(\pi = (\pi_1, \pi_2, \pi_3)\), satisfying \(\pi P = \pi\) and \(\pi_1 + \pi_2 + \pi_3 = 1\). ### **System of Equations** The equations derived from \(\pi P = \pi\) are: \[ \pi_1 = 0 \cdot \pi_1 + \frac{1}{2}\pi_2 + 0 \cdot \pi_3 = \frac{1}{2}\pi_2 \] \[ \pi_2 = 1 \cdot \pi_1 + 0 \cdot \pi_2 + 1 \cdot \pi_3 = \pi_1 + \pi_3 \] \[ \pi_3 = 0 \cdot \pi_1 + \frac{1}{2}\pi_2 + 0 \cdot \pi_3 = \frac{1}{2}\pi_2 \] #### Setting Variables Let \(\pi_1 = x\) and \(\pi_3 = z\). From the first and third equations: - \(\pi_1 = \frac{1}{2}\pi_2 \Rightarrow x = \frac{1}{2}y\) - \(\pi_3 = \frac{1}{2}\pi_2 \Rightarrow z = \frac{1}{2}y\) Now, summing gives: \[ x + y + z = 1 \Rightarrow \frac{1}{2}y + y + \frac{1}{2}y = 1 \Rightarrow 2y = 1 \Rightarrow y = \frac{1}{2} \] Thus: \[ x = \frac{1}{4}, \quad z = \frac{1}{4} \] ### **Final Stationary Distribution** \[ \boxed{\pi = \left( \frac{1}{4}, \frac{1}{2}, \frac{1}{4} \right)} \] --- ## 3. **Is the chain ergodic?** A Markov chain is **ergodic** if it is both irreducible and aperiodic. - Since we determined that the chain is irreducible, we now check for aperiodicity. ### **Periodicity Check** - State 1 returns to itself in 2 steps (1 → 2 → 1). - State 2 can return in 1 or 2 steps. - State 3 also returns in 2 steps (3 → 2 → 3). Since states 1 and 3 have a period of 2, and state 2 can return in both odd and even steps, the overall chain has a period of 2. **Conclusion:** **The chain is NOT ergodic due to periodicity (period = 2).** --- ## **Summary Table** | Part | Answer | |------|--------| | 1 | Yes, irreducible | | 2 | Stationary distribution: \(\left(\frac{1}{4}, \frac{1}{2}, \frac{1}{4}\right)\) | | 3 | Not ergodic (period = 2) |