Question 1

Steady state probabilities are independent of initial state.

Accepted Answer

Steady state probabilities are determined by the system, and are not affected by the initial state. They only depend on the system's transition probabilities, and when the steady state is reached, the probabilities are the same regardless of the initial conditions.

Question 2

The sum of the probabilities in a transition matrix equals the number of rows in the matrix.

Accepted Answer

The sum of probabilities in a transition matrix should be equal to 1, not the number of rows.

Question 3

Transition probabilities are conditional probabilities.

Accepted Answer

Transition probabilities are the probabilities of moving from one state to another, given the current state. Therefore, they are conditional probabilities.

Question 4

For Markov processes having the memoryless property,the prior states of the system must be considered in order to predict the future behavior of the system.

Accepted Answer

Markov processes with memoryless property only depend on the current state of the system to make predictions for the future behavior. Prior states are not considered.

Question 5

If a Markov chain has at least one absorbing state,steady-state probabilities cannot be calculated.

Accepted Answer

When a Markov chain has at least one absorbing state, the chain will eventually reach that state and stay there, making the steady-state probabilities undefined or unstable.

Question 6

A state,i,is an absorbing state if,when i = j,p_ij = 1.

Accepted Answer

This statement refers to the condition of "staying" in state i once it is reached, regardless of any transitions happening in the future. This means that the probability of transitioning from state i to any other state j is 0, and as a consequence, the probability of eventually returning to state i is 1 (since there is no other option). Therefore, p_ij = 1, which makes state i an absorbing state.

Question 7

A state i is a transient state if there exists a state j that is reachable from i,but the state i is not reachable from state j.

Accepted Answer

This is the definition of a transient state in Markov chain theory.

Question 8

State j is an absorbing state if p_ij = 1.

Accepted Answer

An absorbing state is defined as a state that, once entered, cannot be left. The given expression does not clearly define such a state. Instead, an absorbing state is characterized by having a transition probability of 1 to itself and 0 to all other states.

Question 9

A state i is an absorbing state if p_ii = 0.

Accepted Answer

An absorbing state i is defined as a state that, once entered, cannot be left. Therefore, it is characterized by having a transition probability of 1 when transitioning from itself to itself (pii = 1), not 0.

Question 10

If an absorbing state exists,then the probability that a unit will ultimately move into the absorbing state is given by the steady state probability.

Accepted Answer

The probability that a unit will ultimately move into an absorbing state is determined by the transition probabilities leading to that state, not by the steady-state probability, which applies to Markov chains without absorbing states.

Question 11

The fundamental matrix is used to calculate the probability of the process moving into each absorbing state.

Accepted Answer

This statement is true. The fundamental matrix is a component of the absorbing Markov chain, and it is used to calculate the probability of transitioning into each absorbing state.

Question 12

All entries in a matrix of transition probabilities sum to 1.

Accepted Answer

Each row in a matrix of transition probabilities sums to 1, not all entries combined.

Question 13

Markov processes use historical probabilities.

Accepted Answer

Markov processes are based on the concept that the probability of an event occurring depends only on the current state and not on previous states. Therefore, historical probabilities are used to predict future states in a Markov process.

Question 14

All Markov chain transition matrices have the same number of rows as columns.

Accepted Answer

This is true by definition of a Markov chain transition matrix. A Markov chain is a model for a system that transitions between different states, and the transition probabilities are represented by a matrix. In order for this matrix to be a Markov chain transition matrix, it must satisfy certain properties, including having the same number of rows as columns.

Question 15

All Markov chains have steady-state probabilities.

Accepted Answer

Not all Markov chains have steady-state probabilities because some chains do not converge to a steady state. This can happen in chains that are not irreducible or not positive recurrent, where some states might not be reachable from others, or where the chain does not settle into a long-term stable distribution.

Question 16

A Markov chain cannot consist of all absorbing states.

Accepted Answer

A Markov chain can consist of all absorbing states if every state is an absorbing state, meaning once entered, the process will not leave that state. This would technically be a trivial case of a Markov chain.

Question 17

A unique matrix of transition probabilities should be developed for each customer.

Accepted Answer

A unique matrix of transition probabilities is not necessary for each customer; a single matrix can often model the behavior of similar customers or segments, simplifying analysis and reducing complexity.

Question 18

The probability that the system is in state 2 in the 5th period is π₅(2).

Accepted Answer

The given expression does not provide enough information to determine the probability that the system is in state 2 in the 5th period. The correct expression for this probability would involve the transition matrix and initial state vector.

Question 19

All entries in a row of a matrix of transition probabilities sum to 1.

Accepted Answer

This is a fundamental property of transition matrices in probability theory. Each row represents the probabilities of moving from one state to all possible states, so the probabilities must add up to 1.

Question 20

When absorbing states are present,each row of the transition matrix corresponding to an absorbing state will have a single 1 and all other probabilities will be 0.

Accepted Answer

This is because in absorbing states, once the system reaches that state, it remains there and cannot transition to any other state. Therefore, the probability of transitioning to any other state from an absorbing state is 0, leading to a row of all 0's except for a single 1.

Quiz 17: Markov Processes