sojourn times in finite Markov chain
up vote
2
down vote
favorite
Let $X = (X_{n})_{n geq 0}$ be a homogeneous irreducible Markov chain on the state space $E := {1,dots ,N}$, $N in mathbb{N}$, $N geq 2$ and transition probability Matrix $P$. Let $B subset E$ be a proper subset of the state space $E$. For simplicity we assume that $B = {1,2,dots,L}$, $1 leq L < N$.
We call "sojourn of $X$ in $B$" any sequence $X_{m},X_{m+1},dots,X_{m+k}$ where $k geq 1$, $X_{m},X_{m+1},dots,X_{m+k-1} in B$, $X_{m+k} notin B$ and if $m > 0$, $X_{m-1} notin B$. This sojourn begins at time $m$ and finishes at time $m+k$. It lasts $k$.
Now let $V_{n}$, $n geq 1$ be the random variable "state of $B$ in which the $n^{text{th}}$ sojourn of $X$ begins".
Maybe the answer is obvious. I want to show that $(V_{n})_{ngeq 1}$ is a homogeneous Markov chain on the state space $B$. Hope someone can help me. Thanks!
probability markov-chains
edited Nov 17 at 13:10
Bernard
116k637108
asked Nov 17 at 13:07
wayne
377112
add a comment |
up vote
2
down vote
favorite
Let $X = (X_{n})_{n geq 0}$ be a homogeneous irreducible Markov chain on the state space $E := {1,dots ,N}$, $N in mathbb{N}$, $N geq 2$ and transition probability Matrix $P$. Let $B subset E$ be a proper subset of the state space $E$. For simplicity we assume that $B = {1,2,dots,L}$, $1 leq L < N$.
We call "sojourn of $X$ in $B$" any sequence $X_{m},X_{m+1},dots,X_{m+k}$ where $k geq 1$, $X_{m},X_{m+1},dots,X_{m+k-1} in B$, $X_{m+k} notin B$ and if $m > 0$, $X_{m-1} notin B$. This sojourn begins at time $m$ and finishes at time $m+k$. It lasts $k$.
Now let $V_{n}$, $n geq 1$ be the random variable "state of $B$ in which the $n^{text{th}}$ sojourn of $X$ begins".
Maybe the answer is obvious. I want to show that $(V_{n})_{ngeq 1}$ is a homogeneous Markov chain on the state space $B$. Hope someone can help me. Thanks!
probability markov-chains
edited Nov 17 at 13:10
Bernard
116k637108
asked Nov 17 at 13:07
wayne
377112
It is due to the memorylessness of the Markov chain $X_n$. You can formally prove the memorylessness property of $V_n$ by introducing all possible sequences of states between $V_n$ and $V_{n+1}$
– Arnaud Mégret
Nov 17 at 16:37
can you please write it down formally
– wayne
Nov 17 at 16:43
@ArnaudMégret - i agree that this should be "obvious" from the memoryless property, but i also have trouble writing it down formally. also, the OP includes $X$ being irreducible, but my gut feel is that $X$ being irreducible would lead to $V$ being irreducible, but $X$ being not irreducible would still lead to $V$ being a M.C. (albeit also not irreducible). am i missing something subtle here?
– antkam
Nov 17 at 19:58
@antkam I think you are right concerning the irreductible property. I think it is a sufficient condition for an unlimited number of sojourns to be sure to happen.
– Arnaud Mégret
Nov 17 at 21:29
add a comment |
up vote
2
down vote
favorite
up vote
2
down vote
favorite
Let $X = (X_{n})_{n geq 0}$ be a homogeneous irreducible Markov chain on the state space $E := {1,dots ,N}$, $N in mathbb{N}$, $N geq 2$ and transition probability Matrix $P$. Let $B subset E$ be a proper subset of the state space $E$. For simplicity we assume that $B = {1,2,dots,L}$, $1 leq L < N$.
We call "sojourn of $X$ in $B$" any sequence $X_{m},X_{m+1},dots,X_{m+k}$ where $k geq 1$, $X_{m},X_{m+1},dots,X_{m+k-1} in B$, $X_{m+k} notin B$ and if $m > 0$, $X_{m-1} notin B$. This sojourn begins at time $m$ and finishes at time $m+k$. It lasts $k$.
Now let $V_{n}$, $n geq 1$ be the random variable "state of $B$ in which the $n^{text{th}}$ sojourn of $X$ begins".
Maybe the answer is obvious. I want to show that $(V_{n})_{ngeq 1}$ is a homogeneous Markov chain on the state space $B$. Hope someone can help me. Thanks!
probability markov-chains
edited Nov 17 at 13:10
Bernard
116k637108
asked Nov 17 at 13:07
wayne
377112
Let $X = (X_{n})_{n geq 0}$ be a homogeneous irreducible Markov chain on the state space $E := {1,dots ,N}$, $N in mathbb{N}$, $N geq 2$ and transition probability Matrix $P$. Let $B subset E$ be a proper subset of the state space $E$. For simplicity we assume that $B = {1,2,dots,L}$, $1 leq L < N$.
We call "sojourn of $X$ in $B$" any sequence $X_{m},X_{m+1},dots,X_{m+k}$ where $k geq 1$, $X_{m},X_{m+1},dots,X_{m+k-1} in B$, $X_{m+k} notin B$ and if $m > 0$, $X_{m-1} notin B$. This sojourn begins at time $m$ and finishes at time $m+k$. It lasts $k$.
Now let $V_{n}$, $n geq 1$ be the random variable "state of $B$ in which the $n^{text{th}}$ sojourn of $X$ begins".
Maybe the answer is obvious. I want to show that $(V_{n})_{ngeq 1}$ is a homogeneous Markov chain on the state space $B$. Hope someone can help me. Thanks!
probability markov-chains
probability markov-chains
edited Nov 17 at 13:10
Bernard
116k637108
asked Nov 17 at 13:07
wayne
377112
edited Nov 17 at 13:10
Bernard
116k637108
asked Nov 17 at 13:07
wayne
377112
edited Nov 17 at 13:10
Bernard
116k637108
edited Nov 17 at 13:10
Bernard
116k637108
edited Nov 17 at 13:10
Bernard
116k637108
116k637108
asked Nov 17 at 13:07
wayne
377112
asked Nov 17 at 13:07
wayne
377112
asked Nov 17 at 13:07
wayne
377112
377112
It is due to the memorylessness of the Markov chain $X_n$. You can formally prove the memorylessness property of $V_n$ by introducing all possible sequences of states between $V_n$ and $V_{n+1}$
– Arnaud Mégret
Nov 17 at 16:37
can you please write it down formally
– wayne
Nov 17 at 16:43
@ArnaudMégret - i agree that this should be "obvious" from the memoryless property, but i also have trouble writing it down formally. also, the OP includes $X$ being irreducible, but my gut feel is that $X$ being irreducible would lead to $V$ being irreducible, but $X$ being not irreducible would still lead to $V$ being a M.C. (albeit also not irreducible). am i missing something subtle here?
– antkam
Nov 17 at 19:58
@antkam I think you are right concerning the irreductible property. I think it is a sufficient condition for an unlimited number of sojourns to be sure to happen.
– Arnaud Mégret
Nov 17 at 21:29
add a comment |
It is due to the memorylessness of the Markov chain $X_n$. You can formally prove the memorylessness property of $V_n$ by introducing all possible sequences of states between $V_n$ and $V_{n+1}$
– Arnaud Mégret
Nov 17 at 16:37
can you please write it down formally
– wayne
Nov 17 at 16:43
@ArnaudMégret - i agree that this should be "obvious" from the memoryless property, but i also have trouble writing it down formally. also, the OP includes $X$ being irreducible, but my gut feel is that $X$ being irreducible would lead to $V$ being irreducible, but $X$ being not irreducible would still lead to $V$ being a M.C. (albeit also not irreducible). am i missing something subtle here?
– antkam
Nov 17 at 19:58
@antkam I think you are right concerning the irreductible property. I think it is a sufficient condition for an unlimited number of sojourns to be sure to happen.
– Arnaud Mégret
Nov 17 at 21:29
It is due to the memorylessness of the Markov chain $X_n$. You can formally prove the memorylessness property of $V_n$ by introducing all possible sequences of states between $V_n$ and $V_{n+1}$
– Arnaud Mégret
Nov 17 at 16:37
It is due to the memorylessness of the Markov chain $X_n$. You can formally prove the memorylessness property of $V_n$ by introducing all possible sequences of states between $V_n$ and $V_{n+1}$
– Arnaud Mégret
Nov 17 at 16:37
can you please write it down formally
– wayne
Nov 17 at 16:43
can you please write it down formally
– wayne
Nov 17 at 16:43
@ArnaudMégret - i agree that this should be "obvious" from the memoryless property, but i also have trouble writing it down formally. also, the OP includes $X$ being irreducible, but my gut feel is that $X$ being irreducible would lead to $V$ being irreducible, but $X$ being not irreducible would still lead to $V$ being a M.C. (albeit also not irreducible). am i missing something subtle here?
– antkam
Nov 17 at 19:58
@ArnaudMégret - i agree that this should be "obvious" from the memoryless property, but i also have trouble writing it down formally. also, the OP includes $X$ being irreducible, but my gut feel is that $X$ being irreducible would lead to $V$ being irreducible, but $X$ being not irreducible would still lead to $V$ being a M.C. (albeit also not irreducible). am i missing something subtle here?
– antkam
Nov 17 at 19:58
@antkam I think you are right concerning the irreductible property. I think it is a sufficient condition for an unlimited number of sojourns to be sure to happen.
– Arnaud Mégret
Nov 17 at 21:29
@antkam I think you are right concerning the irreductible property. I think it is a sufficient condition for an unlimited number of sojourns to be sure to happen.
– Arnaud Mégret
Nov 17 at 21:29
add a comment |
1 Answer
1
active
oldest
votes
up vote
1
down vote
Let S stands for the sequence of X's states up to (and including) the beginning of the nth sojourn and T be the sequence after S up to (and including) the beginning of the (n+1)th sojourn.
You have $P(V_{n+1}/V_n,V_{n-1},...)=sum_S[P(S/V_n,V_{n-1},...)sum_TP(T,V_{n+1}/S,V_n,V_{n-1},...)]$ (E)
The inner sum is equal to $sum_TP(T,V_n+1/V_n)$ due to the memoryless property so it does not depend on $S$ and the right part of (E) can be rewritten :
$P(V_{n+1}/V_n,V_{n-1},...)=sum_SP(S/V_n,V_{n-1},...).sum_TP(T,V_{n+1}/V_n)]=sum_TP(T,V_{n+1}/V_n)=P(V_{n+1}/V_n)$
answered Nov 17 at 21:19
Arnaud Mégret
351414
thanks! and it's funny how the irreducible property of $X$ is not explicitly used but is needed for $V$ to be well defined. (Otherwise after some $V_n$ there may never be the next $V_{n+1}$.)
– antkam
Nov 18 at 16:04
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
1
down vote
Let S stands for the sequence of X's states up to (and including) the beginning of the nth sojourn and T be the sequence after S up to (and including) the beginning of the (n+1)th sojourn.
You have $P(V_{n+1}/V_n,V_{n-1},...)=sum_S[P(S/V_n,V_{n-1},...)sum_TP(T,V_{n+1}/S,V_n,V_{n-1},...)]$ (E)
The inner sum is equal to $sum_TP(T,V_n+1/V_n)$ due to the memoryless property so it does not depend on $S$ and the right part of (E) can be rewritten :
$P(V_{n+1}/V_n,V_{n-1},...)=sum_SP(S/V_n,V_{n-1},...).sum_TP(T,V_{n+1}/V_n)]=sum_TP(T,V_{n+1}/V_n)=P(V_{n+1}/V_n)$
answered Nov 17 at 21:19
Arnaud Mégret
351414
thanks! and it's funny how the irreducible property of $X$ is not explicitly used but is needed for $V$ to be well defined. (Otherwise after some $V_n$ there may never be the next $V_{n+1}$.)
– antkam
Nov 18 at 16:04
add a comment |
up vote
1
down vote
Let S stands for the sequence of X's states up to (and including) the beginning of the nth sojourn and T be the sequence after S up to (and including) the beginning of the (n+1)th sojourn.
You have $P(V_{n+1}/V_n,V_{n-1},...)=sum_S[P(S/V_n,V_{n-1},...)sum_TP(T,V_{n+1}/S,V_n,V_{n-1},...)]$ (E)
The inner sum is equal to $sum_TP(T,V_n+1/V_n)$ due to the memoryless property so it does not depend on $S$ and the right part of (E) can be rewritten :
$P(V_{n+1}/V_n,V_{n-1},...)=sum_SP(S/V_n,V_{n-1},...).sum_TP(T,V_{n+1}/V_n)]=sum_TP(T,V_{n+1}/V_n)=P(V_{n+1}/V_n)$
answered Nov 17 at 21:19
Arnaud Mégret
351414
thanks! and it's funny how the irreducible property of $X$ is not explicitly used but is needed for $V$ to be well defined. (Otherwise after some $V_n$ there may never be the next $V_{n+1}$.)
– antkam
Nov 18 at 16:04
add a comment |
up vote
1
down vote
up vote
1
down vote
Let S stands for the sequence of X's states up to (and including) the beginning of the nth sojourn and T be the sequence after S up to (and including) the beginning of the (n+1)th sojourn.
You have $P(V_{n+1}/V_n,V_{n-1},...)=sum_S[P(S/V_n,V_{n-1},...)sum_TP(T,V_{n+1}/S,V_n,V_{n-1},...)]$ (E)
The inner sum is equal to $sum_TP(T,V_n+1/V_n)$ due to the memoryless property so it does not depend on $S$ and the right part of (E) can be rewritten :
$P(V_{n+1}/V_n,V_{n-1},...)=sum_SP(S/V_n,V_{n-1},...).sum_TP(T,V_{n+1}/V_n)]=sum_TP(T,V_{n+1}/V_n)=P(V_{n+1}/V_n)$
answered Nov 17 at 21:19
Arnaud Mégret
351414
Let S stands for the sequence of X's states up to (and including) the beginning of the nth sojourn and T be the sequence after S up to (and including) the beginning of the (n+1)th sojourn.
You have $P(V_{n+1}/V_n,V_{n-1},...)=sum_S[P(S/V_n,V_{n-1},...)sum_TP(T,V_{n+1}/S,V_n,V_{n-1},...)]$ (E)
The inner sum is equal to $sum_TP(T,V_n+1/V_n)$ due to the memoryless property so it does not depend on $S$ and the right part of (E) can be rewritten :
$P(V_{n+1}/V_n,V_{n-1},...)=sum_SP(S/V_n,V_{n-1},...).sum_TP(T,V_{n+1}/V_n)]=sum_TP(T,V_{n+1}/V_n)=P(V_{n+1}/V_n)$
answered Nov 17 at 21:19
Arnaud Mégret
351414
answered Nov 17 at 21:19
Arnaud Mégret
351414
answered Nov 17 at 21:19
Arnaud Mégret
351414
answered Nov 17 at 21:19
Arnaud Mégret
351414
351414
thanks! and it's funny how the irreducible property of $X$ is not explicitly used but is needed for $V$ to be well defined. (Otherwise after some $V_n$ there may never be the next $V_{n+1}$.)
– antkam
Nov 18 at 16:04
add a comment |
thanks! and it's funny how the irreducible property of $X$ is not explicitly used but is needed for $V$ to be well defined. (Otherwise after some $V_n$ there may never be the next $V_{n+1}$.)
– antkam
Nov 18 at 16:04
thanks! and it's funny how the irreducible property of $X$ is not explicitly used but is needed for $V$ to be well defined. (Otherwise after some $V_n$ there may never be the next $V_{n+1}$.)
– antkam
Nov 18 at 16:04
thanks! and it's funny how the irreducible property of $X$ is not explicitly used but is needed for $V$ to be well defined. (Otherwise after some $V_n$ there may never be the next $V_{n+1}$.)
– antkam
Nov 18 at 16:04
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3002347%2fsojourn-times-in-finite-markov-chain%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
It is due to the memorylessness of the Markov chain $X_n$. You can formally prove the memorylessness property of $V_n$ by introducing all possible sequences of states between $V_n$ and $V_{n+1}$
– Arnaud Mégret
Nov 17 at 16:37
can you please write it down formally
– wayne
Nov 17 at 16:43
@ArnaudMégret - i agree that this should be "obvious" from the memoryless property, but i also have trouble writing it down formally. also, the OP includes $X$ being irreducible, but my gut feel is that $X$ being irreducible would lead to $V$ being irreducible, but $X$ being not irreducible would still lead to $V$ being a M.C. (albeit also not irreducible). am i missing something subtle here?
– antkam
Nov 17 at 19:58
@antkam I think you are right concerning the irreductible property. I think it is a sufficient condition for an unlimited number of sojourns to be sure to happen.
– Arnaud Mégret
Nov 17 at 21:29