Osmosis Data Sheet

ANSWER

Interpretation of the {ai} as transition rates; the transition rate matrix (infinitesimal generator) Q

Assume here that Pi,i = 0 for all i ∈ S. ai can be interpreted as the transition rate out of state i given that X(t) = i; the intuitive idea being that the exponential holding time will end, independent of the past, in the next dt units of time with probability aidt. This can be made rigorous recalling the little o(t) results for the Poisson process: Let {Ni(t)} denote a Poisson counting process at rate ai . Then P(Ni(h) > 0) = aih + o(h) and P(Ni(h) = 1) = aih + o(h) (and P(Ni(h) > 1) = o(h)).

Thus

lim h↓0 P(X(h) ≠ i | X(0) = i)/h = lim h↓0 P(Ni(h) = 1)/h = lim h↓0 (aih + o(h))/h = ai …….. (1)

More generally, for j ≠i, we additionally use the Markov property asserting that once leaving state i the chain will, independent of the past, next go to state j with probability

Pi,j to obtain

P’ i,j (0) = lim h↓0 Pi,j (h)/h = lim h↓0 P(Ni(h) = 1)Pi,j/h = aiPi,j …….. (2)

ai Pi,j can thus be interpreted as the transition rate from state i to state j given that the chain is currently in state i.

When i = j, Pi,i(h) = 1 − P(X(h) ≠i | X(0) = i) and so

P’ i,i(0) = lim h↓0 (Pi,i(h) − 1)/h = lim h↓0 −P(Ni(h) = 1)/h = −ai ……(3)

Definition 1.2

The matrix Q = P’(0) given explicitly by (2) and (3) is called the transition rate matrix, or infinitesimal generator, of the Markov chain

For example, if S = {0, 1, 2, 3, 4}, then

Q = −a0         a0P0,1          a0P0,2           a0P0,3           a0P0,4

       a1P1,0    −a1                a1P1,2            a1P1,3           a1P1,4

       a2P2,0    a2P2,1          −a2                a2P2,3           a2P2,4

       a3P3,0    a3P3,1          a3P3,2          −a3                 a3P3,4

       a4P4,0    a4P4,3          a4P4,3          a4P4,3           −a4

Note in passing that since we assume that Pi,i = 0, i ∈ S, we conclude that each row of Q sums to 0.

For a (non-explosive) CTMC with transition rate matrix Q = P ‘(0) as in Definition 1.2