Markov generators and stochastic conventions

A continuous-time Markov generator specifies transition rates, not probabilities for a whole time step. This reference fixes the conventions used in the finite-ring TASEP sequence: probability distributions are columns, an off-diagonal matrix entry points from its column to its row, and time evolution preserves positivity and total probability. The formulas below concern a finite state space; infinite systems need additional existence and domain assumptions.

Helpful background. Probability and Markov generators develops the same notation from elementary examples. The TASEP model fixes the allowed particle jumps and periodic boundary.

Rates and the probability generator

Let the states be $a=1,\ldots,d$, and let $w(b\mid a)\ge0$ be the rate of a transition from $a$ to a distinct state $b$. Its units are inverse time. Define the escape rate and generator by

$$q_a=\sum_{b\ne a}w(b\mid a),\qquad Q_{ba}=w(b\mid a)\quad(b\ne a),\qquad Q_{aa}=-q_a.$$

For a probability column $p$, the master equation is

$$\frac{dp}{dt}=Qp,\qquad p_a\ge0,\qquad \boldsymbol 1^{\mathsf T}p=1.$$

Thus columns sum to zero: $\boldsymbol 1^{\mathsf T}Q=0$. Each column describes the losses and gains caused by starting in its state. For small positive $h$,

$$\Pr(X_{t+h}=b\mid X_t=a) =\delta_{ba}+hQ_{ba}+o(h).$$

The diagonal entry is negative because probability leaves $a$; it is not a negative transition probability. Conditional on currently being in $a$ with $q_a\gt0$, the next holding time has survival probability $e^{-q_a t}$, and the next state is $b$ with probability $w(b\mid a)/q_a$. A state with $q_a=0$ is absorbing.

This is the column convention of Golinelli and Mallick 2006, § II.A, pp. 2–3, equations (1)–(2), arXiv v1 PDF. Their Markov matrix is called $M$; here $M$ denotes a particle count and the generator is $Q$.

Observables and row distributions

An observable $f$ assigns a real number $f_a$ to each state. Its expectation is $f^{\mathsf T}p$, so

$$\frac{d}{dt}\mathbb E[f(X_t)] =f^{\mathsf T}Qp =(Q^{\mathsf T}f)^{\mathsf T}p.$$

The backward generator acting on observables is therefore $\mathcal L=Q^{\mathsf T}$:

$$(\mathcal L f)_a =\sum_{b\ne a}w(b\mid a)(f_b-f_a), \qquad \mathcal L\boldsymbol 1=0.$$

Likewise, if the probability distribution is written as a row $\rho=p^{\mathsf T}$, its equation is $\dot\rho=\rho Q^{\mathsf T}$. Changing the orientation requires changing the multiplication order as well.

Object	Evolution or condition
Probability column $p$	$\dot p=Qp$
Probability row $\rho$	$\dot\rho=\rho Q^{\mathsf T}$
Observable $f$	Backward generator $\mathcal L f=Q^{\mathsf T}f$
Conserved total probability	$\boldsymbol 1^{\mathsf T}Q=0$
Stationary column $\pi$	$Q\pi=0$, $\boldsymbol 1^{\mathsf T}\pi=1$

For a two-state process $1\to2$ at rate $a\gt0$ and $2\to1$ at rate $b\gt0$,

$$Q=\begin{pmatrix}-a&b\\a&-b\end{pmatrix}, \qquad \pi=\frac1{a+b}\begin{pmatrix}b\\a\end{pmatrix}.$$

Starting in state 1 gives $\dot p(0)=(-a,a)^{\mathsf T}$. This one-column check detects a transpose immediately when $a\ne b$.

Stationarity, balance and decay modes

Stationarity requires total inflow to equal total outflow at each state. Detailed balance is the stronger pairwise condition

$$w(b\mid a)\pi_a=w(a\mid b)\pi_b \quad\text{for every }a\ne b.$$

A stationary process need not satisfy it. On a TASEP ring, an allowed forward jump usually has no allowed reverse jump. A nonzero stationary current can persist while every configuration probability is constant.

The two-site, one-particle ring is a useful exception: its two configurations exchange at equal rates and do satisfy detailed balance, even though each labelled forward bond records mean count rate $r/2$. The two directed bonds connect the same pair of configurations. Keep configuration-space probability flow distinct from a specified bond-counting observable.

For a finite irreducible continuous-time process, the stationary distribution is unique and strictly positive. Every other generator eigenvalue has negative real part. One way to see the relevant spectral distinction is that $e^{tQ}$, for $t\gt0$, is a strictly positive stochastic matrix: a path with any finite number of jumps has positive probability, as does waiting. Perron–Frobenius then gives a simple eigenvalue 1 and strict contraction of other eigenvalues in modulus. This statement concerns the continuous-time semigroup; the embedded chain observed only at jump times may be periodic.

If $Qv=Ev$ and $E\ne0$, then

$$0=\boldsymbol 1^{\mathsf T}Qv =E\boldsymbol 1^{\mathsf T}v, \qquad\text{so}\qquad \boldsymbol 1^{\mathsf T}v=0.$$

Such a mode is not itself a normalized probability distribution. A real mode can describe a perturbation $p(t)=\pi+\alpha e^{Et}v$ when the initial column is nonnegative. Complex modes enter real distributions in conjugate pairs. For a single eigenmode the decay time is $-1/\operatorname{Re}E$; a defective matrix may also produce polynomial factors in time. Do not assume a symmetric generator, an orthogonal eigenbasis or diagonalizability merely because a matrix conserves probability.

The TASEP Library example has a mode with $E=-3r$ on four sites. Its other eigenvalues include $-r$, so that selected mode does not determine the slowest relaxation.

Bond currents and time units

For homogeneous periodic TASEP, let $\eta_i\in\{0,1\}$ and identify site $N+1$ with site 1. A jump $i\to i+1$ occurs at rate $r$ when $\eta_i(1-\eta_{i+1})=1$. The instantaneous expected bond current is

$$j_i(t)=r\,\mathbb E[\eta_i(1-\eta_{i+1})].$$

It counts forward hops per unit time across that bond. For unit lattice spacing, the total expected jump rate is $\sum_i j_i(t)$; the mean number of jumps per particle per unit time is that sum divided by $M$. These are distinct observables.

Under the uniform stationary distribution at fixed $0\lt M\lt N$,

$$j_i=j=r\frac{M(N-M)}{N(N-1)},\qquad \sum_i j_i=Nj.$$

The stationary-current lesson derives the joint probability by counting configurations. At $N=4,M=2$, the three rates are $j=r/3$ per bond, $4r/3$ for the whole ring, and $2r/3$ per particle. The density expression $r\rho(1-\rho)$, with $\rho=M/N$, omits the factor $N/(N-1)$ in this fixed-particle finite system.

Writing $Q=rQ_0$ and $s=rt$ gives $dp/ds=Q_0p$. Setting $r=1$ means measuring time in units $r^{-1}$; it does not remove the distinction between rates and probabilities. The sources’ unit forward rate and zero backward rate agree with this normalization after replacing their site and particle counts by $N$ and $M$ (Golinelli and Mallick 2004, § 2.1, p. 3, arXiv v1 PDF).

Uniformization and its truncation error

Choose a constant $\nu\ge\max_a q_a$ with $\nu\gt0$, and set

$$P=I+\frac Q\nu.$$

Its entries are nonnegative and its columns sum to one. Since $Q=\nu(P-I)$ and $P$ commutes with $I$,

$$e^{tQ}p(0) =e^{-\nu t}\sum_{k=0}^{\infty} \frac{(\nu t)^k}{k!}P^k p(0).$$

This is a Poisson mixture of stochastic updates, including possible self-transitions. It proves positivity and normalization without diagonalizing $Q$. For the four-site, two-particle TASEP example, the maximum escape rate is $2r$, so $\nu=2r$ is valid.

If the sum stops at $K$, its omitted terms are nonnegative. Because $\|P^kp(0)\|_1=1$, the exact truncation error in the probability-column 1-norm is the Poisson tail

$$\left\|p(t)-p^{(K)}(t)\right\|_1 =\sum_{k=K+1}^{\infty}e^{-\nu t}\frac{(\nu t)^k}{k!}.$$

This equality concerns exact arithmetic and a nonnegative normalized initial column. Floating-point error must be checked separately. Renormalizing a truncated sum changes the approximation and removes the simple interpretation of its missing mass; report whether such a step is taken. If all escape rates vanish, $Q=0$ and $p(t)=p(0)$ directly.

Observation times and jump times

Recording the state at fixed times samples its time-dependent probability distribution. Recording it after every jump samples a different chain. In a stationary ensemble, the mean departure rate from state $a$ is $\pi_a q_a$. For a finite irreducible chain, these rates also determine the long-run empirical frequencies along a trajectory. The embedded jump-chain stationary weights are

$$\widehat\pi_a =\frac{\pi_a q_a}{\sum_b\pi_b q_b},$$

provided the denominator is positive. Indeed, its transition matrix has entries $w(b\mid a)/q_a$ for $b\ne a$, and multiplying by these weights reproduces them by stationary flow balance. States with zero escape rate require separate treatment.

For the six-state TASEP ring in order $(12,13,14,23,24,34)$, the escape rates are $r(1,2,1,1,2,1)$. Thus the stationary time distribution is uniform, but the jump-chain stationary weights are $(1,2,1,1,2,1)/8$. The two alternating configurations appear twice as often at jump instants because they have twice the escape rate. This embedded chain is periodic; its invariant weights and long-run empirical frequencies should not be confused with pointwise convergence of its distribution after exactly $k$ jumps.

The laboratory compares independent trajectories at fixed observation times with uniformization. Statistical sampling error, finite-time relaxation, deterministic truncation error and a wrong transition direction are different sources of discrepancy. Neither a fixed random seed nor a small stationary residual distinguishes them automatically.

References

• Golinelli, Olivier, and Kirone Mallick. “Bethe Ansatz calculation of the spectral gap of the asymmetric exclusion process.” Journal of Physics A: Mathematical and General 37(10), 3321–3331 (2004). DOI. Open PDF, arXiv:cond-mat/0312371v1, submitted 2003; locators use its printed pages.
• Golinelli, Olivier, and Kirone Mallick. “The asymmetric simple exclusion process: an integrable model for non-equilibrium statistical mechanics.” Journal of Physics A: Mathematical and General 39(41), 12679–12705 (2006). DOI. Open PDF, arXiv:cond-mat/0611701v1; locators use its printed pages.