Joe Doob & Change of measures on path-space

Joseph Doob (1910 – 2004)

Consider a continuous time Markov process $X_t$ on the time interval $[0,T]$ and with value in the state space $\mathcal{X}$. This defines a probability $\mathbb{P}$ on the set of $\mathcal{X}$-valued paths. Now, it is often the case that one has to consider a perturbed probability distribution $\mathbb{Q}$ defined as

$$\begin{array}{}\text{(1)}& \frac{d\mathbb{Q}}{d\mathbb{P}}(x_{[0,T]})=\frac{1}{\mathcal{Z}}\exp [g(X_T)]\end{array}$$

for a (typically unknown) normalization constant $\mathcal{Z}$ and some function $g:\mathcal{X}\to \mathbb{R}$. For example, collecting a noisy observation $y_T\sim \mathcal{F}(X_T)+\text{(noise)}$ at time $T$, the distribution $\mathbb{Q}$ defined with the log-likelihood function $g(x)=\log \mathbb{P}(y_T\mid X_T=x)$ describes the dynamics of the Markov process $X_t$ conditioned on the observation $y_T$; we will use this interpretation in the following since this is the most common use case and gives the most intuitive interpretation. Doob h-transforms are a powerful tool to describe the dynamics of the conditioned process.

For convenience, let us use the notation ${\mathbb{E}}_x[\dots ]\equiv \mathbb{E}[\dots \mid x_t=x]$. For a test function $\phi :\mathcal{X}\to \mathbb{R}$ and a time increment $\delta >0$, we have

$$\begin{array}{}\text{(2)}& \begin{array}{rl}\mathbb{E}[\phi (x_{t+\delta })|x_t,y_T]& ={\mathbb{E}}_{x_t}[\phi (x_{t+\delta })\exp (g(x_T))]/{\mathbb{E}}_{x_t}[\exp (g(x_T))]\\ & =\frac{{\mathbb{E}}_{x_t}[\phi (x_{t+\delta })h(t+\delta ,x_{t+\delta })]}{h(t,x)}.\end{array}\end{array}$$

We have introduced the important function $h:[0,T]\times \mathcal{X}\to \mathbb{R}$ defined as

$$h(t,x)=\mathbb{E}[\exp [g(x_T)]\mid x_t=x]=\mathbb{P}(y_T\mid x_t=x).$$

One can readily check that the function $h$ satisfies the Kolmogorov equation

$$({\partial }_t+\mathcal{L})h=0$$

with boundary condition $h(T,x)=\exp [g(x)]$. Furthermore, denoting by $\mathcal{L}$ the infinitesimal generator of the Markov process $X_t$, we have:

$$\begin{array}{}\text{(3)}& \begin{array}{rl}{\mathbb{E}}_{x_t}[\phi (x_{t+\delta })& h(t+\delta ,x_{t+\delta })]\approx \phi (x_t)h(t,x_t)\\ & +\delta ({\partial }_t+\mathcal{L})[h\phi ](t,x_t)+o(\delta ).\end{array}\end{array}$$

The infinitesimal generator ${\mathcal{L}}^{\star }$ of the conditioned process is

$${\mathcal{L}}^{\star }\phi (t,x_t)=\underset{\delta \to 0^{+}}{lim}\frac{\mathbb{E}[\phi (x_{t+\delta })|x_t,y_T]-\phi (x_t)}{\delta }.$$

Plugging Equation 3 within Equation 2 directly gives that

$${\mathcal{L}}^{\star }\phi =\mathcal{L}\phi +\frac{\mathcal{L}[\phi h]}{h}+\phi \frac{{\partial }_{t}h}{h}.$$

The generator ${\mathcal{L}}^{\star }$ describes the dynamics of the conditioned process. In fact, the same computation holds with a more general change of measure of the type $$\begin{array}{}\text{(4)}& \frac{d\mathbb{Q}}{d\mathbb{P}}(x_{[0,T]})=\frac{1}{\mathcal{Z}}\exp \{{\int }_0^{T}f(s,X_s)ds+g(x_T)\}\end{array}$$

for some function $f:[0,T]\times \mathcal{X}\to \mathbb{R}$. One can define the function $h$ similarly as

$$\begin{array}{}\text{(5)}& h(t,x_t)=\mathbb{E}[\exp \{{\int }_t^{T}f(X_s)ds+g(x_T)\}\mid x_t].\end{array}$$

This function satisfies the Feynman-Kac formula $({\partial }_t+\mathcal{L}+f)h=0$ and one obtains entirely similarly that the probability distribution $\mathbb{Q}$ describes a Markov process with infinitesimal generator

$$\begin{array}{}\text{(6)}& {\mathcal{L}}^{\star }\phi =\mathcal{L}\phi +\frac{\mathcal{L}[h\phi ]}{h}+(\frac{{\partial }_{t}h}{h}+f)\phi .\end{array}$$

To see how this works, let us see a few examples:

General diffusions

Consider a diffusion process

$$dX=b(X)dt+\sigma (X)dW$$

with generator $\mathcal{L}\phi =b\mathrm{\nabla }\phi +\frac{1}{2}\sigma {\sigma }^{\mathrm{\top }}:{\mathrm{\nabla }}^2\phi$ and initial distribution ${\mu }_0(dx)$. We are interested in describing the dynamics of the “conditioned” process given by the probability distribution $\mathbb{Q}$ defined in Equation 4. Algebra applied to Equation 5 then shows that

$${\mathcal{L}}^{\star }\phi =\mathcal{L}\phi +\underset{=0}{\underbrace{\frac{\phi ({\partial }_t+\mathcal{L}+f)[h]}{h}}}+\sigma {\sigma }^{\mathrm{\top }}(\mathrm{\nabla }\log h)\mathrm{\nabla }\phi$$

where the function $h$ is described in Equation 5. Since $({\partial }_t+\mathcal{L}+f)h=0$, this reveals that the probability distribution $\mathbb{Q}$ describes a diffusion process $X^{\star }$ with dynamics

$$d{X}^{\star }=b(X^{\star })dt+\sigma (X^{\star })\{dW+u^{\star }(t,X^{\star })dt\}.$$

The additional drift term $\sigma (X^{\star })u^{\star }(t,X^{\star })dt$ is involves a “control” $u^{\star }(t,X^{\star })$ with $$\begin{array}{}\text{(7)}& u^{\star }(t,x)={\sigma }^{\mathrm{\top }}(x)\mathrm{\nabla }\log h(t,x).\end{array}$$

Note that the initial distribution of the conditioned process is

$${\mu }_0^{\star }(dx)=\frac{1}{\mathcal{Z}}{\mu }_0(dx)h(0,x).$$

Unfortunately, apart from a few straightforward cases such as a Brownian motion or an Ornstein-Uhlenbeck process, the function $h$ is generally intractable. However, there are indeed several numerical methods available to approximate it effectively.

Brownian bridge

What about a Brownian motion in ${\mathbb{R}}^D$ conditioned to hit the state $x_{\star }\in {\mathbb{R}}^D$ at time $t=T$, i.e. a Brownian bridge? In that case, the function $h$ is given by

$$h(t,x)=\mathbb{P}(B_T=x_{\star }\mid B_t=x)=\exp \{-\frac{\parallel x-x_{\star }{\parallel }^2}{2(T-t)}\}/{\mathcal{Z}}_{T-t}$$

for some irrelevant normalization constant ${\mathcal{Z}}_{T-t}$ that only depends on $T-t$. Plugging this into Equation 7 gives that the conditioned Brownian $X^{\star }$ motion has dynamics

$$d{X}^{\star }=-\frac{X^{\star }-x_{\star }}{T-t}dt+dB.$$

The additional drift term $-(X^{\star }-x_{\star })/(T-t)$ is intuitive: it points in the direction of $x^{\star }$ and gets increasingly large as $t\to T$.

Positive Brownian motion

What about a scalar Brownian conditioned to stay positive at all times? Let us consider $T$ and let us condition first on the event that the Brownian motion stays positive within $[0,T]$ and later consider the limit $T\to \mathrm{\infty }$. The function $h$ reads

$$h(t,x)=\mathbb{P}(B_t\text{ stays }>0\text{ on }[t,T]\mid B_t=x).$$

This can easily be calculated with the reflection principle. It equals

$$h(t,x)=1-2\mathbb{P}(B_T<0\mid B_T=x)=\mathbb{P}(\sqrt{T-t}\parallel \xi \parallel <x)$$

for a standard Gaussian $\xi \sim \mathcal{N}(0,1)$. Plugging this into Equation 7 gives that the additional drift term is

$$\mathrm{\nabla }\log h(t,x)=\frac{\exp (-x^2/(2(T-t)))}{x}\to \frac{1}{x}$$

as $T\to \mathrm{\infty }$. This shows that a Brownian motion conditioned to stay positive at all times has a upward drift of size $1/x$,

$$d{X}^{\star }=\frac{1}{X^{\star }}+dB.$$

Incidentally, it is the dynamics of a Bessel process of dimension $d=3$, i.e. the law of the modulus of a three-dimensional Brownian motion. More generally, if one conditions a Brownian motion to stay within a closed domain $\mathcal{D}$, the conditioned dynamics exhibit a repulsive drift term of size about $1/\text{dist}(x,\partial \mathcal{D})$ near the boundary $\partial \mathcal{D}$ of the domain, as described below.

Brownian motion staying in a domain

What about a Brownian motion conditioned to stay within a domain $\mathcal{D}$ forever? As before, consider an time horizon $T$ and define the function $h$ as

$$h(t,x)=\mathbb{P}(B_t\text{ stays in }\mathcal{D}\text{ on }[t,T]\mid B_t=x).$$

One can see that the function $h$ satisfies the PDE

$$({\partial }_t+\mathrm{\Delta })h=0$$

and equals zero on the boundary $\partial \mathcal{D}$ of the domain. Furthermore $h(t,x)\to 1$ as $t\to T$ for all $x\in \mathcal{D}$. Consider the eigenfunctions ${\psi }_k$ of the negative Laplacian $-\mathrm{\Delta }$ with Dirichlet boundary conditions on $\partial \mathcal{D}$. Recall that $-\mathrm{\Delta }$ is a positive operator with a discrete spectrum ${\lambda }_1\le {\lambda }_2\le \dots$ of non-negative eigenvalues. The eigenfunction corresponding to the smallest eigenvalue ${\lambda }_1$ is the principal eigenfunction ${\psi }_1$ and it is standard that it is a positive function within the domain $\mathcal{D}$, as a “slight” generalization of the Perron-Frobenius in linear algebra shows it. Expanding $h$ in the basis of eigenfunctions ${\psi }_k$ gives that

$$h(t,x)=\underset{\text{dominant contribution}}{\underbrace{c_1{e}^{-{\lambda }_1(T-t)}{\psi }_1(x)}}+\sum _{k\ge 2}{c}_k{e}^{-{\lambda }_k(T-t)}{\psi }_k(x).$$

Eigenfunctions of the Laplacian

Since we are interested in the regime $T\to \mathrm{\infty }$, it holds that

$${\mathrm{\nabla }}_x\log h(t,x)\to \mathrm{\nabla }\log {\psi }_1(x).$$

This shows that the conditioned Brownian motion has a drift term expressed in terms of the principal eigenfunction ${\psi }_1$ of the Laplacian:

$$d{X}^{\star }=\mathrm{\nabla }\log {\psi }_1(X^{\star })dt+dB.$$

For example, if $\mathcal{D}\equiv [0,L]$ for a 1D Brownian motion, the principal eigenfunction is ${\psi }_1(x)=\sin (\pi x/L)$. This shows that there is a upward drift of size $\sim 1/x$ near $x\approx 0$ and a downward drift of size $\sim 1/(L-x)$ near $x\approx L$.