Consider a Markov chain \((X_t, Y_t)\) with \(X_t \in \mathcal{X}\) and \(Y_t \in \mathcal{Y}\) with initial distribution \(X_0 \in dx_0 = \mu_0(dx_0)\) and described by the dynamics
\[ \begin{align} \left\{ \begin{aligned} (X_{t+1} \in dx_{t+1}) \mid (X_t = x_t) \; &= \; f(x_t, x_{t+1}) \, dx_{t+1}\\ (Y_{t} \in dy_{t}) \mid (X_{t} = x_{t}) \; &= \; g(x_{t}, y_{t}) \, dy_{t} \end{aligned} \right. \end{align} \]
for some transition “density” \(f(\cdot, \cdot)\) and observation density \(g(\cdot, \cdot)\). These functions could be time-dependent but we will assume not for lightening notations.