For continuous data x∈RK, the most basic choice for the forward process is a Gaussian distribution.
In DDPM, the forward process allows us to sample any latent state zt directly from the data z0=x, or transition between two intermediate steps s<t, as follows:
DDIM introduces a hyperparameter σs for the variance of the reverse posterior, generalizing the stochastic process while maintaining the same marginals q(zt∣z0).
The reverse transition is defined by:
While setting σs2=(1−αt)(1−αs)(1−αt∣s) recovers the standard reverse posterior, DDIM generally defines a non-Markovian forward process; the forward transition now depends on z0:
Using a neural network xϕ(zt,t)≈z0, parameterized by ϕ, the approximated reverse process pϕ(zs∣zt) can be written in the same form.
The discrete-time and continuous-time diffusion losses are given by:
zt=αtz0+1−αt−σs21−αszs−αsz0+σsε, where ε∼N(0,I).
Transitioning to continuous-time involves reducing the time interval Δt=t−s to an infinitesimal dt, turning difference equations into differential equations.
By introducing εs=1−αszs−αsz0, βt=Δt1−αt∣s, and σs2=(1−αt)(1−αs)(1−αt∣s)η2, the forward step and its approximation are given by:
where f(z(t),t)=−21βtz(t)is the drift coefficient, g(t)=βt is the diffusion coefficient, and w(t) and wˉ(t)=w(1−t)−w(1) are standard Wiener processes in forward and reverse time, respectively.
Remark.
Here, η=0 corresponds to the deterministic DDIM setup where σs=0; the forward and reverse SDEs collapse into a single PF-ODE, establishing a unique, invertible path between the noise and the data.
On the other hand, η=1 corresponds to the stochastic DDPM setup.
It is worth noting that even in this stochastic regime, the reverse-time SDE does not inject arbitrary random noise; for g(t)=0, the diffusion term remains strictly coupled with the score-driven drift term.
Remark.
Introducing a neural network for ε-prediction, i.e., εϕ(zt,t)≈ε0, is more algebraically concise than x-prediction in this formulation, as the ε-prediction network directly yields the score function via the score-matching identity.
Indeed, it is standard practice for diffusion models to adopt an ε-prediction strategy.
Discrete Diffusion
For discrete data x∈{0,1}K represented as a one-hot vector, the most basic choice for the forward process is a Categorical distribution.
The forward process allows us to sample any latent state zt directly from the data z0=x, or transition between two intermediate steps s<t, as follows:
MDM defines the Kth category as a special [MASK] token, which serves as an absorbing state with a stationary distribution π=m=[0,…,0,1]⊤.
In the forward process, tokens either remain themselves or transition to the mask.
For s<t, the reverse posterior is given by:
Using a neural network xϕ(zt,t)≈z0, parameterized by ϕ and ensured to be ⟨xϕ,m⟩=0, the approximated reverse process pϕ(zs∣zt) can be written in the same form.
The discrete-time and continuous-time diffusion losses are given by:
ReMDM introduces a hyperparameter σt to MDM, enabling transitions to the mask during the reverse process and generalizing the stochastic process while maintaining the same marginals q(zt∣z0).
The reverse transition is defined by:
While setting σt=0 recovers the reverse posterior of MDM, ReMDM generally defines a non-Markovian forward process; the forward transition now depends on z0:
UDM defines the stationary distribution as a uniform across all categories, π=1/K.
In this regime, the forward process allows a token to transition to any other category (including itself), eventually erasing all signal as it converges to uniform noise.
For s<t, the reverse posterior is given by:
Using a neural network xϕ(zt,t)≈z0, parameterized by ϕ, the approximated reverse process can be written in the same form.
The continuous-time diffusion loss is given by:
where xˉ(i)=Kαtz0(i)+(1−αt), xˉϕ(i)=Kαtxϕ(i)(zt,t)+(1−αt), and i=argmaxj∈[K]zt(j).
Continuous-time formulation (CTMC)
Markovian discrete diffusion models randomly transit states on a discrete-time grid:
zt=(1−bt)zs+btvt, where bt∼Bern(1−αt∣s) and vt∼Cat(π).
Transitioning to continuous-time involves reducing the time interval Δt=t−s to an infinitesimal dt, turning difference equations into differential equations.
By introducing βt=Δt1−αt∣s, the transition probability matrix is:
P(s,t)=(1−βtΔt)I+βtΔtπ1⊤.
As the state remains unchanged over zero time, i.e., P(s,s)=I, we obtain the increment:
P(s,t)−P(s,s)=βtΔt(π1⊤−I),
which corresponds to the continuous-time CTMC evolving forward in time with the transition-rate matrix:
Q(t)=βt(π1⊤−I).
The forward evolution of the marginal probability distribution q(t)=αtz0+(1−αt)π is governed by the Kolmogorov forward equation:
q˙(t)=Q(t)q(t).
The relationship between the forward rate matrix Q(t) and the reverse rate matrix Qˉ(t) is governed by the entry-wise probability balance:
Qˉij(t)=Qji(t)qi(t)qj(t).
Reward-tilted Diffusion
The ultimate goal of a diffusion model is to generate samples following the data distribution q(z0).
However, in many applications, we seek samples under a specific reward function r.
To this end, we define the reward-tilted distribution by reweighting the original density with an exponential reward term:
qr(z0)∝q(z0)exp(βr(z0)).
Local search via guidance
In continuous diffusion, steering toward high reward can be achieved by modifying the score function.
By defining an intermediate reward-tilted density qr(zt)∝q(zt)exp(βrt(zt)), involving an intermediate reward function rt, the corresponding score is:
∇ztlogqr(zt)=∇ztlogq(zt)+β∇ztrt(zt),
where the second term acts as a steering gradient.
Since the reward is typically only defined at t=0, one can define the intermediate reward rt as the expected tilted reward conditioned on the current noisy state:
rt(zt):=β1logEz0∼q(⋅∣zt)[exp(βr(z0))].
For discrete diffusion, where gradients are unavailable, we tilt the transition rates of the CTMC.
The reverse evolution is governed by a modified rate matrix Qˉr(t) via Doob's h-transform:
Qˉijr(t)=Qˉij(t)h(i,t)h(j,t), where h(zt,t):=exp(βrt(zt)).
Global search via SMC
While local guidance provides point-wise steering, SMC methods offer a robust global search strategy.
Using the reverse transition of the original diffusion process as the proposal distribution, SMC evolves a population of M particles via:
(Sampling)(Resampling)∀m∈[M]:zsm∼p(zs∣ztm),∀m∈[M]:zsm←zsm′, where m′∼Cat({Gˉ(zti,zsi)}i=1M).
Here, the unnormalized and normalized incremental weights are: