What the Three MNIST Repositories Actually Implement

This audit reads pinned source code, not just repository names or sample galleries. It separates upstream behavior from our controlled reimplementation. The source snapshots and SHA-256 hashes are recorded locally in references/*/snapshot.json. No upstream generator weights are used in our experiments.

Pinned sources

The checked-in configs, source defaults and published weights are distinct evidence. Without a checkpoint's training metadata, we cannot establish that a gallery was generated by exactly the currently checked-in configuration. Nor can gallery appearance alone rank the underlying objectives.

Diffusion: one trained denoiser, two sampling stories

The DDPM repository predicts Gaussian noise with a class-conditional U-Net built from ShuffleNet-style blocks and BatchNorm. It uses a 1,000-step cosine schedule. Its training entry point defaults to 30 epochs, batch size 128 and Adam at 0.001. Images are 28 × 28 and normalized to [-1, 1]. These facts are visible in the training entry point and network implementation.

Its DDPM path drops labels with probability 0.15 during training, then samples with classifier-free guidance scale 1.5. Each guided velocity/noise prediction therefore requires two denoiser forwards: conditional and unconditional. The reverse variance is fixed to beta, a valid modeling choice distinct from the posterior-variance convention in our controlled experiment. See DDPM.py.

The DDIM implementation reuses the same denoiser, but directly uses conditional prediction without that CFG combination. It clips both the predicted clean image and each intermediate state, and its selected-time loop finishes at the stored alpha-bar of index zero rather than explicitly setting the final clean-time alpha-bar to one. These differences mean the upstream DDPM/DDIM comparison changes more than stochasticity and timestep count. They are implementation decisions to inspect, not evidence of two independently trained models. See ddim_sample.

Our diffusion comparison retains cosine corruption and noise regression, but applies the same clean-image clipping, conditioning and selected time grid to both samplers. CFG is one, so one denoiser forward means one NFE. Both have an explicit clean endpoint. We report native 1,000-step DDPM separately from a respaced DDPM at the shared comparison budget.

Flow: distinguish the default, the conditional branch and the solver

The default Hydra configuration selects the unconditional ImageFlowMatcher, not the conditional CFG class. The default training is 10 epochs, batch size 32, Adam at 0.0001 and U-Net width 32. Images become [-1, 1] inside the module. Its CondOTProbPath uses Gaussian noise and real images as endpoints, then regresses path velocity. See default configuration and unconditional model.

The included notebook is a different training entry point: its executable cells instantiate an unconditional width-32 model and request 100 epochs, again at batch size 32. The CLI's 10-epoch default therefore cannot be assumed to describe notebook results. Our 20,000-update, batch-128 base runs use 2.56 million image presentations, about 42.67 dataset-equivalents, and are explicitly a different controlled protocol.

An important sampling detail is its use of adaptive dopri5 with absolute and relative tolerances of 0.0001. The num_steps parameter constructs requested output times. Two requested time points do not mean two model calls; the adaptive solver chooses internal evaluations. A performance comparison must instrument actual neural calls. See the generation method.

The optional class-conditional branch uses class-dependent feature modulation, label dropout with retention probability 0.8, a default CFG scale of 7, and a Hungarian noise/image assignment. Its guidance wrapper evaluates conditional and unconditional branches. Therefore its attractive class-conditioned samples should not be treated as the default unconditional configuration or compared at nominal solver time-grid length. See class_cond_fm.py.

The assignment helper also has a reproducible permutation-direction issue. Its cost matrix has noise samples as rows and data samples as columns; to keep data order fixed, the matched noise must be placed at the matched column. The implementation instead assigns the other direction. With noise [0, 10, 20] and data [20, 0, 10], it returns [10, 20, 0], whose summed distance is 40, although the correct assignment has zero cost. The counterexample was executed against the pinned helper and saved in references/assignment_counterexample.json. This does not invalidate all flow matching; it means that helper does not always deliver its advertised minimum-cost pairing. See models/utils.py.

Our main experiment does not use this optional assignment. Independent pairings give a clean FM/RF1 baseline; reflow then changes one specific object—the coupling—to teacher-generated pairs. Fixed-step Euler makes model evaluations explicit. We do not present conditional Gaussian path transport as a solution to global optimal transport between the full data distributions.

Rectified Flow: the opposite time direction, not a separate baseline loss

The smol implementation passes digit labels to an OpenAI improved-diffusion U-Net. Its script resizes MNIST to 32 × 32 and leaves values in [0, 1]. The checked-in JSON specifies 10,000 updates and learning rate 0.0003; the trainer's default batch size is 16 because that config does not override it. Its preview uses 16 Euler steps, with samples arranged by requested digit. See data loading, configuration and trainer.

Its time zero is data, and time one is noise. The regression target is noise minus data; generation integrates backward with a negative Euler update. Our common convention reverses time and negates that drift. After this substitution, its independent straight-path objective is the same mathematical baseline as the FM construction used here. The equivalence is checked numerically in unified/test_unified.py. See the RectifiedFlow implementation.

The supplied smol training loop does not generate a teacher pair bank or train a second rectification. Calling its model “RF2” would be incorrect. Our RF2 is an explicitly added reflow experiment beyond that repository's single-flow baseline. Its seed is also applied in the trainer after the script constructs the network; our trainer seeds before initialization and records actual initial-weight hashes.

What is unified—and what deliberately remains different?

What remains different is intentional: corruption/path coefficients, prediction target, stochastic versus deterministic sampling, and the reflow endpoint coupling. A universal ranking cannot be extracted from one seed on a shared GPU. The goal is to expose the effect of each change and its cost.

These are controlled reimplementations informed by the repositories, not exact reproductions of their architectures, training runs or published galleries. Upstream sources remain pinned and attributable. The two flow repositories include MIT licenses; no root license file was found in the pinned DDPM snapshot, so our publication links to that source rather than repackaging it.

Read the controlled experiment

The full English tutorial groups the diffusion progression (DDPM → DDIM) before the flow progression (independent FM/RF1 → reflow/RF2). The compact comparison uses the same unfiltered evaluation inputs. Exact measured budgets, checkpoint hashes and costs are in the experiment report.