Materials Genomics
Unit 12: Generative Models & Inverse Design

Prof. Dr. Philipp Pelz

FAU Erlangen-Nürnberg

Where We Stand

Recap of Units 6–11

• Unit 6: local atomic environments + universal MLIPs (MACE-MP-0, M3GNet, CHGNet)
• Unit 7: graphs as the structural language of crystals
• Unit 8: regression with materials-aware splits and OOD discipline
• Unit 9: neural networks as scalable surrogates
• Unit 10–11: representation learning, latent spaces, and what an embedding actually means
• All of the above is forward modelling: structure \(\to\) property
• Today we invert the arrow: property target \(\to\) structure

Open with the arc, not the topic. Six units of machinery — environments, graphs, honest regression, scalable surrogates, latent spaces — and every single one of them answers the same question: given a structure, what does it do? Say that sentence, pause, then flip it: nobody funds a lab to predict properties of structures we already have. The funded question is the inverse one.

One sentence per unit, no more. U6 gave us universal MLIPs (they return as the funnel’s workhorse in Part V — flag that now). U7 gave the graph language every generator’s score network speaks. U8 taught OOD discipline — generated structures are by construction out-of-distribution for our surrogates, so U8’s warnings apply with a vengeance today. U10–11 taught what a latent space is — CDVAE literally samples from one.

Classroom move. Ask: “Name a materials problem where someone hands you the property first.” Collect two or three (battery voltage window, thermoelectric zT, a magnet without rare earths). Every answer is an inverse problem. This lands the motivation better than any slide.

Pacing. ~3 min. Do not re-teach the old units; this is a runway, not a recap exam.

From Prediction to Inverse Design

• A predictor tells you what a given structure will do
• A discovery loop wants the opposite: name a property target, get candidate structures
• Classical inverse design = high-throughput screening + grid search — does not scale
• Modern inverse design = generative models that sample structures conditioned on the target
• Output: a stream of candidate crystals, each with composition, lattice, coordinates, and (optionally) space group
• The candidate stream then enters a filtering funnel (MLIP relax \(\to\) DFT \(\to\) uncertainty \(\to\) experiment)

Why screening does not scale — make the number concrete. Even restricting to quaternary compositions of 60 usable elements gives \(\binom{60}{4} \approx 5\times 10^5\) chemical systems before you pick stoichiometry, prototype, or cell — the structure space per system is effectively unbounded. Grid search dies of combinatorics; high-throughput DFT dies of cost (each relaxation is CPU-days). Screening also has a subtler flaw: it can only rank enumerated candidates, and enumeration is always prototype-substitution in disguise — you will never screen your way to a genuinely new structural motif.

The conceptual pivot, said precisely. A generative model does not invert \(f\); it learns a distribution \(p(x \mid y^\star)\) and lets you sample. That reframing — inverse design as conditional sampling, not root-finding — is the single most examinable sentence of the lecture.

Pre-empt. “Why not gradient descent on \(x\) through the property predictor?” — because \(x\) is discrete-plus-continuous (species + lattice + coordinates), the predictor is only trustworthy on-manifold, and naive optimisation walks straight off it into adversarial garbage. That failure is exactly what U8 called OOD abuse of a regressor.

Plant the funnel image now. The generator is the widest stage of a funnel, not the product. We spend Part V entirely inside that funnel.

Pacing. ~4 min. End Part 0 at ~7 min.

Lecture Roadmap

Part I — foundations of generative modelling for crystals

Part II — diffusion-based crystal generators (CDVAE, DiffCSP, MatterGen)

Part III — flow matching and autoregressive models (FlowMM, CrystaLLM)

Part IV — conditioning and constraints

Part V — downstream filtering, MLIP relaxation, DFT screening, GNoME, the active-learning loop

Closing — open challenges, takeaways, link to Unit 13 (uncertainty-aware discovery)

State the contract. Parts I–III are how the generators work (foundations, diffusion, then the challengers). Parts IV–V are how you actually use them (conditioning, then the funnel). If a student only retains one half, it should be IV–V — that is where materials genomics differs from a computer-vision course.

Say what is deliberately excluded. No molecular generation (different constraints: no lattice, but valence rules), no protein design — we stay with periodic inorganic crystals. Also, no code today; the exercise session runs a pretrained generator through a MACE-MP-0 filter, so the lecture supplies the judgment, the exercise the hands.

Schedule note (for the archived deck). The Unit 13 slot (07.07) was cancelled; its uncertainty-aware-discovery core is folded into Unit 14 (14.07), and the U13 deck stays online as an optional reference. Where this deck says “Unit 13”, read “U13 deck + Unit 14 lecture”.

Pacing. ~1 min. This slide is a table of contents, not content.

The Generative Landscape Today

Year	Model	Family
2018	CrystalGAN (Nouira et al. 2018)	GAN
2020	FTCP (Ren et al. 2022)	VAE-like
2022	CDVAE (Xie et al. 2022)	Diffusion + VAE
2023	DiffCSP / DiffCSP++ (Jiao et al. 2023, 2024)	Diffusion
2023	GNoME (DeepMind) (Merchant et al. 2023)	GNN screening at scale
2024	MatterGen (MSR) (Zeni et al. 2025)	Diffusion + conditioning
2024	CrystaLLM (Antunes et al. 2024)	LLM / autoregressive
2024	FlowMM (Miller et al. 2024)	Flow matching

• The field passed a clear inflection in 2022–2024
• Diffusion currently dominates the headlines for crystal generation
• Flow matching and LLM-style models are closing fast
• Foundation models (MACE-MP-0 (Batatia et al. 2025), MatterSim (Yang et al. 2024), ORB (Neumann et al. 2024), UMA (Wood et al. 2025)) are the scoring layer — generation + universal MLIP is one tightly coupled pipeline

Read the table as a story, not a list. 2018–2020: GANs and VAEs prove the idea but not the yield. 2022: CDVAE is the inflection — first model whose samples survive DFT at a usable rate. 2023–2024: the field professionalises — DiffCSP makes diffusion native to crystals, MatterGen adds industrial-grade conditioning, and two challenger families appear (flow matching, LLM-style autoregression). Dates matter here: this is a field where “two years old” means “one generation behind”.

Flag the odd one out. GNoME is in the table but is not a generative model — it is GNN-scored substitution/search at DeepMind scale. Tell students now so they don’t mis-file it; the full story is in Part V. Exam answers that call GNoME “a diffusion model” lose the point.

The last bullet is the thesis of the lecture. Generators and universal MLIPs co-evolved: nobody would generate \(10^6\) candidates if relaxing them cost DFT prices, and nobody would need MACE-MP-0-scale MLIPs without generators flooding the funnel. Neither is useful alone.

Exam calibration. Students should know each family plus one exemplar (diffusion → MatterGen; flow → FlowMM; autoregressive → CrystaLLM; legacy → CDVAE’s VAE half, CrystalGAN) — not the full table by heart.

Pacing. ~2 min. Close Part 0 at ~10 min.

Part I — Foundations

Forward vs Inverse Problems

• Forward: \(f:\mathcal{X}\to\mathcal{Y}\), i.e. structure \(\to\) property
• Inverse: given target \(y^\star\), find \(x\) with \(f(x)\approx y^\star\)
• Forward is well-posed; inverse is many-to-one (lots of structures share a property) and ill-posed (no closed-form \(f^{-1}\))
• Generative model = a learned distribution \(p(x\mid y^\star)\)
• Sample from \(p\) instead of searching \(\mathcal{X}\) — millions of candidates per GPU-hour

Define ill-posedness properly — students hand-wave this. Hadamard’s criteria: existence, uniqueness, stability. The inverse map fails uniqueness spectacularly (a 2.0 eV bandgap is shared by thousands of structures) and can fail existence (no stable structure may hit an arbitrary target). The many-to-one character is not a nuisance — it is an opportunity: we want the full solution set, because most solutions will die later in the funnel for other reasons (synthesizability, cost, toxicity).

Distribution over point estimate, one more time. A deterministic “inverse network” \(g: y \to x\) would collapse the solution set to one arbitrary representative. \(p(x \mid y^\star)\) keeps the multimodality. Connect backwards: this is the same reason U8 preferred predictive distributions over point regressors — uncertainty and multimodality are information.

Chalkboard moment. Draw \(\mathcal{X}\) as a blob, \(\mathcal{Y}\) as a line, \(f\) as many arrows converging on \(y^\star\). The inverse “map” is the preimage \(f^{-1}(y^\star)\) — a set, not a point. Generative sampling = throwing darts that land preferentially inside that set.

Pre-empt. “Is this just Bayesian inversion?” — yes, structurally: \(p(x \mid y) \propto p(y \mid x)\,p(x)\). The generator amortises the posterior. Students who took MFML will appreciate hearing the connection stated once.

Pacing. ~3 min.

Crystal Structure as Data

A crystal is a structured object with multiple types of variables:

• Composition: which species, how many of each, \(\{Z_i\}\)
• Lattice: 3×3 matrix \(\mathbf{L}\) — six independent parameters \(a,b,c,\alpha,\beta,\gamma\)
• Fractional coordinates: \(\{\mathbf{f}_i\}\in[0,1)^3\) for each atom
• Symmetry: space-group operations that close orbits under translation
• Generators must respect all four — drop any one and the output is unphysical
• Standard datasets supply these as CIFs or PyMatGen Structure objects (Ong et al. 2013)

This slide explains why crystal generation is harder than image generation. An image is a fixed-size tensor of one data type. A crystal mixes categorical variables (species), continuous unconstrained ones (lattice), continuous periodic ones (fractional coordinates live on a 3-torus — 0.99 and 0.01 are neighbours!), and a discrete group (space group). Every model in Parts II–III is, at heart, one answer to “how do you put noise on this mixed object and take it off again.”

List the invariances out loud — they are the exam trap. The same physical crystal has infinitely many representations: rigid rotations of \(\mathbf{L}\), permutations of atom order, translations of all fractional coordinates, and cell choice (any supercell or alternative primitive cell describes the same material). A generator that ignores these wastes capacity learning that Cu in cell A equals Cu in rotated cell B — or worse, counts the same material twice when you measure “uniqueness”. Niggli reduction and primitive-cell canonicalisation exist precisely to kill the cell-choice ambiguity before training.

Connect back. U7’s graph representation already solved rotation + translation invariance for encoding; the new difficulty is doing it for decoding/generation, where you must output coordinates, not just consume them.

Pacing. ~3 min. Worth the time — everything in Part II leans on this.

The Discovery Funnel

1. Generate: sample \(N\sim 10^5\) candidate structures from \(p(x\mid y^\star)\)
2. Pre-filter: drop duplicates, unphysical geometries, exotic compositions
3. Relax with MLIP: MACE-MP-0 (Batatia et al. 2025) or M3GNet (Chen and Ong 2022) relaxes each candidate to a local minimum, much cheaper than DFT
4. DFT verify: validate energy / property predictions for the top few thousand
5. Uncertainty triage: keep the candidates where the surrogate is both good and confident
6. Synthesise the surviving handful in the lab

• Each stage trims by ~10–100×; the top of the funnel must therefore be very wide

First pass only — resist detail. This slide exists so that every model in Parts II–III has a place: generators live at stage 1, MLIPs at stage 3, DFT at stage 4, Unit 13/14’s UQ at stage 5. Walk it in 90 seconds and promise the full treatment in Part V.

The one insight to land now: attrition dictates economics. If each arrow loses 10–100×, then a synthesis campaign of ~5 materials needs \(10^5\)–\(10^6\) candidates at the top. That single multiplication explains (a) why generation must be cheap per sample, (b) why sampling cost differences between diffusion and flow matching (Part III) are strategic rather than cosmetic, and (c) why nobody DFT-relaxes raw generator output.

Classroom move. Ask: “Where would you rather spend an extra €100k — a better generator or a better filter?” There is no clean answer, and the discussion previews Part V’s punchline: the funnel is a system, and the weakest stage sets the discovery rate.

Pacing. ~2 min.

Evaluation Criteria

What makes a generated structure good?

• Validity: charge balance (SMACT-style rules (Davies et al. 2019)), no atomic overlaps, periodic-image consistency
• Novelty: not already in the training set (or any known materials database)
• Uniqueness: distinct from other samples generated in the same batch
• Stability: energy above hull \(\Delta H_{\text{hull}}\leq 0.1\) eV/atom is a common cutoff
• Task fidelity: predicted property close to the conditioning target \(y^\star\)
• All five must hold simultaneously — single-axis benchmarks are misleading

Give each criterion its operational test. Validity: charge balance via SMACT-style oxidation-state rules plus a minimum interatomic-distance check (typically ~0.5 Å tolerance against covalent radii) including periodic images — forgetting images is the classic student bug. Novelty: structure-matcher comparison (PyMatGen StructureMatcher, tolerance-based) against training + reference databases. Uniqueness: the same matcher within the generated batch. Stability: energy above hull after relaxation. Fidelity: \(|\hat y(x) - y^\star|\) under a held-out predictor, not the one used for guidance.

The adversarial example that makes it stick. A generator that memorises the training set scores 100% validity and stability and 0% novelty. One that outputs random noise scores high novelty and zero validity. Any single axis is trivially gameable — which is why the composite (next slide) exists.

Fidelity subtlety worth saying aloud. If you measure fidelity with the same predictor that steered the sampling, you are grading the student with their own answer sheet. Fidelity must be scored by an independent model or, better, DFT. This is the generative version of U8’s data-leakage lecture.

Pacing. ~3 min.

The S.U.N. Metric

Common composite metric: S.U.N. = Stable, Unique, Novel.

• “Stable” = below hull or within tolerance window
• “Unique” within the generated batch
• “Novel” with respect to training / reference databases
• Report as a rate: fraction of samples that pass all three
• Stronger variants: SUNS (Synthesizable) adds a literature-based filter
• Be careful: hull cutoffs depend on the underlying convex hull (MP-2024 vs Alexandria changes the number significantly)

Set the magnitude expectation. For a good 2024-era model, on the order of a few percent to ~10–15% of raw samples are S.U.N., depending brutally on the reference hull and matcher tolerances. Students reading papers should expect numbers in that range; a claimed 60% S.U.N. rate should trigger “which hull? which tolerance?” before admiration.

The hull caveat is the exam-worthy line. “Stable” is always relative to a reference set: adding the Alexandria/GNoME entries to the hull raises it, and structures that were “below hull” against MP-2022 become metastable overnight. Novelty has the same disease in reverse — a bigger reference database deflates it. So S.U.N. rates from two papers are comparable only if hull, database snapshot, and matcher settings match. They rarely do.

War story from the literature. After the GNoME release, several groups’ “novel” hit rates dropped by double-digit percentages when re-evaluated against the updated reference — same models, same samples, different yardstick. The models didn’t get worse; the ruler got longer.

Metastability nuance. \(\Delta H_{\text{hull}} \le 0.1\) eV/atom admits metastable materials on purpose — diamond, most nitrides, and many battery cathodes are above hull. The cutoff is a synthesis-likelihood prior, not a law.

Pacing. ~2 min.

Conditional vs Unconditional Generation

Unconditional

• Sample from the full data distribution
• Useful for exploring the breadth of the materials landscape
• Used for pre-training and ablation studies
• Often produces low S.U.N. unless heavily filtered

Conditional

• Sample from \(p(x\mid y^\star)\)
• Targets a property, composition, symmetry, or full multi-objective spec
• Critical for actual discovery
• Conditioning quality dominates downstream success rate

Why unconditional generation still matters — three legitimate uses. (1) It is the pre-training task: MatterGen trains unconditionally first, conditions later via adapters. (2) It is the diagnostic task: if unconditional samples already fail validity, no amount of conditioning rescues the model. (3) It is the fair ablation baseline for measuring what conditioning buys you.

The asymmetry to emphasise. Unconditional generation answers “what does the materials distribution look like?”; conditional answers “what do I want from it?” Only the second is discovery. A lab pays for \(p(x \mid y^\star)\), not \(p(x)\).

Justify the last bullet — it sounds like a throwaway but is empirical. Downstream, every funnel stage multiplies. If conditioning is sloppy, the fidelity term collapses at the end of the funnel where candidates are most expensive (post-DFT), which is the worst possible place to lose them. Better conditioning moves the attrition to the cheap top of the funnel. That is why Part IV gets its own ten minutes.

Pacing. ~2 min.

Training Data Landscape

• Materials Project (~150 k entries) — DFT relaxations, properties, hull (Jain et al. 2013)
• OQMD (~1.0 M) — broad coverage, less curated (Kirklin et al. 2015)
• Alexandria (~4 M generated/curated) — large, includes many ML-discovered candidates (Schmidt et al. 2024)
• ICSD (~250 k) — experimentally observed structures, the gold standard for “real materials” (Zagorac et al. 2019)
• GNoME (~2.2 M, DeepMind 2023) — ML-discovered stable structures, partially overlaps with Alexandria (Merchant et al. 2023)
• Choice of training corpus strongly shapes the bias of the resulting generator — train on ICSD vs Alexandria and you produce very different distributions

“Data is the prior” — say it exactly like that. A generative model is a lossy photograph of its corpus. ICSD-trained models generate things that look synthesisable because ICSD is the synthesised record — but they inherit its history: oxide-heavy, ambient-pressure, biased toward what crystallographers happened to measure. Alexandria/GNoME-trained models roam wider chemistry but include millions of DFT-only hypotheticals nobody has ever made — the generator happily proposes more of the same.

Practical details students will hit in the exercise. MP entries carry corrections (anion, +U) that shift formation energies — mixing databases without reconciling correction schemes silently corrupts the hull. Most generator papers train on ~20-atom-max subsets (MP-20, Alex-MP-20): small cells keep generation tractable but bias against complex structures — another invisible prior.

Leakage warning (U8 reprise). These databases overlap heavily (GNoME ↔︎ Alexandria; MP ⊂ everything). If your “novelty” reference is a subset of your training corpus’s relatives, novelty is overstated. Deduplicate across all sources before believing any rate.

Classroom move. Ask which corpus they would pick for “discover a synthesisable magnet”. Correct answer: it is a genuine trade-off (ICSD realism vs Alexandria coverage) — and the honest resolution is a synthesizability filter (Part IV) rather than corpus purity.

Pacing. ~3 min. Part I ends here, ~25 min mark.

Part II — Diffusion Models

Diffusion Primer — Forward Process

Start with data \(x_0\), apply a noising schedule (Ho et al. 2020):

\[q(x_t\mid x_{t-1}) = \mathcal{N}\!\left(x_t;\sqrt{1-\beta_t}\,x_{t-1},\beta_t\mathbf{I}\right)\]

After \(T\) steps, \(x_T\approx\mathcal{N}(0,\mathbf{I})\) regardless of \(x_0\).

• Variance schedule \(\{\beta_t\}\) is a hyperparameter
• Closed-form: \(x_t = \sqrt{\bar\alpha_t}\,x_0 + \sqrt{1-\bar\alpha_t}\,\epsilon\)
• For crystals: noising applied to coordinates, lattice, and (categorically) to atomic types

Teach the forward process as “destruction you can control”. Nothing is learned here — that surprises students every year. The forward process is a fixed, hand-designed corruption schedule; all learning happens in reversing it. Analogy that lands: dissolving a crystal into solution step by step, keeping notes on the temperature profile, so you can later ask “what un-dissolves under this profile?”

Chalkboard the closed form. Define \(\alpha_t = 1-\beta_t\), \(\bar\alpha_t = \prod_{s\le t}\alpha_s\); the one-shot formula \(x_t = \sqrt{\bar\alpha_t}\,x_0 + \sqrt{1-\bar\alpha_t}\,\epsilon\) means training never runs the chain — sample \(t\), jump straight to \(x_t\). That trick is why diffusion training is cheap even when sampling is not; the asymmetry returns on the limitations slide.

The crystal-specific twist — one sentence per variable type. Coordinates: Gaussian noise wrapped onto the torus (a wrapped normal), because fractional coordinate 0.99 + noise must be allowed to cross 1.0. Lattice: ordinary Gaussian noise on a suitably parameterised 3×3 (details next slides). Species: you cannot add 0.3 of a Gaussian to “carbon” — discrete corruption à la D3PM (random flips toward a uniform/masked distribution). One slide, three noising processes — that heterogeneity is crystal diffusion.

Pacing. ~4 min. This and the next slide are the mathematical heart; go slow.

Reverse Process — Denoising

Learn \(p_\theta(x_{t-1}\mid x_t)\) — the denoising step.

Training objective: predict the noise \(\epsilon\) that was added to obtain \(x_t\):

\[\mathcal{L} = \mathbb{E}_{t,x_0,\epsilon}\,\|\epsilon - \epsilon_\theta(x_t,t)\|^2\]

• Sampling: start from \(x_T\sim\mathcal{N}(0,\mathbf{I})\), denoise step by step to \(x_0\)
• \(\epsilon_\theta\) is a neural network — for crystals, an equivariant GNN (often a MACE-like or EGNN backbone)
• Inference is iterative: ~50–1000 denoising steps; orders of magnitude slower than a single forward pass

Demystify the objective — it is just regression. For each training crystal: pick a random \(t\), corrupt with known \(\epsilon\), ask the network to recover \(\epsilon\) from \((x_t, t)\), take the MSE. No adversary (contrast GANs), no ELBO gymnastics in the practical form (the simple \(\epsilon\)-loss is a reweighted ELBO — mention for the theory-minded, then move on). This trainability is the reason diffusion beat GANs: a stable, stationary regression target.

Give physical intuition for what \(\epsilon_\theta\) learns. At high noise it recovers coarse order — density, rough composition; at low noise it does the fine work — settling atoms into Wyckoff-like positions, bond distances. The network is effectively a learned relaxation operator, and indeed the denoising trajectory looks like an annealing schedule from melt to crystal. Materials students find this analogy essentially exact.

Why an equivariant GNN and not a transformer on raw coordinates. The noise to be predicted transforms with the structure — rotate the crystal, the noise vector rotates. Equivariance bakes that in (full argument three slides ahead). Backbone families: EGNN, NequIP/MACE-style tensor message passing — the same zoo as U6’s MLIPs, which is not a coincidence: predicting noise on atoms and predicting forces on atoms are formally cousins (both are per-atom equivariant vector fields).

Plant the cost flag. 50–1000 network evaluations per sample × \(10^6\) samples is the funnel’s stage-1 bill. Keep the number in the room; flow matching (Part III) attacks precisely it.

Pacing. ~4 min.

Score-Based View

An equivalent picture: learn the score \(\nabla_x\log p_t(x)\) instead of the noise (Y. Song et al. 2021).

• Forward SDE: \(dx = f(x,t)\,dt + g(t)\,dw\)
• Reverse SDE: \(dx = [f(x,t) - g(t)^2\nabla_x\log p_t(x)]\,dt + g(t)\,d\bar w\)
• Discretising the reverse SDE recovers the denoising step from the previous slide
• Allows fancier samplers (DDIM (J. Song et al. 2021), DPM-Solver (Lu et al. 2022), EDM (Karras et al. 2022)) that converge in 10–50 steps instead of 1000
• Almost all modern crystal diffusion papers cite the score-based formulation rather than the original DDPM derivation

Why bother with a second formulation? Three concrete payoffs. (1) Samplers: once you see sampling as integrating a reverse SDE/ODE, you can swap Euler for higher-order solvers — DDIM, DPM-Solver, EDM — cutting 1000 steps to 10–50 with minor quality loss. (2) Guidance: the score \(\nabla_x \log p_t(x)\) is a gradient field, so conditioning becomes adding gradients — Part IV’s classifier guidance is literally \(\nabla_x \log p_t(x) + \nabla_x \log p(y \mid x)\), one line in score language and incomprehensible in DDPM language. (3) Geometry: scores generalise to manifolds (torus, lattice manifolds), which crystals need and which the wrapped-normal tricks of DiffCSP/FlowMM exploit.

Keep the maths honest but light. Don’t derive Anderson’s reverse-time SDE; state that it exists, that the only unknown in it is the score, and that \(\epsilon\)-prediction and score-prediction are the same thing up to scale: \(\nabla_x \log p_t(x) = -\epsilon/\sqrt{1-\bar\alpha_t}\). Write that identity on the board — it lets students read both notations in papers.

Intuition for “score”. It points from where you are toward higher data density — a compass toward the data manifold at the current noise level. Denoising = following a slowly sharpening compass.

Pacing. ~3 min. If time pressure hits, this is the slide to compress — the identity and the three payoffs suffice.

CDVAE — Crystal Diffusion VAE

Xie et al. 2022 (Xie et al. 2022) — the first practical crystal generator with realistic SUN rates.

• Hybrid: a VAE encodes the crystal into a global latent \(z\), and a diffusion model generates the fine atomic coordinates conditioned on \(z\)
• Decouples coarse (composition, density) from fine (positions)
• Uses periodic-image-aware GNN as the score network
• ICLR 2022 — the model that put diffusion on the crystal-generation map
• Limitations: lattice prediction is weak; struggles with low-symmetry structures

Explain the division of labour — it is the whole design. The VAE half encodes a crystal into one global latent \(z\) and predicts the coarse skeleton from it: composition, number of atoms, lattice. The diffusion half then does what diffusion is good at: Langevin-style denoising of atom positions (and type refinement) conditioned on \(z\). Coarse decisions by regression, fine geometry by iterative refinement. Connect to U10–11: \(z\) is exactly the kind of latent-space object we studied — CDVAE interpolations between two crystals traverse it.

Why 2022 was the inflection. Earlier VAEs decoded coordinates in one shot and produced overlapping atoms and broken stoichiometry. CDVAE’s insight: don’t ask a decoder to nail 3N coordinates at once; ask a score network to nudge noisy coordinates repeatedly with a periodicity-aware GNN. The iterative decoder is what made samples DFT-survivable.

Be precise about the limitation — it motivates the next slide. The lattice comes from the VAE regression, outside the diffusion. If the predicted cell is off, the coordinate denoiser inherits a wrong box it cannot fix. Hence low-symmetry (triclinic/monoclinic) weakness, and hence DiffCSP’s “diffuse the lattice too”.

Historical framing for students. CDVAE is to crystal generation what AlexNet was to vision — superseded on every benchmark, cited by everything, and still the cleanest paper to learn the problem from.

Pacing. ~3 min.

DiffCSP — Joint Lattice + Coords

Jiao et al. 2023 (Jiao et al. 2023) — diffusion model that generates lattice and coordinates jointly.

• Lattice represented in a parameterised form that handles rotations cleanly
• Coordinates are fractional and respect periodic boundaries
• Score network: equivariant GNN with periodic message passing
• Significantly improves stability rate over CDVAE
• Inference cost still dominated by O(100) denoising steps
• DiffCSP is widely used as a baseline in 2024–2025 papers

Name the task shift first — most students miss it. DiffCSP’s headline task is crystal structure prediction: composition is given, the model generates lattice + coordinates. That is a different (and easier) problem than de-novo generation, and it is why DiffCSP numbers and MatterGen numbers are not directly comparable. In papers, watch for “CSP task” vs “ab initio / de novo generation task” — same model families, different exam questions.

The two technical fixes over CDVAE, in plain words. (1) Lattice inside the diffusion: the lattice is noised and denoised jointly with coordinates, in a parameterisation that separates orientation from shape so that the model works with rotation-invariant lattice content rather than an arbitrary 3×3. (2) Fractional-coordinate diffusion done natively on the torus: wrapped noise, periodic score — so an atom at 0.98 can legally drift to 0.02. Together these remove CDVAE’s “wrong box” failure.

Why it became the standard baseline. Clean joint formulation, public code, evaluated on the community’s standard splits (Perov-5, MP-20, MPTS-52) — every 2024–2025 paper tables against it. Students doing theses in this area will run it within their first month.

Pacing. ~3 min.

DiffCSP++ — Symmetry-Constrained

Jiao et al. 2024 (Jiao et al. 2024) — adds space-group conditioning during generation.

• Conditioning on a target space group during the reverse process
• Reduces the search volume drastically — most crystals belong to a handful of space groups
• Improves novelty without sacrificing stability
• Naturally couples to symmetry-aware datasets (Alexandria, GNoME)
• Trade-off: requires you to pick (or sample) the space group up front

Motivate with the statistic. Real crystals are wildly non-uniform over the 230 space groups — a handful (e.g. \(Pnma\), \(Fm\bar 3m\), \(P2_1/c\), \(C2/m\)) cover a large share of known inorganics. Unconstrained diffusion rediscovers this the hard way, spending capacity learning symmetry from scratch and still producing near-symmetric structures that relax into symmetry only approximately. DiffCSP++ instead makes symmetry a constraint: pick a space group, and generation happens within its Wyckoff-position template — atoms move on symmetry-allowed orbits, the lattice respects the crystal system.

Why this helps novelty and stability at once (which sounds too good): the search volume collapses to a symmetry-consistent subspace where physical structures actually live, so fewer samples are wasted on almost-but-not-quite-symmetric junk. You are not restricting creativity; you are removing measure-zero noise.

The trade-off, honestly. Someone must choose the space group. In practice you sample it from the empirical distribution of the training corpus, or sweep the common groups per composition. That is a prior you inject — if your target chemistry crystallises in an unusual group, the constraint becomes a blindfold. Pre-empt the question “can the model choose its own group?” — yes, hierarchical variants first sample a group then a structure; the joint problem is active research.

Pacing. ~2.5 min.

Equivariant Diffusion

Modern crystal diffusion almost always uses equivariant networks.

• E(3)-equivariance: rotating the input rotates the output by the same transform
• Periodic equivariance: a translation of the lattice does not change the predicted distribution
• Networks: EGNN (Satorras et al. 2021), NequIP (Batzner et al. 2022), MACE-style (Batatia et al. 2022) message passing on a periodic graph
• Without equivariance, the model has to learn symmetry from data — usually fails on small training sets
• Same backbone trick that drives the MLIP revolution in Unit 6

Definition with a concrete test. Equivariance: \(f(R x) = R f(x)\) for every rotation \(R\) — rotate the noisy crystal, the predicted noise vectors rotate identically. Invariance is the special case for scalar outputs (energy). The score network needs equivariance because its output is a per-atom vector field. Give the litmus test students can run: feed a structure and its rotated copy through the network; the outputs must match to numerical precision, not approximately.

Why not just data augmentation? — the inevitable question. Augmentation teaches symmetry statistically: coverage of SO(3) is never exact, sample efficiency drops (the network re-learns the same physics in every orientation), and errors show up exactly where generation is sensitive — small distortions that decide space-group assignment. Equivariant architectures make the symmetry exact by construction, which is why they dominate with \(10^5\)–\(10^6\) training structures, tiny by vision standards.

The economy-of-ideas point (say it — it is the course’s through-line). The tensor-message-passing machinery of U6’s MLIPs (NequIP/MACE lineage) is reused unchanged as the diffusion score network. Forces and scores are both equivariant per-atom vector fields; one architecture family serves both. This is why MLIP progress and generator progress track each other so tightly.

Periodic part, one sentence. Messages pass across cell boundaries to periodic images (U7’s periodic graph), so the network cannot tell where the arbitrary cell origin sits — translation invariance handled.

Pacing. ~2.5 min.

MatterGen Architecture

Zeni et al. (Microsoft Research; Nature 2025) (Zeni et al. 2025) — diffusion model for property-conditioned generation.

• Equivariant GNN score network operating on lattice + composition + coordinates
• Trained on ~600 k Alexandria + MP entries
• Property head trained jointly so the model can be conditioned at sample time
• Adapter modules let you condition on new properties without retraining the full model
• Open-source release in 2024 — currently the most-used crystal diffusion baseline

Position MatterGen as the “productisation” of the DiffCSP lineage. Same joint diffusion idea — coordinates (wrapped/periodic), lattice (parameterised, biased toward physically plausible cell shapes), and atom types as a discrete diffusion — but trained bigger (~600k Alex-MP structures), engineered harder, and, crucially, built from day one for conditioning, which the pure-CSP models were not.

The adapter idea deserves two minutes — it is transfer learning, U6-style. The base model is trained unconditionally once (the expensive part). For each new target property, small adapter layers are injected and fine-tuned on a labelled subset — hundreds to thousands of labelled structures, not the full corpus. This changes the economics of inverse design: adding “condition on dielectric constant” becomes a fine-tuning job, not a \(10^4\)-GPU-hour retrain. Students should recognise the pattern from LLM practice (LoRA-style adaptation) — the technique migrated straight across.

Timeline honesty. arXiv preprint December 2023, open-source release 2024, Nature paper January 2025. When a student cites “MatterGen 2024”, they mean the code era; the refereed record is 2025.

Why “most-used baseline” matters practically. Pretrained weights + permissive licence + a maintained repo — for the exercise session this is the checkpoint we sample from. Availability, not just architecture, decides what a field standardises on.

Pacing. ~3 min.

MatterGen — Conditioning and DFT-Validation

• Conditioning targets demonstrated in the paper: bulk modulus, magnetic density, energy density, formation energy
• 2.2× higher rate of DFT-validated stable + unique + novel structures vs prior SOTA
• Lab synthesis: a Microsoft–SIAT collaboration reported the experimental synthesis of MatterGen-proposed TaCr\(_2\)O\(_6\) with a target bulk modulus of 200 GPa
• This was the first widely covered “AI-designed and lab-realised material” story (2024)
• A reminder: the model nominates, the lab validates — and most candidates still fail DFT screening

Walk one conditioning example end-to-end. “Give me structures with bulk modulus ≥ 200 GPa in the Li–Co–O system”: composition constraint + property target, both fed to the conditional score; samples come out biased toward the spec; the measured success is what fraction survive DFT verification of the property. The 2.2× headline is exactly that ratio against prior SOTA — a relative number; absolute yields remain modest, which is the honest reading.

Tell the TaCr₂O₆ story with its full nuance — it is the best cautionary tale in the field. Target: bulk modulus 200 GPa. The synthesised sample measured ~169 GPa — within 20% of spec, remarkable for a designed-from-scratch inorganic. But: the experimental structure came out with Ta/Cr cation disorder, whereas the model predicted an ordered arrangement — reality answered with a disordered variant of the prediction. Both halves matter: the loop closed (design → synthesis → property confirmed to within tolerance), and the perfect-crystal assumption cracked exactly where the limitations slide says it should (disorder). Students should be able to retell both halves.

Guard against the press-release version. “First AI-designed material” claims have a long, contested history (earlier examples exist in molecules and alloys). What is defensible: first widely replicated property-targeted crystal generation → synthesis → property-verification story from a general-purpose generator. Precision here models good scientific hygiene.

Pacing. ~3 min.

Limitations of Diffusion for Crystals

• Sampling cost: O(100) forward passes per candidate
• Mode collapse: heavily over-represents common space groups
• Discrete variables (atomic types) need special handling (categorical / D3PM-style (Austin et al. 2021))
• Magnetic / charged / disordered states are hard
• Quality of evaluation depends heavily on the convex hull cut-off and the reference database
• Most papers do not report failure modes — beware single-number benchmarks

Structure the limitations into three bins so they are memorable: cost (iterative sampling — the flow-matching motivation), coverage (mode collapse onto common space groups and small cells; magnetic, charged, and disordered states effectively out of scope because the training labels don’t encode them), and credibility (hull-reference sensitivity; unreported failure modes).

Make mode collapse concrete. Plot-in-words: histogram the space groups of \(10^4\) samples from any current model against the training histogram — the model’s is sharper, over-concentrated on the popular groups. Conditioning makes it worse (guidance trades diversity for fidelity, Part IV). For discovery this is insidious: the interesting materials are disproportionately in the tails the model under-samples.

The discrete-variable pain, one level deeper. Atom types under D3PM-style corruption lose the clean score interpretation; type-vs-position coupling (swap two species, positions must adjust) is learned only implicitly. Several 2024–2025 failures trace to types “locking in” early in denoising while positions still fit the wrong chemistry.

Credibility bin — connect to S.U.N. caveats. Same lesson as slide “The S.U.N. Metric”: a single leaderboard number hides hull choice, matcher tolerance, cell-size cap. Reviewer’s checklist for reading any generation paper: which hull? which reference DB snapshot? what fraction of failures shown? conditioned or cherry-picked examples?

Pacing. ~3 min. Part II ends here — target the ~50 min mark.

Part III — Beyond Diffusion

Flow Matching Primer

Continuous-time alternative to diffusion (Lipman et al. 2023).

Learn a vector field \(v_\theta(x,t)\) such that the trajectory

\[\dot x = v_\theta(x,t)\]

transports a simple base distribution \(p_0\) to the data distribution \(p_1\).

• Same generative idea as diffusion, but with a deterministic ODE
• Faster sampling (~10–25 steps vs ~100 for diffusion)
• More flexible base distributions (e.g. sample on the lattice manifold directly)
• Becoming the framework of choice for new model families

Teach flow matching by contrast with what they just learned. Diffusion: stochastic corruption, learn to reverse it, sample by simulating a noisy chain. Flow matching: pick the path yourself — typically a straight-line interpolation \(x_t = (1-t)x_0^{\text{noise}} + t\,x_1^{\text{data}}\) — and regress a velocity field \(v_\theta(x,t)\) onto the known path velocity. Training is again plain regression (conditional flow matching), no adversary, no ELBO; sampling is integrating a deterministic ODE.

Why fewer steps, intuitively. Diffusion trajectories are drunken walks that a solver must follow cautiously; straight(ish) learned flows are smooth, so a 10–25-step ODE solver traverses them accurately. Same destination, better-paved road. The determinism also gives exact likelihoods (change-of-variables along the ODE) — occasionally useful for filtering.

The point that matters for crystals — flexible geometry. Nothing in flow matching requires a Gaussian base or Euclidean space. Base distributions can live on the torus for fractional coordinates or respect lattice constraints directly; Riemannian flow matching makes “straight line” mean “geodesic”. This is harder to retrofit into standard diffusion — it is the technical reason the crystal community is migrating.

Historical note for orientation. Lipman et al., ICLR 2023 — notably after the diffusion wave; the field adopted it within a year. When the deck says “likely default by 2026”, that is an extrapolation students should treat as a forecast, not a fact.

Pacing. ~4 min. The primer carries the whole of Part III — invest here.

FlowMM

Miller et al. 2024 (Miller et al. 2024) — flow matching applied to crystals.

• Manifold-respecting flow matching: handles fractional coordinates on the torus, lattice on the SO(3)×\(\mathbb{R}^6\) manifold, and discrete species jointly
• Faster sampling than DiffCSP / MatterGen
• Competitive S.U.N. rates on standard benchmarks
• Naturally trainable from a small data set because the velocity-matching objective is easier to fit than score matching for low-data regimes
• Hot research direction in late 2024 / 2025

FlowMM in one sentence: Riemannian flow matching instantiated for crystals — geodesic paths on the right manifold for each variable block (torus for fractional coordinates, a constrained lattice parameterisation, discrete handling for species), all moved by one learned velocity field. Miller et al., ICML 2024 (Meta FAIR).

The number to quote: roughly an order of magnitude fewer integration steps than DiffCSP-style diffusion at comparable S.U.N./match rates. Recall the funnel arithmetic — \(10^6\) candidates × (steps per sample) — and a 10× step reduction is a 10× cheaper stage 1, i.e. a strategic improvement, not a benchmark vanity metric.

Mention the follow-up that shows where this is going: FlowLLM (same group, NeurIPS 2024) uses a language model as the base distribution and flow matching as the geometric refiner — the LLM proposes a chemically sensible rough draft, the flow polishes geometry. Paradigm fusion, not paradigm war; keep this in mind for the comparison slide.

Temper the “small data” bullet. The claimed low-data advantage (velocity regression being an easier target than scores near \(t\to 0\)) is plausible and reported, but benchmark-dependent — present it as a current-research claim, not settled fact.

Pacing. ~3 min.

Autoregressive Generation

• Treat the crystal as a sequence and predict it token by token
• Order matters — choose a canonicalisation (e.g. Wyckoff-position order)
• Each step: condition on what’s already been generated, predict the next atom / coordinate
• Pros: principled likelihood, simple sampling, easy to integrate with LLMs
• Cons: error compounds along the sequence; long-range dependencies are hard
• This is the family that brings LLM-style models into materials genomics

The factorisation, written once on the board: \(p(x) = \prod_k p(s_k \mid s_{<k})\) — the crystal as a sequence of tokens, each conditioned on everything before. All the machinery of Part II disappears; what replaces it is a design decision that diffusion never faces: in what order do you serialise a 3D periodic object? Composition first, then lattice, then sites? Wyckoff-position order? The canonicalisation is part of the model, and a bad ordering makes long-range constraints (charge balance decided by the last atoms) needlessly hard.

Pros, precisely. Exact log-likelihood per structure (diffusion only bounds it) — usable for ranking and filtering; sampling is standard beam/temperature machinery; and the entire LLM toolbox (prompting, fine-tuning, scaling recipes, KV-caching) transfers for free.

Cons, precisely. Exposure bias: one bad token early (wrong space-group token, absurd lattice parameter) poisons everything after — there is no iterative “go back and fix” like a denoiser has. And 3D geometry is only implicit in token statistics: the model has no built-in notion that two coordinates 0.02 apart collide.

Framing for the audience. Diffusion/flow bake physics into architecture (equivariance); autoregression bets that scale + data substitute for inductive bias. That is the same cultural divide as CNNs-vs-ViTs in vision — students should recognise the debate pattern and expect the answer to be “both, hybridised” (FlowLLM again).

Pacing. ~2.5 min.

CrystaLLM — Language Models for Crystals

Antunes et al. 2024 (Antunes et al. 2024) — train a GPT-style model on CIF text.

• Represent each crystal as a CIF-formatted string and train a decoder LM
• Sampling = generate a CIF, then parse it
• Conditioning by prompting (composition, space group, target property)
• Surprisingly competitive on stability and novelty
• Trivially scales with model + data — inherits the LLM scaling laws
• The “next token = next Wyckoff position” framing is now adopted by several follow-up models

The provocation to open with: CrystaLLM knows nothing about atoms, space, or physics — it is a GPT-2-scale decoder trained on ~2M CIF files as text (Antunes et al., Nature Communications 2024) — and yet its samples are surprisingly often valid, symmetric, and stable. Let the class sit with how strange that is: bond-length regularities and space-group conventions are apparently learnable as string statistics.

Why it works better than it should. CIF is a highly regular format; crystallographic conventions (standardised settings, Wyckoff labels, bounded coordinate precision) act as an implicit canonicalisation. The tokenizer treats numbers digit-wise, so “1.5934” is learned like a word with morphology. Conditioning is literally prompting: start the CIF with the target composition or space group and let it complete — no guidance mathematics at all.

The two honest caveats. (1) Parsing failures and subtly invalid CIFs are a real output mode — the “sanitise” funnel stage earns its keep here. (2) No geometric inductive bias means it leans hardest on data volume of all the families; its scaling-law inheritance is the bet that data will keep winning.

Forward pointer. The follow-up wave (Wyckoff-token models, MCTS-decoded variants, FlowLLM’s LLM-base) all keep the “crystal as sequence” framing and patch the geometry problem in different ways. Sequence models are now a permanent resident of this field, not a curiosity.

Pacing. ~3 min.

VAEs and GANs for Crystals (Legacy)

• VAEs (FTCP (Ren et al. 2022), iMatGen (Noh et al. 2019), …): smooth latent space, easy to interpolate, but lattice geometry is hard to constrain and SUN rates are mediocre
• GANs (CrystalGAN (Nouira et al. 2018), MatGAN (Dan et al. 2020), …): mode collapse and training instability — never matched diffusion / flow rates
• Both still appear in literature for niche tasks: small molecules, doping studies, defect generation
• For crystal generation, diffusion / flow / autoregressive have largely replaced them
• Useful pedagogically: the earlier methods motivated the design choices that later succeeded

Why keep a legacy slide — because the failure modes are the lesson. VAEs: the Gaussian latent + one-shot decoder forces blurry compromises; for crystals that means averaged atom positions, i.e. overlaps — and posterior collapse quietly turns the model into a mean-structure memoriser. GANs: no likelihood, adversarial instability, and mode collapse — for materials, collapse means “generates ten prototypes forever”, the exact opposite of discovery.

Trace the genealogy explicitly — it makes Part II look inevitable rather than fashionable. CDVAE = VAE skeleton + iterative denoising decoder (fixing the one-shot-decoder failure). Diffusion’s stationary regression loss = the answer to adversarial instability. Smooth-latent interpolation (the VAE’s genuine virtue, from U10–11) survives in CDVAE’s global latent. Nothing was wasted; the field metabolised its failures.

Where the old tools still earn their keep. Property-latent visualisation and interpolation studies, doping/defect perturbations around a known host, small-molecule work — places where a smooth, low-dimensional latent matters more than raw sample fidelity.

Pacing. ~2 min. Do not linger; the class needs the comparison slide next.

Comparison: Diffusion vs Flow vs Autoregressive

Family	Sample cost	S.U.N. rate	Conditioning	Notes
Diffusion	high (50–1000 steps)	strongest in 2023–2024	flexible (classifier-free)	dominant today
Flow matching	low (10–25 steps)	catching up fast	deterministic ODE	likely default by 2026
Autoregressive (LLM)	medium (token-by-token)	competitive	prompt-based	exploits LLM scaling
VAE / GAN	low (single pass)	low	limited	legacy / niche

None of these are mutually exclusive — production pipelines often combine paradigms.

How to run this slide: make the class fill the table before revealing it. Ask for each family’s sample cost, then S.U.N., then conditioning story — they have seen every ingredient in the last 25 minutes. Retrieval beats re-presentation.

Add the second-order reading the table can’t show. The columns are coupled: diffusion’s high cost buys its flexible guidance (iterative sampling is what makes gradient steering possible); autoregression’s cheap prompting costs it geometric reliability; flow’s speed comes from determinism, which also removes the noise-injection knobs some guidance tricks exploit. There is no free column.

Date-stamp the table aloud. “Strongest in 2023–2024”, “likely default by 2026” — this table has a half-life of about a year. What is durable: the axes (cost, quality, conditionability). Exam questions will ask students to compare families on these axes, not to recite today’s winner.

The hybrid punchline, with the concrete example. FlowLLM: autoregressive base proposal + flow-matching geometric refinement — cheap chemistry from the LLM, exact geometry from the flow. Expect more such fusions; “which paradigm wins?” is already the wrong question.

Pacing. ~2 min. Part III ends here — target ~65 min.

Part IV — Conditioning & Constraints

Targeting a Property

How do we ask for “a structure with bandgap \(\approx 2.0\) eV”?

• Classifier guidance (Dhariwal and Nichol 2021): train a property predictor \(p(y\mid x)\); use \(\nabla_x\log p(y\mid x)\) during sampling to steer the trajectory
• Classifier-free guidance (Ho and Salimans 2022): jointly train a conditional and unconditional model, mix the gradients at inference: \(\tilde s = (1+w)\,s_{\text{cond}} - w\,s_{\text{uncond}}\)
• Hard conditioning: bake the target into the score network directly (MatterGen approach)
• Strength parameter trades off fidelity to the target against diversity — a constant battle

Set up the mechanics in score language (payoff of the score slide). Bayes at every noise level: \(\nabla_x \log p_t(x \mid y) = \nabla_x \log p_t(x) + \nabla_x \log p_t(y \mid x)\). Classifier guidance implements the second term with a separately trained property predictor evaluated on noisy structures; the sampler simply adds its gradient at each denoising step. Elegant — and fragile: the predictor must be trained on corrupted inputs, and its gradients off-manifold are exactly the adversarial-gradient problem from the “why not gradient descent on \(x\)” discussion.

Classifier-free guidance in one breath. Train one network with the condition randomly dropped (~10% of the time); at sampling, extrapolate: \(\tilde s = (1+w)\,s_{\text{cond}} - w\,s_{\text{uncond}}\). No noisy classifier, one model, one knob \(w\).

Spend real time on the \(w\) trade-off — it is the practitioner’s daily dial. \(w = 0\): on-distribution diversity, weak targeting. Large \(w\): samples slam onto the target — and onto each other (diversity collapse, over-satisfied conditions, validity degradation). For discovery you want diverse hits (the funnel kills most of them for independent reasons), so materials pipelines typically run lower guidance than image generation would — a genuinely non-obvious transfer failure between fields.

Terminology bridge. “Hard conditioning” here = the condition as a network input (MatterGen’s property embedding + adapters), i.e. amortised conditioning rather than sampling-time steering. The three mechanisms compose — MatterGen conditions natively and can be guidance-boosted.

Pacing. ~3 min.

Composition, Symmetry, Synthesizability

• Composition: fix the chemical system (e.g. only Li–Mn–O), or fix exact stoichiometry
• Symmetry / space group: bias toward a specific space group (DiffCSP++)
• Synthesizability: rule out compositions with no known synthesis route — usually via an auxiliary classifier trained on literature data
• Structure prototype: condition on a known structural family (perovskite, spinel, MOF)
• Multi-constraint conditioning is the realistic discovery setting and the hardest to satisfy

Sort the four constraint types by how they are enforced — that is the real content. Composition and space group are hard constraints: enforceable by construction (mask the species vocabulary; generate inside a Wyckoff template à la DiffCSP++). Properties are soft: steered by guidance, verified later. Synthesizability is the odd one — neither hard nor truly soft, because we lack a ground-truth label; it enters as an auxiliary classifier trained on literature-mined synthesis data (and on the negative signal of things repeatedly attempted and never made).

Say the uncomfortable truth about synthesizability classifiers. Their training data is the published record: positives are what people happened to make, negatives are largely absence of evidence. The classifier therefore encodes historical fashion (what chemistries were fundable) as if it were physics. It is still worth using — it filters obvious dead-ends — but treat its score as a prior, not a verdict. This foreshadows the A-Lab feedback loop in Part V, where failed robotic syntheses finally generate honest negatives.

Prototype conditioning, the industry favourite. “Give me new perovskites” is a template constraint — high hit rates because the family is known-synthesisable, at the price of never leaving the family. Perfect for optimisation campaigns; useless for finding new structural motifs. Match the constraint to the mission.

The composition rule of thumb. Fixing the chemical system (Li–Mn–O, any stoichiometry) preserves far more useful diversity than fixing exact stoichiometry — let the generator explore the phase field, let the hull judge.

Pacing. ~2.5 min.

Classifier vs Classifier-Free Guidance

Classifier guidance

• Needs a separate predictor \(p_\phi(y\mid x)\) trained on noisy \(x_t\)
• Gradients \(\nabla_x\log p_\phi(y\mid x)\) steer sampling
• Inherits the predictor’s biases and overfitting

Classifier-free guidance

• One model, two passes (conditional + unconditional)
• No noisy-image classifier to train
• Cleaner; now the default in image and crystal models

For multi-property targets, mixed strategies (CFG + a property predictor head) are common.

This slide is deliberately redundant — use it as consolidation. The mechanics were on “Targeting a Property”; here the job is the decision rule. Have the class articulate it: when do you accept the cost of training a separate noisy-input classifier? Answer: when the property predictor already exists and retraining the generator is off the table, or when you need to steer a frozen pretrained model toward a property it was never conditioned on. Otherwise CFG wins on simplicity and calibration — one model, no classifier-gradient pathologies — which is why it is the default in both image and crystal work.

The failure mode each choice imports. Classifier guidance fails through its classifier: overconfident gradients on off-manifold noisy inputs push samples into adversarial corners (the generator equivalent of U8’s OOD abuse). CFG fails through its knob: crank \(w\) and diversity quietly dies before fidelity does — and you may not notice because every sample looks plausible.

Practical hybrid, said concretely. CFG for the property the base model was trained to condition on + a lightweight classifier gradient for the extra constraint added after the fact (e.g. “also non-magnetic”). Composable because everything is additive in score space — the deep reason the score view owns this field.

Pacing. ~2 min.

Multi-Objective Conditioning

• Real discovery wants several targets at once (low cost, high bandgap, stable, synthesizable)
• Naïve: combine guidance gradients in a weighted sum — works for 2–3 axes
• Sophisticated: Pareto-front exploration via diverse-sample acquisition (Tanimoto / determinantal point processes)
• Constrained generation: enforce hard constraints (e.g. space group) and condition softly on the rest
• Coupled to Unit 13 (now folded into Unit 14): multi-objective UQ + acquisition is the cleanest framework

Open with the reality check. Nobody’s discovery spec has one line. A battery cathode: voltage window and capacity and stability and earth-abundant elements and a plausible synthesis route. Single-property benchmarks are the toy version; this slide is the real game.

Why the naïve weighted sum decays past ~3 objectives. Guidance gradients for different properties point in different directions; a weighted sum finds the compromise direction, and in high dimensions the compromise satisfies nobody — you sample the mediocre middle of the Pareto front. Worse, the weights are in arbitrary units (eV vs GPa vs a classifier logit); tuning them is alchemy.

The better mental model: generate the front, not the point. You want a diverse set spanning the trade-off surface, then let downstream economics pick. Diversity-enforcing selection — determinantal point processes, or simple greedy dissimilarity in a fingerprint space (Tanimoto-style) — applied to the sample pool gets most of the benefit with none of the guidance surgery.

The engineering recipe that actually ships: hard-constrain what can be templated (composition system, space group), CFG-condition the primary property, filter the rest post-hoc in the funnel. Constraints where possible, guidance where necessary, filtering for everything else.

Bridge + schedule note. Choosing which Pareto candidates deserve DFT/synthesis budget is an acquisition-function problem — that is the U13 material, whose lecture slot (07.07) was cancelled and folded into Unit 14 (14.07); point students to the U13 deck as the reference for the formalism.

Pacing. ~2.5 min. Part IV ends here — target ~75 min.

Part V — Downstream Filtering

The Candidate Funnel

A 2025-era production pipeline:

1. Generate \(\sim 10^6\) candidates with a conditional model
2. Sanitise (charge balance, valid composition, no overlaps) \(\to 10^5\)
3. Relax with MLIP (MACE-MP-0 / M3GNet / CHGNet) \(\to 10^4\)
4. Score with MLIP (energy, properties) \(\to 10^3\) above-hull stable
5. DFT verify the top \(10^2\) — energy, bandgap, magnetism
6. UQ filter \(\to 10^1\) trustworthy + on-target
7. Synthesise the 1–10 that survive

• Each \(\to\) is at least 10× — generation needs to over-produce by orders of magnitude

Now walk the funnel slowly — this is the payoff of the whole lecture. Attach a cost and a tool to every stage as you go: generation (GPU-seconds/sample, Parts II–III), sanitisation (milliseconds — SMACT rules, distance checks, dedup by structure matcher), MLIP relax (~0.1 GPU-s, next slide), MLIP scoring (hull distance from U6 machinery), DFT (GPU-hours, two slides on), UQ triage (U13/U14), synthesis (weeks). The cost per candidate rises ~9 orders of magnitude top to bottom; the count falls to match. That inverse pairing is the design principle: cheap tests early, expensive tests late — the same logic as any screening cascade in drug discovery.

Point at the leverage. A 2× better generator (higher raw S.U.N.) doubles the input to every later stage; a 2× better filter at stage 3 halves the waste at stages 4–6. Improvements compound multiplicatively along the funnel — this answers the earlier discussion question about where to spend the €100k.

Common student misconception to kill now. The funnel is not a one-way street; Part V’s active-learning slide bends it into a loop (DFT results retrain the MLIP; failed syntheses retrain the synthesizability filter). Present the linear picture first, then loop it.

Pacing. ~2 min for the overview; details in the next five slides.

MLIP Relaxation

• Universal MLIPs (Unit 6) make this stage cheap: 0.1 s/structure on a GPU
• MACE-MP-0 (Batatia et al. 2025), M3GNet (Chen and Ong 2022), CHGNet (Deng et al. 2023), ORB (Neumann et al. 2024), MatterSim (Yang et al. 2024) all support direct PyMatGen relax APIs
• A typical generation pipeline runs all MLIPs and only keeps structures where they agree (consensus filter)
• Disagreement between MLIPs is a leading indicator of generator-induced distribution shift
• Without MLIPs, this stage would require DFT for every candidate — not feasible at \(10^5\)

Close the loop to U6 explicitly. When we taught MACE-MP-0 and friends, the pitch was “DFT-quality forces at classical-potential cost”. This slide is why that matters: the funnel’s middle is economically impossible without them. 0.1 s vs hours per relaxation is 4–5 orders of magnitude — the difference between filtering \(10^5\) candidates overnight and not doing the project.

The consensus filter is the practical gem — dwell on it. Different universal MLIPs (MACE-MP-0, M3GNet, CHGNet, ORB, MatterSim) share training-data ancestry but differ in architecture and biases. Where they agree, the prediction is probably in well-charted territory; where they diverge, at least one is extrapolating. Ensemble disagreement as a cheap epistemic-uncertainty proxy — precisely the U13-deck idea, implemented with models you already have. And note the built-in irony: generated structures are supposed to be novel, i.e. mildly OOD by design — so some disagreement is the price of ambition; zero disagreement means you are rediscovering the training set.

Two failure modes to warn about. (1) MLIPs relax into their own biases — a structure can be “MLIP-stable” but DFT-unstable; hull distances from MLIP energies are pre-screens, never verdicts. (2) Soft modes and magnetism are weak spots of all current universal potentials; candidates whose stability hinges on either deserve DFT earlier, not later.

Benchmark pointer for self-study. Matbench Discovery ranks universal MLIPs on exactly this use case (stability classification of unseen structures) — worth showing students where to check the current leader.

Pacing. ~2.5 min.

DFT Validation

• A few hundred top candidates per campaign survive to DFT
• VASP / Quantum ESPRESSO / FHI-AIMS run the same protocol used to label the training data
• Critical: use the same XC functional and convergence settings as the training set — otherwise the hull comparison is meaningless
• 8–24 GPU-hours per structure are routine for modest-sized unit cells
• DFT remains the bottleneck of inverse design even with all the ML upstream

The XC-functional bullet is the one that costs real money — expand it. The convex hull is a difference of energies, and energies are only comparable within one level of theory. If the training labels are PBE (+U for transition-metal oxides, with MP’s correction scheme) and your validation runs r²SCAN or different pseudopotentials/cutoffs, your “0.05 eV/atom above hull” is noise. The protocol — functional, pseudopotentials, k-mesh, smearing, corrections — must match the reference database exactly. This is the least glamorous and most violated rule in the pipeline; reviewers now check for it.

Why DFT survives as the arbiter despite everything upstream. It is the level of theory the entire hull infrastructure is built on — MLIPs are trained on it, databases are labelled with it. Replacing it mid-pipeline would redefine “stable” under you. The honest statement: ML moved the bottleneck from “DFT for everything” to “DFT for the top \(10^2\)–\(10^3\)” — it did not remove it.

Give the budget arithmetic. A few hundred candidates × 8–24 GPU-hours ≈ the entire compute cost of training a mid-size generator. Per campaign. This is why UQ triage (next slide) exists: every DFT slot wasted on a candidate the surrogate was merely unsure about is a slot stolen from a genuine contender.

Caveat for honesty. DFT itself errs (band gaps, van der Waals, strong correlation) — “DFT-validated” means “consistent with the reference theory”, not “will exist”. The lab remains the last word, which is where the funnel ends.

Pacing. ~2 min.

Uncertainty-Aware Filtering

• Each surrogate (MLIP energy, property head) ships a predicted value and an uncertainty
• Reject candidates where the surrogate uncertainty is too large to commit to expensive DFT
• Reject candidates where the property estimate is close to the target only because uncertainty is high (false confidence)
• Treated in depth in Unit 13 — UQ is the glue between generation and discovery
• Without UQ, “high yield” generators ship many junk hits

The third bullet is the subtlest idea in Part V — slow down for it. A candidate can look on-target because the surrogate is uncertain: with a wide predictive distribution, the point estimate can land anywhere, including exactly on your spec. Selecting by point estimate therefore preferentially harvests the high-uncertainty junk — selection bias masquerading as success. The fix is to select on something like $P( ) $ under the predictive distribution, not on \(|\hat y - y^\star|\). Students who internalise this one mechanism understand why UQ is load-bearing and not decorative.

Two rejection rules, two different reasons. Reject when uncertainty is too large to justify a DFT slot (budget protection); reject when the apparent fit is uncertainty-driven (the selection-bias trap above). Same \(\sigma\), different logic.

Where the \(\sigma\) comes from, concretely. MLIP committee disagreement (previous slide), ensembles/dropout on the property head, or a GP on a learned representation for small labelled sets — the U13-deck toolbox. No new machinery needed today; the point is where in the funnel it plugs in.

Schedule note, said aloud. The U13 lecture slot was cancelled; the acquisition/calibration formalism arrives inside Unit 14 (14.07), with the U13 deck online as the full reference. Today you need only the role of UQ, not its internals.

Pacing. ~2 min.

The GNoME Story

Merchant et al. (Nature 2023, DeepMind) (Merchant et al. 2023) — graph networks for materials exploration.

• Not strictly generative — used a GNN energy predictor + crystallographic substitution rules to propose candidates
• Validated ~380 k new stable structures against DFT
• An order-of-magnitude jump in the size of the known-stable catalogue overnight
• Sparked the 2023–2025 surge of generative-model papers — every group needed a competitive answer
• Open data release is the standard reference for “novel materials” benchmarks in 2024–2025

Get the mechanism right — students will mislabel GNoME as a generator on the exam. Two candidate mills: symmetry-aware partial substitutions into known prototypes (SAPS) and random structure search; a GNN energy model then triages what goes to DFT; DFT results retrain the GNN; repeat. It is a screening + active-learning machine — brute intelligence applied to enumeration, not sampling from a learned \(p(x)\). Its inclusion in this lecture is as the funnel-at-DeepMind-scale exemplar and as the field’s competitive detonator.

Give the numbers with their fine print. ~2.2M structures below or near the (updated) hull, ~381k labelled “stable” against the MP+OQMD+ICSD reference of the day. Then teach the controversy — it is a gift for critical reading:

• Novelty critique (Cheetham & Seshadri, Chem. Mater. 2024): a large fraction of “new” entries are compositional decorations of known prototypes — e.g. swapping rare-earth neighbours — and many involve ordered arrangements of what are experimentally disordered solid solutions. “New to the database” ≠ “new to chemistry”.
• A-Lab critique (next-slide companion; Leeman et al., PRX Energy 2024): of the headline “41 of 58 synthesised” claims, independent re-analysis of the XRD argued most were misassigned or already-known phases.

The teaching point is not “GNoME bad”. The dataset is genuinely useful and the scale jump real. The point: at \(10^5\)-discoveries scale, validation becomes the science, and database-relative metrics (stability, novelty) inflate quietly. This is the “beware leaderboards” bullet from the limitations slide, wearing a Nature cover.

Pacing. ~3.5 min. Worth every second — this story carries the course’s epistemics.

Active Learning + Generative Loop

• Round 1: generate, MLIP filter, DFT a subset
• Round 2: retrain the MLIP / property head on the new DFT data
• Round 3: regenerate with the improved scorer; smaller funnel attrition
• 3–6 rounds typically halve the cost-per-validated-discovery
• This loop is the operational heart of “AI for materials” platforms in 2025 (Microsoft Quantum, Google DeepMind, A-Lab at LBNL, …)
• Couples directly to Unit 13 on acquisition functions

Why the loop beats the line — the information argument. Every DFT run is a label you paid GPU-hours for; throwing it away after filtering is economic malpractice. Fold it back: the MLIP gets fine-tuned on exactly the chemistry the generator is currently exploring, so the region where the surrogate was weakest — the generator’s frontier — is where it improves fastest. That is active learning with the generator as the acquisition proposal mechanism.

Trace one cycle on the slide, concretely. Round 1: the MLIP disagrees wildly on Na–Mn–F candidates → those go to DFT preferentially (uncertainty sampling, U13-deck vocabulary) → round 2’s MLIP now trusts that region → funnel attrition drops → generation retargets. The “3–6 rounds halve cost-per-discovery” figure is the compounding of exactly this.

Connect to GNoME. GNoME’s engine was this loop, run hard: each DFT round retrained the GNN, and hit-rates rose round over round. The loop is proven at scale; what generative models change is only the proposal stage — from enumeration to sampling.

Pre-empt the fragility question. “Can the loop go wrong?” Yes — feedback loops amplify bias: if the surrogate is systematically wrong in some chemistry and confidently wrong, the loop never samples there and never learns better (the exploration–exploitation knife-edge; U13-deck material). Honest UQ is the loop’s immune system — miscalibration doesn’t just waste one round, it compounds.

Pacing. ~2 min.

Lab Automation Handoff

• Surviving candidates are passed to an autonomous laboratory: A-Lab (LBNL) (Szymanski et al. 2023), MIT Cyborg, GSK / Insitro platforms
• Robotic synthesis: powder mixing, sintering, XRD characterisation
• The synthesizability classifier at the start of the funnel determines whether a candidate even reaches this stage
• Failed syntheses feed back into the synthesizability label, closing a meta-loop
• Cycle time: weeks for inorganic crystals, days for thin films, hours for some MOFs

Describe one A-Lab cycle so it stops being an abstraction. Target arrives with a text-mined recipe prior → robot doses and mixes precursor powders → furnace sinter → automated XRD → ML phase identification against simulated patterns → success/failure verdict → next target. Seventeen days, 58 targets attempted, 41 claimed — and then the community re-analysis (Leeman et al., PRX Energy 2024) argued most claimed successes were XRD misassignments or known phases. Complete the GNoME-slide story here: the robotic loop worked mechanically; the automated characterisation was the weak link. Autonomous synthesis is bottlenecked by autonomous analysis — for this audience (microscopy/diffraction people), that is the career-relevant sentence of the slide.

The meta-loop bullet is quietly the most valuable. Failed syntheses are the only source of honest negative labels for synthesizability models (recall: the literature publishes successes). A robot lab that logs its failures systematically is building a dataset no amount of literature mining can produce. Arguably worth more long-term than the successes.

Calibrate expectations on cycle time. Weeks per inorganic candidate means the lab tests \(10^1\)–\(10^2\) per year while the generator makes \(10^6\) per day — a 4–5 order-of-magnitude impedance mismatch. That is why the funnel must be so aggressive, and why every upstream filter that removes a doomed candidate saves weeks, not seconds.

Pacing. ~2 min. Part V ends here — target ~88 min.

Closing

Open Challenges

• Disorder and defects: every model today assumes a perfect crystal; real materials are partially disordered
• Realistic synthesizability: “predicted stable” \(\ne\) “you can make it next week”
• Property breadth: bandgap and bulk modulus are easy; transport, catalytic activity, magnetism are hard
• Out-of-distribution candidates: the models trust themselves outside the training manifold — UQ is essential (Unit 13)
• Compute cost: a single training run costs \(10^4\)–\(10^5\) GPU-hours; reproducibility is fragile
• Evaluation: no single benchmark captures discovery quality — beware leaderboards

Frame this as the research-frontier slide — where theses live. Each bullet is an open problem with a visible 2025 attack vector; give one sentence of each so ambitious students leave with a direction:

• Disorder: every generator emits an ordered cell; TaCr₂O₆ came back cation-disordered, and most functional materials (battery cathodes, high-entropy alloys) are disordered by design. Representing partial occupancy in a generative model is genuinely unsolved.
• Synthesizability: hull distance is a thermodynamic statement at 0 K; synthesis is kinetics, precursors, and temperature windows. Text-mined recipe models and robot-lab negatives (previous slide) are the current wedge.
• Property breadth: conditioning is only as good as the labels — bulk modulus and bandgap have \(10^5\)-scale DFT labels; catalytic activity and transport need beyond-DFT theory or experiment, so the conditioning simply doesn’t exist yet.
• OOD trust: the generator’s whole purpose is to leave the training distribution, while every scorer downstream degrades exactly there — the structural tension of the field (U8’s lesson, industrialised).
• Compute/reproducibility: nuance the number — full frontier-scale runs sit at the top of that range; a DiffCSP-class model is \(10^2\)–\(10^3\) GPU-hours. The reproducibility pain is less raw compute than stack fragility and unreleased training data.

Classroom move to close the section. Ask which bullet blocks industrial adoption most. Defensible favourite: synthesizability — everything upstream is already faster than the lab.

Pacing. ~1.5 min. It’s a menu, not a meal.

Key Takeaways

• Inverse design = sample from a learned conditional distribution \(p(x\mid y^\star)\)
• Diffusion dominates current crystal generation; flow matching and LLM-style autoregressive are closing fast
• Generated structures must pass S.U.N. + downstream MLIP / DFT / UQ filters before any synthesis claim
• Universal MLIPs (Unit 6) are the indispensable scoring layer of the funnel
• The generative model is one component of a loop that includes UQ (Unit 13), lab automation, and re-training
• Generative + universal MLIP + UQ + autonomous lab is the operational stack of 2025 materials discovery

Read the takeaways as the exam contract — say so explicitly. The five must-know statements for Unit 12:

1. Inverse design = conditional sampling from learned \(p(x \mid y^\star)\), not inversion of a predictor — and why point-inversion fails (many-to-one, off-manifold gradients).
2. A crystal generator must handle four variable types (species: categorical; lattice: continuous; fractional coordinates: periodic/torus; symmetry: discrete group) — this mixed structure is why the architectures look the way they do.
3. Diffusion = learn to reverse a fixed noising process (\(\epsilon\)-prediction ≡ score matching); flow matching = regress a deterministic ODE velocity (fewer steps); autoregression = crystal as token sequence (prompting, exact likelihood, exposure bias).
4. S.U.N. and hull-based stability are reference-relative — no rate is meaningful without naming the hull, database snapshot, and matcher tolerance.
5. Generation is stage one of a funnel (sanitise → MLIP relax → DFT → UQ → lab) whose costs rise ~9 orders of magnitude per candidate — and which closes into an active-learning loop.

The one-sentence version for the road: “The generator proposes; the funnel disposes; the loop learns.”

Pointer for the exercise session. Statement 5 is what they will live on Thursday: sample from a pretrained checkpoint, MACE-MP-0-relax, report S.U.N. against the MP hull, defend a top-10 shortlist. Bring questions about matcher tolerances — everyone hits that wall.

Pacing. ~1 min.

Outlook — Unit 13

• Unit 13: uncertainty-aware discovery and Gaussian Processes — turning “candidate” into “decision”
• Aleatoric vs epistemic uncertainty, calibration, active learning loops
• GPs as the small-data reference; deep ensembles and evidential learning at scale
• Closing the loop: generated candidates \(\to\) UQ filter \(\to\) next experiment
• Unit 14: physical constraints, trust, and outlook — the last word on what ML can and cannot do for materials

Schedule reality (state it plainly). The Unit 13 lecture slot (07.07.2026) is cancelled; its core — uncertainty-aware discovery, GPs, acquisition functions — is folded into the Unit 14 session on 14.07, and the full U13 deck remains online as an optional reference. So the next lecture delivers both the UQ formalism and the constraints/trust closing.

Sell the continuation with today’s own cliffhangers. Three times today we said “you need honest uncertainty for this and deferred it”: the UQ-triage stage of the funnel, the false-confidence selection trap, and the active-learning loop’s immune system. The next session pays those debts: aleatoric vs epistemic, calibration, GPs as the small-data reference, ensembles at scale, and acquisition functions that turn “candidate” into “next experiment”.

End the lecture on the loop, not the model. Final sentence suggestion: “This unit gave you the engine — the generator; next time you get the steering — uncertainty and acquisition. Neither discovers anything alone.”

Pacing. ~0.5 min. Land at 90.

Continue

• ← Previous: Unit 10 — Representation Learning and Feature Discovery
• → Next: Unit 13 — Uncertainty-Aware Discovery & Gaussian Processes (optional reference — lecture cancelled, core folded into Unit 14)
• All courses

References

Antunes, Luis M., Keith T. Butler, and Ricardo Grau-Crespo. 2024. “Crystal Structure Generation with Autoregressive Large Language Modeling.” Nature Communications 15 (10570). https://doi.org/10.1038/s41467-024-54639-7.

Austin, Jacob, Daniel D. Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg. 2021. “Structured Denoising Diffusion Models in Discrete State-Spaces.” Advances in Neural Information Processing Systems 34. https://arxiv.org/abs/2107.03006.

Batatia, Ilyes et al. 2025. “A Foundation Model for Atomistic Materials Chemistry.” The Journal of Chemical Physics 163 (18): 184110. https://doi.org/10.1063/5.0297006.

Batatia, Ilyes, Dávid Péter Kovács, Gregor N. C. Simm, Christoph Ortner, and Gábor Csányi. 2022. “MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields.” Advances in Neural Information Processing Systems 35. https://arxiv.org/abs/2206.07697.

Batzner, Simon, Albert Musaelian, Lixin Sun, et al. 2022. “E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials.” Nature Communications 13: 2453. https://doi.org/10.1038/s41467-022-29939-5.

Chen, Chi, and Shyue Ping Ong. 2022. “A Universal Graph Deep Learning Interatomic Potential for the Periodic Table.” Nature Computational Science 2: 718–28. https://doi.org/10.1038/s43588-022-00349-3.

Dan, Yabo, Yong Zhao, Xiang Li, Shaobo Li, Ming Hu, and Jianjun Hu. 2020. “Generative Adversarial Networks (GAN) Based Efficient Sampling of Chemical Composition Space for Inverse Design of Inorganic Materials.” Npj Computational Materials 6: 84. https://doi.org/10.1038/s41524-020-00352-0.

Davies, Daniel W., Keith T. Butler, Adam J. Jackson, Jonathan M. Skelton, Kazuki Morita, and Aron Walsh. 2019. “SMACT: Semiconducting Materials by Analogy and Chemical Theory.” Journal of Open Source Software 4 (38): 1361. https://doi.org/10.21105/joss.01361.

Deng, Bowen et al. 2023. “CHGNet as a Pretrained Universal Neural Network Potential for Charge-Informed Atomistic Modelling.” Nature Machine Intelligence 5: 1031–41. https://doi.org/10.1038/s42256-023-00716-3.

Dhariwal, Prafulla, and Alexander Nichol. 2021. “Diffusion Models Beat GANs on Image Synthesis.” Advances in Neural Information Processing Systems 34: 8780–94.

Ho, Jonathan, Ajay Jain, and Pieter Abbeel. 2020. “Denoising Diffusion Probabilistic Models.” Advances in Neural Information Processing Systems 33: 6840–51.

Ho, Jonathan, and Tim Salimans. 2022. Classifier-Free Diffusion Guidance. https://arxiv.org/abs/2207.12598.

Jain, Anubhav et al. 2013. “Commentary: The Materials Project: A Materials Genome Approach to Accelerating Materials Innovation.” APL Materials 1 (1): 011002. https://doi.org/10.1063/1.4812323.

Jiao, Rui, Wenbing Huang, Peijia Lin, et al. 2023. “Crystal Structure Prediction by Joint Equivariant Diffusion.” Advances in Neural Information Processing Systems (NeurIPS) 36. https://arxiv.org/abs/2309.04475.

Jiao, Rui, Wenbing Huang, Yu Liu, Deli Zhao, and Yang Liu. 2024. “Space Group Constrained Crystal Generation.” International Conference on Learning Representations (ICLR). https://arxiv.org/abs/2402.03992.

Karras, Tero, Miika Aittala, Timo Aila, and Samuli Laine. 2022. “Elucidating the Design Space of Diffusion-Based Generative Models.” Advances in Neural Information Processing Systems 35. https://arxiv.org/abs/2206.00364.

Kirklin, Scott et al. 2015. “The Open Quantum Materials Database (OQMD): Assessing the Accuracy of DFT Formation Energies.” Npj Computational Materials 1: 15010. https://doi.org/10.1038/npjcompumats.2015.10.

Lipman, Yaron, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. 2023. “Flow Matching for Generative Modeling.” International Conference on Learning Representations (ICLR). https://arxiv.org/abs/2210.02747.

Lu, Cheng, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. 2022. “DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps.” Advances in Neural Information Processing Systems 35. https://arxiv.org/abs/2206.00927.

Merchant, Amil, Simon Batzner, Samuel S. Schoenholz, Muratahan Aykol, Gowoon Cheon, and Ekin Dogus Cubuk. 2023. “Scaling Deep Learning for Materials Discovery.” Nature 624: 80–85. https://doi.org/10.1038/s41586-023-06735-9.

Miller, Benjamin Kurt, Ricky T. Q. Chen, Anuroop Sriram, and Brandon M. Wood. 2024. “FlowMM: Generating Materials with Riemannian Flow Matching.” Proceedings of the 41st International Conference on Machine Learning (ICML), Proceedings of machine learning research, vol. 235: 35664–86.

Neumann, Mark, James Gin, Benjamin Rhodes, et al. 2024. Orb: A Fast, Scalable Neural Network Potential. https://doi.org/10.48550/arXiv.2410.22570.

Noh, Juhwan et al. 2019. “Inverse Design of Solid-State Materials via a Continuous Representation.” Matter 1 (5): 1370–84. https://doi.org/10.1016/j.matt.2019.08.017.

Nouira, Asma, Nataliya Sokolovska, and Jean-Claude Crivello. 2018. CrystalGAN: Learning to Discover Crystallographic Structures with Generative Adversarial Networks. https://arxiv.org/abs/1810.11203.

Ong, Shyue Ping et al. 2013. “Python Materials Genomics (Pymatgen): A Robust, Open-Source Python Library for Materials Analysis.” Computational Materials Science 68: 314–19. https://doi.org/10.1016/j.commatsci.2012.10.028.

Ren, Zekun et al. 2022. “An Invertible Crystallographic Representation for General Inverse Design of Inorganic Crystals with Targeted Properties.” Matter 5 (1): 314–35. https://doi.org/10.1016/j.matt.2021.11.032.

Satorras, Víctor Garcia, Emiel Hoogeboom, and Max Welling. 2021. “E(n) Equivariant Graph Neural Networks.” Proceedings of the 38th International Conference on Machine Learning, Proceedings of machine learning research, vol. 139: 9323–32.

Schmidt, Jonathan et al. 2024. “Improving Machine-Learning Models in Materials Science Through Large Datasets.” Materials Today Physics 48: 101560. https://doi.org/10.1016/j.mtphys.2024.101560.

Song, Jiaming, Chenlin Meng, and Stefano Ermon. 2021. “Denoising Diffusion Implicit Models.” International Conference on Learning Representations. https://openreview.net/forum?id=St1giarCHLP.

Song, Yang, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. 2021. “Score-Based Generative Modeling Through Stochastic Differential Equations.” International Conference on Learning Representations. https://openreview.net/forum?id=PxTIG12RRHS.

Szymanski, Nathan J. et al. 2023. “An Autonomous Laboratory for the Accelerated Synthesis of Inorganic Materials.” Nature 624: 86–91. https://doi.org/10.1038/s41586-023-06734-w.

Wood, Brandon M. et al. 2025. UMA: A Family of Universal Models for Atoms. https://doi.org/10.48550/arXiv.2506.23971.

Xie, Tian, Xiang Fu, Octavian-Eugen Ganea, Regina Barzilay, and Tommi Jaakkola. 2022. “Crystal Diffusion Variational Autoencoder for Periodic Material Generation.” International Conference on Learning Representations (ICLR). https://arxiv.org/abs/2110.06197.

Yang, Han et al. 2024. MatterSim: A Deep Learning Atomistic Model Across Elements, Temperatures and Pressures. https://doi.org/10.48550/arXiv.2405.04967.

Zagorac, D., H. Müller, S. Ruehl, J. Zagorac, and S. Rehme. 2019. “Recent Developments in the Inorganic Crystal Structure Database: Theoretical Crystal Structure Data and Related Features.” Journal of Applied Crystallography 52 (5): 918–25. https://doi.org/10.1107/S160057671900997X.

Zeni, Claudio, Robert Pinsler, Daniel Zügner, et al. 2025. “A Generative Model for Inorganic Materials Design.” Nature 639: 624–32. https://doi.org/10.1038/s41586-025-08628-5.