Github Code
Studon Link
1 Course overview
One 90-minute lecture per week plus one self-study Jupyter notebook per week (runs on a laptop CPU or in Google Colab). Electron microscopy is the application thread throughout: every method is introduced on EM images, spectra, diffraction data or tilt series. Lectures are on Fridays; rooms are announced on StudOn.
Assessment: miniproject (40%, reproducible notebook + short report) and written exam (60%).
2 Week 1: Python crash course & the EM data deluge (23.10.2026)
- Slides: Week 1: Python crash course & the EM data deluge
- Notebook:
week01_python_numpy.ipynb(open in Colab) - Python & NumPy for EM data; the CRISP-DM workflow; why modern detectors make automated analysis unavoidable; course logistics.
3 Week 2: Learning from EM data: signal formation, noise & loss (30.10.2026)
- Slides: Week 2: Learning from EM data: signal formation, noise & loss
- Notebook:
week02_poisson_noise.ipynb(open in Colab) - Learning vs classical analysis; detectors, dose, sampling and aliasing; Poisson/Gaussian noise; how the noise model determines the loss; data quality and metadata.
4 Week 3: Linear algebra, PCA & spectral unmixing (06.11.2026)
- Slides: Week 3: Linear algebra, PCA & spectral unmixing
- Notebook:
week03_pca_nmf_eels.ipynb(open in Colab) - SVD and PCA; conditioning; NMF, MCR-ALS and endmember unmixing; the EELS/EDS preprocessing pipeline.
5 Week 4: Regression, optimisation & honest validation (13.11.2026)
- Slides: Week 4: Regression, optimisation & honest validation
- Notebook:
week04_leakage_demo.ipynb(open in Colab) - Loss minimisation, gradient descent and optimisers; bias–variance and regularisation; splits by specimen, leakage taxonomy and metrics.
6 Week 5: From images to features: descriptors & tree ensembles (20.11.2026)
- Slides: Week 5: From images to features: descriptors & tree ensembles
- Notebook:
week05_features_trees.ipynb(open in Colab) - Particle and grain descriptors; atom-column finding and local-environment descriptors; random forests and gradient boosting; permutation importance and SHAP.
7 Week 6: Neural networks & backpropagation (27.11.2026)
- Slides: Week 6: Neural networks & backpropagation
- Notebook:
week06_tiny_mlp.ipynb(open in Colab) - MLPs, softmax and cross-entropy; autograd and backpropagation; initialisation, normalisation and training diagnostics.
8 Week 7: CNNs & U-Nets for microscopy (04.12.2026)
- Slides: Week 7: CNNs & U-Nets for microscopy
- Notebook:
week07_cnn_segmentation.ipynb(open in Colab) - Convolution and receptive fields; VGG to ResNet; U-Net segmentation; invariance vs equivariance; saliency, Grad-CAM and shortcut learning.
9 Week 8: Small data: augmentation, transfer & self-supervision (11.12.2026)
- Slides: Week 8: Small data: augmentation, transfer & self-supervision
- Notebook:
week08_embeddings_transfer.ipynb(open in Colab) - Physics-respecting augmentation; transfer learning and sim-to-real; SimCLR, MAE, DINO; foundation models (SAM, µSAM); linear probes.
10 Week 9: Unsupervised learning, autoencoders & latent spaces (18.12.2026)
- Slides: Week 9: Unsupervised learning, autoencoders & latent spaces
- Notebook:
week09_autoencoder_vae.ipynb(open in Colab) - k-means and GMMs; autoencoders and VAEs; rVAE for atomic-resolution images; t-SNE/UMAP and how latent spaces mislead.
11 Week 10: Attention, transformers & graph networks for EM (08.01.2027)
- Slides: Week 10: Attention, transformers & graph networks for EM
- Notebook:
week10_attention_gnn.ipynb(open in Colab) - Self-attention and vision transformers; tokenising diffraction patterns and spectra; atom-column graphs and message passing.
12 Week 11: Uncertainty, Gaussian processes & autonomous EM (15.01.2027)
- Slides: Week 11: Uncertainty, Gaussian processes & autonomous EM
- Notebook:
week11_gp_bo.ipynb(open in Colab) - Aleatoric vs epistemic uncertainty; ensembles, calibration and conformal prediction; Gaussian processes, Bayesian optimisation and automated 4D-STEM.
13 Week 12: Inverse problems I: regularisation, tomography & sensor fusion (22.01.2027)
- Slides: Week 12: Inverse problems I: regularisation, tomography & sensor fusion
- Notebook:
week12_inverse_deblurring.ipynb(open in Colab) - Forward models and ill-posedness; Tikhonov, TV and plug-and-play; tomography and the Fourier-slice theorem; HAADF + EDS sensor fusion.
14 Week 13: Inverse problems II: ptychography, generative priors & synthesis (29.01.2027)
- Slides: Week 13: Inverse problems II: ptychography, generative priors & synthesis
- Notebook:
week13_ptychography.ipynb(open in Colab) - Ptychography and multislice; gradient-descent reconstruction; GAN and diffusion priors; physics constraints; course synthesis and exam preparation.
15 Exam revision (05.02.2027)
- Repetition and preparation for the exam, based on the must-know list
16 Miniproject
Apply the full pipeline from the course (data → model → uncertainty → explainability) to an electron-microscopy or materials-characterisation dataset, as a fully reproducible notebook plus a 4–6 page report. Choose one of five options:
A. STEM/SEM image segmentation (U-Net or foundation-model features) B. Spectral denoising and phase clustering (PCA / NMF / VAE) C. Materials property regression with honest validation and uncertainty D. A small imaging inverse problem (deblurring or limited-angle tomography) E. Atom-column descriptors and defect classification (tree ensembles, optionally a GNN)
Milestones: topic selection from Week 1, proposal around Week 6, progress check-in around Week 10, final submission in the exam period. The full task descriptions and grading rubric are on StudOn.
Archive: SS 25 lectures
The previous (9-lecture) version of the course is kept here for reference.
16.1 Lecture 1: Intro (13.05.2025)
- Lecture slides: Lecture 1: Introduction
- d2l Chapter 2: Preliminaries
16.2 Lecture 2: Optimization, Regression, Sensor Fusion (20.05.2025)
16.3 Lecture 3: CNNs (03.06.2025)
- Lecture slides: Lecture 3: CNNs
- d2l Chapter 7: CNNs
- d2l Chapter 8: CNNs
16.4 Lecture 4: Classification, Segmentation, AutoEncoders (10.06.2025)
- Lecture slides: Lecture 4: Classification, Segmentation, AutoEncoders
- d2l Chapter 4: Classification
- d2l Chapter 14.9: Segmentation
- Segmentation
- Dimensionality Reduction
- PCA
- Autoencoder
- Variational Autoencoder
16.5 Miniproject (10.6. - 24.6.2025) concurrent to lectures
In the miniproject, you will test multiple deep neural network architectures on one of four microscopy-related tasks. You should summarize your results in a short presentation (5 minutes + 2 minutes discussion) and deliver a Jupyter Notebook with your code and results. The miniproject will be graded and will count as 40% towards your final grade.
Segmentation Task
We will use the HRTEM dataset from “A robust synthetic data generation framework for machine learning in high-resolution transmission electron microscopy (HRTEM)” by Rangel DaCosta et al. (2024) to implement a segmentation model. The goal is to segment nanoparticles in HRTEM images.
Please use the article “A robust synthetic data generation framework for machine learning in high-resolution transmission electron microscopy (HRTEM)” by Rangel DaCosta et al. (2024) as a starting point for your implementation.
The datast contains pairs of HRTEM images and ground truth segmentations.
VAE & Dimensionality Reduction
We will use the dataset from “Uncovering material deformations via machine learning combined with four-dimensional scanning transmission electron microscopy” by Shi et al. (2022) to implement a dimensionality reduction model and cluster 4DSTEM data.
The goal is to learn a mapping from 4DSTEM data to a lower-dimensional embedding where you can perform clustering to identify different deformation modes.
Please use the article “Uncovering material deformations via machine learning combined with four-dimensional scanning transmission electron microscopy” by Shi et al. (2022) as a starting point for your implementation.
Denoising
We will use the dataset from “Unsupervised deep denoising for four-dimensional scanning transmission electron microscopy” by Sadri et al. (2024) to implement a denoising model for 4DSTEM data.
The goal is to learn a mapping from noisy to clean 4DSTEM data.
Please use the article “Unsupervised deep denoising for four-dimensional scanning transmission electron microscopy” by Sadri et al. (2024) as a starting point for your implementation.
The article contains pytorch code for the model.
Learn how to adapt it to your needs and try to replicate the results on the SrTiO3_High_mag_Low_dose.npy and SrTiO3_High_mag_High_dose.npy datasets.
Image-to-Image Translation
We will use a simulated X-ray image dataset with pairs of projected thickness and phase contrast images to implement an Image to image translation model.
The goal is to learn a mapping from phase contrast images to projected thickness images.
This is usually a task that is solved with multiple measurements and a physical model of the imaging process.
Here we will try to learn this mapping from simulated data. Please use the article “Multi-resolution convolutional neural networks for inverse problems” by Wang et al. (2020) as a starting point for your implementation.
16.6 Lecture 5: Mixed Bag (24.06.2025)
- Lecture slides: Lecture 5: Advanced AutoEncoder, GANs, and more
- Project presentations
- Generative Adversarial Networks d2l Chapter 20: Generative Adversarial Networks
16.7 Lecture 6: Gaussian Processes Introduction (01.07.2025)
- Lecture slides: Lecture 6: Gaussian Processes Introduction
- Introduction to Gaussian Processes
- d2l Chapter 18: Gaussian Processes
16.8 Lecture 7: Gaussian Processes Applications (08.07.2025)
- Lecture slides: Lecture 7: Gaussian Processes Applications
- Bayesian Optimization
- Active Learning
- Deep Kernel Learning
16.9 Lecture 8: Imaging Inverse Problems 1 (15.07.2025)
- Lecture slides: Lecture 8: Imaging Inverse Problems 1
- Imaging Inverse Problems: Introduction
16.10 Lecture 9: Imaging Inverse Problems 2 (22.07.2025)
- Lecture slides: Lecture 9: Imaging Inverse Problems 2
References
Citation
@online{pelz2026,
author = {Pelz, Philipp},
title = {Lecture {Data} {Science} for {Electron} {Microscopy} {WS}
26/27},
date = {2026-09-27},
langid = {en},
abstract = {This is the website for the Data Science for Electron
Microscopy Lecture}
}