1 Mathematical Foundations of AI & ML – Unified Syllabus Overview (with ML-PC & MG)
Legend
- ★ First serious use — concept must be introduced in MFML before being used in ML-PC or MG
- ◎ Reinforcement / application — concept is applied or deepened, but not introduced
- (R) Refresher — topic was covered in a prior course and is only briefly revisited
- MFML Mathematical Foundations of AI & ML
- ML-PC Machine Learning in Materials Processing & Characterization
- MG Materials Genomics
| Week | MFML – Mathematical Foundations (revised) | ML-PC – ML in Materials Processing & Characterization (revised) | MG – Materials Genomics (revised) | Exercise (90 min, Python-based) | Dependency Logic |
|---|---|---|---|---|---|
| 1 | Learning vs data analysis; models, loss functions, prediction vs explanation | Role of ML in processing & characterization; ML vs physics models | Role of ML in materials discovery; databases & targets | NumPy refresher; vectors, dot products, simple loss (MSE) | MFML defines “learning” as optimization, not statistics |
| 2 | Linear algebra refresher for learning: covariance, PCA/SVD (R) | PCA as a tool for spectra & images (◎) | PCA & low-D structure in materials spaces (◎) | PCA refresher on known dataset; visualize variance directions | PCA assumed known; MFML aligns notation & geometry |
| 3 | Regression as loss minimization; linear models revisited | Regression as surrogate modeling for processes & properties (★) | Regression & correlation in materials datasets (★) | Linear regression from scratch via loss minimization | Regression reframed explicitly as learning problem |
| 4 | Neural networks early: neuron, activations, universal approximation | NN regression for materials properties (★) | NN models for structure–property relations (★) | Single-neuron + activation functions (manual forward pass) | MFML must precede any NN usage |
| 5 | Backpropagation, gradients, training dynamics | NN training stability & convergence (★) | NN training pitfalls in materials data (◎) | Manual backprop for shallow NN | MFML supplies chain rule & gradient flow |
| 6 | Loss landscapes, conditioning, optimization behavior | Hyperparameters, robustness, convergence issues (★) | Model robustness & sensitivity (◎) | Gradient descent experiments: learning rate & conditioning | Optimization treated as learning dynamics |
| 7 | Generalization, bias–variance, regularization | Overfitting control in models (★) | Limits of high-D regression (★) | Overfitting demo: polynomial vs NN models | Critical conceptual gate for both applied courses |
| 8 | Probabilistic view of learning: noise & likelihood | Noise-aware modeling & error propagation (◎) | Noise & uncertainty in materials datasets (★) | Noise injection; likelihood vs MSE comparison | MFML reframes probability for ML |
| 9 | Representation learning: learned vs engineered features | Feature learning in signals & images (★) | Descriptor learning vs hand-crafted features (★) | Feature learning with simple NN | Transition from classical descriptors |
| 10 | Latent spaces: autoencoders & embeddings | Compression & anomaly detection in processes (★) | Latent materials spaces & embeddings (★) | Autoencoder with framework (PyTorch/Keras) | Core week for Materials Genomics |
| 11 | Unsupervised learning revisited (objectives, not algorithms) | Clustering & process drift detection (◎) | Clustering vs discovery in materials space (◎) | Compare clustering vs AE embeddings | Students reinterpret known clustering methods |
| 12 | Uncertainty in predictions (aleatoric vs epistemic); Gaussian Processes (conceptual) | Trust & confidence in ML-assisted decisions; surrogate models (★) | Discovery & screening with uncertainty; exploration vs exploitation (★) | Predictive uncertainty: GP regression vs NN ensembles | Enables responsible ML & accelerator concepts |
| 13 | Physics-informed & constrained learning | Physics-informed ML for processes & characterization (★) | Physical constraints in materials ML (◎) | Constrained NN / penalty-based PINN demo | MFML leads constraints & PINN concepts |
| 14 | Explainability, limits, scientific trust | Integrated case studies & failure modes | Limits & ethics of data-driven discovery | Mini end-to-end synthesis project | All courses converge conceptually |
2 Recommended readings
We base much of the lecture on the following books:
- Neuer (2024), Machine Learning for Engineers: Introduction to Physics-Informed, Explainable Learning Methods for AI in Engineering Applications. Springer Nature.
- McClarren (2021), Machine Learning for Engineers: Using Data to Solve Problems for Physical Systems. Springer.
Tangentially, we also recommend the following books:
- Murphy (2012), Machine Learning: A Probabilistic Perspective. MIT Press.
- Bishop (2006), Pattern Recognition and Machine Learning. Springer Science+ Business Media, LLC Berlin, Germany.
| MFML Week | MFML Lecture Focus (Revised) | Neuer – Required Reading | Neuer – Optional / Skim | McClarren – Contextual / Optional | Bishop – Targeted Depth (Optional) |
|---|---|---|---|---|---|
| 1 | Learning vs data analysis; models, loss functions | Ch. 1.1 Data-Based Modeling; 1.1.1 Concept of Model | 1.1.3 Criticism of Data-Based Modeling | Ch. 1 Introduction (ML in physical systems) | Ch. 1 §1.1–1.2 (what is a model, pattern recognition view) |
| 2 | Linear algebra refresher; covariance, PCA/SVD (R) | Ch. 5.2 PCA (skim, notation & geometry only) | PCA implementation details | Ch. 5 Dimension Reduction (ROM intuition) | Ch. 12 §12.1–12.2 PCA derivation (selective) |
| 3 | Regression as loss minimization | Ch. 4.2.2 Regression; Ch. 4.4.1 LMS theory | LMS algorithm variants | Ch. 4 Regression (physical meaning of regression) | Ch. 3 §3.1–3.3 Linear regression, least squares geometry |
| 4 | Neural networks early: neuron & activations | Ch. 4.5.1 Neuron; 4.5.3 Activation Functions | Framework-specific NN sections | Ch. 8 Neural Networks (surrogate perspective) | Ch. 5 §5.1–5.2 Neural network basics |
| 5 | Backpropagation & gradient flow | Ch. 4.5.4 Training of Neural Networks | Advanced NN variants | Ch. 7 Optimization (inverse-problem framing) | Ch. 5 §5.3 Backpropagation (conceptual) |
| 6 | Loss landscapes & optimization behavior | Ch. 4.4.6 Hyperparameters; Ch. 4.5.5 Optimization | Detailed optimizer variants | Ch. 7 Optimization | Ch. 3 §3.4 Regularization; §3.5 Bayesian view (skim) |
| 7 | Generalization, bias–variance, regularization | Ch. 4.5.9 Overfitting & Cross-Validation | — | Ch. 6 Model Selection & Validation | Ch. 3 §3.2 Bias–variance decomposition |
| 8 | Probabilistic view of learning; noise | Ch. 2.2 Distinguishing Uncertainties; Ch. 6.4 Uncertainty | Bayesian details | Ch. 3 Error and Uncertainty | Ch. 2 §2.1–2.3 Gaussian distributions & moments |
| 9 | Representation learning; features vs learned reps | Ch. 5.5 Autoencoder (intro & motivation) | AE uncertainty extensions | Ch. 5 Dimension Reduction | Ch. 12 §12.3 Nonlinear PCA / latent variables |
| 10 | Latent spaces; embeddings | Ch. 5.5.1–5.5.3 Autoencoder & Latent Space | AE architectures | Ch. 5 Dimension Reduction | Ch. 12 §12.3–12.4 Latent variable intuition |
| 11 | Unsupervised learning revisited (objectives) | Ch. 5.3 K-Means (objective-based view) | t-SNE, advanced clustering | Ch. 9 Classification (decision boundaries) | Ch. 9 Mixture Models & EM (conceptual only) |
| 12 | Uncertainty in predictions | Ch. 6.4 Stochastic Methods for Uncertainty | Advanced stochastic methods | Ch. 3 Error and Uncertainty | Ch. 3 §3.5 Bayesian regularization (skim) |
| 13 | Physics-informed & constrained learning | Ch. 6.1–6.3 Physics-Informed Learning | Semantic technologies | Ch. 11 Physics-Informed & Hybrid Models | Ch. 1 §1.6 Model complexity & Occam’s razor |
| 14 | Explainability, limits, scientific trust | Ch. 7 Explainability (discussion & outlook) | — | Ch. 12 Limitations and Outlook | Ch. 1 §1.1–1.2 Reflection on model limits |
References
Bishop, Christopher M. 2006. Pattern Recognition and Machine Learning by Christopher m. Bishop. Vol. 400. Springer Science+ Business Media, LLC Berlin, Germany:
McClarren, Ryan G.. 2021. Machine Learning for Engineers: Using Data to Solve Problems for Physical Systems. Springer.
Murphy, Kevin P. 2012. Machine Learning: A Probabilistic Perspective. MIT press.
Neuer, Marcus J. 2024. Machine Learning for Engineers: Introduction to Physics-Informed, Explainable Learning Methods for AI in Engineering Applications. Springer Nature.