Spring 2026
This commit is contained in:
David Allemang
206
00-collated-results.md
Normal file
206
00-collated-results.md
Normal file
@@ -0,0 +1,206 @@
|
||||
# The Structural Integration of Physical Laws in Neural Architectures: A Multi-Paradigm Survey of Physics-Informed Machine Learning
|
||||
|
||||
The intersection of classical physics and contemporary artificial intelligence
|
||||
has given rise to a transformative field known as Physics-Informed Machine
|
||||
Learning (PIML). For decades, scientific discovery relied on the dichotomy of
|
||||
"first-principles" mathematical modeling and purely empirical observation.
|
||||
However, the modern data landscape, characterized by high-dimensional
|
||||
observations from sensors and simulations, has outpaced the capabilities of
|
||||
traditional numerical solvers while simultaneously highlighting the fragility of
|
||||
standard "black-box" neural networks.[1][1] PIML seeks to synthesize these
|
||||
approaches by treating physical laws not merely as external benchmarks, but as
|
||||
foundational constraints within the learning pipeline. This synthesis addresses
|
||||
the chronic data scarcity in scientific domains, enhances the generalizability
|
||||
of models across unseen regimes, and ensures that the outputs of deep learning
|
||||
models remain physically plausible.[3][3] This report provides an exhaustive
|
||||
analysis of the four primary paradigms of physical integration: data- and
|
||||
loss-embedded physics, architecture-embedded physics, operator-embedded physics,
|
||||
and system-embedded physics.
|
||||
|
||||
## II. Architecture-Embedded Physics: Hard Constraints and Geometric Deep Learning
|
||||
|
||||
Architecture-embedded physics represents a paradigm shift from "soft" to "hard"
|
||||
physical constraints. Instead of hoping the loss function steers the model
|
||||
toward physical reality, architecture-embedded physics bakes physical laws
|
||||
directly into the network's topology.[2][2] This approach ensures that the model
|
||||
natively respects symmetries (like rotation or translation invariance),
|
||||
conservation laws (like mass or energy), and structural interaction patterns
|
||||
(like $N$-body dynamics).[15][15]
|
||||
|
||||
### **Representative State-of-the-Art in Architecture-Embedded Physics**
|
||||
|
||||
State-of-the-art developments in this category focus on atomic and electronic
|
||||
structure modeling, where the interaction between particles is modeled as a
|
||||
graph where nodes are atoms and edges are bonds.[15] Models like DeepH-E3 and
|
||||
NequIP have demonstrated the ability to predict electronic Hamiltonians and
|
||||
interatomic potentials with sub-meV accuracy, outperforming traditional solvers
|
||||
while being orders of magnitude faster.[15]
|
||||
|
||||
#### Equivariant Tensor Networks
|
||||
|
||||
In scientific domains, physical systems are defined by their geometric
|
||||
structure. For instance, the forces acting on a molecule must rotate exactly as
|
||||
the molecule rotates. Standard MLPs must learn this through data
|
||||
augmentation (training on thousands of rotated examples), which is
|
||||
computationally wasteful and prone to error. In contrast, Equivariant Tensor
|
||||
Networks are architecturally designed so that their internal feature
|
||||
representations transform according to the underlying symmetry group, such as
|
||||
E(3) (Euclidean) or SO(3) (Rotation). This ensures the neural mapping fθ
|
||||
satisfies fθ(g⋅x)=g⋅fθ(x) for any group action g. This "inductive bias" makes
|
||||
the model coordinate-blind, leading to exceptional data efficiency and
|
||||
robustness.
|
||||
|
||||
Unified Formulation: ∂ is a Symmetry Operator. The network uθ is restricted such
|
||||
that uθ(partial(x))=partial(uθ(x)).
|
||||
|
||||
| Paper Reference | Core Innovation | Key Strengths | Identified Weaknesses |
|
||||
|:--------------------------------|:-----------------------------|:--------------------------------------------------------------------------------|:------------------------------------------------------------------------------|
|
||||
| [Group-Equivariant Survey][18] | Group Representation Theory | Establishes the mathematical standard for SO(3) and Lorentz group networks. | Abstract theoretical nature; few practical code implementations provided. |
|
||||
| [NequIP / MACE (2021/2024)][15] | E(3)-Equivariant Potentials | Unprecedented data efficiency; captures higher-order multi-body interactions. | Complex implementation of tensor products can lead to high inference latency. |
|
||||
| [DeepH-E3 (2023)][16] | Equivariant DFT Hamiltonian | Preserves Euclidean symmetry for supercells $\>10^4$ atoms; ab initio accuracy. | Significant memory consumption for high-rank tensor representations. |
|
||||
| [Atomic-site ENN (2024)][15] | Lattice Symmetry-Aware | Bridges microscopic electronic processes to mesoscale behavior in solids. | Computationally intensive for very large supercells. |
|
||||
| [QHNet (2023)][19] | Efficient SE(3)-Equivariance | Reduces the number of tensor products by 92% compared to previous SOTA. | Trade-off between structural simplicity and expressive capacity. |
|
||||
|
||||
#### Hamiltonian Networks
|
||||
|
||||
Physical systems are governed by fundamental conservation laws (energy,
|
||||
momentum, mass). In this subcategory, the architecture moves from predicting a
|
||||
field u to predicting a scalar Hamiltonian or energy potential H. By embedding
|
||||
the symplectic structure of physics into the layers, the model cannot produce a
|
||||
result which violates the conserved property because the final output is derived
|
||||
by the predicted energy surface. This ensures that the system stays on a valid
|
||||
physical manifold during long-term time-series simulations, avoiding the
|
||||
"explosion" of values common in black-box models.
|
||||
|
||||
Unified Formulation: ∂ is a Conservation Operator (like the Gradient or Curl).
|
||||
The model is forced to output a scalar "Energy" first, and the actual physical
|
||||
state is derived by taking the derivative of that energy.
|
||||
|
||||
| Paper Reference | Core Innovation | Key Strengths | Identified Weaknesses |
|
||||
|:------------------------------------------|:------------------------------|:-------------------------------------------------------------------------|:-----------------------------------------------------------------------------|
|
||||
| [Neural Hamiltonian Diffusion (2025)][20] | Manifold Hamiltonian Learning | Unifies stochastic diffusion and Hamiltonian mechanics on curved spaces. | Requires a-priori knowledge of the Riemannian manifold's metric. |
|
||||
| [Deep Potentials (2021)][17] | Density-based Descriptors | High-speed molecular dynamics with quantum mechanical fidelity. | Struggles with systems undergoing chemical reactions (bond breaking). |
|
||||
| [SpinGNN (2025)][17] | Heisenberg/Spin-Lattice GNN | Preserves symmetries of exchange and spin-lattice couplings for magnets. | Specialized architecture that lacks general-purpose utility for soft matter. |
|
||||
| [Heisenberg Edge GNN][17] | Equivariant Message Passing | Specifically captures tensorial quantities like spin Hall conductivity. | Performance is sensitive to the cutoff radius for atomic interactions. |
|
||||
|
||||
#### Basis-Expansion Networks
|
||||
|
||||
Complexity in physics often arises from the interaction of multiple particles
|
||||
becoming exponentially difficult to calculate. Rather than forcing a neural
|
||||
network to learn these complex interactions from raw data, Basis-Expansion
|
||||
Networks limit the network’s "vocabulary" to a set of physically proven
|
||||
templates. By projecting the problem onto a mathematically complete basis set (
|
||||
like Atomic Cluster Expansion), the network only needs to learn the weights of
|
||||
these basis functions. This turns the neural network into a "Neural Code" or a
|
||||
differentiable version of a classical physics solver, combining the flexibility
|
||||
of AI with the rigor of analytical physics.
|
||||
|
||||
Unified Formulation: ∂ is a Projection Operator. The network uθ is a weighted
|
||||
sum of physical templates: uθ=∑wiϕi, where ϕi are fixed, physically valid
|
||||
functions.
|
||||
|
||||
| Paper Reference | Core Innovation | Key Strengths | Identified Weaknesses |
|
||||
|:---------------------------|:-----------------------------|:--------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|
|
||||
| [ACE Framework (2024)][15] | Atomic Cluster Expansion | Hierarchical basis for symmetry-adapted invariants; mathematically complete. | Steep learning curve for researchers not versed in group representation theory. |
|
||||
| [AI2DFT (2024)][22] | Differential DFT Neural Code | First unsupervised physics-informed learning framework for DFT quantities. | Stability depends on the quality of the variational energy functional. |
|
||||
| [Timrov et al. (2025)][21] | Hubbard Parameter ENN | Speeds up Hubbard $U$ and $V$ calculations via equivariant occupation matrices. | Transferability is high but confined to the specific lattice structures trained. |
|
||||
|
||||
### **Implications of Hard Constraints on Emergent Behavior**
|
||||
|
||||
The integration of Hamiltonian mechanics into neural architectures (Hamiltonian
|
||||
Neural Networks or HNNs) structurally guarantees energy conservation, a feat
|
||||
that is nearly impossible for data-embedded PINN models over long simulation
|
||||
times.[20][20] By deriving the dynamics from a learned scalar Hamiltonian
|
||||
function $H\_\\theta$, the model respects the symplectic structure of phase
|
||||
space, preventing the "energy drift" commonly seen in standard RNNs or
|
||||
Transformers used for physics simulation.[20][20]
|
||||
|
||||
A deep insight from recent ENN literature is the realization that strict
|
||||
equivariance might be too restrictive for certain "broken symmetry" systems.
|
||||
This has led to the development of relaxed-symmetry models that allow for small,
|
||||
learnable deviations from perfect equivariance, which is critical for modeling
|
||||
materials under stress or in non-equilibrium states.[17] Furthermore, the move
|
||||
toward "unsupervised" learning in models like AI2DFT suggests that the
|
||||
variational principles of physics (like minimizing total energy) can serve as
|
||||
the ultimate loss function, potentially bypassing the need for labeled DFT data
|
||||
entirely.[22]
|
||||
|
||||
#### **Works cited**
|
||||
|
||||
[0]: https://www.researchgate.net/publication/391540378_When_physics_meets_machine_learning_a_survey_of_physics-informed_machine_learning
|
||||
|
||||
[1]: https://arxiv.org/html/2408.09840v2
|
||||
|
||||
[2]: https://arxiv.org/html/2506.13777v1
|
||||
|
||||
[3]: https://arxiv.org/html/2501.06572v1
|
||||
|
||||
[4]: https://www.ejpam.com/ejpam/article/view/6334/2750
|
||||
|
||||
[5]: https://www.mdpi.com/2227-7390/13/20/3289
|
||||
|
||||
[6]: https://www.articsledge.com/post/physics-informed-neural-networks-pinns
|
||||
|
||||
[7]: https://www.researchgate.net/publication/388357372_A_Review_of_Physics-Informed_Neural_Networks
|
||||
|
||||
[8]: https://towardsdatascience.com/essential-review-papers-on-physics-informed-neural-networks-a-curated-guide-for-practitioners/
|
||||
|
||||
[9]: https://ieeexplore.ieee.org/iel8/8784343/10845831/10843279.pdf
|
||||
|
||||
[10]: https://arxiv.org/html/2408.06650v1
|
||||
|
||||
[11]: https://github.com/Event-AHU/PINN_Paper_List
|
||||
|
||||
[12]: https://www.frontiersin.org/journals/artificial-intelligence/articles/10.3389/frai.2026.1717117/full
|
||||
|
||||
[13]: https://www.emergentmind.com/topics/physics-inspired-kolmogorov-arnold-network-pikan
|
||||
|
||||
[14]: https://arxiv.org/html/2601.04104v1
|
||||
|
||||
[15]: https://pmc.ncbi.nlm.nih.gov/articles/PMC10199065/
|
||||
|
||||
[16]: https://www.oaepublish.com/articles/jmi.2025.17
|
||||
|
||||
[17]: https://pmc.ncbi.nlm.nih.gov/articles/PMC12541325/
|
||||
|
||||
[18]: https://proceedings.mlr.press/v202/yu23i/yu23i.pdf
|
||||
|
||||
[19]: https://neurips.cc/virtual/2025/poster/117646
|
||||
|
||||
[20]: https://communities.springernature.com/posts/machine-learning-hubbard-parameters-with-equivariant-neural-networks
|
||||
|
||||
[21]: https://arxiv.org/pdf/2403.11287
|
||||
|
||||
[22]: https://www.researchgate.net/publication/394511831_Gaussian_Splashing_Unified_Particles_for_Versatile_Motion_Synthesis_and_Rendering
|
||||
|
||||
[23]: https://arxiv.org/html/2512.24986v1
|
||||
|
||||
[24]: https://www.researchgate.net/publication/382119336_A_Review_of_Differentiable_Simulators
|
||||
|
||||
[25]: https://www.emergentmind.com/topics/differentiable-simulation-engines
|
||||
|
||||
[26]: https://arxiv.org/html/2203.00806v5
|
||||
|
||||
[27]: https://www.semanticscholar.org/paper/A-Review-of-Differentiable-Simulators-Newbury-Collins/b3a10024b9ad159a6dc68d3acce36dffc464dd67
|
||||
|
||||
[28]: https://www.azooptics.com/News.aspx?newsID=30558
|
||||
|
||||
[29]: https://www.researchgate.net/publication/372961478_Spatially_Varying_Nanophotonic_Neural_Networks
|
||||
|
||||
[30]: https://pubs.acs.org/doi/10.1021/acsphotonics.4c01874
|
||||
|
||||
[31]: https://www.mdpi.com/2304-6732/12/12/1187
|
||||
|
||||
[32]: https://pmc.ncbi.nlm.nih.gov/articles/PMC12758549/
|
||||
|
||||
[33]: https://opg.optica.org/oe/abstract.cfm?uri=oe-34-2-2197
|
||||
|
||||
[34]: https://www.spiedigitallibrary.org/conference-proceedings-of-spie/PC13585.toc
|
||||
|
||||
[35]: https://www.spiedigitallibrary.org/conference-proceedings-of-spie/0/PC139061/Diffractive-deep-neural-networks-with-multimode-fibers/10.1117/12.3079670.full
|
||||
|
||||
[36]: https://opg.optica.org/optcon/fulltext.cfm?uri=optcon-4-8-1810
|
||||
|
||||
[37]: https://www.spiedigitallibrary.org/journals/advanced-photonics-nexus/volume-4/issue-2/026009/Compressed-meta-optical-encoder-for-image-classification/10.1117/1.APN.4.2.026009.pdf
|
||||
|
||||
[38]: https://www.researchgate.net/publication/339555300_Deep_Tensor_ADMM-Net_for_Snapshot_Compressive_Imaging
|
||||
Reference in New Issue
Block a user