207 lines
13 KiB
Markdown
207 lines
13 KiB
Markdown
# The Structural Integration of Physical Laws in Neural Architectures: A Multi-Paradigm Survey of Physics-Informed Machine Learning
|
||
|
||
The intersection of classical physics and contemporary artificial intelligence
|
||
has given rise to a transformative field known as Physics-Informed Machine
|
||
Learning (PIML). For decades, scientific discovery relied on the dichotomy of
|
||
"first-principles" mathematical modeling and purely empirical observation.
|
||
However, the modern data landscape, characterized by high-dimensional
|
||
observations from sensors and simulations, has outpaced the capabilities of
|
||
traditional numerical solvers while simultaneously highlighting the fragility of
|
||
standard "black-box" neural networks.[1][1] PIML seeks to synthesize these
|
||
approaches by treating physical laws not merely as external benchmarks, but as
|
||
foundational constraints within the learning pipeline. This synthesis addresses
|
||
the chronic data scarcity in scientific domains, enhances the generalizability
|
||
of models across unseen regimes, and ensures that the outputs of deep learning
|
||
models remain physically plausible.[3][3] This report provides an exhaustive
|
||
analysis of the four primary paradigms of physical integration: data- and
|
||
loss-embedded physics, architecture-embedded physics, operator-embedded physics,
|
||
and system-embedded physics.
|
||
|
||
## II. Architecture-Embedded Physics: Hard Constraints and Geometric Deep Learning
|
||
|
||
Architecture-embedded physics represents a paradigm shift from "soft" to "hard"
|
||
physical constraints. Instead of hoping the loss function steers the model
|
||
toward physical reality, architecture-embedded physics bakes physical laws
|
||
directly into the network's topology.[2][2] This approach ensures that the model
|
||
natively respects symmetries (like rotation or translation invariance),
|
||
conservation laws (like mass or energy), and structural interaction patterns
|
||
(like $N$-body dynamics).[15][15]
|
||
|
||
### **Representative State-of-the-Art in Architecture-Embedded Physics**
|
||
|
||
State-of-the-art developments in this category focus on atomic and electronic
|
||
structure modeling, where the interaction between particles is modeled as a
|
||
graph where nodes are atoms and edges are bonds.[15] Models like DeepH-E3 and
|
||
NequIP have demonstrated the ability to predict electronic Hamiltonians and
|
||
interatomic potentials with sub-meV accuracy, outperforming traditional solvers
|
||
while being orders of magnitude faster.[15]
|
||
|
||
#### Equivariant Tensor Networks
|
||
|
||
In scientific domains, physical systems are defined by their geometric
|
||
structure. For instance, the forces acting on a molecule must rotate exactly as
|
||
the molecule rotates. Standard MLPs must learn this through data
|
||
augmentation (training on thousands of rotated examples), which is
|
||
computationally wasteful and prone to error. In contrast, Equivariant Tensor
|
||
Networks are architecturally designed so that their internal feature
|
||
representations transform according to the underlying symmetry group, such as
|
||
E(3) (Euclidean) or SO(3) (Rotation). This ensures the neural mapping fθ
|
||
satisfies fθ(g⋅x)=g⋅fθ(x) for any group action g. This "inductive bias" makes
|
||
the model coordinate-blind, leading to exceptional data efficiency and
|
||
robustness.
|
||
|
||
Unified Formulation: ∂ is a Symmetry Operator. The network uθ is restricted such
|
||
that uθ(partial(x))=partial(uθ(x)).
|
||
|
||
| Paper Reference | Core Innovation | Key Strengths | Identified Weaknesses |
|
||
|:--------------------------------|:-----------------------------|:--------------------------------------------------------------------------------|:------------------------------------------------------------------------------|
|
||
| [Group-Equivariant Survey][18] | Group Representation Theory | Establishes the mathematical standard for SO(3) and Lorentz group networks. | Abstract theoretical nature; few practical code implementations provided. |
|
||
| [NequIP / MACE (2021/2024)][15] | E(3)-Equivariant Potentials | Unprecedented data efficiency; captures higher-order multi-body interactions. | Complex implementation of tensor products can lead to high inference latency. |
|
||
| [DeepH-E3 (2023)][16] | Equivariant DFT Hamiltonian | Preserves Euclidean symmetry for supercells $\>10^4$ atoms; ab initio accuracy. | Significant memory consumption for high-rank tensor representations. |
|
||
| [Atomic-site ENN (2024)][15] | Lattice Symmetry-Aware | Bridges microscopic electronic processes to mesoscale behavior in solids. | Computationally intensive for very large supercells. |
|
||
| [QHNet (2023)][19] | Efficient SE(3)-Equivariance | Reduces the number of tensor products by 92% compared to previous SOTA. | Trade-off between structural simplicity and expressive capacity. |
|
||
|
||
#### Hamiltonian Networks
|
||
|
||
Physical systems are governed by fundamental conservation laws (energy,
|
||
momentum, mass). In this subcategory, the architecture moves from predicting a
|
||
field u to predicting a scalar Hamiltonian or energy potential H. By embedding
|
||
the symplectic structure of physics into the layers, the model cannot produce a
|
||
result which violates the conserved property because the final output is derived
|
||
by the predicted energy surface. This ensures that the system stays on a valid
|
||
physical manifold during long-term time-series simulations, avoiding the
|
||
"explosion" of values common in black-box models.
|
||
|
||
Unified Formulation: ∂ is a Conservation Operator (like the Gradient or Curl).
|
||
The model is forced to output a scalar "Energy" first, and the actual physical
|
||
state is derived by taking the derivative of that energy.
|
||
|
||
| Paper Reference | Core Innovation | Key Strengths | Identified Weaknesses |
|
||
|:------------------------------------------|:------------------------------|:-------------------------------------------------------------------------|:-----------------------------------------------------------------------------|
|
||
| [Neural Hamiltonian Diffusion (2025)][20] | Manifold Hamiltonian Learning | Unifies stochastic diffusion and Hamiltonian mechanics on curved spaces. | Requires a-priori knowledge of the Riemannian manifold's metric. |
|
||
| [Deep Potentials (2021)][17] | Density-based Descriptors | High-speed molecular dynamics with quantum mechanical fidelity. | Struggles with systems undergoing chemical reactions (bond breaking). |
|
||
| [SpinGNN (2025)][17] | Heisenberg/Spin-Lattice GNN | Preserves symmetries of exchange and spin-lattice couplings for magnets. | Specialized architecture that lacks general-purpose utility for soft matter. |
|
||
| [Heisenberg Edge GNN][17] | Equivariant Message Passing | Specifically captures tensorial quantities like spin Hall conductivity. | Performance is sensitive to the cutoff radius for atomic interactions. |
|
||
|
||
#### Basis-Expansion Networks
|
||
|
||
Complexity in physics often arises from the interaction of multiple particles
|
||
becoming exponentially difficult to calculate. Rather than forcing a neural
|
||
network to learn these complex interactions from raw data, Basis-Expansion
|
||
Networks limit the network’s "vocabulary" to a set of physically proven
|
||
templates. By projecting the problem onto a mathematically complete basis set (
|
||
like Atomic Cluster Expansion), the network only needs to learn the weights of
|
||
these basis functions. This turns the neural network into a "Neural Code" or a
|
||
differentiable version of a classical physics solver, combining the flexibility
|
||
of AI with the rigor of analytical physics.
|
||
|
||
Unified Formulation: ∂ is a Projection Operator. The network uθ is a weighted
|
||
sum of physical templates: uθ=∑wiϕi, where ϕi are fixed, physically valid
|
||
functions.
|
||
|
||
| Paper Reference | Core Innovation | Key Strengths | Identified Weaknesses |
|
||
|:---------------------------|:-----------------------------|:--------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|
|
||
| [ACE Framework (2024)][15] | Atomic Cluster Expansion | Hierarchical basis for symmetry-adapted invariants; mathematically complete. | Steep learning curve for researchers not versed in group representation theory. |
|
||
| [AI2DFT (2024)][22] | Differential DFT Neural Code | First unsupervised physics-informed learning framework for DFT quantities. | Stability depends on the quality of the variational energy functional. |
|
||
| [Timrov et al. (2025)][21] | Hubbard Parameter ENN | Speeds up Hubbard $U$ and $V$ calculations via equivariant occupation matrices. | Transferability is high but confined to the specific lattice structures trained. |
|
||
|
||
### **Implications of Hard Constraints on Emergent Behavior**
|
||
|
||
The integration of Hamiltonian mechanics into neural architectures (Hamiltonian
|
||
Neural Networks or HNNs) structurally guarantees energy conservation, a feat
|
||
that is nearly impossible for data-embedded PINN models over long simulation
|
||
times.[20][20] By deriving the dynamics from a learned scalar Hamiltonian
|
||
function $H\_\\theta$, the model respects the symplectic structure of phase
|
||
space, preventing the "energy drift" commonly seen in standard RNNs or
|
||
Transformers used for physics simulation.[20][20]
|
||
|
||
A deep insight from recent ENN literature is the realization that strict
|
||
equivariance might be too restrictive for certain "broken symmetry" systems.
|
||
This has led to the development of relaxed-symmetry models that allow for small,
|
||
learnable deviations from perfect equivariance, which is critical for modeling
|
||
materials under stress or in non-equilibrium states.[17] Furthermore, the move
|
||
toward "unsupervised" learning in models like AI2DFT suggests that the
|
||
variational principles of physics (like minimizing total energy) can serve as
|
||
the ultimate loss function, potentially bypassing the need for labeled DFT data
|
||
entirely.[22]
|
||
|
||
#### **Works cited**
|
||
|
||
[0]: https://www.researchgate.net/publication/391540378_When_physics_meets_machine_learning_a_survey_of_physics-informed_machine_learning
|
||
|
||
[1]: https://arxiv.org/html/2408.09840v2
|
||
|
||
[2]: https://arxiv.org/html/2506.13777v1
|
||
|
||
[3]: https://arxiv.org/html/2501.06572v1
|
||
|
||
[4]: https://www.ejpam.com/ejpam/article/view/6334/2750
|
||
|
||
[5]: https://www.mdpi.com/2227-7390/13/20/3289
|
||
|
||
[6]: https://www.articsledge.com/post/physics-informed-neural-networks-pinns
|
||
|
||
[7]: https://www.researchgate.net/publication/388357372_A_Review_of_Physics-Informed_Neural_Networks
|
||
|
||
[8]: https://towardsdatascience.com/essential-review-papers-on-physics-informed-neural-networks-a-curated-guide-for-practitioners/
|
||
|
||
[9]: https://ieeexplore.ieee.org/iel8/8784343/10845831/10843279.pdf
|
||
|
||
[10]: https://arxiv.org/html/2408.06650v1
|
||
|
||
[11]: https://github.com/Event-AHU/PINN_Paper_List
|
||
|
||
[12]: https://www.frontiersin.org/journals/artificial-intelligence/articles/10.3389/frai.2026.1717117/full
|
||
|
||
[13]: https://www.emergentmind.com/topics/physics-inspired-kolmogorov-arnold-network-pikan
|
||
|
||
[14]: https://arxiv.org/html/2601.04104v1
|
||
|
||
[15]: https://pmc.ncbi.nlm.nih.gov/articles/PMC10199065/
|
||
|
||
[16]: https://www.oaepublish.com/articles/jmi.2025.17
|
||
|
||
[17]: https://pmc.ncbi.nlm.nih.gov/articles/PMC12541325/
|
||
|
||
[18]: https://proceedings.mlr.press/v202/yu23i/yu23i.pdf
|
||
|
||
[19]: https://neurips.cc/virtual/2025/poster/117646
|
||
|
||
[20]: https://communities.springernature.com/posts/machine-learning-hubbard-parameters-with-equivariant-neural-networks
|
||
|
||
[21]: https://arxiv.org/pdf/2403.11287
|
||
|
||
[22]: https://www.researchgate.net/publication/394511831_Gaussian_Splashing_Unified_Particles_for_Versatile_Motion_Synthesis_and_Rendering
|
||
|
||
[23]: https://arxiv.org/html/2512.24986v1
|
||
|
||
[24]: https://www.researchgate.net/publication/382119336_A_Review_of_Differentiable_Simulators
|
||
|
||
[25]: https://www.emergentmind.com/topics/differentiable-simulation-engines
|
||
|
||
[26]: https://arxiv.org/html/2203.00806v5
|
||
|
||
[27]: https://www.semanticscholar.org/paper/A-Review-of-Differentiable-Simulators-Newbury-Collins/b3a10024b9ad159a6dc68d3acce36dffc464dd67
|
||
|
||
[28]: https://www.azooptics.com/News.aspx?newsID=30558
|
||
|
||
[29]: https://www.researchgate.net/publication/372961478_Spatially_Varying_Nanophotonic_Neural_Networks
|
||
|
||
[30]: https://pubs.acs.org/doi/10.1021/acsphotonics.4c01874
|
||
|
||
[31]: https://www.mdpi.com/2304-6732/12/12/1187
|
||
|
||
[32]: https://pmc.ncbi.nlm.nih.gov/articles/PMC12758549/
|
||
|
||
[33]: https://opg.optica.org/oe/abstract.cfm?uri=oe-34-2-2197
|
||
|
||
[34]: https://www.spiedigitallibrary.org/conference-proceedings-of-spie/PC13585.toc
|
||
|
||
[35]: https://www.spiedigitallibrary.org/conference-proceedings-of-spie/0/PC139061/Diffractive-deep-neural-networks-with-multimode-fibers/10.1117/12.3079670.full
|
||
|
||
[36]: https://opg.optica.org/optcon/fulltext.cfm?uri=optcon-4-8-1810
|
||
|
||
[37]: https://www.spiedigitallibrary.org/journals/advanced-photonics-nexus/volume-4/issue-2/026009/Compressed-meta-optical-encoder-for-image-classification/10.1117/1.APN.4.2.026009.pdf
|
||
|
||
[38]: https://www.researchgate.net/publication/339555300_Deep_Tensor_ADMM-Net_for_Snapshot_Compressive_Imaging
|