4.1 KiB
I have reviewed a number of papers relating to physics-informed machine learning principles.
- Deep Tensor ADMM-Net for Snapshot Compressive Imaging
- End-to-End Optimization of Optics and Image Processing
- NeRF Basics
- Implicit Surfaces via Volume Rendering
- Continuum-aware NeRF (PAC-NeRF)
- NeRF in Scattering Media
- Lens Design with Differentiable Ray Tracing
- Hybrid Lens Design with Differentiable Wave Optics
- Diffractive Deep Neural Networks
- Spatially Varying Nanophotonic Neural Networks
- 3D Gaussian Splatting
- Physics Integrated Gaussians
- Intro to Graph Neural Networks
- Interaction Networks for Learning Physics
- GNNs as Learnable Physics Engines
- Graph-based Physics Simulators
- Deep Image Prior
- GNNs and Generative Priors for Solving Inverse Problems
- Invertible Generative Models
- Diffusion Posterior Sampling
Broadly, I have categorized these papers into four groups:
- Data and Loss embedded physics. Physical constraints are applied in the training data and/or loss functions; any physical accuracy in the results is implicitly learned from these.
- Architecture embedded physics. Physical constraints are applied in the model architecture. Typically this involves some sub-stage of the model which decodes a latent vector, applies some analytical physical formulae to it, then re-encodes it to a new latent vector for downstream processing. For example PAC-NeRF.
- Operator embedded physics. The machine learning model does not directly produce outputs, but rather it produces some state representation which is then processed by an analytical physical model. For example differentiable implicit renderers as in NeRF or Gaussian Splatting.
- System embedded physics. Some stage of the model involves a hardware physical step, such as optical neural networks or robotic feedback mechanisms.
Synthesize a rough outline for a literature review paper which explores, generalizes, and unifies these four categories of Physics-Informed Neural Networks (PINNs).
In particular, for each category, provide:
- Brief introduction of the category
- Unified formulation of architectures within the category
- List of criteria to search for papers which describe models of the category
- A coarse outline of that section of the review paper.
We have already built some notions for unified formulation and outline for the first category, data- and loss-embedded physics, listed below.
(note that we have not yet expressed robotic feedback mechanisms in this common formulation)
The driving formulation for all these models is that there is some general time-dependent nonlinear PDE of the form:
d_t u(x, t) + N_u(x, t) = 0
where N denotes a possibly nonlinear spatial differential operator, and u is the unknown physical field. The neural network u_theta approximates u, and from this we can solve inverse problems or forward inference.
PINNs define a physics residual by substituting the neural approximation u_theta into the governing equation:
r_theta(x, t) = d_t u_theta(x, t) + N_u_theta(x, t)
The governing equation is satisfied at a point (x, t) when r_theta(x, t) = 0, so the physics loss penalizes based on this residual at various observations (x_j, t_j).
That is, we optimize for the model parameters theta by
argmin_theta | N_theta(x, t) - u_theta(x, t) |^2_2
In architecture embedded physics, the model approximation u_theta
incorporates some physics-based differential possibly nonlinear operator
partial which informs the result.
argmin_theta | N_theta(partial, x, t) - u_theta(partial, x, t) |^2_2
In operator-embedded physics physics, the differential operator is applied to the model result.
argmin_theta | partial(N_theta, x, t) - partial(u_theta, x, t) |^2_2
Finally, in system-embedded physics, the model parameters and model are expressed in such a way that inference can be realized by a physical system. For example, of optical neural networks, the parameters theta are expressed in terms of diffraction gratings or nanophotonics. In this sense, the parameters are optimized by operator-embedded physics where partial is a free-space wave propagation operator.