COMP-790-175/prompt-01-coarse-research.md

I have reviewed a number of papers relating to physics-informed machine learning principles.

• Deep Tensor ADMM-Net for Snapshot Compressive Imaging
• End-to-End Optimization of Optics and Image Processing
• NeRF Basics
• Implicit Surfaces via Volume Rendering
• Continuum-aware NeRF (PAC-NeRF)
• NeRF in Scattering Media
• Lens Design with Differentiable Ray Tracing
• Hybrid Lens Design with Differentiable Wave Optics
• Diffractive Deep Neural Networks
• Spatially Varying Nanophotonic Neural Networks
• 3D Gaussian Splatting
• Physics Integrated Gaussians
• Intro to Graph Neural Networks
• Interaction Networks for Learning Physics
• GNNs as Learnable Physics Engines
• Graph-based Physics Simulators
• Deep Image Prior
• GNNs and Generative Priors for Solving Inverse Problems
• Invertible Generative Models
• Diffusion Posterior Sampling

Broadly, I have categorized these papers into four groups, listed below.

Review the literature and identify, for each category, a dozen or so papers. These papers must be current (or expository) and be representative of the strengths and weaknesses of the state-of-the-art in each of these categories.

---

I. Data- and Loss-Embedded Physics

Brief Introduction

This category represents the most common paradigm of Physics-Informed Neural Networks (PINNs). In these models, the neural network architecture itself remains a standard black-box (e.g., an MLP), but physical laws are introduced as "soft constraints." The model implicitly learns physical accuracy because violations of governing analytical equations (like PDEs) are heavily penalized during training via the loss function, or because the training data itself is heavily curated by physical simulators.

Unified Formulation

The driving formulation is that there is some general time-dependent nonlinear PDE:

\partial_t u(x, t) + \mathcal{N}[u](x, t) = 0

where \mathcal{N} denotes a spatial differential operator and u is the unknown physical field. The neural network u_\theta approximates u. We define a physics residual by substituting u_\theta into the governing equation:

r_\theta(x, t) = \partial_t u_\theta(x, t) + \mathcal{N}[u_\theta](x, t)

Because the governing equation is satisfied when r_\theta(x, t) = 0, the physics loss penalizes this residual alongside data observations. Utilizing your framework's notation, we optimize the model parameters \theta by minimizing the discrepancy:

\arg\min_\theta \| \mathcal{N}_\theta(x, t) - u_\theta(x, t) \|_2^2

Search Criteria

To find literature in this category, filter for papers discussing:

• Keywords: Physics-Informed Neural Networks (PINNs), soft constraints, PDE residual loss, physics-guided machine learning, deep image prior, regularization via physics.
• Methodology: Models that use standard MLPs or CNNs but modify the training regime or loss landscape (e.g., adding a term to Mean Squared Error that computes gradients using Autograd to enforce physical laws).

Coarse Outline

1. Introduction to PINNs: The shift from purely data-driven to physics-guided learning.
2. Formulating the Physics Loss: Calculating derivatives and residuals via automatic differentiation.
3. Forward vs. Inverse Problems: Using data/loss-embedded models to discover unknown PDE parameters versus simulating known PDEs.
4. Generative Priors: Deep Image Prior and Diffusion Posterior Sampling as implicit data-driven physical regularizers.
5. Limitations: The "soft constraint" problem (models can still output physically impossible results if the loss isn't perfectly balanced) and optimization failures (stiff PDEs).

---

II. Architecture-Embedded Physics

Brief Introduction

Architecture-embedded physics moves from "soft constraints" to "hard constraints." Instead of relying on the loss function to penalize non-physical behavior, the physical constraints are baked directly into the neural network's topology or internal operations. The model decodes latent vectors, applies analytical physics formulations internally, and re-encodes them, ensuring the output natively respects invariances, symmetries, or conservation laws.

Unified Formulation

The model approximation u_\theta incorporates some physics-based, possibly nonlinear, differential operator \partial that structurally dictates the result. The internal forward pass is strictly bound by this operator:

\arg\min_\theta \| \mathcal{N}_\theta(\partial, x, t) - u_\theta(\partial, x, t) \|_2^2

Search Criteria

To find literature in this category, filter for papers discussing:

• Keywords: Hard constraints, physics-encoded architecture, invariant/equivariant neural networks, Graph Neural Networks (GNNs) for physics, Hamiltonian Neural Networks, PAC-NeRF.
• Methodology: Models containing custom layers, inductive biases, message-passing topologies matching physical interactions (like GNNs modeling particle systems), or internal latent physics solvers.

Coarse Outline

1. Transitioning to Hard Constraints: Overcoming the optimization challenges of standard PINNs.
2. Graph-based Physics Simulators: How GNNs naturally mirror $N$-body interactions and physical meshes (e.g., Interaction Networks).
3. Latent Physics Solvers: Embedding physical formulae in the latent space (e.g., PAC-NeRF incorporating continuum mechanics into the NeRF MLP).
4. Symmetry and Equivariance: Ensuring physical laws (like rotation or translation invariance) are structurally guaranteed.
5. Trade-offs: The balance between expressivity (neural capacity) and strict adherence to physical priors.

---

III. Operator-Embedded Physics

Brief Introduction

In operator-embedded physics, the neural network acts as a state generator rather than a direct solution generator. The ML model outputs a continuous or discrete state representation (like a radiance field, volume density, or Gaussian splat parameters). This state is then passed through an external, fixed analytical physical operator (like a differentiable renderer) to produce the final output. The physical simulator is the lens through which the neural network learns.

Unified Formulation

The differential or physical operator \partial (e.g., volume rendering integral, ray tracing) is applied after the model result, to the model's state representation. The loss is computed on the output of this operator:

\arg\min_\theta \| \partial(\mathcal{N}_\theta, x, t) - \partial(u_\theta, x, t) \|_2^2

Search Criteria

To find literature in this category, filter for papers discussing:

• Keywords: Differentiable rendering, differentiable physics engines, implicit neural representations (INRs), NeRF, 3D Gaussian Splatting, inverse graphics, differentiable wave optics.
• Methodology: Systems where an MLP predicts properties (color, density, index of refraction) at coordinates, which are subsequently integrated or propagated via classical physics equations.

Coarse Outline

1. The State-Representation Paradigm: Decoupling the neural representation from the physical observation.
2. Differentiable Simulators: How standard physics engines (ray tracers, wave propagators) are made differentiable to allow backpropagation to the neural state.
3. Implicit Neural Representations: NeRFs, Gaussian Splatting, and volume rendering as prime examples of operator-embedded physics.
4. Optical Inverse Problems: Lens design and scattering media models utilizing differentiable wave optics.

---

IV. System-Embedded Physics

Brief Introduction

System-embedded physics takes the computational graph off the traditional silicon chip and places it into the physical world. Here, inference is realized by physical hardware. The trainable parameters are physical objects (e.g., the thickness of a nanophotonic lens, the diffraction grating, or robotic actuators). The physical world itself computes the forward pass.

Unified Formulation

The parameters \theta represent physical properties, optimized via operator-embedded principles (where \partial is the natural physical environment, such as free-space wave propagation or Newtonian mechanics). For optical networks, optimization mirrors Category III. For robotic feedback mechanisms (which you noted needed formulation), the operator \partial acts as the real-world state transition dynamics given a physically embedded action a_\theta:

\text{State}_{t+1} = \partial(\text{State}_t, a_\theta)

The optimization minimizes the discrepancy between the physical system's true hardware performance and the desired objective:

\arg\min_\theta \| \partial_{\text{hardware}}(\mathcal{N}_\theta) - \text{Target} \|_2^2

Search Criteria

To find literature in this category, filter for papers discussing:

• Keywords: Optical neural networks, diffractive deep neural networks, nanophotonic neural networks, hardware-in-the-loop optimization, embodied AI, neuromorphic photonics, physical neural networks.
• Methodology: End-to-end optimization of physical sensors, meta-optics, analog physical computing, or robots using differentiable reality or real-world feedback to update hardware-level parameters.

Coarse Outline

1. From Software to Hardware: The physical world as the ultimate physics engine.
2. Optical and Photonic Neural Networks: Diffractive layers and spatially varying nanophotonics functioning as neural weights.
3. Robotic and Embodied Feedback: Hardware-in-the-loop training and optimizing physical control parameters.
4. End-to-End Co-design: Simultaneously optimizing the physical hardware (\theta) alongside downstream digital image processing.
5. The Future of PIML: Analog computing and overcoming the memory/power bottlenecks of traditional digital GPUs.