Abstract

Data sparsity is still the main challenge to apply machine learning models to solve complex scientific and engineering problems. The root cause is the “curse of dimensionality” in training these models. Training algorithms need to explore and exploit in a very high-dimensional parameter space to search the optimal parameters for complex models. In this study, a new scheme of multifidelity physics-constrained neural networks with minimax architecture is proposed to improve the data efficiency of training neural networks by incorporating physical knowledge as constraints and sampling data with various fidelities. In this new framework, fully connected neural networks with two levels of fidelities are combined to improve the prediction accuracy. The low-fidelity neural network is used to approximate the low-fidelity data, whereas the high-fidelity neural network is adopted to approximate the correlation function between the low-fidelity and high-fidelity data. To systematically search the optimal weights of various losses for reducing the training time, the Dual-Dimer algorithm is adopted to search high-order saddle points of the minimax optimization problem. The proposed framework is demonstrated with two-dimensional heat transfer, phase transition, and dendritic growth problems, which are fundamental in materials modeling. With the same set of training data, the prediction error of the multifidelity physics-constrained neural network with minimax architecture can be two orders of magnitude lower than that of the multifidelity neural network with minimax architecture.

References

1.
Lisboa
,
P. J.
, and
Taktak
,
A. F. G.
,
2006
, “
The Use of Artificial Neural Networks in Decision Support in Cancer: A Systematic Review
,”
Neural Netw.
,
19
(
4
), pp.
408
415
.
2.
Schmidhuber
,
J.
,
2015
, “
Deep Learning in Neural Networks: An Overview
,”
Neural Netw.
,
61
, pp.
85
117
.
3.
Wang
,
X.
,
Lin
,
X.
, and
Dang
,
X.
,
2020
, “
Supervised Learning in Spiking Neural Networks: A Review of Algorithms and Evaluations
,”
Neural Netw.
,
125
, pp.
258
280
.
4.
Lobo
,
J. L.
,
Del Ser
,
J.
,
Bifet
,
A.
, and
Kasabov
,
N.
,
2020
, “
Spiking Neural Networks and Online Learning: An Overview and Perspectives
,”
Neural Netw.
,
121
, pp.
88
100
.
5.
Johnson
,
N. S.
,
Vulimiri
,
P. S.
,
To
,
A. C.
,
Zhang
,
X.
,
Brice
,
C. A.
,
Kappes
,
B. B.
, and
Stebner
,
A. P.
,
2020
, “
Invited Review: Machine Learning for Materials Developments in Metals Additive Manufacturing
,”
Addit. Manuf.
,
36
, p.
101641
.
6.
Yang
,
Z.
,
Li
,
X.
,
Brinson
,
L. C.
,
Choudhary
,
A. N.
,
Chen
,
W.
, and
Agrawal
,
A.
,
2018
, “
Microstructural Materials Design via Deep Adversarial Learning Methodology
,”
ASME J. Mech. Des.
,
140
(
11
), p.
111416
.
7.
Yang
,
Z.
,
Yabansu
,
Y. C.
,
Jha
,
D.
,
Liao
,
W. K.
,
Choudhary
,
A. N.
,
Kalidindi
,
S. R.
, and
Agrawal
,
A.
,
2019
, “
Establishing Structure-Property Localization Linkages for Elastic Deformation of Three-Dimensional High Contrast Composites Using Deep Learning Approaches
,”
Acta Mater.
,
166
, pp.
335
345
.
8.
Li
,
J.
,
Sage
,
M.
,
Guan
,
X.
,
Brochu
,
M.
, and
Zhao
,
Y. F.
,
2020
, “
Machine Learning-Enabled Competitive Grain Growth Behavior Study in Directed Energy Deposition Fabricated Ti6Al4V
,”
JOM
,
72
(
1
), pp.
458
464
.
9.
Dissanayake
,
M. W. M. G.
, and
Phan-Thien
,
N.
,
1994
, “
Neural-Network-Based Approximations for Solving Partial Differential Equations
,”
Commun. Numer. Methods Eng.
,
10
(
3
), pp.
195
201
.
10.
Mai-Duy
,
N.
, and
Tran-Cong
,
T.
,
2001
, “
Numerical Solution of Navier-Stokes Equations Using Multiquadric Radial Basis Function Networks
,”
Int. J. Numer. Methods Fluids
,
37
(
1
), pp.
65
86
.
11.
Jianyu
,
L.
,
Siwei
,
L.
,
Yingjian
,
Q.
, and
Yaping
,
H.
,
2003
, “
Numerical Solution of Elliptic Partial Differential Equation Using Radial Basis Function Neural Networks
,”
Neural Netw.
,
16
(
5–6
), pp.
729
734
.
12.
Souza De Cursi
,
J. E.
, and
Koscianski
,
A.
,
2007
,
Advances and Innovations in Systems, Computing Sciences and Software Engineering
,
Springer
, pp.
567
572
.
13.
Raissi
,
M.
,
Perdikaris
,
P.
, and
Karniadakis
,
G. E.
,
2019
, “
Physics-Informed Neural Networks: A Deep Learning Framework for Solving Forward and Inverse Problems Involving Nonlinear Partial Differential Equations
,”
J. Comput. Phys.
,
378
, pp.
686
707
.
14.
Dwivedi
,
V.
, and
Srinivasan
,
B.
,
2022
, “
A Normal Equation-Based Extreme Learning Machine for Solving Linear Partial Differential Equations
,”
ASME J. Comput. Inf. Sci. Eng.
,
22
(
1
), p.
014502
.
15.
Oommen
,
V.
, and
Srinivasan
,
B.
,
2022
, “
Solving Inverse Heat Transfer Problems Without Surrogate Models: A Fast, Data-Sparse, Physics Informed Neural Network Approach
,”
ASME J. Comput. Inf. Sci. Eng.
,
22
(
4
), p.
041012
.
16.
Liu
,
D.
, and
Wang
,
Y.
,
2019
, “
Multi-Fidelity Physics-Constrained Neural Network and Its Application in Materials Modeling
,”
ASME J. Mech. Des.
,
141
(
12
), p.
121403
.
17.
Dourado
,
A.
, and
Viana
,
F. A. C.
,
2020
, “
Physics-Informed Neural Networks for Missing Physics Estimation in Cumulative Damage Models: A Case Study in Corrosion Fatigue
,”
ASME J. Comput. Inf. Sci. Eng.
,
20
(
6
), p.
061007
.
18.
Liu
,
D.
, and
Wang
,
Y.
,
2021
, “
A Dual-Dimer Method for Training Physics-Constrained Neural Networks with Minimax Architecture
,”
Neural Netw.
,
136
, pp.
112
125
.
19.
Kennedy
,
M.
, and
O’Hagan
,
A.
,
2000
, “
Predicting the Output From a Complex Computer Code When Fast Approximations Are Available
,”
Biometrika
,
87
(
1
), pp.
1
13
.
20.
Fernández-Godino
,
M. G.
,
Park
,
C.
,
Kim
,
N.-H.
, and
Haftka
,
R. T.
,
2016
, “
Review of Multi-Fidelity Models
,” arXiv:1609.07196. https://arxiv.org/abs/1609.07196
21.
Peherstorfer
,
B.
,
Willcox
,
K.
, and
Gunzburger
,
M.
,
2018
, “
Survey of Multifidelity Methods in Uncertainty Propagation, Inference, and Optimization
,”
SIAM Rev.
,
60
(
3
), pp.
550
591
.
22.
Xiong
,
Y.
,
Chen
,
W.
, and
Tsui
,
K. L.
,
2008
, “
A New Variable-Fidelity Optimization Framework Based on Model Fusion and Objective-Oriented Sequential Sampling
,”
ASME J. Mech. Des.
,
130
(
11
), p.
111401
.
23.
Zhou
,
Q.
,
Wang
,
Y.
,
Choi
,
S. K.
,
Jiang
,
P.
,
Shao
,
X.
, and
Hu
,
J.
,
2017
, “
A Sequential Multi-Fidelity Metamodeling Approach for Data Regression
,”
Knowl. Based Syst.
,
134
, pp.
199
212
.
24.
Zhou
,
Q.
,
Wang
,
Y.
,
Choi
,
S. K.
,
Jiang
,
P.
,
Shao
,
X.
,
Hu
,
J.
, and
Shu
,
L.
,
2018
, “
A Robust Optimization Approach Based on Multi-Fidelity Metamodel
,”
Struct. Multidiscipl. Optim.
,
57
(
2
), pp.
775
797
.
25.
Shi
,
R.
,
Liu
,
L.
,
Long
,
T.
,
Wu
,
Y.
, and
Gary Wang
,
G.
,
2020
, “
Multi-Fidelity Modeling and Adaptive Co-Kriging-Based Optimization for All-Electric Geostationary Orbit Satellite Systems
,”
ASME J. Mech. Des.
,
142
(
2
), p.
021404
.
26.
Meng
,
X.
, and
Karniadakis
,
G. E.
,
2020
, “
A Composite Neural Network That Learns From Multi-Fidelity Data: Application to Function Approximation and Inverse PDE Problems
,”
J. Comput. Phys.
,
401
, p.
109020
.
27.
Paszke
,
A.
,
Gross
,
S.
,
Massa
,
F.
,
Lerer
,
A.
,
Bradbury
,
J.
,
Chanan
,
G.
,
Killeen
,
T.
, et al
,
2019
, “PyTorch: An Imperative Style, High-Performance Deep Learning Library,”
Advances in Neural Information Processing Systems
, pp.
8024
8035
.
28.
Wang
,
S.
,
Teng
,
Y.
, and
Perdikaris
,
P.
,
2021
, “
Understanding and Mitigating Gradient Pathologies in Physics-Informed Neural Networks
,”
SIAM J. Sci. Comput.
,
43
(
5
), pp.
A3055
A3081
.
29.
Wang
,
S.
,
Wang
,
H.
, and
Perdikaris
,
P.
,
2021
, “
On the Eigenvector Bias of Fourier Feature Networks: From Regression to Solving Multi-Scale PDEs with Physics-Informed Neural Networks
,”
Comput. Methods Appl. Mech. Eng.
,
384
, p.
113938
.
30.
Wang
,
Y.
,
2015
, “
Uncertainty in Materials Modeling, Simulation, and Development for ICME
,”
Proceedings of 2015 Materials Science & Technology
,
Columbus, OH
, pp.
1295
1305
.
31.
Wang
,
Y.
, and
McDowell
,
D. L.
,
2020
,
Uncertainty Quantification in Multiscale Materials Modeling
,
Woodhead Publishing
,
Cambridge, UK
, pp.
1
40
.
You do not currently have access to this content.