Abstract

Bayesian optimization (BO) is an efiective surrogate-based method that has been widely used to optimize simulation-based applications. While the traditional Bayesian optimization approach only applies to single-fidelity models, many realistic applications provide multiple levels of fidelity with various computational complexity and predictive capability. In this work, we propose a multi-fidelity Bayesian optimization method for design applications with both known and unknown constraints. The proposed framework, called sMF-BO-2CoGP, is built on a multi-level CoKriging method to predict the objective function. An external binary classifier, which we approximate using a separate CoKriging model, is used to distinguish between feasible and infeasible regions. The sMF-BO-2CoGP method is demonstrated using a series of analytical examples, and a fiip-chip application for design optimization to minimize the deformation due to warping under thermal loading conditions.

References

References
1.
Digabel
,
S. L.
, and
Wild
,
S. M.
,
2015
, “
A Taxonomy of Constraints in Simulation-Based Optimization
,” .
2.
Kennedy
,
M. C.
, and
O'Hagan
,
A.
,
2000
,
Predicting the Output From a Complex Computer Code When Fast Approximations Are Available
,”
Biometrika
,
87
(
1
), pp.
1
13
. 10.1093/biomet/87.1.1
3.
Tran
,
A.
,
Wildey
,
T.
, and
McCann
,
S.
,
2019
, “
sBF-BO-2CoGP: A Sequential Bi-Fidelity Constrained Bayesian Optimization for Design Applications
,”
Proceedings of the ASME 2019 IDETC/CIE, Volume 1: 39th Computers and Information in Engineering Conference of International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
,
Anaheim, CA
,
Aug. 18–21
,
American Society of Mechanical Engineers
, p.
V001T02A073
.
4.
Brochu
,
E.
,
Cora
,
V. M.
, and
De Freitas
,
N.
,
2010
, “
A Tutorial on Bayesian Optimization of Expensive Cost Functions, With Application to Active User Modeling and Hierarchical Reinforcement Learning
”, .
5.
Shahriari
,
B.
,
Swersky
,
K.
,
Wang
,
Z.
,
Adams
,
R. P.
, and
de Freitas
,
N.
,
2016
, “
Taking the Human out of the Loop: A Review of Bayesian Optimization
,”
Proc. IEEE
,
104
(
1
), pp.
148
175
. 10.1109/JPROC.2015.2494218
6.
Frazier
,
P. I.
,
2018
, “
A Tutorial on Bayesian Optimization
,” .
7.
Jones
,
D. R.
,
Schonlau
,
M.
, and
Welch
,
W. J.
,
1998
, “
Eficient Global Optimization of Expensive Black-Box Functions
,”
J. Global Optim.
,
13
(
4
), pp.
455
492
. 10.1023/A:1008306431147
8.
Le Gratiet
,
L.
, and
Garnier
,
J.
,
2014
, “
Recursive Co-Kriging Model for Design of Computer Experiments With Multiple Levels of Fidelity
,”
Int. J. Uncertainty Quantif.
,
4
(
5
), pp.
365
386
. 10.1615/Int.J.UncertaintyQuantification.2014006914
9.
Raissi
,
M.
, and
Karniadakis
,
G.
,
2016
, “
Deep Multi-Fidelity Gaussian Processes
,” .
10.
Raissi
,
M.
,
Perdikaris
,
P.
, and
Karniadakis
,
G. E.
,
2017
, “
Machine Learning of Linear Difierential Equations Using Gaussian Processes
,”
J. Comput. Phys.
,
348
, pp.
683
693
. 10.1016/j.jcp.2017.07.050
11.
Perdikaris
,
P.
,
Venturi
,
D.
,
Royset
,
J.
, and
Karniadakis
,
G.
,
2015
, “
Multi-Fidelity Modelling Via Recursive Co-Kriging and Gaussian{Markov Random Fields
,”
Proc. R. Soc. A
, Vol.
471
,
The Royal Society
, p.
20150018
.
12.
Perdikaris
,
P.
,
Raissi
,
M.
,
Damianou
,
A.
,
Lawrence
,
N.
, and
Karniadakis
,
G. E.
,
2017
, “
Nonlinear Information Fusion Algorithms for Data-Eficient Multi-Fidelity Modelling
,”
Proc. R. Soc. A: Math., Phys. Eng. Sci.
,
473
(
2198
), p.
20160751
. 10.1098/rspa.2016.0751
13.
Poloczek
,
M.
,
Wang
,
J.
, and
Frazier
,
P.
,
2017
, “
Multi-Information Source Optimization
,”
Advances in Neural Information Processing Systems
, pp.
4288
4298
.
14.
Swersky
,
K.
,
Snoek
,
J.
, and
Adams
,
R. P.
,
2013
, “
Multi-Task Bayesian Optimization
,”
Advances in Neural Information Processing Systems
, pp.
2004
2012
.
15.
Ghoreishi
,
S. F.
, and
Allaire
,
D.
,
2019
, “
Multi-Information Source Constrained Bayesian Optimization
,”
Struct. Multidiscip. Optim.
,
59
(
3
), pp.
977
991
. 10.1007/s00158-018-2115-z
16.
Ghoreishi
,
S. F.
, and
Allaire
,
D. L.
,
2018
, “
A Fusion-Based Multi-Information Source Optimization Approach Using Knowledge Gradient Policies
,”
2018 AIAA/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference
, p.
1159
.
17.
Ghoreishi
,
S. F.
, and
Allaire
,
D. L.
,
2018
, “
Gaussian Process Regression for Bayesian Fusion of Multi-Fidelity Information Sources
,”
2018 Multidisciplinary Analysis and Optimization Conference
, p.
4176
.
18.
Ghoreishi
,
S. F.
,
Molkeri
,
A.
,
Srivastava
,
A.
,
Arroyave
,
R.
, and
Allaire
,
D.
,
2018
, “
Multi-Information Source Fusion and Optimization to Realize ICME: Application to Dual-Phase Materials
,”
ASME J. Mech. Des.
,
140
(
11
), p.
111409
. 10.1115/1.4041034
19.
Xiao
,
M.
,
Zhang
,
G.
,
Breitkopf
,
P.
,
Villon
,
P.
, and
Zhang
,
W.
,
2018
, “
Extended Co-Kriging Interpolation Method Based on Multi-Fidelity Data
,”
Appl. Math. Comput.
,
323
, pp.
120
131
.
20.
Couckuyt
,
I.
,
Forrester
,
A.
,
Gorissen
,
D.
,
De Turck
,
F.
, and
Dhaene
,
T.
,
2012
, “
Blind Kriging: Implementation and Performance Analysis
,”
Adv. Eng. Software
,
49
(
1
), pp.
1
13
. 10.1016/j.advengsoft.2012.03.002
21.
Couckuyt
,
I.
,
Dhaene
,
T.
, and
Demeester
,
P.
,
2013
,
ooDACE Toolbox, A Matlab Kriging Toolbox: Getting Started
,
Universiteit Gent
,
Ghent, Belgium
, pp.
3
15
.
22.
Couckuyt
,
I.
,
Dhaene
,
T.
, and
Demeester
,
P.
,
2014
, “
ooDACE Toolbox: a Flexible Object-Oriented Kriging Implementation
,”
J. Mach. Learn. Res.
,
15
(
1
), pp.
3183
3186
.
23.
Forrester
,
A. I.
,
Sóbester
,
A.
, and
Keane
,
A. J.
,
2007
, “
Multi-Fidelity Optimization Via Surrogate Modelling
,”
Proc. R. Soc. London A: Math., Phys. Eng. Sci.
,
463
(
2088
), pp.
3251
3269
. 10.1098/rspa.2007.1900
24.
Hernández-Lobato
,
J. M.
,
Hofiman
,
M. W.
, and
Ghahramani
,
Z.
,
2014
, “
Predictive Entropy Search for Eficient Global Optimization of Black-Box Functions
,”
Advances in Neural Information Processing Systems
, pp.
918
926
.
25.
Hernández-Lobato
,
J. M.
,
Gelbart
,
M.
,
Hofiman
,
M.
,
Adams
,
R.
, and
Ghahramani
,
Z.
,
2015
, “
Predictive Entropy Search for Bayesian Optimization With Unknown Constraints
,”
International Conference on Machine Learning
, pp.
1699
1707
.
26.
Hernández-Lobato
,
D.
,
Hernández-Lobato
,
J.
,
Shah
,
A.
, and
Adams
,
R.
,
2016
, “
Predictive Entropy Search for Multi-Objective Bayesian Optimization
,”
International Conference on Machine Learning
, pp.
1492
1501
.
27.
Hennig
,
P.
, and
Schuler
,
C. J.
,
2012
, “
Entropy Search for Information-Eficient Global Optimization
,”
J. Mach. Learn. Res.
,
13
(
Jun
), pp.
1809
1837
.
28.
Wang
,
Z.
,
Zhou
,
B.
, and
Jegelka
,
S.
,
2016
, “
Optimization As Estimation With Gaussian Processes in Bandit Settings
,”
Artificial Intelligence and Statistics
, pp.
1022
1031
.
29.
Shahriari
,
B.
,
Wang
,
Z.
,
Hofiman
,
M. W.
,
Bouchard-Côté
,
A.
, and
de Freitas
,
N.
,
2014
, “
An Entropy Search Portfolio for Bayesian Optimization
,” .
30.
Kushner
,
H. J.
,
1964
, “
A New Method of Locating the Maximum Point of An Arbitrary Multipeak Curve in the Presence of Noise
,”
ASME J. Basic Eng.
,
86
(
1
), pp.
97
106
. 10.1115/1.3653121
31.
Mockus
,
J.
,
1975
, “
On Bayesian Methods for Seeking the Extremum
,”
Optimization Techniques IFIP Technical Conference
,
Springer
, pp.
400
404
.
32.
Mockus
,
J.
,
1982
, “The Bayesian Approach to Global Optimization,”
System Modeling and Optimization
,
M.
Hazewinkel
, ed.,
Kluwer Academic
,
Dordrecht
, pp.
473
481
.
33.
Bull
,
A. D.
,
2011
, “
Convergence Rates of Eficient Global Optimization Algorithms
,”
J. Mach. Learn. Res.
,
12
(
Oct
), pp.
2879
2904
.
34.
Snoek
,
J.
,
Larochelle
,
H.
, and
Adams
,
R. P.
,
2012
, “
Practical Bayesian Optimization of Machine Learning Algorithms
,”
Advances in Neural Information Processing Systems
, pp.
2951
2959
.
35.
Srinivas
,
N.
,
Krause
,
A.
,
Kakade
,
S. M.
, and
Seeger
,
M.
,
2009
, “
Gaussian Process Optimization in the Bandit Setting: No Regret and Experimental Design
,” .
36.
Srinivas
,
N.
,
Krause
,
A.
,
Kakade
,
S. M.
, and
Seeger
,
M. W.
,
2012
, “
Information-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting
,”
IEEE Trans. Inf. Theory
,
58
(
5
), pp.
3250
3265
. 10.1109/TIT.2011.2182033
37.
Daniel
,
C.
,
Viering
,
M.
,
Metz
,
J.
,
Kroemer
,
O.
, and
Peters
,
J.
,
2014
, “
Active Reward Learning
,”
Robotics: Science and Systems
,
Berkeley, CA
,
July 12–16
.
38.
Tran
,
A.
,
Sun
,
J.
,
Furlan
,
J. M.
,
Pagalthivarthi
,
K. V.
,
Visintainer
,
R. J.
, and
Wang
,
Y.
,
2019
, “
pBO-2GP-3B: A Batch Parallel Known/unknown Constrained Bayesian Optimization With Feasibility Classification and Its Applications in Computational Fluid Dynamics
,”
Comput. Methods Appl. Mech. Eng.
,
347
, pp.
827
852
. 10.1016/j.cma.2018.12.033
39.
Tran
,
A.
,
Tran
,
M.
, and
Wang
,
Y.
,
2019
, “
Constrained Mixed-Integer Gaussian Mixture Bayesian Optimization and Its Applications in Designing Fractal and Auxetic Metamaterials
,”
Struct. Multidiscip. Optim.
,
59
(
6
), pp.
2131
2154
.
40.
Tran
,
A.
,
Scott
,
M.
,
Furlan
,
J. M.
,
Pagalthivarthi
,
K. V.
,
Visintainer
,
R. J.
, and
Wildey
,
T.
,
2019
, “
aphBO-2GP-3B: A Budgeted Asynchronously-Parallel Multi-Acquisition for Known/Unknown Constrained Bayesian Optimization on High-Performing Computing Architecture
,”
Reliab. Eng. Syst. Saf.
, https://arxiv.org/abs/2003.09436
41.
Bentley
,
J. L.
,
1975
, “
Multidimensional Binary Search Trees Used for Associative Searching
,”
Commun. ACM
,
18
(
9
), pp.
509
517
. 10.1145/361002.361007
42.
Hastie
,
T.
,
Rosset
,
S.
,
Zhu
,
J.
, and
Zou
,
H.
,
2009
, “
Multi-Class AdaBoost
,”
Stat. Interface
,
2
(
3
), pp.
349
360
. 10.4310/SII.2009.v2.n3.a8
43.
Breiman
,
L.
,
2001
, “
Random Forests
,”
Mach. Learn.
,
45
(
1
), pp.
5
32
. 10.1023/A:1010933404324
44.
Hearst
,
M. A.
,
Dumais
,
S. T.
,
Osuna
,
E.
,
Platt
,
J.
, and
Scholkopf
,
B.
,
1998
, “
Support Vector Machines
,”
IEEE Intell. Syst. Appl.
,
13
(
4
), pp.
18
28
. 10.1109/5254.708428
45.
Suykens
,
J. A.
, and
Vandewalle
,
J.
,
1999
, “
Least Squares Support Vector Machine Classifiers
,”
Neural Process. Lett.
,
9
(
3
), pp.
293
300
. 10.1023/A:1018628609742
46.
Rasmussen
,
C. E.
,
2004
, “Gaussian Processes in Machine Learning,”
Advanced Lectures on Machine Learning
,
Springer
, pp.
63
71
.
47.
LeCun
,
Y.
,
Bengio
,
Y.
, and
Hinton
,
G.
,
2015
, “
Deep Learning
,”
Nature
,
521
(
7553
), p.
436
. 10.1038/nature14539
48.
Hansen
,
N.
, and
Ostermeier
,
A.
,
2001
, “
Completely Derandomized Self-Adaptation in Evolution Strategies
,”
Evol. Comput.
,
9
(
2
), pp.
159
195
. 10.1162/106365601750190398
49.
Hansen
,
N.
,
Müller
,
S. D.
, and
Koumoutsakos
,
P.
,
2003
, “
Reducing the Time Complexity of the Derandomized Evolution Strategy With Covariance Matrix Adaptation (CMA-ES)
,”
Evol. Comput.
,
11
(
1
), pp.
1
18
. 10.1162/106365603321828970
50.
Kandasamy
,
K.
,
Dasarathy
,
G.
,
Schneider
,
J.
, and
Poczos
,
B.
,
2017
, “
Multi-Fidelity Bayesian Optimisation With Continuous Approximations
,”
Proceedings of the 34th International Conference on Machine Learning-Volume 70
, pp.
1799
1808
.
51.
Xiong
,
S.
,
Qian
,
P. Z.
, and
Wu
,
C. J.
,
2013
, “
Sequential Design and Analysis of High-Accuracy and Low-Accuracy Computer Codes
,”
Technometrics
,
55
(
1
), pp.
37
46
. 10.1080/00401706.2012.723572
52.
Surjanovic
,
S.
, and
Bingham
,
D.
,
Virtual Library of Simulation Experiments: Test Functions and Datasets
, https://www.sfu.ca/~ssurjano/optimization.html
53.
Datta
,
D.
, and
Figueira
,
J. R.
,
2011
, “
A Real-Integer-Discrete-Coded Particle Swarm Optimization for Design Problems
,”
Appl. Soft Comput.
,
11
(
4
), pp.
3625
3633
. 10.1016/j.asoc.2011.01.034
54.
McCann
,
S.
,
Ostrowicki
,
G. T.
,
Tran
,
A.
,
Huang
,
T.
,
Bernhard
,
T.
,
Tummala
,
R. R.
, and
Sitaraman
,
S. K.
,
2017
, “
Determination of Energy Release Rate Through Sequential Crack Extension
,”
ASME J. Electron. Packag.
,
139
(
4
), p.
041003
. 10.1115/1.4037334
55.
Darveaux
,
R.
,
2000
, “
Efiect of Simulation Methodology on Solder Joint Crack Growth Correlation
,”
Proceedings of the 50th Electronic Components & Technology Conference, IEEE
, pp.
1048
1058
.
56.
Huang
,
D.
,
Allen
,
T. T.
,
Notz
,
W. I.
, and
Miller
,
R. A.
,
2006
, “
Sequential Kriging Optimization Using Multiple-Fidelity Evaluations
,”
Struct. Multidiscip. Optim.
,
32
(
5
), pp.
369
382
. 10.1007/s00158-005-0587-0
57.
Gauthier
,
B.
, and
Pronzato
,
L.
,
2014
, “
Spectral Approximation of the IMSE Criterion for Optimal Designs in Kernel-Based Interpolation Models
,”
SIAM/ASA J. Uncertainty Quantif.
,
2
(
1
), pp.
805
825
. 10.1137/130928534
58.
Gauthier
,
B.
, and
Pronzato
,
L.
,
2017
, “
Convex Relaxation for IMSE Optimal Design in Random-Field Models
,”
Comput. Stat. Data Anal.
,
113
, pp.
375
394
. 10.1016/j.csda.2016.10.018
59.
Silvestrini
,
R. T.
,
Montgomery
,
D. C.
, and
Jones
,
B.
,
2013
, “
Comparing Computer Experiments for the Gaussian Process Model Using Integrated Prediction Variance
,”
Qual. Eng.
,
25
(
2
), pp.
164
174
. 10.1080/08982112.2012.758284
60.
Hofiman
,
M. D.
,
Brochu
,
E.
, and
de Freitas
,
N.
,
2011
, “
Portfolio Allocation for Bayesian Optimization
,”
UAI, Citeseer
, pp.
327
336
.
61.
Travaglino
,
S.
,
Murdock
,
K.
,
Tran
,
A.
,
Martin
,
C.
,
Liang
,
L.
,
Wang
,
Y.
, and
Sun
,
W.
,
2020
, “
Computational Optimization Study of Transcatheter Aortic Valve Leafiet Design Using Porcine and Bovine Leafiets
,”
ASME J. Biomech. Eng.
,
142
(
1
), p.
011007
. 10.1115/1.4044244
62.
Lawrence
,
N. D.
,
2004
, “
Gaussian Process Latent Variable Models for Visualisation of High Dimensional Data
,”
Advances in Neural Information Processing Systems
, pp.
329
336
.
63.
Tran
,
A.
,
He
,
L.
, and
Wang
,
Y.
,
2018
, “
An Eficient First-Principles Saddle Point Searching Method Based on Distributed Kriging Metamodels
,”
ASCE-ASME J. Risk Uncertainty Eng. Syst., Part B: Mech. Eng.
,
4
(
1
), p.
011006
. 10.1115/1.4037459
64.
Tran
,
A.
,
Furlan
,
J. M.
,
Pagalthivarthi
,
K. V.
,
Visintainer
,
R. J.
,
Wildey
,
T.
, and
Wang
,
Y.
,
2019
, “
WearGP: A Computationally Eficient Machine Learning Framework for Local Erosive Wear Predictions Via Nodal Gaussian Processes
,”
Wear
,
422
, pp.
9
26
. 10.1016/j.wear.2018.12.081
65.
Snelson
,
E.
, and
Ghahramani
,
Z.
,
2006
, “
Sparse Gaussian Processes Using Pseudo-Inputs
,”
Advances in Neural Information Processing Systems
, pp.
1257
1264
.
66.
Kersting
,
K.
,
Plagemann
,
C.
,
Pfafi
,
P.
, and
Burgard
,
W.
,
2007
, “
Most Likely Heteroscedastic Gaussian Process Regression
,”
Proceedings of the 24th International Conference on Machine Learning, ACM
, pp.
393
400
.
67.
Snoek
,
J.
,
Rippel
,
O.
,
Swersky
,
K.
,
Kiros
,
R.
,
Satish
,
N.
,
Sundaram
,
N.
,
Patwary
,
M. M. A.
,
Prabhat
,
M.
, and
Adams
,
R. P.
,
2015
, “
Scalable Bayesian Optimization Using Deep Neural Networks
,”
ICML
, pp.
2171
2180
.
You do not currently have access to this content.