Abstract

An industrial design process is often highly iterative. With unclear relationships between the quantity of interest (QoI) trade-offs and the design solution, the definition of the cost function usually undergoes several modifications that mandate a continued interaction between the designer and the client to encode all design and mission requirements into an optimization-friendly mathematical formulation. Such an iterative process is time consuming and computationally expensive. An efficient way to accelerate this process is to derive data-driven mappings between the design/mission and QoI spaces to provide visual insights into the interactions among different QoIs as related to their corresponding simulation parameters. In this paper, we propose Shared-Gaussian process (GP), a generative model for the design process that is based on a Gaussian process latent variable model. Shared-GP learns correlations within and across multiple, but implicitly correlated, data spaces considered in the design process (i.e., the simulation parameter space, the design space, and the QoI spaces) to provide data-driven mappings across these data spaces via efficient inference. Shared-GP also provides a structured low-dimensional representation shared among data spaces (some of which are of very high dimension) that the designer can use to efficiently explore the design space without the need for costly simulations.

References

1.
Zitzler
,
E.
, and
Thiele
,
L.
,
1999
, “
Multiobjective Evolutionary Algorithms: A Comparative Case Study and the Strength Pareto Approach and Applications
,”
IEEE Trans. Evol. Comput.
,
3
(
4
), pp.
257
271
.
2.
Knowles
,
J.
, and
Corne
,
D.
,
1999
, “
The Pareto Archived Evolution Strategy: A New Baseline Algorithm for Pareto Multiobjective Optimisation
,”
Congress on Evolutionary Computation (CEC99)
,
Washington, DC
,
July 10–16
, Vol.
1
, pp.
98
105
.
3.
Chen
,
W.
,
Fuge
,
M.
, and
Chazan
,
J.
,
2017
, “
Design Manifolds Capture the Intrinsic Complexity and Dimension of Design Spaces
,”
ASME J. Mech. Des.
,
139
(
5
), p.
051102
. 10.1115/1.4036134
4.
Burnap
,
A.
,
Liu
,
Y.
,
Pan
,
Y.
,
Lee
,
H.
,
Gonzalez
,
R.
, and
Papalambros
,
P. Y.
,
2016
, “
Estimating and Exploring the Product Form Design Space Using Deep Generative Models
,”
ASME 2016 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
,
Charlotte, NC
,
Aug. 21–24
,
American Society of Mechanical Engineers
, p.
V02AT03A013
.
5.
Sedlmair
,
M.
,
Heinzl
,
C.
,
Bruckner
,
S.
,
Piringer
,
H.
, and
Möller
,
T.
,
2014
, “
Visual Parameter Space Analysis: A Conceptual Framework
,”
IEEE Trans. Visualization Comput. Graph.
,
20
(
12
), pp.
2161
2170
.
6.
Torsney-Weir
,
T.
,
Saad
,
A.
,
Moller
,
T.
,
Hege
,
H.-C.
,
Weber
,
B.
,
Verbavatz
,
J.-M.
, and
Bergner
,
S.
,
2011
, “
Tuner: Principled Parameter Finding for Image Segmentation Algorithms Using Visual Response Surface Exploration
,”
IEEE Trans. Visualization Comput. Graph.
,
17
(
12
), pp.
1892
1901
. 10.1109/TVCG.2011.248
7.
Averkiou
,
M.
,
Kim
,
V. G.
,
Zheng
,
Y.
, and
Mitra
,
N. J.
, “
Shapesynth: Parameterizing Model Collections for Coupled Shape Exploration and Synthesis
,”
Computer Graphics Forum
,
Seoul
,
Oct. 8–10
, pp.
125
134
.
8.
Xing
,
W.
,
Triantafyllidis
,
V.
,
Shah
,
A.
,
Nair
,
P.
, and
Zabaras
,
N.
,
2016
, “
Manifold Learning for the Emulation of Spatial Fields From Computational Models
,”
J. Comput. Phys.
,
326
, pp.
666
690
. 10.1016/j.jcp.2016.07.040
9.
Apley
,
D. W.
,
Liu
,
J.
, and
Chen
,
W.
,
2006
, “
Understanding the Effects of Model Uncertainty in Robust Design With Computer Experiments
,”
ASME J. Mech. Des.
,
128
(
4
), pp.
945
958
. 10.1115/1.2204974
10.
Wang
,
G. G.
, and
Shan
,
S.
,
2007
, “
Review of Metamodeling Techniques in Support of Engineering Design Optimization
,”
ASME J. Mech. Des.
,
129
(
4
), pp.
370
380
. 10.1115/1.2429697
11.
Jeong
,
S.
,
Murayama
,
M.
, and
Yamamoto
,
K.
,
2005
, “
Efficient Optimization Design Method Using Kriging Model
,”
J. Aircr.
,
42
(
2
), pp.
413
420
. 10.2514/1.6386
12.
Bessa
,
M.
,
Bostanabad
,
R.
,
Liu
,
Z.
,
Hu
,
A.
,
Apley
,
D. W.
,
Brinson
,
C.
,
Chen
,
W.
, and
Liu
,
W.
,
2017
, “
A Framework for Data-Driven Analysis of Materials Under Uncertainty: Countering the Curse of Dimensionality
,”
Comput. Methods Appl. Mech. Eng.
,
320
, pp.
633
667
. 10.1016/j.cma.2017.03.037
13.
Kennedy
,
M. C.
, and
O’Hagan
,
A.
,
2001
, “
Bayesian Calibration of Computer Models
,”
J. R. Stat. Soc.: Ser. B (Statistical Methodology)
,
63
(
3
), pp.
425
464
. 10.1111/1467-9868.00294
14.
D’Agostino
,
D.
,
Serani
,
A.
,
Campana
,
E. F.
, and
Diez
,
M.
,
2018
, “
Deep Autoencoder for Off-Line Design-Space Dimensionality Reduction in Shape Optimization
,”
2018 AIAA/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference
,
Florida
,
Jan. 8–12
, p.
1648
.
15.
Cang
,
R.
,
Yao
,
H.
, and
Ren
,
Y.
,
2019
, “
One-Shot Generation of Near-Optimal Topology Through Theory-Driven Machine Learning
,”
Comput. Aided Des.
,
109
, pp.
12
21
. 10.1016/j.cad.2018.12.008
16.
Sosnovik
,
I.
, and
Oseledets
,
I.
,
2019
, “
Neural Networks for Topology Optimization
,”
Russ. J. Numer. Anal. Math. Model.
,
34
(
4
), pp.
215
223
. 10.1515/rnam-2019-0018
17.
Gal
,
Y.
,
Chen
,
Y.
, and
Ghahramani
,
Z.
,
2015
, “
Latent Gaussian Processes for Distribution Estimation of Multivariate Categorical Data
,”
International Conference on Machine Learning
,
Lille, France
.
18.
Oakley
,
J.
, and
O’Hagan
,
A.
,
2002
, “
Bayesian Inference for the Uncertainty Distribution of Computer Model Outputs
,”
Biometrika
,
89
(
4
), pp.
769
784
. 10.1093/biomet/89.4.769
19.
Girard
,
A.
,
Rasmussen
,
C. E.
,
Candela
,
J. Q.
, and
Murray-Smith
,
R.
,
2003
, “
Gaussian Process Priors With Uncertain Inputs Application to Multiple-Step Ahead Time Series Forecasting
,”
Advances in Neural Information Processing Systems
,
Vancouver, British Columbia, Canada
,
Dec. 8–13
, pp.
545
552
.
20.
Shon
,
A.
,
Grochow
,
K.
,
Hertzmann
,
A.
, and
Rao
,
R. P.
,
2006
, “
Learning Shared Latent Structure for Image Synthesis and Robotic Imitation
,”
Advances in Neural Information Processing Systems
,
Vancouver, British Columbia, Canada
,
Dec. 4–7
, pp.
1233
1240
.
21.
Yew
,
O.
,
Nair
,
P. B.
, and
Keane
,
A. J.
,
2001
, “
Evolutionary optimization of computationally expensive problems via surrogate modeling
,”
AIAA J.
,
41
(
4
), pp.
687
696
.
22.
Jiang
,
Z.
,
Chen
,
W.
,
Fu
,
Y.
, and
Yang
,
R.-J.
,
2013
, “
Reliability-Based Design Optimization With Model Bias and Data Uncertainty
,”
SAE Int. J. Mater. Manuf.
,
6
(
3
), pp.
502
516
. 10.4271/2013-01-1384
23.
Eleftheriadis
,
S.
,
Rudovic
,
O.
, and
Pantic
,
M.
,
2013
, “
Shared Gaussian Process Latent Variable Model for Multi-View Facial Expression Recognition
,”
International Symposium on Visual Computing
,
Rethymnon, Crete, Greece
,
July 29–31
,
Springer
,
New York
, pp.
527
538
.
24.
Su
,
H.
,
Maji
,
S.
,
Kalogerakis
,
E.
, and
Learned-Miller
,
E.
,
2015
, “
Multi-View Convolutional Neural Networks for 3d Shape Recognition
,”
Proceedings of the IEEE International Conference on Computer Vision
,
Santiago, Chile
,
Dec. 7–13
, pp.
945
953
.
25.
Ge
,
L.
,
Liang
,
H.
,
Yuan
,
J.
, and
Thalmann
,
D.
,
2016
, “
Robust 3d Hand Pose Estimation in Single Depth Images: From Single-View CNN to Multi-View CNNS
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
,
Las Vegas, NV
,
June 26–July 1
, pp.
3593
3601
.
26.
Ek
,
C. H.
,
Jaeckel
,
P.
,
Campbell
,
N.
,
Lawrence
,
N. D.
, and
Melhuish
,
C.
,
2009
, “
Shared Gaussian Process Latent Variable Models for Handling Ambiguous Facial Expressions
,”
AIP Conference Proceedings
,
Ann Arbor, MI
,
July 29–30
, pp.
147
153
.
27.
Chen
,
W.
, and
Fuge
,
M.
,
2019
, “
Synthesizing Designs With Interpart Dependencies Using Hierarchical Generative Adversarial Networks
,”
ASME J. Mech. Des.
,
141
(
11
), pp.
11
24
.
28.
Conti
,
S.
, and
OHagan
,
A.
,
2010
, “
Bayesian Emulation of Complex Multi-Output and Dynamic Computer Models
,”
J. Stat. Plann. Inference
,
140
(
3
), pp.
640
651
. 10.1016/j.jspi.2009.08.006
29.
Rasmussen
,
C. E.
, and
Williams
,
C. K. I.
,
2006
,
Gaussian Processes for Machine Learning
(Adaptive Computation and Machine Learning)
,
MIT Press
,
Cambridge, MA
,
OCLC: ocm61285753
.
30.
Wilson
,
A. G.
,
Hu
,
Z.
,
Salakhutdinov
,
R.
, and
Xing
,
E. P.
,
2016
, “
Deep Kernel Learning
,”
International Conference on Artificial Intelligence and Statistics
,
Cadiz, Spain
.
31.
Wilson
,
A. G.
,
2014
, “
Covariance Kernels for Fast Automatic Pattern Discovery and Extrapolation With Gaussian Processes
,”
University of Cambridge
,
Cambridge, UK
.
32.
Hinton
,
G. E.
, and
Salakhutdinov
,
R. R.
,
2007
, “
Using Deep Belief Nets to Learn Covariance Kernels for Gaussian Processes
,”
Advances in Neural Information Processing Systems
,
Vancouver, British Columbia, Canada
,
Dec. 8–11
, p.
8
.
33.
Bishop
,
M.
, and
Whitlock
,
P. A.
,
2007
, “
Monte Carlo Simulation of Hard Hyperspheres in Six, Seven and Eight Dimensions for Low to Moderate Densities
,”
J. Stat. Phys.
,
126
(
2
), pp.
299
314
. 10.1007/s10955-006-9266-9
34.
Higdon
,
D.
,
Gattiker
,
J.
,
Williams
,
B.
, and
Rightley
,
M.
,
2008
, “
Computer Model Calibration Using High-Dimensional Output
,”
J. Am. Stat. Assoc.
,
103
(
482
), pp.
570
583
. 10.1198/016214507000000888
35.
Bayarri
,
M. J.
,
Berger
,
J. O.
,
Cafeo
,
J.
,
Garcia-Donato
,
G.
,
Liu
,
F.
,
Palomo
,
J.
,
Parthasarathy
,
R. J.
,
Paulo
,
R.
,
Sacks
,
J.
, and
Walsh
,
D.
,
2007
, “
Computer Model Validation With Functional Output
,”
Ann. Stat.
,
35
(
5
), pp.
1874
1906
, . 10.1214/009053607000000163
36.
Ma
,
X.
, and
Zabaras
,
N.
,
2011
, “
Kernel Principal Component Analysis for Stochastic Input Model Generation
,”
J. Comput. Phys.
,
230
(
19
), pp.
7311
7331
. 10.1016/j.jcp.2011.05.037
37.
Xing
,
W.
,
Shah
,
A. A.
, and
Nair
,
P. B.
,
2015
, “
Reduced Dimensional Gaussian Process Emulators of Parametrized Partial Differential Equations Based on Isomap
,”
Proceedings of the Royal Society of London A: Mathematical, Physical and Engineering Sciences
,
471
. 10.1098/rspa.2014.0697
38.
Gadd
,
C.
,
Xing
,
W.
,
Nezhad
,
M. M.
, and
Shah
,
A.
,
2018
, “
A Surrogate Modelling Approach Based on Nonlinear Dimension Reduction for Uncertainty Quantification in Groundwater Flow Models
,”
Transp. Porous Media
,
126
(
1
), pp.
39
77
. 10.1007/s11242-018-1065-7
39.
Wackernagel
,
H.
,
1995
,
Multivariate Geostatistics
,
Springer
,
Berlin
.
40.
Zhang
,
H.
,
2007
, “
Maximum-Likelihood Estimation for Multivariate Spatial Linear Coregionalization Models
,”
Environmetrics
,
18
(
2
), pp.
125
139
. 10.1002/env.807
41.
Fricker
,
T.
,
Oakley
,
J.
, and
Urban
,
N.
,
2013
, “
Multivariate Gaussian Process Emulators With Nonseparable Covariance Structures
,”
Technometrics
,
55
(
1
), pp.
47
56
. 10.1080/00401706.2012.715835
42.
Konomi
,
B.
,
Karagiannis
,
G.
,
Sarkar
,
A.
,
Sun
,
X.
, and
Lin
,
G.
,
2014
, “
Bayesian Treed Multivariate Gaussian Process With Adaptive Design: Application to a Carbon Capture Unit
,”
Technometrics
,
56
(
2
), pp.
145
158
. 10.1080/00401706.2013.879078
43.
Wilson
,
A. G.
,
Gilboa
,
E.
,
Nehorai
,
A.
, and
Cunningham
,
J. P.
,
2014
, “
Fast Kernel Learning for Multidimensional Pattern Extrapolation
,”
Advances in Neural Information Processing Systems
,
Montreal, Quebec, Canada
,
Dec. 8–13
, pp.
3626
3634
.
44.
Zhe
,
S.
,
Xing
,
W.
, and
Kirby
,
M.
,
2019
, “
Scalable High-Order Gaussian Process Regression
,”
International Conference on Artificial Intelligence and Statistics
,
Naha, Okinawa, Japan
,
Apr. 16–18
.
45.
Wilson
,
A. G.
, and
Nickisch
,
H.
,
2015
, “
Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)
,”
international conference on machine learning
,
Lille, France
,
July 6-11
.
46.
Alvarez
,
M. A.
,
Rosasco
,
L.
, and
Lawrence
,
N. D.
,
2012
, “
Kernels for Vector-Valued Functions: A Review
,”
Found. Trends Mach. Learn.
,
4
(
3
), pp.
195
266
. 10.1561/2200000036
47.
Lawrence
,
N.
,
2005
, “
Probabilistic Non-Linear Principal Component Analysis With Gaussian Process Latent Variable Models
,”
J. Mach. Learn. Res.
,
6
, pp.
1783
1816
.
48.
Zhe
,
S.
,
Xu
,
Z.
,
Chu
,
X.
,
Qi
,
Y.
, and
Park
,
Y.
,
2015
, “
Scalable Nonparametric Multiway Data Analysis
,”
International Conference on Artificial Intelligence and Statistics
,
San Diego, CA
,
May 9–12
, pp.
1125
1134
.
49.
Blei
,
D. M.
,
Jordan
,
M. I.
, et al.,
2006
, “
Variational Inference for Dirichlet Process Mixtures
,”
Bayesian Anal.
,
1
(
1
), pp.
121
143
. 10.1214/06-BA104
50.
Sobol
,
I.
,
1976
, “
Uniformly Distributed Sequences With an Addition Uniform Property
,”
USSR Comput. Maths. Math. Phys.
,
16
(
5
), pp.
236
242
. 10.1016/0041-5553(76)90154-3
51.
Jin
,
R.
,
Chen
,
W.
, and
Sudjianto
,
A.
,
2003
, “
An Efficient Algorithm for Constructing Optimal Design of Computer Experiments
,”
ASME 2003 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
,
Chicago, IL
,
Sept. 2–6
, pp.
545
554
.
52.
Sobol’
,
I. M.
,
1967
, “
On the Distribution of Points in a Cube and the Approximate Evaluation of Integrals
,”
Zhurnal Vychislitel’noi Matematiki i Matematicheskoi Fiziki
,
7
(
4
), pp.
784
802
.
53.
Bates
,
R.
,
Buck
,
R.
,
Riccomagno
,
E.
, and
Wynn
,
H.
,
1996
, “
Experimental Design and Observation for Large Systems
,”
J. R. Stat. Soc., Ser. B (Methodological)
,
58
(
1
), pp.
77
94
.
54.
Andreassen
,
E.
,
Clausen
,
A.
,
Schevenels
,
M.
,
Lazarov
,
B. S.
, and
Sigmund
,
O.
,
2011
, “
Efficient Topology Optimization in Matlab Using 88 Lines of Code
,”
Struct. Multidiscip. Optim.
,
43
(
1
), pp.
1
16
. 10.1007/s00158-010-0594-7
55.
Bendsoe
,
M. P.
, and
Sigmund
,
O.
,
2004
,
Topology Optimization: Theory, Methods and Applications
,
Springer
,
New York
.
56.
Bruns
,
T. E.
, and
Tortorelli
,
D. A.
,
2001
, “
Topology Optimization of Non-Linear Elastic Structures and Compliant Mechanisms
,”
Comput. Methods Appl. Mech. Eng.
,
190
(
26
), pp.
3443
3459
. 10.1016/S0045-7825(00)00278-4
57.
O’Hagan
,
A.
, and
Kingman
,
J. F. C.
,
1978
, “
Curve Fitting and Optimal Design for Prediction
,”
J. R. Stat. Soc., Ser. B (Methodological)
,
40
(
1
), pp.
1
42
.
58.
Wilson
,
A.
, and
Adams
,
R.
,
2013
, “
Gaussian Process Kernels for Pattern Discovery and Extrapolation
,”
International Conference on Machine Learning
,
Miami, FL
,
Dec. 4–7
, pp.
1067
1075
.
59.
Teckentrup
,
A. L.
,
Jantsch
,
P.
,
Webster
,
C. G.
, and
Gunzburger
,
M.
,
2015
, “
A Multilevel Stochastic Collocation Method for Partial Differential Equations With Random Input Data
,”
SIAM/ASA J. Uncertainty Quantification
,
3
(
1
), pp.
1046
1074
. 10.1137/140969002
60.
Damianou
,
A.
, and
Lawrence
,
N.
,
2013
, “
Deep Gaussian Processes
,”
International Conference on Artificial Intelligence and Statistics
,
Scottsdale, AZ
,
Apr. 29–May 1
, pp.
207
215
.
61.
Friedman
,
N.
, and
Nachman
,
I.
,
2000
, “
Gaussian Process Networks
,”
Proceedings of the Sixteenth Conference on Uncertainty in Artificial Intelligence
,
Stanford, CA
,
June 30–July 3
, pp.
211
219
.
62.
Wilson
,
A. G.
,
Hu
,
Z.
,
Salakhutdinov
,
R.
, and
Xing
,
E. P.
,
2016
, “
Deep Kernel Learning
,”
International Conference on Artificial Intelligence and Statistics
,
Cadiz, Spain
,
May 9–11
, pp.
370
378
.
You do not currently have access to this content.