Abstract

This research proposes an iterative dynamic programing (IDP) algorithm that generates an optimal supervisory control policy for hybrid electric vehicles (HEVs) considering transient powertrain dynamics. The proposed algorithm tries to solve the “curse of dimensionality” and the “curse of modeling” of conventional dynamic programing (DP). The proposed IDP algorithm iteratively updates the DP formulation using a machine learning-based powertrain model. The machine learning model is recursively trained using the outputs from the driving cycle simulation with a high-fidelity model. Once the reduced model converges to the high-fidelity model accuracy, the resulting control policy yields a 9.1% fuel economy (FE) improvement compared to the baseline nonpredictive rule-based control for the urban dynamometer driving schedule (UDDS) driving cycle. A conventional DP control strategy based on a quasi-static powertrain model and a perfect preview of future power demand yields 14.2% FE improvement. However, the FE improvement reduces to 5.7% when the policy is validated with the high-fidelity model. It is concluded that capturing the transient powertrain dynamics is critical to generating a realistic fuel economy prediction and relevant powertrain control policy. The proposed IDP strategy employs targeted state-space exploration to leverage the improving state trajectory from previous iterations. Compared to conventional fixed state-space sampling methods, this method improves the accuracy of the DP policy against discretization error. It also significantly reduces the computational load of the relatively high number of states of the transient powertrain model.

References

1.
Sciarretta
,
A.
, and
Guzzella
,
L.
,
2007
, “
Control of Hybrid Electric Vehicles
,”
IEEE Control Syst. Mag.
,
27
(
2
), pp.
60
70
.
2.
Serrao
,
L.
,
Onori
,
S.
, and
Rizzoni
,
G.
,
2011
, “
A Comparative Analysis of Energy Management Strategies for Hybrid Electric Vehicles
,”
ASME J. Dyn. Syst. Meas. Control
,
133
(
3
), p. 031012.10.1115/1.4003267
3.
Hui-Ce
,
Y.
,
Chong
,
Z.
,
Xun
,
G.
,
Yun-Feng
,
H.
, and
Hong
,
C.
,
2020
, “
Energy Management Strategy of Fuel Cell Hybrid Electric Vehicle Based on Dynamic Programming
,” Chinese Automation Congress (
CAC
),
IEEE
, Shanghai, China, Nov. 6–8, pp.
3134
3139
10.1109/CAC51589.2020.9327615.
4.
Hu
,
J.
,
Li
,
J.
,
Hu
,
Z.
,
Xu
,
L.
, and
Ouyang
,
M.
,
2021
, “
Power Distribution Strategy of a Dual-Engine System for Heavy-Duty Hybrid Electric Vehicles Using Dynamic Programming
,”
Energy
,
215
, p.
118851
.10.1016/j.energy.2020.118851
5.
Kim
,
N.
,
Cha
,
S.
, and
Peng
,
H.
,
2011
, “
Optimal Control of Hybrid Electric Vehicles Based on Pontryagin's Minimum Principle
,”
IEEE Trans. Control Syst. Technol.
,
19
(
5
), pp.
1279
1287
.
6.
Serrao
,
L.
,
Onori
,
S.
, and
Rizzoni
,
G.
,
2009
, “
ECMS as a Realization of Pontryagin's Minimum Principle for HEV Control
,”
Proceedings of the American Control Conference
, St. Louis, MO, June 10–12, pp.
3964
3969
.10.1109/ACC.2009.5160628
7.
Borhan
,
H.
,
Vahidi
,
A.
,
Phillips
,
A.
,
Kuang
,
M.
,
Kolmanovsky
,
I.
, and
Cairano
,
S.
,
2012
, “
MPC-Based Energy Management of a Power-Split Hybrid Electric Vehicle
,”
IEEE Trans. Control Syst. Technol.
,
20
(
3
), pp.
593
603
.10.1109/TCST.2011.2134852
8.
Zeng
,
X.
, and
Wang
,
J.
,
2015
, “
A Parallel Hybrid Electric Vehicle Energy Management Strategy Using Stochastic Model Predictive Control With Road Grid Preview
,”
IEEE Trans. Control Syst. Technol.
,
23
(
6
), pp.
2416
2423
.10.1109/TCST.2015.2409235
9.
Musardo
,
C.
,
Rizzoni
,
G.
, and
Staccia
,
B.
,
2005
, “
A-ECMS: An Adaptive Algorithm for Hybrid Electric Vehicle Energy Management
,”
Proceedings of the 44th IEEE Conference on Decision and Control
, Seville, Spain, pp.
1816
1823
.
10.
Sciarretta
,
A.
,
Back
,
M.
, and
Guzzella
,
L.
,
2004
, “
Optimal Control of Parallel Hybrid Electric Vehicles
,”
IEEE Trans. Control Syst. Technol.
,
12
(
3
), pp.
352
363
.10.1109/TCST.2004.824312
11.
Zhu
,
Q.
,
Song
,
S.
,
Tan
,
X.
,
Song
,
C.
, and
Prucka
,
R.
,
2018
, “
Control Optimization of a Charge Sustaining Hybrid Powertrain for Motorsports
,”
SAE
Paper No. 2018-01-0416.10.4271/2018-01-0416
12.
Maino
,
C.
,
Misul
,
D.
,
Musa
,
A.
, and
Spessa
,
E.
,
2021
, “
Optimal Mesh Discretization of the Dynamic Programming for Hybrid Electric Vehicles
,”
Appl. Energy
,
292
, p.
116920
.10.1016/j.apenergy.2021.116920
13.
Harselaar
,
W. V.
,
Schreuders
,
N.
,
Hofman
,
T.
, and
Rinderknecht
,
S.
,
2019
, “
Improved Implementation of Dynamic Programming on the Example of Hybrid Electric Vehicle Control
,”
IFAC-PapersOnLine
,
52
(
5
), pp.
147
152
.10.1016/j.ifacol.2019.09.024
14.
Liu
,
J.
, and
Peng
,
H.
,
2008
, “
Modeling and Control of a Power-Split Hybrid Vehicle
,”
IEEE Trans. Control Syst. Technol.
,
16
(
6
), pp.
1242
1251
.
15.
Zhang
,
C.
, and
Vahidi
,
A.
,
2012
, “
Route Preview in Energy Management of Plug-in Hybrid Vehicles
,”
IEEE Trans. Control Syst. Technol.
,
20
(
2
), pp.
546
553
.10.1109/TCST.2011.2115242
16.
Bertsekas
,
D.
, and
Tsitsiklis
,
J.
,
1996
,
Neuro-Dynamic Programming
, 1st ed.,
Athena Scientific
, Nashua, NH.
17.
Xu
,
B.
,
Tang
,
X.
,
Hu
,
X.
,
Lin
,
X.
,
Li
,
H.
,
Rathod
,
D.
, and
Wang
,
Z.
,
2021
, “
Q-Learning-Based Supervisory Control Adaptability Investigation for Hybrid Electric Vehicles
,”
IEEE
Trans. Intell. Transport. Syst., epub.10.1109/TITS.2021.3062179
18.
Liu
,
T.
,
Wang
,
B.
, and
Yang
,
C.
,
2018
, “
Online Markov Chain-Based Energy Management for a Hybrid Tracked Vehicle With Speedy Q-Learning
,”
Energy
,
160
, pp.
544
555
.10.1016/j.energy.2018.07.022
19.
Sutton
,
R.
, and
Barto
,
A.
,
2018
,
Reinforcement Learning
, 2nd ed.,
The MIT Press
,
Cambridge, MA
, pp.
321
337
.
20.
Han
,
X.
,
He
,
H.
,
Wu
,
J.
,
Peng
,
J.
, and
Li
,
Y.
,
2019
, “
Energy Management Based on Reinforcement Learning With Double Deep Q-Learning for a Hybrid Electric Tracked Vehicle
,”
Appl. Energy
,
254
, p.
113708
.10.1016/j.apenergy.2019.113708
21.
Hu
,
Y.
,
Li
,
W.
,
Xu
,
K.
,
Zahid
,
T.
,
Qin
,
F.
, and
Li
,
C.
,
2018
, “
Energy Management Strategy for a Hybrid Electric Vehicle Based on Deep Reinforcement Learning
,”
Appl. Sci.
,
8
(
2
), p.
187
.10.3390/app8020187
22.
Liao
,
L. Z.
, and
Shoemaker
,
C. A.
,
1992
,
Advantages of Differential Dynamic Programming Over Newton's Method for Discrete-Time Optimal Control Problems
,
Cornell University
,
Ithaca, NY
.
23.
Betts
,
J.
,
1998
,
A Survey of Numerical Methods for Trajectory Optimization
,
Boeing Information and Support Services
,
Seattle, WA
.
24.
Stryk
,
O.
, and
Bulirsch
,
R.
,
1992
, “
Direct and Indirect Methods for Trajectory Optimization
,”
Ann. Oper. Res.
,
37
(
1
), pp.
357
373
.10.1007/BF02071065
25.
Todorov
,
E.
, and
Li
,
W.
,
2005
, “
A Generalized Iterative LQG Method for Locally-Optimal Feedback Control of Constrained Nonlinear Stochastic Systems
,”
Proceedings of the American Control Conference
, Portland, OR, June 8–10, pp.
300
306
.10.1109/ACC.2005.1469949
26.
Tassa
,
Y.
,
Mansard
,
N.
, and
Todorov
,
E.
,
2014
, “
Control-Limited Differential Dynamic Programming
,”
Proceedings of IEEE International Conference on Robotics and Automation
, Hong Kong, China, May 31–June 7, pp.
1168
1175
.10.1109/ICRA.2014.6907001
27.
Jacobson
,
D. H.
, and
Mayne
,
D. Q.
,
1970
,
Differential Dynamic Programming
,
Elsevier
, American Elsevier Publishing Company.
28.
Ruxton
,
D. J. W.
,
1991
,
Differential Dynamic Programming and Optimal Control of Quality Constrained Continuous Dynamic Systems
,
Department of Mathematics and Computing
,
University of Central Queensland
, Norman Gardens, Australia.
29.
Guzzella
,
L.
, and
Sciarretta
,
A.
,
2007
,
Vehicle Propulsion Systems-Introduction to Modeling and Optimization
, 2nd ed.,
Springer
,
Berlin, Germany
.
30.
Moawad
,
A.
,
Balaprakash
,
P.
,
Rousseau
,
A.
, and
Wild
,
S.
,
2016
, “
Novel Large Scale Simulation Process to Support DOT's CAFE Modeling System
,”
Int. J. Automot. Technol.
,
17
(
6
), pp.
1067
1077
.10.1007/s12239-016-0104-z
31.
Lennart
,
L.
,
1999
,
System Identification: Theory for the User
, 2nd ed.,
Prentice-Hall
, Upper Saddle River, NJ.
32.
Miro-Padovani
,
T.
,
Colin
,
G.
,
Ketfi-Cherif
,
A.
, and
Chamaillard
,
Y.
,
2016
, “
Implementation of an Energy Management Strategy for Hybrid Electric Vehicles Including Drivability Constraints
,”
IEEE Trans. Veh. Technol.
,
65
(
8
), pp.
5918
5929
.10.1109/TVT.2015.2476820
33.
Kim
,
N.
, and
Rousseau
,
A. P.
,
2011
, “
Comparison Between Rule-Based and Instantaneous Optimization for a Single-Mode, Power-Split HEV
,”
SAE
Paper No. 2011-01-0873.10.4271/2011-01-0873
You do not currently have access to this content.