Abstract
This work examines a class of distributed linear systems that fit a tree type of hierarchical structure. Therefore, unidirectional information flow from a subsystem higher in the hierarchy impacts subsystems lower in the hierarchy. While prior efforts at control focused on feedback solutions to these systems, this effort introduces Iterative Learning Control as a feedforward controller. This is termed hierarchical iterative learning control (HILC). This HILC can be implemented in parallel with feedback algorithms or can be used in a series feedforward manner assuming stable, or stabilized, subsystems. An augmentation of the standard learning update operators provides stability and monotonic convergence for the proposed approach. A simulation case study highlights the performance of the proposed design approach.
1 Introduction
Several classes of engineering system can be represented in a hierarchical distributed format. These include manufacturing systems, electric power systems, and many others. Usually, the capabilities of the systems vary at each level of the hierarchy. In general, higher levels of a hierarchy are concerned with slower and more complex systems spread over a large range of operating conditions. These upper levels are concerned with global resource allocation and optimization. Lower levels of a hierarchy tend to focus more on servotype regulation or tracking relevant to a given subsystem. By design, the actuation and sensing ability of the upper levels are quite different than those of the lower levels, and this difference enables complex systems to be decomposed and managed more effectively. There have been significant advances in the control of these types of systems including Scattolini, Koeln and Alleyne, Boyle and Stockar, Bhattacharjee et al., and Aksland et al. [1–5]. To date, much of the effort in the control of distributed hierarchical systems is in the form of feedback. That is, information is gathered at individual layers or levels of a hierarchy and passed up to levels above. Decisions are made at each level with corresponding commands or references sent down to lower levels. Oftentimes, optimization techniques such as model predictive control are used when the information gathering and decision-making happen on-line during process operation [1,2]. What is less prevalent is the use of feedforward techniques passed from one level of the hierarchy to another below. The goal of this work is to introduce a specific type of feedforward termed iterative learning control (ILC) [6–8] to these hierarchical systems.
Iterative Learning Control considers systems that operate the same activity repeatedly, usually with a break and a reset of initial conditions between each activity and trial. As outlined in this work, we consider the case where information is gathered offline between trials at all levels of the hierarchy. Then this information is used to provide feedforward signals to individual subsystems at each layer. MIMO ILC methods can be employed to address varying degrees of coupling in multivariable systems [9,10]. However, standard MIMO ILC approaches do not consider potential mismatches between the actuator capabilities and the given tasks at each level of a hierarchy. Without some form of intelligent allocation of separation, typical MIMO designs can allocate undesired amounts of actuation to the wrong levels [11]. Using an electrical grid example, this would be analogous to asking the base load provider power plant to participate in a rapid demand following at a local substation level. This misallocation of control effort can be detrimental. Such scenarios lead to actuator saturation in these faster actuation stages, which typically have less control input amplitude, with the potential to cause saturation and possible subsystem failures.
This work represents an extension and generalization of prior work [12] that considered a series hierarchical iterative learning control approach. In the prior work [12], there was one subsystem at each layer in the hierarchy. In the current work, multiple subsystems exist within each layer of the hierarchy requiring a more generalized analysis of convergence.
The remainder of this paper is organized as follows: Section 2 describes the class of distributed hierarchical systems under consideration. Section 2 also introduces the nomenclature and framework for system representation. Section 3 develops the hierarchical ILC controller. Section 4 provides analysis of the control strategy in terms of both stability and monotonic convergence, which is a highly desirable characteristic. Section 5 provides a simulation example on a two-level hierarchy with a single subsystem at level 1 and two subsystems at level 2. A conclusion discusses the results presented and offers concluding remarks.
2 Preliminaries
Section 2.1 presents standard ILC nomenclature in a time–domain lifted framework. This representation is generic to linear time invariant (LTI) systems. Subsequently, Sec. 2.2 introduces the particular tree-type hierarchical framework for systems of interest and uses this framework within the context of ILC.
2.1 Norm-Optimal Iterative Learning Control.
The lifted system in Eq. (2), , is a lower triangular Toeplitz matrix that contains the finite-impulse response Markov parameters, , of closed-loop system P. The vector, , of Eq. (2) is a sample lifted vector; the element of a lifted vector is the corresponding signal's value at time-step . In this work, a serial ILC is used [7] where exogenous reference augmentations are made to stable or stabilized subsystems.
where, are, respectively, the lifted vector forms of the plant output, desired output (reference), ILC control input, and repetitive system disturbances. The term is the ILC input to be determined. For standard ILC problems, the reference signal and disturbance are iteration-invariant. Therefore, there is no indexing of these terms with respect to j in Eq. (3).
In Eq. (4), is introduced to simplify expressions that appear in later sections and the operator present in Eq. (5) is simply the iteration–domain difference function, i.e., . Since neither the reference trajectory nor the exogenous disturbances to the system are iteration-varying, both terms are canceled out within the first iteration of the delta operator and do not carry through to future iterations.
The lifted approach results in two well-known conditions for stability and monotonic convergence.
2.2 Class of Tree-Structure Hierarchical Systems.
This work focuses on a class of systems that exhibit a tree-distributed type network in the connection of their subsystems. Additionally, these systems of interest have a unidirectional coupling between specific subsystems in the sense of physical couplings, flow of information, or both. Furthermore, this class of systems can be organized (implicitly or explicitly) into a multilevel hierarchy of their subsystems and/or the operational tasks performed by said subsystems. By representing each subsystem as a specific node and assigning the coupling between subsystems to the directed edges allows such tree-distributed hierarchical systems to be described by directed graphs like the one depicted below in Fig. 1.
A useful feedback control architecture for multistage systems is the dual-feedback architecture given in Ref. [17] and built upon in Ref. [12]. For these systems, the subsystem at the upper level, defined as level 1, receives a desired output as the reference signal to track. The subsystem in level 2 is then tasked with tracking the feedback error that accumulates in the previous level, which becomes the anticipated unidirectional coupling between subsystems.
Here, j is the iteration index, is the discrete-time index, and is the number of samples per iteration. Figure 2 illustrates this multifeedback architecture. The introduction of the iteration domain in Eq. (14) sets up the discussion of applying ILC to these multilevel, tree-distributed systems in Sec. 3.
3 Hierarchical Iterative Learning Control Design for Tree Distributed Systems
Section 3.1 classifies the systems of interest and describes the feedback architecture for this class of systems, laying the notational foundation for future sections. Section 3.2 applies the lifted system representation to the distributed hierarchy of subsystems described by (14) to derive the iteration–domain error evolution equations for levels l > 1 of the hierarchy. Section 3.3 redefines the extended error term previously reported in Ref. [12] and introduces an extended tracking error formulation used in the analyses of Sec. 4. Section 3.4 develops the generalized form of HILC update laws and learning filters for each ILC stage within the HILC network.
3.1 Lifted Representation of Multi-Level, Distributed Subsystems.
3.2 Extended Error Terms.
Prior work reported a novel extended error term in Ref. [12] for generalized class of multistage actuator systems in which only one subsystem existed per level. In this subsection, we revisit the concept of an extended error term based on the unidirectional coupling between levels with a modification to the grouping of terms from Ref. [12] that enables an analysis of stability and monotonic convergence conditions.
3.3 Hierarchical Iterative Learning Control.
The predicted change in the extended error term, , appears in the HILC update laws (25), identical to its appearance in (23). Further, the learning operators in (26) are the same as the operators given in Eq. (9).
Inspection of the extended error term definition (22) and comparison with the HILC update laws (25) provide the following insight: the HILC update laws contain current iteration inputs from levels of previous HILC stages, i.e., . These inclusions provide a single iteration preview horizon for lower HILC stages. This passing of information from one level down to the next facilitates a unidirectional coupling of the HILC stages, creating a structural agreement between the HILC system as a whole and the distributed hierarchical system of interest.
Remark. Due to the offline nature of ILC algorithm updates, calculating HILC inputs sequentially by level affords an engineer the availability of the desired future HILC input(s), , from previous levels when calculating the current HILC input(s), . In this way, the proposed method leverages the offline nature of ILC algorithms to incorporate current iteration information to all dependent subsystems. This is depicted schematically in Fig. 3.

A graph-based representation of the HILC system architecture for a system with three levels of subsystems and varying numbers of subsystems per level. Each node in the HILC architecture is an individual ILC stage denoted by L*.
4 Stability and Convergence Analysis
4.1 Hierarchical Iterative Learning Control Stability.
This completes the proof.
4.2 Error Convergence.
This completes the proof.
It is necessary to appropriately choose the learning operators to achieve the stability and convergence criteria specified in Theorems 1 and 2. This is achieved through tuning.
4.3 Hierarchical Iterative Learning Control Tuning.
The goal of tuning of the HILC system is to satisfy the stability and monotonic convergence conditions of Theorems 1 and 2 by adjusting the scalar weightings, , of the HILC cost functions (24). Tuning of an individual stage follows the NOILC tuning guide provided in Ref. [18] such that generate appropriate learning operators.
Briefly, set , , and ; reduce until the system is unstable, then double ; then increase until the error bound is satisfactory. Each level of the HILC is tuned sequentially, beginning with l = 1. While tuning any stage of some level, , the HILC stages of all levels, , are kept inactive . In contrast, stages in all preceding levels, , are kept active to afford control designers the ability to observe the “upstream” HILC effects. The individual stages within a given level require no specific order when being tuned; however, the engineer should give attention to completing the tuning of every stage of the previous level to avoid incomplete or misleading information concerning the stability and/or convergence trends of not only individual stages but also the entire system. The overall approach is summarized in the flowchart given in Fig. 4.
5 Case Study
To examine the behavior of the proposed hierarchical approach, this work applies the hierarchical ILC to a simplified version of a fine steering mirror plant similar to those found on segmented telescopes [19] or lithography stages [20]. The simplified system model is presented, which includes feedback controllers on the open-loop system models. Then the HILC controller design is implemented with this numerical example. A set of simulations result and discussions finish the case study.
The target plant consists of two levels of actuation: a coarse actuating linear motor as its first level (l = 1) and two fine stage actuators assigned to level l = 2, attached to a reflecting mirror. It is assumed that there is a direct force as an actuation input for each of the levels. At second level, the two inputs are attached at opposite ends of the mirror, equidistant from its center, to enable control over the angular position of the mirror as shown in Fig. 5.
Therefore, the linear position of the mirror assembly is based on an implicit midline calculation of the second level subsystem outputs. The task space for this class of system can be partitioned into linear and angular regimes.
In this example, the level 1 actuator is assigned to the linear regime while the level 2 actuators affect both the linear and angular system outputs.
The subsystems from Eq. (33) are stabilized by simple PID controllers and Table 1 provides the gain parameters for each controller, where is the derivative filter coefficient in the pseudo-derivative term: .
Plant feedback controller PID gains
Feedback controller | ||||
---|---|---|---|---|
0.01553 | 0.02233 | 0.00254 | 0.00733 | |
13.023 | 279.62 | 0.13783 | 0.00052 | |
9.6158 | 140.90 | 0.16149 | 0.00250 |
Feedback controller | ||||
---|---|---|---|---|
0.01553 | 0.02233 | 0.00254 | 0.00733 | |
13.023 | 279.62 | 0.13783 | 0.00052 | |
9.6158 | 140.90 | 0.16149 | 0.00250 |
Combining systems with closed-loop controllers results in the stable plants, , that will interact with the HILC. After tuning, the level 1 coarse stage plant, , has a closed-loop bandwidth of 7.1 Hz. Closed-loop bandwidths for the second level fine stage plants, and , are approximately 79.3 Hz and 82.9 Hz, respectively. This gives the levels more than a decade of separation in the frequency domain. Plant uncertainty is introduced to the simulations by including unmodeled plant dynamics in the form of complex poles and zeros, which represent common resonances in the structural components of electromechanical servosystems [20]. Bode plots comparing the ideal versus uncertain closed-loop models for FSM subsystems are provided in Fig. 6.
The first level is tasked with tracking the iteration-invariant reference profile, , which is a 5 Hz triangle wave with an amplitude of 10 mm. Both fine stages in level 2 adopt the feedback error of as their reference profiles, —at least in part. The second level (l = 2) references are then partitioned into the two regimes of linear and angular tracking.
Since the second level of the FSM is completely responsible for the angular task regime, both fine subsystems, and , are given iteration-invariant angular command signals as a portion of their reference profiles. Here, the angular command signal, , is a 60 Hz sine wave with 15 degrees of maximum tilt amplitude that is assigned to subsystem to track while the negation of the signal is passed to subsystem . For this simulation case study, the spacing X in Fig. 5 is set to 5 mm between the level 2 fine actuators. The angular command signals become linear position commands with amplitude 0.6685 mm under a small angle assumption. From the construction above, it is understood that the fine actuators receive oppositely signed commands at each time instance. The reference trajectories for both levels are given below in Fig. 7.
Both subsystems at level 2 adopt the iteration-varying feedback error of as the linear portion of their reference profiles. Thus, each subsystem in level 2 is tasked with linear and angular tasks within their references, , . The goal for of the overall system is to track the lower-frequency components of via the first level subsystem, , while the second level stages track any high-frequency components through the unidirectional coupling enforced in Eq. (13). Further, level 2 of the system must track all portions of the angular commands, , in addition to their unidirectional coupling from . Although the dual-feedback architecture between levels of the FSM creates a natural frequency separation of this linear regime reference signal, saturation within the second level of actuators remains a significant real-world concern. For this reason, level 2 subsystems have the added goal of remaining within a saturation limit of 3 mm.
As pointed out in Ref. [12], the saturation limits on level 2 subsystems preclude using an overall MIMO ILC that accounts for all the dynamics. A traditionally implemented MIMO ILC, which did not account for a hierarchical decomposition of the plant, would appropriate the fast level 2 subsystems to address more of the tracking tasks than they are able and would easily run them into saturation. The separation into a hierarchical format avoids that.
Simulations are run in the MATLAB/Simulink environment for a total time of 0.5 s using a fixed time-step of one millisecond. The HILC system is designed in discrete time with = 2 levels and a total of three HILC stages corresponding to each subsystem. The goal of the HILC system is to improve total tracking performance of each iteration-invariant reference and allocate appropriate frequency separation of the linear reference, . Additionally, the second level subsystems must be kept within their saturation limits of ± 3 mm.
The distributed nature of the HILC method proposed in Sec. 3 is compared against two additional ILC configurations. The first and second comparative cases are Series HILC (SHILC) controllers from Ref. [12] applied to single branches of the overall system. Namely, the first ILC configuration is the SHILC case that excludes subsystem (referred to as: SHILC on branch), whereas the second configuration excludes subsystem (referred to as: SHILC on branch). These first and second cases amount to a hierarchical implementation of ILC comprising a top level and only one subsystem with the other subsystem remaining solely in its feedback configuration. Finally, a third set of results present the distributed HILC method as applied to this configuration of the FSM system.
Each ILC configuration is run for 25 iterations with iteration j = 0 representing the initial system performance. The initial conditions of the system are reset at the beginning of each trial. Table 2 provides the tuning parameters used for each ILC stage with corresponding stability/convergence criteria values. The error weighting parameter is the same for all ILC stages and is subsequently omitted from the table. White noise is added as measurement disturbance to each subsystem independently in an iteration-varying manner.
ILC cost function weights and stability/convergence values
ILC stage | |||
---|---|---|---|
l = 1, n = 1 | 1.6464 × 10−3 | 0.4 | 0.9959 |
l = 2, n = 1 | 1.7168 × 10−2 | 2.0 | 0.9906 |
l = 2, n = 2 | 2.6656 × 10−2 | 2.0 | 0.9868 |
ILC stage | |||
---|---|---|---|
l = 1, n = 1 | 1.6464 × 10−3 | 0.4 | 0.9959 |
l = 2, n = 1 | 1.7168 × 10−2 | 2.0 | 0.9906 |
l = 2, n = 2 | 2.6656 × 10−2 | 2.0 | 0.9868 |
Figures 8 and 9 show the partitioned task space results of each ILC configuration after 25 iterations. These are normalized figures with the 0th trial representing unity. Therefore, all results are presented as improvements over the 0th trial. Figure 8 provides the normalized root-mean square (RMS) error convergence of the linear task regime for the system, which is measured as the average true error of level 2 subsystems, . Figure 9 provides the normalized RMS error convergence of the angular task regime for the system. Collectively, both figures showcase the advantages of the proposed distributed HILC method within the partitioned task space regimes over either of the single-branch SHILC methods.
![Tracking performances of each subsystem at the final iteration, j = 25. [Top] Subsystem P1,1. [Middle] Subsystem P2,1. [Bottom] Subsystem P2,2.](https://asmedc.silverchair-cdn.com/asmedc/content_public/journal/dynamicsystems/146/3/10.1115_1.4064174/1/m_ds_146_03_031002_f010.png?Expires=1742316898&Signature=sS-e-RfoJYLTPUa8Y9DCbyJGbKuugEfpZMS0H-TOo9AkanMtas67jDkOtLAu7~JsNSRpt10ByaRX9q-6Ki9Hy35NWJMUwI8a1Ggk00ZhmTbSTNSzscV9emUhbQH9j7OP6wpcX5Ov2oZTQUTdihNvKiqFLmuwO-Nj5bDn-3tH8Gk5mVnOXIbPgSeiWgt4ViWo729jipO9-c6eLR9IVL9sBjPunthfNI07YKxVfMlA03yI~VPGkrKr0-0yotaZySeE0smp2XRBNh0HeQQ4vgVDdn5gp2t1P5deNlwPCq0pW55WTLaZn~4h-fzJlOukmQfDh9CUb8KRDunWp0XlnKkIeg__&Key-Pair-Id=APKAIE5G5CRDK6RD3PGA)
Tracking performances of each subsystem at the final iteration, j = 25. [Top] Subsystem P1,1. [Middle] Subsystem P2,1. [Bottom] Subsystem P2,2.
As shown in Fig. 8, both SHILC configurations initially outperform the single-stage SISO NOILC configuration before eventually climbing back up to their converged RMS linear error values of approximately 65% and 60% of initial system performance, or 35% and 40% reduction, respectively. Neither of them exhibit monotonic convergence. The SISO NOILC configuration outperforms these values by the final iteration with an error reduction of roughly 50% in the linear task space. From Fig. 8, it is evident that the distributed HILC method outperforms the other configurations in both convergence rate and final linear error reduction. HILC surpasses 60% reduction of RMS linear error in just two iterations (j = 2) and achieves a reduction of over 80% by the final iteration at j = 25 while maintaining a monotonically decreasing trend modulo the effect of the sensor noise that leads to standard iteration to iteration variability of the error.
The angular tracking error resulting in Fig. 9 also indicates a benefit to the HILC approach for this type of system. None of the other configurations achieves a reduction of the normalized RMS angular error over 50%. SHILC on the branch and SHILC on branch result in monotonically decreasing error trends to final reductions of approximately 42% and 47%, respectively, for this metric. Each can improve the system behavior but without coordination, their benefit is limited. Ultimately, the distributed HILC system controllers, which can coordinate between level 1 and level 2 as well as within level 2 subsystems, surpass the 90% reduction mark of the normalized RMS angular error by the tenth iteration (j = 10) and converge to a reduction between 93% and 94% by the final iteration, despite the measurement disturbances introduced to the subsystems.
Figure 10 provides each subsystem's tracking performance at the final iteration of the ILC implementations. The top plot of the figure shows the final performance of the coarse actuator subsystem, , which is shared across all configurations; the middle and bottom plots give a 0.2 s window into final performances of the first and second fine actuating subsystems, and , respectively. As shown in the middle and bottom subplots, when there is coordination between level 1 and level 2 for a single level 2 subsystem, that level 2 subsystem performs similarly well to the distributed HILC. However, the lack of coordination within level 2 affects the overall system performance as seen in Figs. 8 and 9. It is this coordination that the distributed HILC provides.
Collectively, the plots reconfirm early findings of Igram and Alleyne [12] that inclusion of the extended error term in ILC update laws of the HILC system enforces unidirectional coupling of its stages to achieve the desired frequency separation between intralevel subsystems. Further case studies and results of design can be found in Ref. [21].
6 Conclusions
This paper provides an approach for Iterative Learning Control that can be applied to distributed systems in a tree type of hierarchical structure. Subsystems of the considered plants can be organized into multilevel distributed hierarchies, where each level of the hierarchy can contain multiple subsystems connected to some subsystem from the level above. Utilizing the NOILC framework, ILC update laws are derived for each level. For each local ILC stage, the extended error term is shared among different subsystems of the encompassing HILC controller that mirror the couplings in the controlled MIMO system, thereby connecting them in a distributed hierarchical control manner. This provides a preview horizon of the downstream iteration-domain actions for ILC stages from the level above and reinforces the hierarchical structure of the distributed HILC algorithm. Analysis of the HILC system shows no dependence on extended error terms for the stability and convergence properties, reducing the analysis conditions for the global HILC controller to the worst-case SISO condition among any local ILC stage.
Future work would consider robustness of the overall approach to uncertainties that occur within each level of the hierarchy. In particular, if the levels of subsystem uncertainty were heterogeneous within a hierarchy level, we desire to develop approaches that would perform better than a worst-case uncertainty across the entire level.
Funding Data
Sandia National Laboratories (award ID: SNL 1737490; Funder ID: 10.13039/100006234).
University of Illinois.
Data Availability Statement
The datasets generated and supporting the findings of this article are obtainable from the corresponding author upon reasonable request.
Nomenclature
- e =
measured error from reference
- j =
iteration index
- J =
norm-optimal cost function
- k =
discrete-time index
- l =
level index
- L =
learning operators in lifted domain
- n,m =
subsystem number of level l
- Nl =
total number of levels
- Nk =
total number of samples per iteration
- P =
discrete-time subsystem descriptor
- =
continuous-time subsystem descriptor
- =
cost function weights
- s/S =
real-valued signals & functions
- s/S =
vector-valued signals/S.S. matrices
- s/S =
lifted vectors/lifted systems & matrices
- u =
feedback control input
- uj =
ILC input in iteration-domain
- y =
measured output
- yd =
desired output/reference signal
- =
ILC input stability Lipschitz constant
- =
ILC error convergence Lipschitz constant
- =
single iteration difference function
- =
ILC tracking error proxy
- =
novel ILC extended error term
- =
- =