Gaussian Process Repetitive Control Beyond Periodic Internal Models through Kernels

Mooren, Noud, Gert Witvoet, and Tom Oomen. "Gaussian process repetitive control: Beyond periodic internal models through kernels." Automatica 140 (2022): 110273.

ABSTRACT: Repetitive control enables the exact compensation of periodic disturbances if the internal model is appropriately selected. The aim of this paper is to develop a novel synthesis technique for repetitive control (RC) based on a new more general internal model. By employing a Gaussian process internal model, asymptotic rejection is obtained for a wide range of disturbances through an appropriate selection of a kernel. The implementation is a simple linear time-invariant (LTI) filter that is automatically synthesized through this kernel. The result is a user-friendly design approach based on a limited number of intuitive design variables, such as smoothness and periodicity. The approach naturally extends to reject multi-period and non-periodic disturbances, exiting approaches are recovered as special cases, and a case study shows that it outperforms traditional RC in both convergence speed and steady-state error.

1. Introduction

Repetitive control is only applicable to periodic signals with a known period due to the traditional delay-based buffer as an internal disturbance model. A key assumption to achieving good performance is that the delay size matches the known period of the disturbance. As a result, RC is very sensitive to small variations in the disturbance period and non-periodic disturbances are even amplified.

Parametric internal models for RC enable rejection of a wider class of disturbances, e.g., matched basis functions and adaptive RC approaches. In this approach, a set of basis functions is defined by selecting all relevant frequencies in the error, subsequently, the corresponding coefficients are learned. This allows to learn multi-period disturbances and non-periodic disturbances, but it requires each specific frequency and its harmonics to be selected a priori.

In view of generic internal models for RC, recent developments in kernel-based approaches such asGaussian Process (GP) regression have shown to be promising. GP regression is a non-parametric approach that allows learning a wide range of functions, more specifically, a distribution over functions is learned, by specifying prior knowledge in the sense of a kernel function through hyperparameters. In contrast to parametric internal models for RC, where the basis functions have to be selected explicitly, the GP is a non-parametric approach that only requires selecting a periodic kernel function with a few intuitive tuning parameters.

The aim of this paper is to present a generic internal model for RC that efficiently uses Gaussian Processes to enable the rejection of a wide variety of disturbances, including, single-period, multi-period, and non-periodic disturbances, by specifying disturbance properties in a kernel function. By learning a continuous function, the disturbance period is not restricted to be an integer multiple of the sample time allowing for rational disturbance periods.

2. Problem formulation

2.1. Control setting

is a discrete-time linear time-invariant (LTI) system, is a stabilizing feedback controller, and is an add-on type repetitive controller (RC) that is specified in the forthcoming sections. The goal is to reject the input disturbance , a sampled version of a continuous disturbance . Without loss of generality the sample time is scaled to . Furthermore, noise is present that follows an independent, identically distributed (i.i.d.) Gaussian distribution with zero mean.

Definition 1. The control goal is to asymptotically reject the disturbance-induced error , given by for , i.e., by designing . In the case that R is LTI, then where is the modifying sensitivity, that is a measure for the performance improvement through , and is the process sensitivity when 0.

2.2. Internal model control

The internal model principle states that asymptotic disturbance rejection is obtained by including a model of the disturbance generating system in a stable feedback loop.

By the final value theorem, it can be shown that a constant disturbance with transform is asymptotically rejected with a factor in the open-loop .

Table of Laplace and Transforms:

https://lpsa.swarthmore.edu/LaplaceZTable/LaplaceZFuncTable.html

For periodic disturbances with period , a model of the disturbance generating system consists of a delay-based buffer , with , in a feedback loop, i.e., However, disturbances with a rational period time , do not fit in these traditional buffers and require additional interpolation.

Example: a continuous time disturbance with period from which discrete samples with sample frequency 1 Hz are taken, i.e., the discrete time sequence is non-periodic for all while the continuous time signal is periodic with the period time .

The following general class of disturbances is considered in this paper.

Definition 2. The continuous-time disturbance is defined as which is a multi-period disturbance consisting of periodic scalar-valued signals that are smooth and satisfy with , and is the period time of the h component.

The disturbance is a single-period disturbance if or a multi-period disturbance with ; in the latter case is either periodic with a period equal to the least common multiple (LCM,最小公倍数) or is non-periodic if there is no least common multiple.

2.3. Gaussian process RC setup

is a learning filter and the proposed GP-based internal model of the disturbance generating system is given by with the GP-based memory. Moreover, is a delay line that accumulates the past of its input , where is the state, and which results in the vector valued signal . Finally, is a vector of, possibly time-varying, coefficient that are designed and formally introduced in the forthcoming sections.

3. Gaussian process buffer in repetitive control

3.1. Gaussian process repetitive control setup

A sample of is parameterized as are, in general, time-varying coefficients that follow from GP regression.

The data used for GP-regression is given by the noisy data samples in to estimate a continuous function of the true disturbances for compensation. To compose the data set for GP regression, define the vector with corresponding time instances: constituting the data set that contains pairs of observations. At each sample the data is used to perform GP regression resulting in the coefficients .

3.2. Gaussian process disturbance model

First, consider the prior disturbance model given by a GP that is a distribution over functions which is completely determined by its prior mean function and prior covariance function with and the size of and respectively.

Next, it is shown how the prior knowledge and the data is used to compute . (需要搞清楚GPR的输入是什么，输出是什么，训练数据是哪些？感觉目前这个论文里没有说的很清楚，符号有些乱)

The data set DN contains noisy observations as (observed output): Predictions of the disturbance model for compensation can be made at arbitrary , denoted by . Moreover, for the application in RC, predictions are made at the current time, i.e., the test point becomes since . The joint prior distribution (这里，训练数据有t(k)，预测同样是t(k)，这个不是很清楚怎么实现？？？) defines the correlation between the data and the test point with It follows that the predictive posterior distribution at the test point becomes where Then, we have In contrast to traditional RC with internal disturbance model, GPRC enables compensation within the first period (这里是为什么？). Furthermore, by using a GP function estimator a more general setting is established in which also multi-period and non-periodic disturbances can be captured with suitable prior.

3.3. LTI representation of GPRC

In this section, conditions are presented under which the coefficients are time invariant.

Theorem 1. The repetitive controller is LTI and given by where the GP buffer is a finite impulse response (FIR) filter with time-invariant coefficients .

Proof. The stationary function of the kernel , It follows that . (因为输入的是时间，对于固定的采样频率，任意输入是固定不变的)。 Similarly for obtained by evaluating at all pairs given by With the assumption: the test point with constant, and are time-invariant, so are and are time invariant.

Consequently, the RC output is given by the following FIR operation In addition, the internal disturbance model is now also LTI and given by This framework then facilitates the construction of appropriate FIR coefficients , through which it enables efficient implementation of GPs in RC, allowing for larger flexibility, and offers superior performance in the first period due to continuous updating (这个需要注意).

3.4. Stability analysis

Theorem 2. Consider repetitive controller in Theorem 1, a specified kernel function and a buffer size . Suppose all poles of and are in the open unit disk, and the feedback loop is asymptotically stable, then the closed-loop is stable if and only if the image of :

• makes no encirclements （不包围）around the point −1, and
• does not pass through the point −1.

If the resulting closed-loop is unstable, e.g., due to modeling errors, the following slightly more conservative frequency-domain condition is provided to tune for stability.

Corollary 1. Theorem 2 is satisfied if for all .

4. Design methodology for Gaussian process RC

4.1. Learning filter design

The learning filter L in the repetitive controller is designed as Direct inversion of may lead to an unstable or non-causal inverse, e.g., if contains non-minimum phase zeros. By using Zero-Phase-Error-Tracking-Control (ZPETC), we have where is causal and with is a possible finite preview.

A practical implementation for the non-causal filter is presented, where the error is filtered with the causal part yielding This delay is compensated by a preview in the memory , i.e., the test point becomes , to implement the non-causal part of . (这里非常重要，需要特别注意！)

4.2. Prior selection

In this section, a suitable covariance function that specifies prior knowledge for the class of disturbances in Definition 2 is presented.

The additive structure in Definition 2 is imposed on the disturbance model by parameterizing it as a sum of periodic functions with periods , i.e., where are samples from independent GPs with periodic covariance function . Hence, it is referred to as an additive GP with an additive covariance function The periodic covariance function is of the form with hyperparameters where

• is the period of the th component;
• ​ is the smoothness of , i.e., choosing large implies less higher harmonics and vise versa; and
• ​ is a gain relative to the other components and the noise variance .

Example: Left plot shows three examples of periodic covariance functions as a function of with hyperparameters in and with . This shows that if is close to zero or the period , then and are highly correlated. Also the sum of two periodic kernels with periods and is shown yielding a nonperiodic kernel function. The bottom plot shows random samples taken from the distributions , these samples both periodic and have smoothness, or become non-periodic with a non-periodic kernel that is the sum of two periodic kernels.

This allows to capture both period and non-periodic disturbances in the GP-based internal disturbance model. In GPRC with kernel function the disturbance period is included through the hyperparameter that may be rational, in contrast to traditional RC, allowing to reject disturbances with a rational period time.

4.3. Design procedure

The following procedure summarizes the design steps that are required to implement GP-based RC.

Procedure 1 (GPRC design).

Given a measured frequency response function (FRF) and a parametric model SP, perform;

1. Invert to obtain and non-causal part with , e.g., using ZPETC.

2. Determine , e.g., using a PSD estimate of the error. Then, set and repeat the following:

​ (a) Choose the period , smoothness and gain for ;

​ (b) until , set and repeat step 2a.

3. Choose a buffer size , e.g., a good starting point is which yields sufficient design freedom, although smaller buffer sizes are possible with appropriate prior.

4. Define and evaluate in for and to obtain and respectively.

5. Compute FIR coefficient as and verify stability with using Theorem 2 or Corollary 1.

5. Performance and robustness

5.1. Recovering traditional RC

GPRC recovers traditional RC for a specific type of prior, i.e., a periodic kernel without smoothness. In traditional RC the buffer is a pure delay, hence, the output is simply a delayed version of the input.

Theorem 3. Under the conditions in Theorem 1, then with , a periodic kernel where , the memory , recovers traditional RC.

Hence, by setting smoothness to zero and the kernel period limited to an integer, the traditional RC memory is recovered.

Example: Modifying sensitivity function (left plot) and impulse response of MGP (bottom plot) for GPRC without smoothness and , and with smoothness with . Including smoothness yield that many FIR coefficients are non-zero for automatic interpolation, which enables suppression at and higher harmonics, whereas traditional RC performance in much worse

5.2. GPRC for discrete-time non-periodic disturbances

Traditional RC is not applicable to rational period times as in Definition 2 with which are non-periodic in discrete time, for these disturbances additional interpolation is required. In contrast, it is shown that GPRC can suppress disturbances that have a rational period time.

In GPRC the disturbance period is specified through the kernel function (29) where , and is not necessarily related to the buffer size as in traditional RC. In the case that is rational then is not directly available, i.e., it is in-between two samples, but it is estimated from the available inputs using a smoothness also estimating the disturbance in-between samples. Hence, smoothness enables interpolation for disturbances with a rational period time.

5.3. Recovering HORC

GPRC can improve the robustness of RC with respect to noise or uncertain period times similar to HORC, where buffers are combined.

Lemma 1. Consider GPRC under Assumption 1, then for all and the kernel matrix if and only if its inverse .

Lemma 2. Under Assumption 1, then with the kernel where , the FIR filter is of the form , with weights .

5.3.1. GPs for period-time robust RC

A form of HORC improves robustness for uncertain period times, which is recovered by GPRC through a locally periodic kernel, that allows for slight variations in the disturbance estimate and is given by where is the periodic kernel and ls the local smoothness.

Example: GPRC closely recovers period-time robust RC using a suitable kernel function with a specific smoothness.

5.3.2. GPs for noise robust RC

GPRC can improve noise robustness with respect to traditional RC by using smoothness in a periodic kernel, even outperforming noise-robust HORC with a smaller buffer size.

HORC is recovered without smoothness and an appropriate kernel, furthermore, introducing smoothness yields additional design freedom to improve noise robustness with a much smaller buffer size than HORC. However, including smoothness also leads to less disturbance attenuation at high frequencies.

Examples: Modifying sensitivity with GPRC for a periodic kernel with three different levels of smoothness , and , showing that smoothness improves robustness for noise and reduces disturbance rejection at higher frequencies.

5.4. GPs for multi-period RC

The periodic kernel also enables rejection of multi-period disturbances. Using a multi-period kernel GPRC suppresses the disturbance at specific frequencies instead of all harmonics of the common multiple, resulting in less amplification of non-periodic errors.

By only introducing disturbance suppression where this is required, less amplification of noise at intermediate frequencies is obtained, due to Bode’s Sensitivity integral.

Examples: (Left) Modifying sensitivity for multi-period GPRC with samples with buffer size samples and samples. As a comparison, the traditional RC with is also shown.

​ (Right) FIR coefficients with a multi-period kernel where , yielding non-zero FIR coefficients at and the difference between them for and with .

5.5. Robustness for model errors

Robustness for model errors in RC is often improved by designing a robustness filter , typically a low-pass filter, that is placed in series with the buffer . Next, it is shown that robustness is naturally included in GPRC by increasing smoothness.

From an intuitive point of view, higher smoothness yields a smoother disturbances estimate , and thereby less high-frequency content in the RC output . Hence, learning is limited in the high-frequency range, i.e., where the model is not reliable, having a similar effect as a filter in traditional approaches. Hence, smoothness also imposes an upper bound on the frequency content of the disturbance that can be learned.

Example: Magnitude of for a periodic kernel with and and three settings of smoothness, i.e., and . This shows that smoothness leads to a low-pass characteristic in , which in turn increases robustness.

6. Implementation aspects: dealing with initial conditions

In this section, performance in the first samples is improved even further by taking into account the initial conditions of the buffer , which may limit performance in the LTI case.

6.1. Limitations of the LTI case

The problem that arises in the LTI case is that the initial condition of the buffer , which is zero by default, appears as observations of the disturbance in the training data set during the first samples. Performing GP regression with these incorrect observations gives a worse disturbance estimate. After the first samples, the initial condition of disappears from the buffer. To improve GPRC in the first samples, the following two solutions are provided.

6.2. Discarding observations

A simple solution is to discard the first observations from the data set that correspond with the initial conditions of the memory . This is done by introducing a time-varying selection matrix such that where with the time-varying number of samples that are used for GP regression. After samples thus such that the LTI case in Theorem 1 is recovered.

Note that this approach requires computing GP at each sample during the first samples, which is computationally demanding. Therefore, an alternative solution is introduced next.

6.3. Time-varying kernel to improve learning

A second solution is to choose a sufficiently high noise variance for the undesired inputs such that these are reflected less in the RC output. This can be done by modifying the matrix by replacing the diagonal matrix with noise variances with the following time-varying diagonal matrix: such that after samples and the LTI case is recovered.

The time-varying matrix is diagonal with noise variance for GP inputs that correspond to the initial condition of , and the variance is n for the GP inputs that represent the disturbance. In this way, the observations with a large variance have negligible influence on the posterior mean, resulting in a significant improvement in convergence during the first N samples if smoothness is included.

7. Conclusions

A generic repetitive control framework for asymptotic rejection of single-period, and multi-period disturbances, with potentially rational period times, is enabled through a Gaussian process (GP) based internal model. The presented GP-based approach also enables compensation within the first period, in contrast to many existing RC approaches. The disturbance is modeled using GP regression,which is a non-parametric approach that combines data with prior knowledge. Prior knowledge is included in the form of a kernel function with periodicity and smoothness, which allows modeling a wide range of disturbances by specifying intuitive tuning parameters. It appears that under mild assumptions the GP-based RC approach is LTI and more specifically given by an FIR filter, such that it is computationally inexpensive, stability conditions can be provided and several existing approaches are recovered as a special case. Moreover, applying GP-based RC for non-linear systems is conceptually possible following the developments in this paper by reformulating the stability conditions for the non-linear case which is a part of future research. Ongoing work focuses on utilizing the posterior variance of the disturbance model to improve robustness against model errors and incorrect prior.