TITLE:  Shrinkage in Change Detection and Uncertainty Quantification

ABSTRACT:

In the information age, many real-world applications such as biosurveillance, manufacturing systems, physical and computer experiments often involve data that are massive, high-dimension or have complicated structures. In some cases it is cheap to collect large-scale data, while in other cases it may be costly or time-consuming to collect them. In either case, it is often non-trivial to extract information from these types of data to make useful decisions.

This dissertation makes methodology contributions to three important subfields of statistics: (i) Large-scale multi-stream quickest change detection, (ii) multichannel profile monitoring and (iii) global optimization of expensive functions. A common feature of the thesis work is the use of shrinkage to the respective subfields to address the challenges of high-dimensional or complicated data. However, since different subfields and applications have different features and challenges, details of the shrinkage techniques vary with the subfield.

This dissertation consists of three chapters. In Chapter 1, we study the problem of online monitoring large-scale data streams, which has many important applications from biosurveillance and quality control to finance and security in modern information age. While many classical quickest change detection methods can be extended from one-dimensional to any K-dimensional, their performances are rather poor when monitoring large K of data streams. This motives us to investigate the effects of dimensionality on the performance of quickest change detection methods. We found out through theoretical analysis that the classical quickest change detection methods often over-emphasize the first-order term of the detection delays and overlook the second-order terms of the detection delays, where the latter often increases linearly as a function of the dimension K. When K is large (e.g., hundreds), the second-order term of the detection delay will likely be comparable to the first-order term, which implies that the nice first-order asymptotic optimality properties have little practical meaning for large K. We propose a novel approach to lessen the dimensionality effects by introducing some shrinkage estimators of the unknown post-change parameters. In addition, we also illustrate the challenge of Monte Carlo simulation of the average run length to false alarm in the context of online monitoring large-scale data streams.

In Chapter 2, we consider the problem of monitoring multichannel profiles that has important applications in manufacturing systems improvement. A concrete motivating example of this work is from a forging process, in which multichannel load profiles measure exerted forces in each column of the forging machine. While various methods have been developed for univariate profile monitoring, they often cannot easily be extended to multichannel profiles. There are two main challenges when monitoring multichannel profiles. The first one is that profiles are high-dimensional functional data with intrinsic inner- and inter-channel correlations, and the second, probably more fundamental, challenge is that the functional structure of multi-channel profiles might change over time, and thus the dimension reduction method should be capable of taking into account the potential unknown change. We develop a novel thresholded multivariate principal component analysis (PCA) method for multi-channel profile monitoring. Our proposed method consists of two steps of dimension reduction: It first applies the functional PCA to extract a reasonable large number of features under the in-control state, and then uses the shrinkage techniques to functional PCAs to further select significant features capturing profile information in the out-of-control state. The choice of tuning parameter for soft-thresholding is provided based on asymptotic analysis, and extensive simulation studies are conducted to illustrate the efficacy of our proposed methodology.

In Chapter 3, we study the problem of global optimization of expensive functions. In modern physical and computer experiments, one often deals with expensive functions in the sense that it may take days or months to evaluate their values at a single input setting. An important problem is how to choose an appropriate setting of the input variables so as to optimize the output. To tackle this question, our proposed method involves two main components: one is the construction of a surrogate model to approximate the true function with much cheaper computation, and the other is the determination of a new input setting for function evaluation based on the surrogate model. After iteratively updating these two components, we optimize the latest surrogate model, which yields the approximation to the optima of the original expensive function. To be specific, we propose an adaptive Radial Basis Function (RBF) based global optimization framework via uncertainty quantification. For the surrogate model, we construct an adaptive RBF-based normal mixture Bayesian surrogate model, where the parameters in the RBFs can be adaptively updated each time a new point is explored. It is crucial to employ the normal mixture Bayesian structure which leads to a more stable surrogate model and avoid over-fitting. Its use can be regarded as a ridge-type regression estimate of model coefficients. For the selection of input setting, we propose a novel criterion to assess the input setting based on the surrogate model, and we choose the inputs that maximize the criterion. Our criterion incorporates the expected improvement (EI) of the function prediction to effectively identify promising areas for the global optima, and its uncertainties to efficiently explore the unknown regions. We conduct numerical studies with standard test functions to understand and compare the empirical performance of our proposed method with a prominent existing method.