DATA PRE-PROCESSING TECHNIQUES FOR MULTIVARIATE ANALYSIS TO TREAT INDUSTRIAL OPERATING DATA FOR RETROFIT DESIGN

Abstract

Multivariate Analysis (MVA), a statistical design tool for dealing with very large datasets, was applied to historical data from a Thermo-Mechanical Pulp (TMP) newsprint mill in Eastern Canada. Partial Least Squares (PLS) type MVA models were created to identify significant correlations between operating parameters in the woodchip refining section and variations in pulp quality. Understanding these relationships is of crucial importance to any eventual retrofit design for this process. This paper focusses on pre-selecting and pre-treating the raw process data, including infrequently measured variables, to maximize the realism and usefulness of the MVA black-box models. Key methods explored were ways of selecting low-production periods for removal, techniques for identifying and eliminating major outliers using MVA outputs, and noise filtering. A major conclusion of this work was that the PLS models were significantly improved by pre-treating the data. This paper recommends an overall design approach for applying MVA to industrial operating data, involving stringent removal of dubious periods of operation such as aberrant process behaviour, and an aggressive Exponentially Weighted Moving Average (EWMA) filtering of all dependent and independent variables.