Data Pre-Processing and Modeling Factors

Abstract

This chapter covers several important pre-processing steps. Before implementing a demand prediction method, it is crucial to process the raw data in order to extract as much predictive power as possible from the different features available in the data. We discuss how to deal with missing data and how to test for outliers in the context of demand prediction. We then cover various concepts related to feature engineering for demand prediction, such as accounting for time effects and constructing lag-price variables. We end this chapter by discussing the practice of scaling features, and how to sort and export the resulting processed dataset. Each step is illustrated using the accompanying dataset.