Intelligent Automated Workload Analysis for Database Replatforming

Abstract

Performing a detailed workload analysis is a crucial step in determining the feasibility, timeline and cost of a major data warehouse replatforming project, i.e., migration from one platform to another. A large company's data warehouse applications may include millions of queries, some of which will use features that are unsupported or have different semantics in the new warehouse, or may have poor performance there. In this paper we present qInsight, a workload analyzer that Datometry has used in data warehouse replatforming efforts for dozens of major clients. qInsight leverages Datometry's Hyper-Q to obtain insights from a workload, including SQL features and workload structural information that could not be obtained without deep query analysis. qInsight uses the identified features and a weighting scheme based on human expert judgments to assess the difficulty of rewriting each application in the workload via traditional migration methods. Datometry's clients find this information useful in planning their projects, including the order in which to migrate applications. We present a qInsight-based data warehouse usage analysis of over 1.7 billion queries from real-world workloads.