Development of Algorithm for Treatment of Extreme Outliers in Numerical Data, Conditional on Joint Distribution Relationship

Abstract

The paper describes the development of a model for treating extreme outliers in numerical data variables, using an alternative point of view-relationship conditional on data variables' joint distribution. The mentioned algorithm finds its roots in existing practices like Winsorization (or truncation) of extreme outliers independently for each variable. After establishing an overview of historical outlier treating, the authors turn their attention towards common erroneous assumptions, that, while often, being made, bring research analysis and conclusions towards big caveats. A common example is a joint distribution relationship shape's shift from that of a circular towards a rectangular shape, unexplainable or indefensible by statistics. Next the paper makes some propositions how to fix this prominent problem, by looking at existing joint distribution relationship's shape and how it could give insight on outlier treatment. Following that, the authors try to circumscribe a methodology for application of the proposed fix by going through the algorithm's mathematical specifics and assumptions. Lastly the paper studies the algorithm's application results, including possibilities for future studies, including interdisciplinary application within programing environment, for the possible purpose of creating an actual software solution.