Normalized compression distance for DNA classification

Abstract

Analyzing the origin and diversity of numerous genomic sequences, such as those sampled from the human microbiome, is an important first step in genomic analysis. The use of normalized compression distance (NCD) has demonstrated capabilities in the field of text classification as a low-resource alternative to deep neural networks (DNNs) by leveraging compression algorithms to approximate Kolmogorov information distance. In an effort to apply this technique toward genomics tasks akin to tools such as Many-against-Many sequence searching (MMseqs) and Kraken2, we have explored the use of a gzip-based NCD combination in both gene labeling of open reading frames (ORFs) and taxonomic classification of short reads. Our implementation achieved 0.89 accuracy and 0.88 macro-F1 on human gene classification, surpassing similar NCD-based approaches. In prokaryotic gene labeling tasks, NCD shows superior classification accuracy to traditional alignment or exact-match tools in out-of-distribution settings, while also outperforming comparable sequence-embedding methods in in-distribution classification. However, the computational complexity of O(MN) (in standard big-O notation, where M and N denote the sizes of the training and test databases, respectively) constrains scalability to very large datasets, though these findings nonetheless demonstrate that compression-based approaches provide an effective alternative for genomic sequence classification, particularly in low-data environments.