Finding transcripts associated with prostate cancer gleason stages using next generation sequencing and machine learning techniques

Abstract

Prostate cancer is a leading cause of death world-widely and the third leading cause of cancer death in Northen American men. Prostate cancer causes parts of the prostate cells to lose normal control of growth and division. The Gleason classification system is one of the known systems used to grade the aggressiveness of the prostate progression. In this study, an RNA-Seq dataset of 104 prostate cancer patients with different Gleason stages is analyzed using machine learning techniques to identify transcripts that are linked to prostate progression. The proposed method utilizes information gain as a ranker for feature selection to overcome the curse of dimensionality, because of dealing with a large number of features (41,971 transcripts). Minimum redundancy maximum relevance (MRMR) feature selection was applied on a one-versus-all hierarchical classification model to find the best subset of transcripts that predicts each stage. The Naive Bayes classifier was used at each node of the hierarchical model. Naive Bayes is compared with support vector machine (SVM) for accuracy as performance measure. The results suggest that Naive Bayes outperforms SVM as a classifier in the hierarchical model. Several transcripts are found to be highly associated with different Gleason stages in prostate cancer patients.