Image-Based TF Colorization with CNN for Direct Volume Rendering

Abstract

In the direct volume rendering (DVR), it often takes a long time for a novice to manipulate the transfer function (TF) and analyze the volume data. To lessen the difficulty in volume rendering, several researchers have developed deep learning techniques. However, the existing techniques are not easy to apply directly to existing DVR pipelines. In this study, we propose an image-based TF colorization with CNN to automatically generate a direct volume rendering image (DVRI) similar to a target image. Our system includes CNN model training, TF labeling, image-based TF generation, and volume rendering by matching the target image. We introduce a technique for training CNN and labeling the TF with images similar to the input volume dataset. Moreover, we extract the primary colors from the target image according to the labels classified with the CNN model. We render the volume data with the TF to produce the DVRI reproducing the prominent colors in the target image.