Deep Learning 2019/2020, EEMCS TU Delft
Authors: Anastasis Alexopoulos, Guilherme Fonseca and Miguel Cardoso
In this blog post, we aim at describing the process behind reproducing the results from the paper "Deep learning with multimodal representation for pancancer prognosis predictions" by Anika Cheerla and Olivier Gevaert, published in July 2019 [1]. In this work, the researchers developed neural networks to predict the survival time of cancer patients by combining data from various sources.
In the following sections, we present in more details the technical aspects of the aforementioned paper and the steps we implemented in order to reproduce its results. In fact, our purpose is to describe our attempts, successes and failures, as well as explain every technical decision we had to make throughout this study.
To learn more about other reproducibility project like this one, click here.
Disclaimer: The goal of this blog post is to assess the reproducibility of the paper. In no way we intend to discredit the results obtained by the authors.
In a clinical setting, physicians have access to a wide range of different data in order to estimate the potential survival of a cancer patient. The high-dimensionality of the data together with the availability of vast information makes it difficult to decipher any robust survival pattern. This is where Machine Learning techniques, and in particular Deep Learning ones, can be helpful in finding directions of clinical interest.
The main focus of the research paper published by Cheerla and Gevaert [1] was to develop a Deep Learning algorithm that computes a prognosis for the survival of cancer patients using data from different types of cancer. The different data examined in this publication are highly heterogenous, combining gene expressions, miRNA expressions, clinical data and histopathological whole slide images (WSI). Gene and miRNA expressions are molecular information, the clinical data includes patients' details such as gender, age, tumor stage, and the WSI are high-resolution tissue images. The complexity of these datasets was exploited by developing a multimodal neural network that includes a specific architecture for each data-type and combines them into a unique feature vector. This feature vector is then used for prognosis prediction. The authors acquired the data from the publicly available database of the "The Cancer Genome Atlas" project [2].
Four neural networks were suggested and implemented by the authors. The goal was to design a structure which could couple the different data modalities (miRNA, gene expression, clinical data, WSI), while being able to process different input dimensions as well as handling any missing data. Each network outputs a 512 feature vector, as shown in Figure 1.