Select-ProtoNet: Learning to Select
for Few-Shot Disease Subtype Prediction

Ziyi Yang Affiliation: Macau University of Science and Technology Email: yangziyi091100@163.com Jun Shu Affiliation: Xi’an Jiaotong University Email: xjtushujun@gmail.com Yong Liang Affiliation: Macau University of Science and Technology Email: yliang@must.edu.mo Deyu Meng Affiliation: Xi’an Jiaotong University Email: dymeng@mail.xjtu.edu.cn Zongben Xu Affiliation: Xi’an Jiaotong University Email: zbxu@mail.xjtu.edu.cn

Abstract

Current machine learning has made great progress on computer vision and many other fields attributed to the large amount of high-quality training samples, while it does not work very well on genomic data analysis, since they are notoriously known as small data. In our work, we focus on few-shot disease subtype prediction problem, identifying subgroups of similar patients that can guide treatment decisions for a specific individual through training on small data. In fact, doctors and clinicians always address this problem by studying several interrelated clinical variables simultaneously. We attempt to simulate such clinical perspective, and introduce meta learning techniques to develop a new model, which can extract the common experience or knowledge from interrelated clinical tasks and transfer it to help address new tasks. Our new model is built upon a carefully designed meta-learner, called Prototypical Network [35], that is a simple yet effective meta learning machine for few-shot image classification. Observing that gene expression data have specifically high dimensionality and high noise properties compared with image data, we proposed a new extension of it by appending two modules to address these issues. Concretely, we append a feature selection layer to automatically filter out the disease-irrelated genes and incorporate a sample reweighting strategy to adaptively remove noisy data, and meanwhile the extended model is capable of learning from a limited number of training examples and generalize well. Simulations and real gene expression data experiments substantiate the superiority of the proposed method for predicting the subtypes of disease and identifying potential disease-related genes.

1 Introduction

Disease subtype prediction is to identify subgroups of similar patients that can guide treatment decisions for a specific individual [29, 40]. For instance, in the past 15 years, five subtypes of breast cancer have been identified and intensively studied [36]. At the level of molecular biology, the use of gene expression data to predict disease subtypes is of great significance for improving the accuracy of disease diagnosis and identifying potential disease-related genes. However, one of the challenging problem is that the gene expression data are notoriously known as small data [33], i.e., we only have a relatively small number of samples for each disease subtype. The small data learning or few-shot learning has recently attracted many researches in machine learning community [33, 35]. This inspires us to bring the state-of-the-art algorithms of few-shot learning to genomic data analysis.

In our paper, we attempt to deal with few-shot disease subtype prediction problem. The most used method in genomic data is to augment sample size by aggregating data from multiple studies under comparable conditions or treatments [17, 10, 46]. This strategy commonly encounters bottlenecks due to the complex properties of gene expression data, in which the aggregation of data from different platforms or experiments inevitably suffers from batch effects, heterogeneity, and other sources of bias [14]. Besides, we observe that the current methods usually consider only the subtype prediction task for a particular disease, without taking into account several clinical variables simultaneously that are often of concern to both doctors and clinicians. To simulate the process of doctors and clinicians studying disease subtype prediction, we introduce the meta learning techniques to develop a new data efficient model, which can extract the shared experience or knowledge from a series of related tasks, and rapidly transfer it to the new tasks [5, 34, 9]. Therefore, the basic idea of the proposed new model is to learn from interrelated clinical tasks through meta learning techniques to extract the valuable information to help model generalize to the disease subtype prediction task well.

Prototypical Network (ProtoNet) [35] is a carefully designed meta-learner that is a simple and yet effective meta learning machine for few-shot image classification. It tries to learn a metric space in which classification can be performed by computing distances to prototype representations of each class. However, unlike images, gene expression data are much harder to analyze due to their high-dimensional and high-noise properties. The curse of dimensionality problems tends to make predictions become more challenging since a large number of redundant features involving in decision. Besides, existing evidence suggests that a high level of technical or biological noise inevitably exists in gene expression data [22, 31], which tends to easily occur overfitting issue and lead to poor generalization performance. To address these issues, we propose Select-ProtoNet, a new extension of ProtoNet by appending two modules, feature selection layer and sample selection net, making it filter out the disease-irrelated genes and remove noisy data towards few-shot disease subtype prediction.

Our proposed method is built on top of ProtoNet, and we additionally append a feature selection layer to automatically cherry-pick the disease-related genes, and incorporate a sample reweighting strategy to adaptively suppress the negative influence of noisy data. With these two modules, our method can perform robust to high-dimensional and high-noise gene expression data, and allows for better generalization on small data benefited from ProtoNet.

Refer to caption — Figure 1: *(Left)* (a) Diagram of the Select-ProtoNet structure. The weighted prototype $\mathbf{c}_{n}$ is calculated by weighted representation of all the embedded support samples for class $n$ . The blue boxes represent the newly appending modules to ProtoNet. *(Right)* Qualitative example of how Select-ProtoNet works. Assume a query sample $x$ belongs to class $i$ . (b) The prototypes $\mathbf{c}_{n}$ are calculated by the mean of embedded support samples for each class. The closet prototype to the query sample $x$ is $\mathbf{c}_{j}$ . (c) The prototypes $\mathbf{c}_{n}$ are calculated by weighted representation of all the embedded support samples for each class. The closet prototype to the query sample is now the one of the class $i$ .

The key contributions of this paper can be summarized as follows:

•

We propose a new extensions of ProtoNet, called Select-ProtoNet, for few-shot disease subtype prediction, as shown in Fig. 1(a). To our best knowledge, this should be the first work to use meta-learning techniques for Bioinformatics. By additionally appending two modules, feature selection layer and sample selection net, our model can adaptively select important features and clean samples, as well as learn from small data and generalize well.
•

The additionally appending modules and the vanilla ProtoNet are updated in a unified framework, which can be easily implemented on the basis of the ProtoNet.
•

To address the absence of benchmark datasets for testing few-shot classification in the field of genomics, we devise a new dataset, miniTCGA Meta-Dataset, which is derived from TCGA Meta-Dataset [28]. This dataset can be used as a benchmark for meta learning methods in genomics applications under few-shot learning setting.
•

We experimentally show the superiority of the proposed method for predicting the subtypes of disease and identifying potential disease-related genes.

The remaining sections are organized as follows. Section 2 reviews the related work. Section 3 detailedly introduces the proposed method. Section 4 demonstrates simulation and real data experimental results and the conclusion is finally made.

2 Related Work

Data Integration. Genomics data integration is usually used method to deal with small sample problem [13]. It can be roughly performed through two different strategies: (1) ‘meta-analysis’ [27, 26, 42] analyzes each dataset independently and finally combines the statistic results to identify disease-related genes; (2) ‘integrative analysis via data merging’ [32, 16, 10] first aggregates samples from different datasets into a unified large dataset and then analyzes the new integrated dataset. However, integrating data from different experiments or platforms for integrative analysis easily suffers from batch effects and heterogeneity, which is still very challenging in computational biology.

Few-shot Learning. Meta-learning has a prominent history in machine learning [1, 30, 37], and many meta learning methods have been applied to address the small data learning or few-shot learning [34]. They can be roughly divided into two main types: (1) gradient-based optimization methods [5, 6, 23, 15] learn a meta-learner in the outer loop to initialize a base-learner for the inner loop that is then trained on a novel few-shot task; (2) metric-based methods [39, 35, 24, 41, 43] learn a metric space of sample features in which classification can be efficient with few samples. Most current methods rely solely on image classification tasks. There is a lack of systematic researches about using meta-learning for few-shot disease subtype prediction problems. In this work, we extend a well-known meta-learner, Prototypical Networks [35], to address biological problems. Our model appends two modules that performs robust to high-dimensional and high-noise gene expression data.

Learning with High-Dimensional Features. Feature extraction and feature selection are two commonly used manners to address the curse of dimensionality problems. Feature extraction transforms the original features into a set of new features through subspace learning [21]. However, a reasonable biological interpretation is difficult to obtain from the learned feature subspace. Feature selection removes irrelevant, redundant data, and selects a set of important features that are significantly related to the objective [14]. Regularization technique is a typical feature selection method [38, 18, 44, 49]. However, they usually have assumptions about the prior distribution of the data [45] and inevitably involve hyper-parameters (i.e., regularization parameters) required to be tuned by cross-validation.

Sample Reweighting for Noisy Samples. The main idea of sample reweighting strategy is to impose weights on samples based on their reliability for training. Typically methods include self-paced learning (SPL) [12] and its variations [11, 47], iterative reweighting [3, 48], FWL [4], L2RW [25], and Meta-Weight-Net (MW-Net) [33]. They incline to suppress the effects of samples with extremely large loss values, possibly with high noisy samples.

3 Proposed Select-ProtoNet for Few-Shot Disease Subtype Prediction.

In this section, we first present preliminaries in the meta learning setting with independent episodic tasks and the prototypical networks (ProtoNet) method for few-shot learning, then give an overview of our approach, finally elaborate the details of our method.

3.1 Meta-Learning for Few-Shot Learning

Few-shot learning (FSL) involves building a model using available training data of seen classes that can classify unseen novel classes using only few examples. Under the FSL setting, we have a large sample set $\mathcal{D}_{s}$ from a set of source classes $\mathcal{C}_{s}$ , while a few-shot sample set $\mathcal{D}_{t}$ from a set of target classes $\mathcal{C}_{t}$ , and a test set $\mathcal{T}$ from $\mathcal{C}_{t}$ , where $\mathcal{C}_{s}\cap\mathcal{C}_{t}=\emptyset$ . The goal of the FSL is to training a classification model with $\mathcal{C}_{s}$ that can generalize well to $\mathcal{T}$ .

We then define the episodic training strategy widely used by existing meta-learning based FSL models [39, 35]. Concretely, we can define a series of $n$ -way $k$ -shot tasks randomly sampled from $\mathcal{D}_{s}$ , and each $n$ -way $k$ -shot task is defined as an episode $D=(S,Q)$ . Generally, we call $S$ the support set containing $n$ classes and $k$ samples per class, and $Q$ the query set with the same $n$ classes. In our paper, we try to construct a new meta dataset of FSL for disease subtype prediction in biological field. The episode $D$ can be constructed by the following process: we first select a small set of source class containing $n$ classes $\mathcal{C}=\{\mathcal{C}_{i}|i=1,2,\cdots,n_{c}\}$ from $\mathcal{C}_{s}$ , and then generate $S$ and $Q$ by randomly sampling $k$ support samples and $q$ query samples from each class in $C$ , respectively. Therefore, we have $S=\{(x_{i},y_{i})|y_{i}\in C,i=1,2,\cdots,m_{s}\}$ , and $Q=\{(x_{i},y_{i})|y_{i}\in C,i=1,2,\cdots,m_{q}\}$ , where $m_{s}=n\times k$ , $m_{q}$ is the number of samples for query set in each episodic, and $S\cap Q=\emptyset$ .

We denote the classifier we want to learn as $p_{\phi}(y|x)$ , with parameter $\phi$ , which outputs a probability of a data point belonging to the class $y$ given the data sample $x$ . The classifier is trained between its predicted labels and the ground truth labels over the query set $Q$ with different episodes such that it can generalize to other datasets, the objective function is written as follows:

\phi=\arg\min_{\phi}\mathbb{E}_{\mathcal{D}\subset\mathcal{D}_{s}}\sum_{(x,y)\in Q}-\log p_{\phi}(y|x,S),

(1)

where different forms of $p_{\phi}(y|x,S)$ define different kinds of meta learning algorithm.

3.2 Review of Prototypical Networks

Here, we will give a introduction to the prototypical networks (ProtoNet), which is a well-known meta-learning algorithm, and our model is built on top of it. ProtoNet learns a prototype of each class in the support set $S$ and classifies each sample in the query set $Q$ based on its distances to different prototypes. Thus $p_{\phi}(y|x,S)$ in ProtoNet is defined as follows:

\displaystyle p_{\phi}(y|x,S)=\frac{\exp\left(-d\left(g_{\phi}(x),\mathbf{c}_{n}\right)\right)}{\sum_{n^{\prime}}\exp\left(-d\left(g_{\phi}(x),\mathbf{c}_{n^{\prime}}\right)\right)},

(2)

where $d$ represents the Euclidean distance metric in the feature space, and $\mathbf{c}_{n}$ is prototype for every episode. Eq.2 means that ProtoNet produces the class distributions of a query sample $x$ based on the softmax output w.r.t. the distance between $g_{\phi}(x)$ and the class prototype $\mathbf{c}_{n}$ , where $g_{\phi}(x)$ is the embedding function which maps a sample $x$ to the feature vector $g_{\phi}(x)$ . Each prototype $\mathbf{c}_{n}$ is calculated by averaging the vectors of all the embedded support samples $\mathcal{S}_{n}$ belonging to the class $n$ , given by the following form:

\mathbf{c}_{n}=\frac{1}{\left|\mathcal{S}_{n}\right|}\sum_{\left(x_{i},y_{i}\right)\in\mathcal{S}_{n}}g_{\phi}\left(x_{i}\right),

(3)

where $\mathcal{S}_{n}\in S$ denotes the set of support samples that belong to the class $n$ .

3.3 Learning to Select Important Features and Clean Data for Disease Subtype Prediction

We attempt to introduce ProtoNet to disease subtype prediction problem to enhance the capability of few-shot recognition. However, the gene expression data is harder to deal with compared with image dataset, since for its high dimensionality and high noise properties. To further overcome these issues, we proposed to append two modules (as shown in Fig.1 (a)), i.e., a feature selection layer (FS-Layer) to automatically filtering out the disease-irrelated genes and incorporate an adaptive sample reweighting strategy (MW-NetV2) against data noise, building up on the ProtoNet.

3.3.1 Feature Selection Layer

For each sample $x\in\mathbb{R}^{p}$ of gene expression data, the feature dimension $p$ is high. Generally, feature selection methods try to find a selection vector $\beta=(\beta_{1},\beta_{2},\cdots,\beta_{p})$ , which is element-wise multiplication with data $x$ , to filter out the useless feature and obtain a new representation of data $x_{new}$ to help the following tasks perform well, i.e.,

\displaystyle x_{new}=\beta\odot x,\beta_{j}\in[0,1].

(4)

The regularization technique, the effective method on this problem, often manually set a specific form of regularization under a certain assumption on training data, which is infeasible when we know little knowledge underlying gene expression data. To overcome this, we model the selection vector as a Softmax network layer as follows, which can learn an adaptive feature weighting vector from data,

\displaystyle x_{new}=\beta(\theta)\odot x\triangleq f_{\theta}(x),\beta_{i}(\theta)=\frac{\exp(\theta)_{i}}{\sum_{j}\exp(\theta)_{j}},\sum_{i}\beta_{i}(\theta)=1,

(5)

where $\theta\in\mathbb{R}^{p}$ is the parameter vector of Softmax layer, and $\exp$ is the element-wise exponential operator. It can be be easily embedded into Eq.(2),

\displaystyle p_{\phi,\theta}(y|x,S)=\frac{\exp\left(-d\left(g_{\phi}(f_{\theta}(x)),\mathbf{c}_{n}\right)\right)}{\sum_{n^{\prime}}\exp\left(-d\left(g_{\phi}(f_{\theta}(x)),\mathbf{c}_{n^{\prime}}\right)\right)},

(6)

where $\mathbf{c}_{n}$ can be rewritten as:

\mathbf{c}_{n}=\frac{1}{\left|\mathcal{S}_{n}\right|}\sum_{\left(x_{i},y_{i}\right)\in\mathcal{S}_{n}}g_{\phi}\left(f_{\theta}(x_{i})\right),

(7)

This formulation needs no expert-level knowledge to character $\beta$ , and $\theta$ can efficiently learned by automatic differentiation techniques as $\phi$ does, which makes it easy to scale to other problems.

Algorithm 1 Training episode loss computation for Select-ProtoNet.

N

is the number of examples in the training set,

K

is the number of classes in the training set,

n_{c}\leq K

is the number of classes per episode,

k

is the number of support examples per class,

q

is the number of query examples per class. RandomSample(

S

n

) denotes a set of

n

elements chosen uniformly at random from set

S

, without replacement. This algorithm is built upon ProtoNet [35], and the revised parts are shown in blue.

0: Training set

\mathcal{D}=\{(x_{1},y_{1}),\ldots,(x_{N},y_{N})\}

, where each

y_{i}\in\{1,\ldots,K\}

\mathcal{D}_{k}

denotes the subset of

\mathcal{D}

containing all elements

(x_{i},y_{i})

such that

y_{i}=n

0: The loss

\mathcal{J}(\phi,\theta,\Theta)

for a randomly generated training episode.

V\leftarrow\textsc{RandomSample}(\{1,\dots,K\},n_{c})

\triangleright

Select class indices for episode

2: for

n

in {

1,\dots,n_{c}

} do

\mathcal{S}_{n}\leftarrow\textsc{RandomSample}(\mathcal{D}_{V_{n}},k)

\triangleright

Select support examples

\mathcal{Q}_{n}\leftarrow\textsc{RandomSample}(\mathcal{D}_{V_{n}}\setminus\mathcal{S}_{n},q)

\triangleright

Select query examples

\mathbf{c}_{n}\leftarrow\frac{1}{\left|\mathcal{S}_{n}\right|}\sum_{\left(x_{i},y_{i}\right)\in\mathcal{S}_{n}}\mathcal{V}(g_{\phi}(f_{\theta}(x_{i}));\Theta)\cdot g_{\phi}\left(f_{\theta}(x_{i})\right)

\triangleright

Compute the weighted prototype

6: end for

\mathcal{J}(\phi,\theta,\Theta)\leftarrow 0

\triangleright

Initialize loss

8: for

n

in {

1,\dots,n_{c}

} do

9: for

(x,y)

\mathcal{Q}_{n}

10:

\mathcal{J}(\phi,\theta,\Theta)\leftarrow\mathcal{J}(\phi,\theta,\Theta)+\frac{1}{n_{c}\cdot q}\left[d\left(g_{\phi}\left(f_{\theta}(x)\right),\mathbf{c}_{n}\right)+\log\sum_{n^{\prime}}\exp\left(-d\left(g_{\phi}\left(f_{\theta}(x)\right),\mathbf{c}_{n^{\prime}}\right)\right)\right]

\triangleright

Update loss

11: end for

12: end for

3.3.2 Sample Selection Net

There exist high noise in gene expression data, which can easily lead to poor performance in generalization. To overcome this, instead of the simple average strategy in ProtoNet, we attempt to assign weights to support samples to character the clean confidence of data, expecting to suppress the effects of samples with extremely noise. Specifically, the final prototype $\mathbf{c}_{n}$ should be calculated by weighted representation of all the embedded support samples for class $n$ , i.e.,

\displaystyle\mathbf{c}_{n}=\frac{1}{\left|\mathcal{S}_{n}\right|}\sum_{\left(x_{i},y_{i}\right)\in\mathcal{S}_{n}}v_{i}\cdot g_{\phi}\left(f_{\theta}(x_{i})\right),v_{i}\in[0,1],

(8)

where $v_{i}$ reflect the confidence of the support sample $x_{i}$ belonging to clean data. In general, large weights $v$ are more likely to be high-confident ones with clean data.

To determine the $v$ , inspired by current adaptive sample weighting strategy Meta-Weight-Net [33], we attempt to learn a weighting function to distinguish clean and noisy data. Specifically, MW-Net models the sample weights $v$ as an MLP network $\mathcal{V}(\ell;\Theta)$ with only one hidden layer, which is a universal approximator for almost any continuous function and thus can fit a wide range of weighting functions. Its input is the loss function of the sample, and output is the weight to this sample. Since ProtoNet does not compute the losses of the support samples, we use the embedding representation $g_{\phi}\left(f_{\theta}(x)\right)$ as input of MW-Net. Therefore, the Eq.(8) function can be furtherly rewritten as:

\displaystyle\mathbf{c}_{n}=\frac{1}{\left|\mathcal{S}_{n}\right|}\sum_{\left(x_{i},y_{i}\right)\in\mathcal{S}_{n}}\mathcal{V}(g_{\phi}(f_{\theta}(x_{i}));\Theta)\cdot g_{\phi}\left(f_{\theta}(x_{i})\right),

(9)

where $\Theta$ represents the parameters of MW-NetV2 (to distinction with the original MW-Net), and the architecture of MW-NetV2 is same with MW-Net, except for the input. The input of MW-NetV2 is the embedding representation for each sample instead of the loss in MW-Net.

3.3.3 Learning Algorithm

By additionally appending feature selection layer (FS-Net) and sample selection net (MW-NetV2) to vanilla ProtoNet, the proposed method, Select-ProtoNet, can simultaneously cherry-pick the disease-related genes helping classification and suppress the negative influence of noisy data. The learning procedure is similar to vanilla ProtoNet, and the parameters of FS-Net and MW-NetV2 are simultaneously updated with the parameters of ProtoNet. The final loss function is:

	$\displaystyle\phi,\theta,\Theta=\arg\min_{\phi,\theta,\Theta}\mathbb{E}_{\mathcal{D}\subset\mathcal{D}_{s}}$	$\displaystyle\sum_{(x,y)\in Q}-\log p_{\phi,\theta,\Theta}(y\|x,S),$		(10)
	$\displaystyle p_{\phi,\theta,\Theta}(y\|x,S)=\frac{\exp\left(-d\left(g_{\phi}(f_{\theta}(x)),\mathbf{c}_{n}\right)\right)}{\sum_{n^{\prime}}\exp\left(-d\left(g_{\phi}(f_{\theta}(x)),\mathbf{c}_{n^{\prime}}\right)\right)},$	$\displaystyle\mathbf{c}_{n}=\frac{1}{\left\|\mathcal{S}_{n}\right\|}\sum_{\left(x_{i},y_{i}\right)\in\mathcal{S}_{n}}\mathcal{V}(g_{\phi}(f_{\theta}(x_{i}));\Theta)\cdot g_{\phi}\left(f_{\theta}(x_{i})\right).$		(11)

Fig. 1(right) illustrates an example on how the proposed method works. The pseudocode for calculating the episode training loss $\mathcal{J}(\phi,\theta,\Theta)$ is provided in Algorithm 1.

4 Experiments

To evaluate the capability of the proposed method, Select-ProtoNet, we conduct experiments on simulated data sets and real gene expression datasets. We show that Select-ProtoNet outperforms its backbone methods and several conventional methods.

4.1 Simulation Experiments

Simulated Datasets. We construct two sets of data $\mathcal{D}_{train}$ and $\mathcal{D}_{test}$ with distinct sets of classes under the FSL setting. The generation of simulated data is refer to the work in [20]. For more detailed information on generating simulated datasets, see Appendix A.

Noise and Feature Dimension Settings. We consider four-level settings of corrupted class labels on the support set. The class label of each support sample is independently changed to a random class with probability $p$ , where $p=0\%,10\%,30\%,50\%$ . Moreover, we consider four-level settings of irrelevant feature with $d$ dimension, where $d=100,500,1000,2000$ .

Baselines. We compare against two family of methods. The first is metric-based meta learning methods: ProtoNet and ProtoNet with either of the two appended modules. ‘SelectF-ProtoNet’ and ‘SelectS-ProtoNet’ are models with feature selection layer and sample selection net, respectively. The second is the conventional methods, usually used to analyze gene expression data to predict disease subtypes. The conventional comparison methods include: Support Vector Machines (SVM), Naive Bayes, Logistic Regression, Logistic regression with Lasso penalty (Logistic_lasso), Random Forest, NeuralNet [20], and AffinityNet [20]. Details of baseline implementations, training and test procedures can be found in Appendix B.

Results. Table 1 shows the accuracy of Select-ProtoNet and its backbones on simulated datasets under different experiment settings with $30$ random runs. It can be seen that the appended two modules both make contributions to improve the performance, and the proposed method achieves the best results. With the increase of noise rate, the advantages of Select-ProtoNet over ProtoNet become more obvious. As shown in Fig. 4, the curves of training loss and accuracy comparison between ProtoNet and Select-ProtoNet. We can observe that ProtoNet takes almost twice the time of our model to achieve the best classification accuracy. To illustrate the effect of two appended modules of our model, we plot the weight distribution of clean and noisy training support samples in Fig. 4. It can be seen that almost all lager weights belong to clean samples, and the weight value of noisy samples is less than that of clean samples, which implies that MW-NetV2 can distinguish clean and noisy samples. Fig. 5(c) shows the feature weights learned by Select-ProtoNet. It can be observed that the weights of important features are higher than those of irrelevant features, and almost all the weights of irrelevant features are less than a certain threshold value, indicating that the feature selection layer can select important features.

We also compare Select-ProtoNet with state of the art conventional methods. Fig. 4 shows the test accuracy of all competing methods on the few selected training samples (only 1% of datasets). It can be seen that our model is significantly superior to all other competing methods. There are big performance gaps between our model and conventional methods (i.e., Random Forest, NeuralNet, SVM, and Logistic Regression) when training data is small. Particularly, NeuralNet, is the worst performer because the training pool is quite small that the power of deep learning can only be manifested when a large amount of data is available. In addition, we plot the weights of features learned by NeuralNet and AffinityNet, are shown in Fig 5. We can observe that feature selection performance of our model outperforms competing methods. We further increase the feature dimension to evaluate the classification accuracy of all comparison methods. The results are shown in Table 2 with $30$ random runs. As can be seen, Select-ProtoNet can improve the accuracy compared with ProtoNet, and outperforms all other conventional comparison methods. With the increase of the feature dimension, the performance gaps between the proposed method and all competing methods increase gradually.

Table 1: Few-shot classification accuracy (%) comparison on simulated data with varying experiment settings. The best results are highlighted in bold.

Noise rate	Model	5 way 5 shot	5 way 10 shot	10 way 5 shot	10 way 10 shot
0%	ProtoNet	91.93 $\pm$ 3.08	92.30 $\pm$ 2.61	93.01 $\pm$ 1.81	93.40 $\pm$ 1.30
	SelectS-ProtoNet	92.74 $\pm$ 1.42	93.29 $\pm$ 1.39	93.70 $\pm$ 1.21	94.30 $\pm$ 1.40
	SelectF-ProtoNet	96.55 $\pm$ 1.75	96.78 $\pm$ 1.02	97.42 $\pm$ 0.48	97.21 $\pm$ 0.51
	Select-ProtoNet	96.69 $\pm$ 1.08	96.94 $\pm$ 0.98	97.44 $\pm$ 0.41	97.62 $\pm$ 0.47
10%	ProtoNet	83.23 $\pm$ 4.90	82.25 $\pm$ 4.10	85.27 $\pm$ 3.41	86.38 $\pm$ 4.00
	SelectS-ProtoNet	84.30 $\pm$ 4.44	84.49 $\pm$ 3.73	88.26 $\pm$ 2.81	89.89 $\pm$ 2.59
	SelectF-ProtoNet	91.17 $\pm$ 3.59	92.29 $\pm$ 2.05	92.62 $\pm$ 2.24	92.96 $\pm$ 2.00
	Select-ProtoNet	93.56 $\pm$ 2.78	94.40 $\pm$ 1.34	94.67 $\pm$ 1.55	94.31 $\pm$ 1.73
30%	ProtoNet	72.33 $\pm$ 7.42	73.98 $\pm$ 6.59	74.02 $\pm$ 6.43	74.52 $\pm$ 6.13
	SelectS-ProtoNet	75.74 $\pm$ 6.21	76.34 $\pm$ 4.92	77.64 $\pm$ 7.53	77.81 $\pm$ 6.00
	SelectF-ProtoNet	86.56 $\pm$ 4.48	87.07 $\pm$ 5.49	87.72 $\pm$ 4.53	86.29 $\pm$ 6.22
	Select-ProtoNet	88.95 $\pm$ 3.74	89.66 $\pm$ 4.02	89.87 $\pm$ 3.85	89.60 $\pm$ 4.38
50%	ProtoNet	64.35 $\pm$ 7.99	65.95 $\pm$ 8.01	66.93 $\pm$ 7.60	66.84 $\pm$ 6.86
	SelectS-ProtoNet	68.69 $\pm$ 6.42	68.79 $\pm$ 7.99	70.22 $\pm$ 7.75	70.90 $\pm$ 6.30
	SelectF-ProtoNet	79.04 $\pm$ 9.33	80.85 $\pm$ 7.44	81.30 $\pm$ 7.08	81.47 $\pm$ 8.95
	Select-ProtoNet	83.42 $\pm$ 6.06	83.21 $\pm$ 5.59	84.49 $\pm$ 5.33	84.43 $\pm$ 5.54

Figure 2: Training loss and accuracy (%) vs. number of iterations under unbiased data.

Figure 3: Sample weight distribution on training data under 30% noise rate.

Figure 4: Box-plot diagram of test accuracy (%) of all competing methods over

30

repetitions.

(a) NeuralNet

(b) AffinityNet

Figure 5: Plots of feature weights learned by the competing methods on unbiased simulated data. The red bar represents the true features and the blue bar represents the irrelevant features.

Table 2: Test accuracy (%) comparison on simulated data with varying feature dimension settings. The best results are highlighted in bold.

Methods	100	500	1000	2000
SVM	40.79 $\pm$ 1.47	30.84 $\pm$ 0.82	29.34 $\pm$ 1.08	27.95 $\pm$ 0.73
Naive Bayes	76.91 $\pm$ 5.03	54.19 $\pm$ 5.34	48.17 $\pm$ 3.07	40.47 $\pm$ 3.10
Random Forest	52.23 $\pm$ 9.47	26.43 $\pm$ 2.48	26.21 $\pm$ 2.71	25.73 $\pm$ 1.63
Logistic Regression	41.59 $\pm$ 1.86	30.83 $\pm$ 0.92	29.36 $\pm$ 1.07	27.97 $\pm$ 0.56
Logistic_lasso	72.96 $\pm$ 4.72	59.24 $\pm$ 4.78	49.65 $\pm$ 5.94	42.31 $\pm$ 5.60
NeuralNet	32.42 $\pm$ 1.50	27.61 $\pm$ 0.75	26.29 $\pm$ 0.85	25.92 $\pm$ 0.66
AffinityNet	66.06 $\pm$ 25.13	39.17 $\pm$ 13.30	27.55 $\pm$ 6.98	34.18 $\pm$ 10.66
ProtoNet	80.48 $\pm$ 4.68	71.90 $\pm$ 2.64	54.09 $\pm$ 10.95	48.66 $\pm$ 6.29
Select-ProtoNet	96.08 $\pm$ 0.35	93.13 $\pm$ 2.44	85.40 $\pm$ 5.12	81.49 $\pm$ 6.10

4.2 Experiments on miniTCGA Meta-Dataset

Real Datasets. TCGA Meta-Dataset [28] is a publicly available benchmark dataset in the field of gene expression analysis, containing 174 clinical tasks derived from The Cancer Genome Atlas (TCGA) [19], which can be used in a multi-task learning framework. However, it cannot be directly used as a few-shot learning benchmark dataset because some tasks have extremely imbalance classes. Thus, after carefully selecting samples, we devise a new dataset, mini-TCGA Meta-Dataset, consisting of 68 TCGA benchmark clinical tasks, where each task has two classes and each class has at least 60 samples. More details about datasets can be found in Appendix D.

Baselines. We compare against the following state-of-art methods: ProtoNet, the majority class prediction (Majority), Logistic Regression, and Neural Network. The implementations of three conventional methods are the same as the work in [28]. For details on implementations, training and test procedures, see Appendix E.

Results. The classification accuracy of miniTCGA Meta-Dataset with varying noise rate settings are reported in Table 3. As can be seen, our model improves on its backbone method with a large margin for all noise rate settings, and outperforms the three conventional supervised methods.

Table 4 shows the lung cancer subtype prediction accuracy of all competing methods on the TCGA Meta-Dataset with task id (‘Expression_Subtype’, ‘LUNG’). Especially, ProtoNet and Select-ProtoNet consider the interrelationship of clinical tasks, leveraging samples from all clinical tasks of miniTCGA Meta-Dataset to build shared experience or knowledge and transfer it to help predict the lung cancer subtype. The conventional methods, however, only consider the lung cancer subtype task. As shown in Table 4, we can observe that our model achieves the best results, with an accuracy gain of more than 20% compared with its backbone, and over 29% against the best result of the conventional methods. It implies that our model significantly improves the prediction performance of disease subtypes by incorporating various interrelated clinical tasks. Fig. 6 shows the Kaplan-Meier (KM) survival curves of the 20 top-ranked significant genes selected by Select-ProtoNet on the lung cancer subtype task of TCGA Meta-Dataset. By using bBioPortal [2, 7], the survival analysis of significant genes is done based on the Pan-Cancer Atlas dataset [8]. As shown in Fig. 6, the two curves without intersect (logrank test $p$ -value $=5.801e-4$ ). The patients without alterations in the selected genes (blue line) have long survival times.

Table 3: Test accuracy (%) comparison on miniTCGA Meta-Dataset with varying noise rate settings. The mean accuracy (

\pm

std) over 30 repetitions are reported. The best results are highlighted in bold.

Noise Rate	Conventional supervised methods			Select-ProtoNet and its backbone
Noise Rate	Majority	Logistic Regression	Neural Network	ProtoNet	Select-ProtoNet
0%	64.10 $\pm$ 8.83	67.96 $\pm$ 12.80	68.30 $\pm$ 11.63	72.01 $\pm$ 19.91	84.71 $\pm$ 4.49
10%	63.63 $\pm$ 9.38	65.08 $\pm$ 10.49	64.85 $\pm$ 9.98	66.69 $\pm$ 19.54	81.33 $\pm$ 2.32
30%	61.98 $\pm$ 9.94	60.89 $\pm$ 8.00	61.50 $\pm$ 8.89	64.18 $\pm$ 19.48	79.89 $\pm$ 2.40
50%	59.73 $\pm$ 9.56	57.06 $\pm$ 5.72	58.73 $\pm$ 7.59	60.72 $\pm$ 17.31	78.15 $\pm$ 2.30

Table 4: Accuracy (%) comparison on the lung cancer subtype task of TCGA Meta-Dataset.

Method	Majority	Logistic Regression	Neural Network	ProtoNet	Select-ProtoNet
Accuracy	38.00 $\pm$ 0.00	60.64 $\pm$ 8.23	68.20 $\pm$ 1.60	77.78 $\pm$ 27.22	97.78 $\pm$ 3.44

5 Conclusion

In this paper, we propose a novel meta learning method for few-shot disease subtype prediction problem. The proposed method, carefully designs to append two modules to the vanilla Prototypical Network, making it able to address the high dimensionality and high noise issues in gene expression data compared with image data. Meanwhile, it possesses the ability of Prototypical Network to extract the shared experience or knowledge from interrelated clinical tasks, capable of learning from a limited number of training examples and generalize well. Synthetic and a new released benchmark dataset for disease subtype prediction, mini-TCGA Meta-Dataset experiments substantiate the superiority of the proposed method for predicting the subtypes of disease and identifying potential disease-related genes. In future work, we need to further consider the interaction between genes and integrate gene regulatory networks into meta-learning techniques and further promote the application of meta-learning techniques in the field of few-shot biological genomics, thus bringing us closer to the clinic of the future.

References

[1] Samy Bengio, Yoshua Bengio, Jocelyn Cloutier, and Jan Gecsei. On the optimization of a synaptic learning rule. In Preprints Conf. Optimality in Artificial and Biological Neural Networks, volume 2. Univ. of Texas, 1992.
[2] Ethan Cerami, Jianjiong Gao, Ugur Dogrusoz, Benjamin E Gross, Selcuk Onur Sumer, Bülent Arman Aksoy, Anders Jacobsen, Caitlin J Byrne, Michael L Heuer, Erik Larsson, et al. The cbio cancer genomics portal: an open platform for exploring multidimensional cancer genomics data, 2012.
[3] Fernando De La Torre and Michael J Black. A framework for robust subspace learning. International Journal of Computer Vision, 54(1-3):117–142, 2003.
[4] Mostafa Dehghani, Arash Mehrjou, Stephan Gouws, Jaap Kamps, and Bernhard Schölkopf. Fidelity-weighted learning. arXiv preprint arXiv:1711.02799, 2017.
[5] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 1126–1135. JMLR. org, 2017.
[6] Chelsea Finn, Kelvin Xu, and Sergey Levine. Probabilistic model-agnostic meta-learning. In Advances in Neural Information Processing Systems, pages 9516–9527, 2018.
[7] Jianjiong Gao, Bülent Arman Aksoy, Ugur Dogrusoz, Gideon Dresdner, Benjamin Gross, S Onur Sumer, Yichao Sun, Anders Jacobsen, Rileen Sinha, Erik Larsson, et al. Integrative analysis of complex cancer genomics and clinical profiles using the cbioportal. Sci. Signal., 6(269):pl1–pl1, 2013.
[8] Katherine A Hoadley, Christina Yau, Toshinori Hinoue, Denise M Wolf, Alexander J Lazar, Esther Drill, Ronglai Shen, Alison M Taylor, Andrew D Cherniack, Vésteinn Thorsson, et al. Cell-of-origin patterns dominate the molecular classification of 10,000 tumors from 33 types of cancer. Cell, 173(2):291–304, 2018.
[9] Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. Meta-learning in neural networks: A survey. arXiv preprint arXiv:2004.05439, 2020.
[10] Jacob J Hughey and Atul J Butte. Robust meta-analysis of gene expression using the elastic net. Nucleic acids research, 43(12):e79–e79, 2015.
[11] Lu Jiang, Deyu Meng, Teruko Mitamura, and Alexander G Hauptmann. Easy samples first: Self-paced reranking for zero-example multimedia search. In Proceedings of the 22nd ACM international conference on Multimedia, pages 547–556, 2014.
[12] M Pawan Kumar, Benjamin Packer, and Daphne Koller. Self-paced learning for latent variable models. In Advances in Neural Information Processing Systems, pages 1189–1197, 2010.
[13] Cosmin Lazar, Stijn Meganck, Jonatan Taminau, David Steenhoff, Alain Coletta, Colin Molter, David Y Weiss-Solís, Robin Duque, Hugues Bersini, and Ann Nowé. Batch effect removal methods for microarray gene expression data integration: a survey. Briefings in bioinformatics, 14(4):469–490, 2013.
[14] Cosmin Lazar, Jonatan Taminau, Stijn Meganck, David Steenhoff, Alain Coletta, Colin Molter, Virginie de Schaetzen, Robin Duque, Hugues Bersini, and Ann Nowe. A survey on filter techniques for feature selection in gene expression microarray analysis. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 9(4):1106–1119, 2012.
[15] Yoonho Lee and Seungjin Choi. Gradient-based meta-learning with learned layerwise metric and subspace. arXiv preprint arXiv:1801.05558, 2018.
[16] Jeffrey T Leek, Robert B Scharpf, Héctor Corrada Bravo, David Simcha, Benjamin Langmead, W Evan Johnson, Donald Geman, Keith Baggerly, and Rafael A Irizarry. Tackling the widespread and critical impact of batch effects in high-throughput data. Nature Reviews Genetics, 11(10):733–739, 2010.
[17] Quefeng Li, Sijian Wang, Chiang-Ching Huang, Menggang Yu, and Jun Shao. Meta-analysis based variable selection for gene expression data. Biometrics, 70(4):872–880, 2014.
[18] Yong Liang, Cheng Liu, Xin-Ze Luan, Kwong-Sak Leung, Tak-Ming Chan, Zong-Ben Xu, and Hai Zhang. Sparse logistic regression with a l 1/2 penalty for gene selection in cancer classification. BMC bioinformatics, 14(1):198, 2013.
[19] Jianfang Liu, Tara Lichtenberg, Katherine A Hoadley, Laila M Poisson, Alexander J Lazar, Andrew D Cherniack, Albert J Kovatich, Christopher C Benz, Douglas A Levine, Adrian V Lee, et al. An integrated tcga pan-cancer clinical data resource to drive high-quality survival outcome analytics. Cell, 173(2):400–416, 2018.
[20] Tianle Ma and Aidong Zhang. Affinitynet: semi-supervised few-shot learning for disease type prediction. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 1069–1076, 2019.
[21] Fotios D Mandanas and Constantine L Kotropoulos. Subspace learning and feature selection via orthogonal mapping. IEEE Transactions on Signal Processing, 2020.
[22] Jonathan M Raser and Erin K O’Shea. Noise in gene expression: origins, consequences, and control. Science, 309(5743):2010–2013, 2005.
[23] Sachin Ravi and Hugo Larochelle. Optimization as a model for few-shot learning. 2016.
[24] Mengye Ren, Eleni Triantafillou, Sachin Ravi, Jake Snell, Kevin Swersky, Joshua B Tenenbaum, Hugo Larochelle, and Richard S Zemel. Meta-learning for semi-supervised few-shot classification. arXiv preprint arXiv:1803.00676, 2018.
[25] Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urtasun. Learning to reweight examples for robust deep learning. arXiv preprint arXiv:1803.09050, 2018.
[26] Daniel R Rhodes and Arul M Chinnaiyan. Integrative analysis of the cancer transcriptome. Nature genetics, 37(6s):S31, 2005.
[27] Daniel R Rhodes, Jianjun Yu, K Shanker, Nandan Deshpande, Radhika Varambally, Debashis Ghosh, Terrence Barrette, Akhilesh Pandey, and Arul M Chinnaiyan. Large-scale meta-analysis of cancer microarray data identifies common transcriptional profiles of neoplastic transformation and progression. Proceedings of the National Academy of Sciences, 101(25):9309–9314, 2004.
[28] Mandana Samiei, Tobias Würfl, Tristan Deleu, Martin Weiss, Francis Dutil, Thomas Fevens, Geneviève Boucher, Sebastien Lemieux, and Joseph Paul Cohen. The tcga meta-dataset clinical benchmark. arXiv preprint arXiv:1910.08636, 2019.
[29] Suchi Saria and Anna Goldenberg. Subtyping: What it is and its role in precision medicine. IEEE Intelligent Systems, 30(4):70–75, 2015.
[30] Jürgen Schmidhuber. Evolutionary principles in self-referential learning, or on learning how to learn: the meta-meta-… hook. PhD thesis, Technische Universität München, 1987.
[31] Jörn M Schmiedel, Lucas B Carey, and Ben Lehner. Empirical mean-noise fitness landscapes reveal the fitness impact of gene expression noise. Nature communications, 10(1):1–12, 2019.
[32] Andrey A Shabalin, Håkon Tjelmeland, Cheng Fan, Charles M Perou, and Andrew B Nobel. Merging two gene-expression studies via cross-platform normalization. Bioinformatics, 24(9):1154–1160, 2008.
[33] Jun Shu, Qi Xie, Lixuan Yi, Qian Zhao, Sanping Zhou, Zongben Xu, and Deyu Meng. Meta-weight-net: Learning an explicit mapping for sample weighting. arXiv preprint arXiv:1902.07379, 2019.
[34] Jun Shu, Zongben Xu, and Deyu Meng. Small sample learning in big data era. arXiv preprint arXiv:1808.04572, 2018.
[35] Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In Advances in Neural Information Processing Systems, pages 4077–4087, 2017.
[36] Bo Hwa Sohn, Jun-Eul Hwang, Hee-Jin Jang, Hyun-Sung Lee, Sang Cheul Oh, Jae-Jun Shim, Keun-Wook Lee, Eui Hyun Kim, Sun Young Yim, Sang Ho Lee, et al. Clinical significance of four molecular subtypes of gastric cancer identified by the cancer genome atlas project. Clinical Cancer Research, 23(15):4441–4449, 2017.
[37] Sebastian Thrun. Lifelong learning algorithms. In Learning to learn, pages 181–209. Springer, 1998.
[38] Robert Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodological), 58(1):267–288, 1996.
[39] Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. In Advances in neural information processing systems, pages 3630–3638, 2016.
[40] Bo Wang, Aziz M Mezlini, Feyyaz Demir, Marc Fiume, Zhuowen Tu, Michael Brudno, Benjamin Haibe-Kains, and Anna Goldenberg. Similarity network fusion for aggregating data types on a genomic scale. Nature methods, 11(3):333, 2014.
[41] Yu-Xiong Wang, Ross Girshick, Martial Hebert, and Bharath Hariharan. Low-shot learning from imaginary data. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7278–7286, 2018.
[42] Pratyaksha Wirapati, Christos Sotiriou, Susanne Kunkel, Pierre Farmer, Sylvain Pradervand, Benjamin Haibe-Kains, Christine Desmedt, Michail Ignatiadis, Thierry Sengstag, Frédéric Schütz, et al. Meta-analysis of gene expression profiles in breast cancer: toward a unified understanding of breast cancer subtyping and prognosis signatures. Breast Cancer Research, 10(4):R65, 2008.
[43] Chen Xing, Negar Rostamzadeh, Boris Oreshkin, and Pedro OO Pinheiro. Adaptive cross-modal few-shot learning. In Advances in Neural Information Processing Systems, pages 4848–4858, 2019.
[44] Zongben Xu, Hai Zhang, Yao Wang, XiangYu Chang, and Yong Liang. L 1/2 regularization. Science China Information Sciences, 53(6):1159–1169, 2010.
[45] Bing Xue, Mengjie Zhang, Will N Browne, and Xin Yao. A survey on evolutionary computation approaches to feature selection. IEEE Transactions on Evolutionary Computation, 20(4):606–626, 2015.
[46] Zi-Yi Yang, Xiao-Ying Liu, Jun Shu, Hui Zhang, Yan-Qiong Ren, Zong-Ben Xu, and Yong Liang. Multi-view based integrative analysis of gene expression data for identifying biomarkers. Scientific reports, 9(1):1–15, 2019.
[47] Zi-Yi Yang, Liang-Yong Xia, Hui Zhang, and Yong Liang. Mspl: Multimodal self-paced learning for multi-omics feature selection and data integration. IEEE Access, 7:170513–170524, 2019.
[48] Zhilu Zhang and Mert Sabuncu. Generalized cross entropy loss for training deep neural networks with noisy labels. In Advances in neural information processing systems, pages 8778–8788, 2018.
[49] Hui Zou and Trevor Hastie. Regularization and variable selection via the elastic net. Journal of the royal statistical society: series B (statistical methodology), 67(2):301–320, 2005.

Select-ProtoNet: Learning to Select for Few-Shot Disease Subtype Prediction