Greetings! This is Bo Li, member of Team Winterfell. We have noticed some issues with the current evaluation metric used for the quantification subchallenge when we ran the data provided from round 1. The evaluation metric used is a Spearman's rank correlation over all annotated isoforms. In our test, we found this metric generally gives poor correlations because it includes many low expressors which we do not have enough data to get reliable estimation. Therefore, we wonder if the committee can consider some alternative metrics that might be better for the evaluation purpose. For example, one commonly used metric for evaluating quantifiers is the log-transformed Pearson correlation. Plots of percent errors, which are used in the [RSEM paper](http://bmcbioinformatics.biomedcentral.com/articles/10.1186/1471-2105-12-323)), can also be applied. Or we can calculate the area under curve values from ROC curves, as [Irizarry lab](https://genomebiology.biomedcentral.com/articles/10.1186/s13059-016-0940-1) had done. Irizarray lab's recent paper, [A benchmark for RNA-seq quantification pipelines](https://genomebiology.biomedcentral.com/articles/10.1186/s13059-016-0940-1), gives a very good discussion about different metrics for evaluating RNA-Seq transcript quantifiers. Best, Bo