Let's continue [this thread](syn6156761/discussion/threadId=1073) here -- long threads don't always load correctly on synapse.

Created by Daniel Marbach daniel.marbach
Hi Daniel, I hope that the DREAM Conference was conducted successfully. I would be grateful if you can now share the scripts for evaluation as per your earlier suggestion. Regards Raghvendra
Congratulations to the best player. I wrote a short note about spectral clustering to help myself understand it. https://fairmiracle.github.io/st/spectr.html Even after many years spectral clustering is still proven to be efficient.
Congratulations to the winners. [This](https://arxiv.org/abs/1103.0368) paper can be tried to combine networks in sub-challenge 2. I find the premise very promising. It'd be great for one (or all) of the top performing teams to test their methods by combining networks using methods outlined in this paper.
yes Ragh- we have cleaned up scripts to be a single bashable file from input to sanity check and submission; a brief description and location was also added in write up.
Hi Yuanfang, Congratulations on your runner up position. I was going through your write up and found that no where you have mentioned the link to your codes. I think one of the terms for a team to be declared a winner (or runner-up) is to have their code available for others to see and reproduce results. I would be grateful if you can share your code. My team (Resham) is also using Louvain method as base algorithm and performing several strategies on top of it and we have a decent performance in the Final leaderboard but not as good as yours. So, we just wanted to compare with you method and understand the parameters you have used. Regards, Raghvendra
thanks daniel. That explains a lot.   I am involved with this club for several years. As I said in my write-up, we did the best this spring, and trained the only finalist in INTEL talent search in Michigan State (we are not MA or california, and obviously we cannot compete with their education system. ), who is now in Harvard undergrad. In general, I found it a much more enjoyable and much less complicated process to work with kids than working with grown-ups. This is one of the things I am actively exploring if i am truly interested in, in case i lose job from university because of performance, economy, earthquake, nuclear war, etc.. even Richard Feynman once taught small kids what is infinity, at least in the film...   This high school club participated in DREAM several years ago. They once ranked #5 in one of the sub-challenges and **beat over the majority of the PhD. teams including my own team**. This challenge should be the best one they ranked so far (**and thank you again for the runner-up; this recognition is very important for us**), and we will pick a relatively easy one next summer, and to break this record.  
Yuanfang, thank you, I think it's great that you are mentoring high school students. The subsampling only shows the variation when you subsample the GWAS set used for evaluation, but the input data (networks) and predicted modules are kept fixed (i.e., there is no need to re-run Pascal). If we would subsample the networks and re-run the module identification and scoring with Pascal, the variation would be much higher (as your approach also showed). However, just by switching java versions we would not get a different best performer: Team tusk performed best at each FDR cutoff, both on leaderboard and final GWAS set, and also was among the top performers in sub-challenge 2. How is it possible that the subsampling gives a Bayes factor >5 with Causality? It seems the score of Team Tusk is more robust to removing GWASs than the score of Causality. The score is the number of modules that show significant enrichment in *at least one* GWAS. This means that the significant modules of Team Tusk often show enrichment in more than one GWAS, so when you remove one of the GWAS, the module is still counted as a hit. We'll have to analyze this in more detail. Raghvendra, unfortunately I'm very busy with other things this week, but we'll try to make the scripts and data available ASAP, hopefully in the first week of November. Best Daniel
Hi Daniel, Thanks a lot for all your co-operation post the completion of the Challenge. I am the lead of team Resham which is next to Triple AHC (runner up of competition) at the official FDR cutoff and in the top 10 corresponding to 2.5% and 1% FDR cutoffs. We have made few major modifications to our method and wanted to ask if you can make the evaluation scripts (before the conference) available so that we can conduct an independent follow up research. This is because we are sure that the modifications we are making now will help to improve the method that we used as Final DREAM submission. Thanks a lot for your co-operation in advance. Regards, Raghvendra
Dear Daniel,   I represent all runner-ups to thank you.   I had to do this challenge under great emotional stress from other issues in the last few days and with scratch teaching of a high school student. Thus we are absolutely satisfied with the results. Here I represent my teammate, as well as the Ann Arbor Area High School Science Club GIDAS (Genes In Diseases And Symptoms), and Community High School to thank you as well.   But, (well, there is always a but), **I do not think the subsampling was done correctly**. **Because when you change from Java 7 to java 8, you can have 2 modules different. How is it even possible that CAUSALITY has 2 lower than top team, but has a Bayes factor >5? ** As I have said before, if it was in a different Java version, maybe CAUSALITY would have been top.   Thanks a ton for organizing this great challenge. Yuanfang

*** Best Performers *** (continued) page is loading…