Good evening,
I have read through the submission guide, and I think we need to have a folder called "submission" under our team folder. Then, we need to put our final model code workbook and the output prediction dataset of the model in the same folder, and a report as well. However, I'm wondering whether we can put additional files in the folder? I've described my questions in detail below.
As said in the report template, to include the model interpretation figure/table in the report, we need to directly "Send to report" on the enclave platform. Can we put the workbook/contour/data_output that generates the interpretation figure and table also in the team submission folder, or just in the team main folder? Or, must we implement the transforms of model testing/figure/table generating also on the submitted model workbook (all in the master branch)?
I am asking this because most interpretation "explainers" such as Shapley need to be fitted on the training set and explain the prediction/feature importance on an independent test set. Throughout the model-building process, we also need a test set to tune the model. So, we left out part of the censored training data as our own "test set". Although our model will be refitted on the whole censored training set before submission, our interpretation figures/tables can only be generated using our own "test set" since the provided censored test set cannot be used (too small and no label). Therefore, I'm wondering should we also put the workbook/contour/data_output, where we tune our model and generate the interpretation methods, in the submission folder?
Also, is the official end date 11:59PM December 15 EST?
Thanks a lot in advance for your help.
Created by Changyu Yin yinchangyu Hi @yinchangyu,
That should be fine, so long as you're specific about which transforms in the supplementary code workbook need to be run in order to complete the main submission.
Thanks!
Tim Hi @trberg ,
Thanks for your prompt reply!
Based on your suggestion, could please help see whether the following is acceptable:
(1) We put another code workbook in the "submission" folder and call it maybe like "[team name]_supplementary" where we did feature selection, model tuning, model testing, and interpretation.
(2) As for the main submission model, we put a main submission code workbook in the "submission" folder, in which
1. It is a clean version where we fitted the model on the entire censored training set and put the censored test set as a placeholder.
2. We just use the hyperparameter and features selected directly we got from (1)
3. We'll clearly document them in the report.
I am also wondering if we can import a dataset into (2) generated in (1) indicating which features were selected?
Thanks again!
Best,
Changyu Hi @yinchangyu,
First, the official end date is 5PM EST on December 15th.
You **do not** have to implement your interpretation methods in your main submitted model workbook, they can be in a separate workbook/contour/data_output. But this separate implementation does need to be in your "submission" folder. If you do decide to create a separate interpretation analysis, please make sure you clearly title your workbooks and include instructions in your write up.
Let me know if you have further questions!
Tim
Drop files to upload
Put intermediate output/workbook in submission folder? page is loading…