Hi @PitVis2023Participants,
The team participation instructions are as follows:
(1) Create a team on Synapse by clicking the team icons menu on the left panel.
(2) Complete the team participation form: [team participation form](https://forms.gle/jAZQqk353RV1fevK7).
For (1), the team creation instructions can be downloaded here: [team participation instructions](https://www.synapse.org/#!Synapse:syn52386208), or the written instructions can be found in the next post of this thread.
The final models will be submitted via docker submissions on Synapse. Further instructions will be given next week, and the submission deadline for this will be at least one week after these instructions have been provided. The aim is Tue 12 Sep for instructions, with a Tue 19 Sep deadline for submission. A model description form must also be filled in at that point, this can be found here: [model description form](https://forms.gle/NMo17MadvpMoE2Mo9).
Created by Adrito Das dreets @PitVis2023Participants Submissions are now finalised. Thank you to all those who have submitted.
I will be running your models on the test data and will let you know the final scores next week. - @Stephen17 9740966 is now set to "OPEN".
- @a.perezr20 9741007 and 9741008 are correct submissions. I noticed you no longer have submitted for task 2 - please confirm this is correct.
- @simongeek 9741012 and 9741011 are correct submissions. For 9741009 (task 2) the following error is given:
File "/opt/submission_inference_resnet_lstm_tools.py", line 115, in
SystemExit(main(pt_input=args["pt_input"], pt_output=args["pt_output"]))
File "/opt/submission_inference_resnet_lstm_tools.py", line 42, in main
model = ResnetLSTM(num_frames= 5) # HERE total frames number
File "/opt/models/resnet_lstm.py", line 25, in __init__
self.feature_extractor = FeatureExtractor()
File "/opt/models/resnet_lstm.py", line 64, in __init__
state_dict = torch.load("/home/jk/EndoVis_PitVis2023/code/models/checkpoints/embeddings_66_0.5465_0.5614_0.5319_0.5995.pth")
File "/usr/local/lib/python3.9/site-packages/torch/serialization.py", line 791, in load
with _open_file_like(f, 'rb') as opened_file:
File "/usr/local/lib/python3.9/site-packages/torch/serialization.py", line 271, in _open_file_like
return _open_file(name_or_buffer, mode)
File "/usr/local/lib/python3.9/site-packages/torch/serialization.py", line 252, in __init__
super().__init__(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: '/home/jk/EndoVis_PitVis2023/code/models/checkpoints/embeddings_66_0.5465_0.5614_0.5319_0.5995.pth' @dreets The status of my latest submission 9740966 is still "received", and has not been updated to "Open". @PitVis2023Participants
All runs are complete - please check https://www.synapse.org/#!Synapse:syn51232283/wiki/621588 for the status of your submissions, please comment below if you see any mistakes. Re-submissions to fix errors by those who have already submitted will be allowed until 23:59 BST on Mon 25 Sep 2023, all other submission will be excluded. Please also make sure you have completed the google form otherwise your submission will not be valid.
For task 1 there are a total of 8 valid submissions (2 fixes); for task 2 there 6 (2 fixes); for task 3 there are 5 (2 fixes).
@simongeek (9740984, 9740976, 9740983) Apologies, I believe I sent you the wrong command before, could you please try "docker build --platform linux/amd64 -t {image_name}" and re-submit.
@a.perezr20 (9740986, 9741006, 9741002) The following error is received (below is for task 1 but similar for the others):
Namespace(batch_size=2, num_workers=10, pt_input='/home/adrito/workspace/pitvis/docker_example/code/data/inputs/01', pt_output='/home/adrito/workspace/pitvis/docker_example/code/data/outputs/01.csv')
[09/24 23:49:46][INFO] checkpoint.py: 251: Loading network weights from ./model_weights/checkpoint_best_phases.pyth.
[09/24 23:49:46][INFO] ava_helper.py: 115: Finished loading image paths from: /home/adrito/workspace/pitvis/docker_example/code/data/inputs/01
[09/24 23:49:46][INFO] ava_dataset.py: 75: === PitVis dataset summary ===
[09/24 23:49:46][INFO] ava_dataset.py: 76: Split: val
[09/24 23:49:46][INFO] ava_dataset.py: 77: Number of videos: 1
[09/24 23:49:46][INFO] ava_dataset.py: 81: Number of frames: 5
[09/24 23:49:46][INFO] ava_dataset.py: 82: Number of key frames: 5
Traceback (most recent call last):
File "I_steps_recognition/run_net.py", line 29, in
main()
File "I_steps_recognition/run_net.py", line 26, in main
launch_job(cfg=cfg, init_method=args.init_method, func=test)
File "/opt/I_steps_recognition/utils/misc.py", line 313, in launch_job
func(cfg=cfg)
File "/opt/I_steps_recognition/test_net.py", line 198, in test
test_loader = loader.construct_loader(cfg, "val")
File "/opt/I_steps_recognition/datasets/loader.py", line 87, in construct_loader
dataset.__getitem__(343)
File "/opt/I_steps_recognition/datasets/ava_dataset.py", line 187, in __getitem__
video_idx, sec = self._keyframe_indices[idx]
IndexError: list index out of range @PitVis2023Participants
Hi all,
Unfortunately I have had to change table permissions so the docker_checker.py script will not work. Please submit before the end of the day, and I will run the checker script myself. You will then have 24 hours to correct any issues.
If you run your own docker command, as outlined in the pdf, and it works there, it should work on our end. Hello @dreets we have the same problem with permissions
synapseclient.core.exceptions.SynapseHTTPError: 403 Client Error:
User lacks READ_PRIVATE_SUBMISSION access to Evaluation 9615415 Hello @dreets
We are getting permission errors when trying to run our submissions:
synapseclient.core.exceptions.SynapseHTTPError: 403 Client Error:
User lacks READ_PRIVATE_SUBMISSION access to Evaluation 9615415
@dreets Thanks. But I notice that the status of submission **9740966** has not been updated to **"Open"**. For today's run:
- @litianbin (9740971, 9740972, 9740973): Please re-submit following the correct naming convention as outlined in the submission pdf. Check the other names on https://www.synapse.org/#!Synapse:syn51232283/wiki/621588 if unsure.
- @YouPANG (9740975): Correct submission.
- @Stephen17 (9740966, 9740967, 9740968): Correct submissions.
- @simongeek (9740970): Failed submission - the following error was detected: "Docker run output: standard_init_linux.go:228: exec user process caused: exec format error. Docker run error: WARNING: The requested image's platform (linux/arm64/v8) does not match the detected host platform (linux/amd64) and no specific platform was requested" To solve this please re-build your docker with the following command: "docker build --platform linux/arm64 -t pitvis_task2_sano:v3 ." and re-submit. For tasks 2 & 3:
- @simongeek (9740923): Please re-submit following the correct naming convention as outlined in the submission pdf.
- @SKJP (9740935, 9740934): Correct submissions.
- @Stephen17 (9740946, 9740947): Correct submissions.
For task 1:
- @YouPANG (9740937): The outputs are a 9-integer vector, can you please check this.
- @Stephen17 (9740941): Correct submission.
- @abdulkcl (9740948): Please re-submit following the correct naming convention as outlined in the submission pdf. This is also true for your task 2 (9740949) submission.
@PitVis2023Participants Due to requests from several groups, we have extended the submissions to **Sun 24 Sep 2023 23:59 BST**.
I would also like to confirm that "out_of_patient" frames will not be used as inputs in the test dataset.
I have updated the docker_checker.py script such that now you can check your specific docker submission id. Please pull the most recent version of https://github.com/dreets/pitvis and run docker_checker.py in the code directory with (str_user, str_pass, int_id) inputs - you can check your input id in the "Id" column in https://www.synapse.org/#!Synapse:syn51232283/wiki/621588.
@dreets
I've updated the submission. But, please ignore ID9740932 in Task2 and ID9740933 in Task3. These were uploaded by mistake.
@PitVis2023Participants
For all participants, please pull the most recent version of https://github.com/dreets/pitvis and run docker_checker.py in the code directory to make sure your script complies before submission.
I have run the docker script and you should have received automated emails:
- Submissions are from @TSONCT and @Uniandes23 are valid.
- @SKJP please check your submission names, and replace "-" with "_".
Hi @PitVis2023Participants - docker submissions are now open!
(1) Submit your docker image on Synapse: [PitVis docker submission instructions.](https://www.synapse.org/#!Synapse:syn52418202)
(2) Complete the [model description form](https://forms.gle/NMo17MadvpMoE2Mo9).
Submission are open until **Thu 21 Sep 2023 23:59 BST**. Submissions after this deadline will **not** be accepted.
For each submission, only the latest submitted entry by the team lead will be considered as the final submission.
We will be running the docker test script once per day - please check your submission on the tables: https://www.synapse.org/#!Synapse:syn51232283/wiki/621588. An email notification will also be sent after this script is run. Thanks @SKJP - I have changed the permission, hopefully this should work now. Dear organizers,
I cannot find the"create a new team" button in "PitVis teams" page. Also, I cannot download "PitVis team participation instructions.pdf" file. Would you check the status ? Instructions on creating a team in Synapse, written:
(1) From the PitVis synapse homepage [PitVis home](https://www.synapse.org/#!Synapse:syn51232283/wiki/), click the teams icon on the left panel.
(2) Click on ?create a new team?, found on the right side of this page.
(3) To invite a new user click on ?Team Actions? found in the top-right of the page, and click ?Invite User?.
(4) Type in the name or email address of the synapse user, click on their name in dropdown menu, and click ?Send Invitation(s)?.
(5) All other team actions can be found under ?Team Actions? in the top-right menu.
(6) Once you have created your team, please complete the team participation google form, found here: [team participation form](https://forms.gle/jAZQqk353RV1fevK7).
A visual guide to these instructions can be found here: [team participation instructions](https://www.synapse.org/#!Synapse:syn52386208).