Hi @allawayr ,
Is there a docker size limit of this challenge or synapse system? I failed to upload an 8GB docker to synapse and docker push keeps retrying. (It works well when I push a smaller docker e.g. 1GB.) Any suggestions about this?
Thanks,
Hongyang
Created by Hongyang Li arcanum449 Thank you very much, @brucehoff @allawayr ! I can push large docker images to synapse now. Thanks, @brucehoff ! Dear All: We have fixed the problem with large Docker images. You can now push an image larger than 10GB. If there are further issues, please let us know. Hi Hongyang,
This is not a limit specific to this challenge, but rather a limit on the Synapse Docker repository. I think that it is unlikely that our platform team will be able to change this limit within the timeframe of the challenge.
Are your large models pre-trained on the training data provided for this challenge? We have configured the challenge queues to allow people to train on the scoring server, so if this is a possibility, it may allow you to submit a smaller container that can be trained during evaluation. I'll see if there are any other workarounds, but I'm not optimistic there are any that we'll be able to implement before this challenge closes.
Thanks,
Robert
Hi Robert,
I have some large models and I'd like to test them on the leaderboard. Is it possible to expand the size limit of docker to e.g. 10-20GB for this challenge? Thanks!
Best,
Hongyang Hi there,
I don't yet have a direct answer from our platform team, but looking through old bug reports it sounds like there is a limit that affects containers down to 7 gb in size that causes the issue you are describing. The actual limit is 10 GB, but I think there are differences in how this size is determined on a docker repository vs the local output of `docker images`.
My suggestion, if possible, would be to try to reduce the size of your container to less than 7 GB and try again.
Thanks,
Robert
Thanks, @allawayr
It did not report an error, but got stuck during pushing. I can see the progress of uploading e.g. 0GB/8GB -> 4GB/8GB -> ... Then at the final step 8GB/8GB, it could not finish and said "Retrying in 8 seconds", then started from 0GB/8GB again..
It is similar to this post:
[docker push keeps retrying and failing](https://github.com/docker/distribution/issues/2042)
but I'm not sure how to solve it. Hi there,
Do you get any error message? I don't think there should be a size limit but let me confer with the people who manage the synapse docker repo.