For example: ``` bash> curl -O http://adult.psychencode.org/Datasets/Pipeline/HiC_matrices/Header_PIP-01_DLPFC.10kb.txt.tar.gz 25666 Jan 14 09:05 Header_PIP-01_DLPFC.10kb.txt.tar.gz bash > tar ztvf Header_PIP-01_DLPFC.10kb.txt.tar.gz -rw-rw-r-- 0 hwon hwon 4592772900 Aug 1 2017 DLPFC.chr10.10kb.txt tar: Truncated input file (needed 4592773120 bytes, only 0 available) tar: Error exit delayed from previous errors. ``` It is a safe bet that a 25k tarball will not contain a 4.5G text file!

Created by Paul Shannon paul-shannon
Hi Ramil, In the Supplementary section, we direct the reader to a previous publication where the analysis is described in greater detail: H. Won, L. de la Torre-Ubieta, J. L. Stein, N. N. Parikshak, J. Huang, C. K. Opland, M. J. Gandal, G. J. Sutton, F. Hormozdiari, D. Lu, C. Lee, E. Eskin, I. Voineagu, J. Ernst, D. H. Geschwind, Chromosome conformation elucidates regulatory relationships in developing human brain. Nature 538, 523?527 (2016). doi:10.1038/nature19847 Please take a look at this. If there are still additional questions, I can forward them to my colleague who carried out the processing and we will get back to you. Best wishes, Prashant.
@prashantemani - can you field this question as well? Appreciate your help
Sorry, I have found the indication of hg19 assembly. The paper is "Comprehensive functional genomic resource and integrative model for the human brain" The authors used hiclib (https://bitbucket.org/mirnylab/hiclib) to process the data as they mentioned in Supplementary data. However there is no information about the exact commands they run, how they normalized the matrices.
Dear Colleagues I have small followup questions. I searched any HiC data from brain tissue. I have found these matrices at http://resource.psychencode.org/ and downloaded them. Can you please share with me information how these data were generated? Any link to pipeline, genome version, sample type. Yours sincerely Ramil Nurtdinov, PhD
Hi Paul, I apologize for the long delay! Apart from being side-tracked, I have also been trying to get new versions of all the files uploaded. Please try the new files in: http://resource.psychencode.org/Datasets/Pipeline/HiC_matrices/ Also, please note that the Header-* files are just that: simple extractions of the 1st 20 lines of one of the sub files, put up on the server to give the user a sense for the format of the underlying files. The PIP-01 and PIP02 files are the full set of Hi-C matrices. Please do let me know if there are any issues accessing these new files. I shall endeavor to correct any issues. Best wishes, Prashant.
Thanks, Prashant. I might be missing something obvious here. Forgive me if that is so! I just downloaded Header_PIP-02_DLPFC.40kb.txt 14-Jan-2019 16:28 1.6M after which I run into the same problems as before: ``` khaleesi.tmp> uname -a Linux khaleesi.systemsbiology.net 4.4.0-141-generic #167-Ubuntu SMP Wed Dec 5 10:40:15 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux khaleesi.tmp> curl -O http://adult.psychencode.org/Datasets/Pipeline/HiC_matrices/Header_PIP-02_DLPFC.40kb.txt % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1655k 100 1655k 0 0 1262k 0 0:00:01 0:00:01 --:--:-- 1261k khaleesi.tmp> file Header_PIP-02_DLPFC.40kb.txt Header_PIP-02_DLPFC.40kb.txt: POSIX tar archive (GNU) khaleesi.tmp> tar tvf Header_PIP-02_DLPFC.40kb.txt -rw-r--r-- hjwon/root 287133025 2018-02-24 02:04 DLPFC.chr10.40kb.txt tar: Unexpected EOF in archive tar: Error is not recoverable: exiting now ``` So it appears that version 02, despite its file extension, is a gnu tar archive - but one which is malformed. Or :} I am seriously confused! - Paul
Hi Paul, The repaired versions are available at: http://resource.psychencode.org/Datasets/Pipeline/HiC_matrices/ Let me know if you have any issues accessing them. What I can do is to put those repaired files explicitly on our website, resource.psychencode.org, for general use. Best wishes, Prashant.
Thank you, Prashant! I obtained the old, flawed files from http://adult.psychencode.org/Datasets/Pipeline/HiC_matrices/ Where can I find your repaired versions within synapse? - Paul P.S. If you have a communication channel with the PsychENCODE team, perhaps they would like to use your fixed versions, replacing their old flawed ones?
The files are named: Header_PIP-01_DLPFC.10kb.txt Header_PIP-02_DLPFC.40kb.txt
I looked into the issue. To be clear, those tar.gz files were not put on the website explicitly and were left over in the folders of the website from an earlier iteration, and thus were not checked and updated. However, I have removed those files and directly put the text versions of the Header files in the same folder. These files should work.
@prashantemani - can you please take a look at this issue. This is direct download from the adult.psychencode.org site. Here is the complete error http://adult.psychencode.org/Datasets/Pipeline/HiC_matrices/ http://adult.psychencode.org/Datasets/Pipeline/HiC_matrices/Header_PIP-01_DLPFC.10kb.txt.tar.gz curl -O http://adult.psychencode.org/Datasets/Pipeline/HiC_matrices/Header_PIP-01_DLPFC.10kb.txt.tar.gz 25666 Jan 14 09:05 Header_PIP-01_DLPFC.10kb.txt.tar.gz file Header_PIP-01_DLPFC.10kb.txt.tar.gz Header_PIP-01_DLPFC.10kb.txt.tar.gz: gzip compressed data, last modified: Tue Aug 1 19:58:47 2017, from Unix, original size 170901504 tar ztvf Header_PIP-01_DLPFC.10kb.txt.tar.gz -rw-rw-r-- 0 hwon hwon 4592772900 Aug 1 2017 DLPFC.chr10.10kb.txt tar: Truncated input file (needed 4592773120 bytes, only 0 available) tar: Error exit delayed from previous errors.

Pipeline/HiC_matrices/Header_PIP*.tar.gz files appear to be ill-formed page is loading…