I would like to verify the timestam values for the dataset.
The timestamp value for sensor measurements is Greenwich time (+00:00) and the timestamp provided for all the events input by the patient is in local time ( -05:00).
Am I correct?
Created by Tomasz Gutowski tgutowski Hi @ukhan.mscs18seecs can you create your own thread instead of jumping onto this one? It will help others who might have the same questions as you. @phil is the following correct to read the timestamp
pd.to_datetime(out['timestamp'],unit='s')
??
Also, while dealing with data-I, can we join on timestamp ? let me explain, so e.g i am getting data for subject = x , on day = 1 , and i want all the data from all three sensors in one pandas dataframe, so here i read the 3 different file (one for each sensor), and joined them on timestamp. Is this approach correct ? After refamiliarizing myself with the data, it seems that both of us were misled by the data format. So to readdress your first question:
> I noticed that in some cases the medicine intake times are not always provided in the same timezone as provided in Metadata of onboarding and visits
The timestamps in the Medication Diary table were correctly computed using either EST or EDT, respective of the date of medication. The `timezone` field in the Metadata of Patient Onboarding table has nothing to do with the timezone of the medications. If we reference the [dictionary](syn18376585) for the Metadata of Patient Onboarding table, we see that `timezone` is the recording (device) timezone. The devices were set to a fixed timezone to avoid ambiguous timestamps, so we have recorded that timezone here.
Likewise, the `timezone` field in the Metadata of Patient Onboarding table records the device timezone, as does the `timezone` field in the Metadata of Laboratory Visits table. I've clarified that this timezone information is for the devices in the respective data dictionary tables. Yes, for Medication Diary I have switch to the date and time columns, however, the Laboratory Visits and Patient Onboarding contain only the timestamp for medicine intake - in some cases the hour is provided in the comments but this is not always the case. The date and the 24-hour clock time were pulled directly from the study data (as was the timezone information in the Metadata of Laboratory Visits table, but we know the timezone is correct since we know how daylight savings time works). The timestamp was derived from the date and 24 hour clock time. --But it appears that DST wasn't taken into account by the method we used.-- We converted to unix timestamps in the metadata using Python's `dateutil.tz.gettz("America/New_York")`, but this does--n't-- take into account the date that it's operating on - i.e., whether that date is within daylight savings time or not. --I'm still trying to figure out where else this may have affected metadata timestamps, but until we get this fixed it is useful to keep in mind that the unix timestamps in the metadata were derived from the date and clock times, not the other way around.-- Thank you for the provided answers.
The example for #1 is 7_BOS with -5 one of the medication intake times is 18:00 and the timestamp provided in UTC is 22:00 which shows a 4h difference.
The information I provided "For one of the patients, the summer time change took place during the study." is wrong, I did not notice that the time change is different among countries. However, patient 2_BOS that participated before time change and patient 14_BOS who participated after have the same timezone assigned.
My question #2 is motivated by the fact when I filter the signal in the tremor range I sometimes get the following results :
For tremor of score = 0:
${imageLink?synapseId=syn25400708&align=None&scale=100&responsive=true&altText=Tremor 0}
For tremor of score = 2:
${imageLink?synapseId=syn25400707&align=None&scale=100&responsive=true&altText=Tremor 2}
Sometimes the values are correlated with the expectations but due to the small number of registered higher score values of tremor I am wondering what can be the reason to these differences. Can you provide a specific example or two for question #1? I will look into this.
Question #2 will need to be answered by someone who was involved in the testing, but I can answer question #3. The subject diary was parsed as-is. If the participant marked troublesome dyskinesia, but didn't mark dyskinesia, we didn't change the data to account for that. Thank you very much, I have three more questions.
1. I noticed that in some cases the medicine intake times are not always provided in the same timezone as provided in Metadata of onboarding and visits. In some cases the assigned timezone (-4 or -5) does not correspond to the actual time (Daylight Time). For one of the patients, the summer time change took place during the study. Are the provided timestamps according to the actual timezone or always according to the timezone provided in onboarding and visits?
2. The second question is regarding the Task Scores table. How were the scores for tremor and the presence of dyskinesia and bradykinesia assigned during the tasks? Especially sitting and standing, since these provide a clear tremor component? The reason why I am asking is that sometimes the tremor acquired based on the signal analysis seems to be very different than the assigned value.
3. The last question is about the Subject Diary of home tasks. The answers provided by the patients about the symptoms seem to be inconsistent sometimes. Especially the questions regarding dyskinesia and troublesome dyskinesia. Disregarding the answer provided for the question about the dyskinesia the answers provided for troublesome_dyskinesia are both true and false. So the patients were telling that they experience troublesome dyskinesia even if they did not experience the dyskinesia. Are these inconsistencies due to patients errors or was the troublesome dyskinesia question regarding to the dyskinesia experienced in general not only at the measurement time?
Thank you That's more or less correct. The devices record the timestamp as [Unix time](https://www.wikiwand.com/en/Unix_time) so we kept the time in that format. The patient sleep data was recorded in their local time ( -05:00) and so we kept the times in that format again.