BIDS to NIMH Data Archive (NDA)
Last updated
Last updated
To help users comply with data sharing requirements from the NIMH, we are extending a tool from the BIDS folks that automatically converts BIDS-formatted MR data into the format required by the NIMH. This tool is a work in progress, so please get in touch with any issues or suggested features!
If you are using XNAT and have worked with us to set up your protocol to be BIDS-friendly, follow these steps to pull your data from XNAT to Oscar and automatically convert it into BIDS format.
If you are converting fMRI task data, you will need to do some additional work to also convert your behavioral data into the _events.tsv files required by BIDS. You can find an example using our demodat dataset here.
If you have stimulus files that you would also like to include in your upload to the NDA, make sure to include stimulus information in the stim_file column of your _events.tsv files like this:
The BIDS spec assumes that these files live in a "stimuli" folder at the top level of your BIDS directory like this:
If you want to organize the stimuli further, you can have subfolders within your "stimuli" folder, and then include the subfolder in the stim_file field. For example, if you want to put all the stimuli sub-001 saw in a folder called "001" within the "stimuli" folder, your events.tsv file would have 001/001_stim1.png, etc. in the stim_file column.
NIMH requires that each participant be assigned a "Global Unique Identifier" or GUID, created with their tool. Once you have done this for each participant, you need to create a text file that contains each of your BIDS subject IDs and the associated GUIDs. If my BIDS directory has folders for sub-001 and sub-002, my GUID mapping file would look like this:
001 - LJFOIJWEL 002 - LKJSFIJLW
Save this file (i.e. guids.txt) in the same folder as your BIDS directory, but not inside the BIDS directory.
The NDA requires that every experiment be defined, and when approved, it will be assigned an ID number. Each task that your participants complete will need its own approved experiment ID (including resting state). Once you have received your experiment ID(s), you will need to create another mapping text file - this one providing the mapping between the task names in your fMRI BIDS filenames (i.e. sub-001_ses-01_task-checks_run-01_bold.nii.gz) and your approved experiment ID numbers. For example, if my participants completed a "checks" task that was assigned an ID of 9990, and a "rest" resting state scan that was assigned an ID of 9991, my experiment ID mapping file would look like this:
checks - 9990 rest - 9991
Save this file (i.e. expIDs.txt) in the same folder as your BIDS directory, but not inside the BIDS directory. If you don't provide this file in the BIDS2NDA conversion, the experiment_id column in the output image03.csv file will be left blank and you will need to fill it in manually for any fMRI rows.
Sometimes there is information that is missing or incorrect from the automatic BIDS conversion, that you may have available from another source. For example, the BIDS conversion approximates the interview_age in months from the age in years in the BIDS data. This age needs to exactly match any other data uploaded for the same participant (i.e. ndar_subject01.csv), so our bids2nda conversion tool allows you to specify a csv with additional data you would like to use to modify the image03.csv file.
Save this csv in the same folder as your BIDS directory, but not inside the BIDS directory.
To install in a Python virtual environment on Oscar:
Change to your home directory
cd ~
Create a new python environment called "bids2nda"
python -m venv bids2nda
Activate the new environment
source ~/bids2nda/bin/activate
Install our version of the BIDS to NDA conversion tool
pip install https://github.com/brown-bnc/bids2nda/archive/master.zip
Install the NDA validator that will let us test whether the data is NDA-compliant
pip install nda-tools
First, make sure that you are in your bids2nda environment. It should say (bids2nda) in front of your terminal command prompt. If it does not, activate the environment with
source ~/bids2nda/bin/activate
Change to your directory that contains your BIDS directory, GUID mapping file, and experiment ID mapping file.
Make a new directory for your NDA-formatted data
mkdir nda_output
Launch the BIDS to NDA converter
bids2nda bids guids.txt nda_output -e expIDs.txt --lookup_csv ndar_subject01.csv --lookup_fields interview_age
The -e, --lookup_csv, and --lookup_fields are optional. In this example, we are grabbing the interview age in months from the ndar_subject01.csv file and using it to fill in the interview_age column in the image03.csv file.
If successful, you should see the message "Metadata extraction complete.", and the nda_output folder should contain one image03.csv file and a series of .metadata.zip files. These zip files are referenced in the data_file2 column of the image03.csv and will ultimately be uploaded with your data. They contain:
The BIDS json sidecars that go along with each of your NIFTIs
the _events.tsv behavioral files
any stimulus files you supplied
We are working on a fix, but for now the photomet_interpret field of the image03.csv file will need to be filled in manually if you are converting any "enhanced" DICOM files (i.e. data collected post- scanner upgrade to XA30). The correct value is likely MONOCHROME2, but double-check your DICOM header for the (0028,0004) | Photometric Interpretation field to make sure this is true for your data.
To run the validator tool supplied by the NDA:
Change directory to your new NDA output folder
cd nda_output
Run the validator, passing in the new image03.csv file
vtcmd image03.csv
The tool will print information about whether or not your data passed validation. If it failed, open the validation report and address any identified issues.
Make sure your dataset is BIDS-valid. The conversion tool will attempt to add any files in your BIDS directory that follow the sub-*.nii.gz pattern, so any extra files in your BIDS directory (from analyses, etc.) may cause errors.
If you are uploading data to the NDA in batches, you'll need to temporarily copy just the data for the current upload into a separate BIDS directory, so that the tool doesn't attempt to parse any previously uploaded data. When you do that, make sure to also copy over the participants.tsv file and edit it to only contain the participants for the current upload.
If you manually modify the participants.tsv file (in MATLAB especially), be careful that it retains its tab-delimited format. Otherwise, it will not be read in properly.
