{ "cells": [ { "cell_type": "markdown", "id": "907c5c49", "metadata": {}, "source": [ "# Modality agnostic files" ] }, { "cell_type": "code", "execution_count": 1, "id": "782c4c48", "metadata": { "tags": [ "remove-cell" ] }, "outputs": [], "source": [ "from myst_nb import glue\n", "from IPython.display import display, Markdown, HTML\n", "from schemacode import render, schema, utils\n", "from markdown import markdown\n", "\n", "# Load the schema path\n", "schemapath = utils.get_schema_path()\n", "schema_obj = schema.load_schema(schemapath)" ] }, { "cell_type": "markdown", "id": "86830e39", "metadata": {}, "source": [ "## Dataset description\n", "\n", "Templates:\n", "\n", "- `dataset_description.json`\n", "- `README`\n", "- `CHANGES`\n", "- `LICENSE`\n", "\n", "### `dataset_description.json`\n", "\n", "The file `dataset_description.json` is a JSON file describing the dataset.\n", "Every dataset MUST include this file with the following fields:" ] }, { "cell_type": "code", "execution_count": 2, "id": "79214366", "metadata": { "tags": [ "remove-input" ] }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
Key nameRequirement LevelData typeDescription
{term}Name (metadata)REQUIREDstringName of the dataset.
{term}BIDSVersion (metadata)REQUIREDstringThe version of the BIDS standard that was used.
{term}HEDVersion (metadata)RECOMMENDEDstringIf HED tags are used: The version of the HED schema used to validate HED tags for study.
{term}DatasetType (metadata)RECOMMENDEDstringThe interpretation of the dataset. For backwards compatibility, the default value is \"raw\".

Must be one of: \"raw\", \"derivative\".
{term}License (metadata)RECOMMENDEDstringThe license for the dataset. The use of license name abbreviations is RECOMMENDED for specifying a license (see Appendix II). The corresponding full license text MAY be specified in an additional LICENSE file.
{term}Authors (metadata)OPTIONALarray of stringsList of individuals who contributed to the creation/curation of the dataset.
{term}Acknowledgements (metadata)OPTIONALstringText acknowledging contributions of individuals or institutions beyond those listed in Authors or Funding.
{term}HowToAcknowledge (metadata)OPTIONALstringText containing instructions on how researchers using this dataset should acknowledge the original authors. This field can also be used to define a publication that should be cited in publications that use the dataset.
{term}Funding (metadata)OPTIONALarray of stringsList of sources of funding (grant numbers).
{term}EthicsApprovals (metadata)OPTIONALarray of stringsList of ethics committee approvals of the research protocols and/or protocol identifiers.
{term}ReferencesAndLinks (metadata)OPTIONALarray of stringsList of references to publications that contain information on the dataset. A reference may be textual or a URI.
{term}DatasetDOI (metadata)OPTIONALstringThe Digital Object Identifier of the dataset (not the corresponding paper). DOIs SHOULD be expressed as a valid URI; bare DOIs such as 10.0.2.3/dfjj.10 are DEPRECATED.
{term}GeneratedBy (metadata)RECOMMENDEDarray of objectsUsed to specify provenance of the dataset.
{term}SourceDatasets (metadata)RECOMMENDEDarray of objectsUsed to specify the locations and relevant attributes of all source datasets. Valid keys in each object include \"URL\", \"DOI\" (see URI), and \"Version\" with string values.
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "field_info = {\n", " \"Name\": \"REQUIRED\",\n", " \"BIDSVersion\": \"REQUIRED\",\n", " \"HEDVersion\": \"RECOMMENDED\",\n", " \"DatasetType\": \"RECOMMENDED\",\n", " \"License\": \"RECOMMENDED\",\n", " \"Authors\": \"OPTIONAL\",\n", " \"Acknowledgements\": \"OPTIONAL\",\n", " \"HowToAcknowledge\": \"OPTIONAL\",\n", " \"Funding\": \"OPTIONAL\",\n", " \"EthicsApprovals\": \"OPTIONAL\",\n", " \"ReferencesAndLinks\": \"OPTIONAL\",\n", " \"DatasetDOI\": \"OPTIONAL\",\n", " \"GeneratedBy\": \"RECOMMENDED\",\n", " \"SourceDatasets\": \"RECOMMENDED\",\n", "}\n", "\n", "table = render.make_metadata_table(schema_obj, field_info)\n", "display(HTML(markdown(table.to_markdown(), extensions=['tables'])))" ] }, { "cell_type": "markdown", "id": "8a244b76", "metadata": {}, "source": [ "Each object in the `GeneratedBy` array includes the following REQUIRED, RECOMMENDED\n", "and OPTIONAL keys:\n", "\n", "| **Key name** | **Requirement level** | **Data type** | **Description** |\n", "|--------------|-----------------------|---------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n", "| Name | REQUIRED | [string][] | Name of the pipeline or process that generated the outputs. Use `\"Manual\"` to indicate the derivatives were generated by hand, or adjusted manually after an initial run of an automated pipeline. |\n", "| Version | RECOMMENDED | [string][] | Version of the pipeline. |\n", "| Description | OPTIONAL | [string][] | Plain-text description of the pipeline or process that generated the outputs. RECOMMENDED if `Name` is `\"Manual\"`. |\n", "| CodeURL | OPTIONAL | [string][] | URL where the code used to generate the dataset may be found. |\n", "| Container | OPTIONAL | [object][] | Used to specify the location and relevant attributes of software container image used to produce the dataset. Valid keys in this object include `Type`, `Tag` and [`URI`][uri] with [string][] values. |\n", "\n", "Example:\n", "\n", "```JSON\n", "{\n", " \"Name\": \"The mother of all experiments\",\n", " \"BIDSVersion\": \"1.6.0\",\n", " \"DatasetType\": \"raw\",\n", " \"License\": \"CC0\",\n", " \"Authors\": [\n", " \"Paul Broca\",\n", " \"Carl Wernicke\"\n", " ],\n", " \"Acknowledgements\": \"Special thanks to Korbinian Brodmann for help in formatting this dataset in BIDS. We thank Alan Lloyd Hodgkin and Andrew Huxley for helpful comments and discussions about the experiment and manuscript; Hermann Ludwig Helmholtz for administrative support; and Claudius Galenus for providing data for the medial-to-lateral index analysis.\",\n", " \"HowToAcknowledge\": \"Please cite this paper: https://www.ncbi.nlm.nih.gov/pubmed/001012092119281\",\n", " \"Funding\": [\n", " \"National Institute of Neuroscience Grant F378236MFH1\",\n", " \"National Institute of Neuroscience Grant 5RMZ0023106\"\n", " ],\n", " \"EthicsApprovals\": [\n", " \"Army Human Research Protections Office (Protocol ARL-20098-10051, ARL 12-040, and ARL 12-041)\"\n", " ],\n", " \"ReferencesAndLinks\": [\n", " \"https://www.ncbi.nlm.nih.gov/pubmed/001012092119281\",\n", " \"Alzheimer A., & Kraepelin, E. (2015). Neural correlates of presenile dementia in humans. Journal of Neuroscientific Data, 2, 234001. doi:1920.8/jndata.2015.7\"\n", " ],\n", " \"DatasetDOI\": \"doi:10.0.2.3/dfjj.10\",\n", " \"HEDVersion\": \"8.0.0\",\n", " \"GeneratedBy\": [\n", " {\n", " \"Name\": \"reproin\",\n", " \"Version\": \"0.6.0\",\n", " \"Container\": {\n", " \"Type\": \"docker\",\n", " \"Tag\": \"repronim/reproin:0.6.0\"\n", " }\n", " }\n", " ],\n", " \"SourceDatasets\": [\n", " {\n", " \"URL\": \"s3://dicoms/studies/correlates\",\n", " \"Version\": \"April 11 2011\"\n", " }\n", " ]\n", "}\n", "```\n", "\n", "#### Derived dataset and pipeline description\n", "\n", "As for any BIDS dataset, a `dataset_description.json` file MUST be found at the\n", "top level of every derived dataset:\n", "`/derivatives//dataset_description.json`.\n", "\n", "In contrast to raw BIDS datasets, derived BIDS datasets MUST include a\n", "`GeneratedBy` key:" ] }, { "cell_type": "code", "execution_count": 3, "id": "89c170c4", "metadata": { "tags": [ "remove-input" ] }, "outputs": [], "source": [ "field_info = {\n", " \"GeneratedBy\": \"REQUIRED\",\n", "}\n", "\n", "table = render.make_metadata_table(schema_obj, field_info)" ] }, { "cell_type": "markdown", "id": "f393cd11", "metadata": {}, "source": [ "If a derived dataset is stored as a subdirectory of the raw dataset, then the `Name` field\n", "of the first `GeneratedBy` object MUST be a substring of the derived dataset directory name.\n", "That is, in a directory `/derivatives/[-]/`, the first\n", "`GeneratedBy` object should have a `Name` of ``.\n", "\n", "Example:\n", "\n", "```JSON\n", "{\n", " \"Name\": \"FMRIPREP Outputs\",\n", " \"BIDSVersion\": \"1.6.0\",\n", " \"DatasetType\": \"derivative\",\n", " \"GeneratedBy\": [\n", " {\n", " \"Name\": \"fmriprep\",\n", " \"Version\": \"1.4.1\",\n", " \"Container\": {\n", " \"Type\": \"docker\",\n", " \"Tag\": \"poldracklab/fmriprep:1.4.1\"\n", " }\n", " },\n", " {\n", " \"Name\": \"Manual\",\n", " \"Description\": \"Re-added RepetitionTime metadata to bold.json files\"\n", " }\n", " ],\n", " \"SourceDatasets\": [\n", " {\n", " \"DOI\": \"doi:10.18112/openneuro.ds000114.v1.0.1\",\n", " \"URL\": \"https://openneuro.org/datasets/ds000114/versions/1.0.1\",\n", " \"Version\": \"1.0.1\"\n", " }\n", " ]\n", "}\n", "```\n", "\n", "### `README`\n", "\n", "A REQUIRED text file, `README`, SHOULD describe the dataset in more detail.\n", "The `README` file MUST be either in ASCII or UTF-8 encoding and MAY have one of the extensions:\n", "`.md` ([Markdown](https://www.markdownguide.org/)),\n", "`.rst` ([reStructuredText](https://docutils.sourceforge.io/rst.html)),\n", "or `.txt`.\n", "A BIDS dataset MUST NOT contain more than one `README` file (with or without extension)\n", "at its root directory.\n", "BIDS does not make any recommendations with regards to the\n", "[Markdown flavor](https://www.markdownguide.org/extended-syntax/#lightweight-markup-languages)\n", "and does not validate the syntax of Markdown and reStructuredText.\n", "The `README` file SHOULD be structured such that its contents can be easily understood\n", "even if the used format is not rendered.\n", "A guideline for creating a good `README` file can be found in the\n", "[bids-starter-kit](https://github.com/bids-standard/bids-starter-kit/blob/master/templates/README).\n", "\n", "### `CHANGES`\n", "\n", "Version history of the dataset (describing changes, updates and corrections) MAY\n", "be provided in the form of a `CHANGES` text file. This file MUST follow the\n", "[CPAN Changelog convention](https://metacpan.org/pod/release/HAARG/CPAN-Changes-0.400002/lib/CPAN/Changes/Spec.pod).\n", "The `CHANGES` file MUST be either in ASCII or UTF-8 encoding.\n", "\n", "Example:\n", "\n", "```Text\n", "1.0.1 2015-08-27\n", " - Fixed slice timing information.\n", "\n", "1.0.0 2015-08-17\n", " - Initial release.\n", "```\n", "\n", "### `LICENSE`\n", "\n", "A `LICENSE` file MAY be provided in addition to the short specification of the\n", "used license in the `dataset_description.json` `\"License\"` field.\n", "The `\"License\"` field and `LICENSE` file MUST correspond.\n", "The `LICENSE` file MUST be either in ASCII or UTF-8 encoding.\n", "\n", "## Participants file\n", "\n", "Template:\n", "\n", "```Text\n", "participants.tsv\n", "participants.json\n", "```\n", "\n", "The purpose of this RECOMMENDED file is to describe properties of participants\n", "such as age, sex, handedness, species and strain.\n", "If this file exists, it MUST contain the column `participant_id`,\n", "which MUST consist of `sub-