Download Data

Public data files on ESS-DIVE can be accessed by anyone through unique dataset landing pages, where they are accompanied by relevant metadata. There is no login requirement to download data.

View Dataset Metadata

Once you have found a dataset using the search capabilities of the main data portal, click on the dataset title. This will bring up the dataset landing page, which is a page with the dataset citation at the top, a list of included data files, and accompanying metadata fields (Figure 1). Metadata provides important information about the purpose for and collection of the data included and helps to support data reuse. Scroll below the files section to view the metadata fields for the dataset.

Figure 1. Dataset landing page with data file table and accompanying metadata.

Below are the metadata fields available on each ESS-DIVE landing page and their descriptions. Only fields that have been completed by the data contributor will appear on the landing page. For more information about the metadata fields required by ESS-DIVE, please visit our Package Level Metadata Guide.

• Identifier/Alternate Identifiers: Identifiers in ESS-DIVE or other systems for this dataset (includes the DOI if applicable)

• Abstract: Brief description of the dataset

• Keywords and variables: Categorical Keywords that indicate the general themes of this dataset, and data variables associated with the package

• Publication Date: Date this package was published

• Data Table, Image and Other Data Details: The details about the files uploaded, including files sources and derivations

• People and Associated Parties: Creators, Contact, Contributors, and related Funding agencies.

• Geographic Region: Information about where the data were collected, along with a map view

• Temporal Coverage: Date period the data spans

• Project Information: Information about the DOE project the data is associated with

• Methods: Methods that were used to produce the data, including processing, QA/QC, site information etc.

• Usage Rights: The usage rights under which this dataset is released. Anyone using the data must comply with the usage rights specified on the package.

Download Data Files

Data files are displayed in a table at the top of the dataset landing page, below the citation (Figure 2). Here you can view the name, file type, and size of each datafile and select which ones you would like to download. Alternatively, you can select the download icon at the top to download all files as a zip file.

Figure 2. View of the data file table with options for downloading.

Download Large Data on Tier 2

Tier 2 data files are accessible for download via the external link table at the top of the dataset metadata landing page (Figure 3). In this table, there are two options to choose from when downloading Tier 2 data: HTTP or Globus.

Figure 3: Tier 1 dataset landing page where metadata and data can be discovered and downloaded. Access to files on Tier 2 are provided via external link.

Tier 2 (HTTP):

The HTTP access link can be used to download individual files directly from your browser. For those familiar with command line tools, it can be used to download data in bulk.

Figure 4: Tier 2 landing page for large file exploration and download. Too see the dataset metadata, select the "Access via ESS-DIVE" link.

1. Scroll down to the "External Links to Data and Metadata" table and select the URL next to the external link titled "ESS-DIVE Tier 2 (HTTP)"; this will redirect you to the Tier 2 data page (Figure 4).

2. All data files will be located in the data/ folder.

3. You can browse and download files individually as needed by selecting the file.

If you are familiar with command line tools, you can initiate a bulk download with wget or curl by pulling all the files in the data folder. The manifest-md5.txt contains a list of all files with MD5 sums.

• Example code coming soon

Tier 2 (Globus):

The Globus access link can be used to download multiple files at once from your browser. Additionally, you can choose to download the files locally or to transfer them to an existing cloud storage service, if applicable. To download files locally, it is necessary to install Globus Connect Personal.

Figure 5: Tier 2 data files can be browsed and downloaded from the Globus file manager.

1. Scroll down to the "External Links to Data and Metadata" table (Figure 3) and select the URL next to the external link titled "ESS-DIVE Tier 2 (Globus)"; this will direct you to Globus.

2. You will be prompted to login to access the data on Globus (Figure 5). If you are new to Globus, see ESS-DIVE's instructions to Create a Globus Account.

   1. Individual file download: At this point, you have the option to select individual files for download. Continue reading to download multiple files at once or entire folders.

3. If you are new to Globus, see ESS-DIVE's instructions to Install Globus Connect Personal on your desktop. Return to the File Manager in your browser (Figure 5) when you have finished setup.

4. In the Globus File Manager, use the panel toggle in the top right to change your view to two panels (Figure 6).

5. In the empty panel, select the “Collection” search bar and navigate to the “Your Collections” tab (Figure 7). Then select your desktop collection. Alternatively, you can select any other existing cloud endpoint you may have on Globus.

6. Now you’re ready to download (i.e. transfer) the data. On your desktop panel, navigate to where you want to download the files. On the dataset panel, select some or all files to download then either use the “Sync transfer...” button or the "Start" button to start the transfer (Figure 8). You can also drag the selected files to your desktop.

7. This will start a transfer job and you will see a green popup window on your screen confirming that the transfer has started (Figure 9).

In many cases the transfer is quick, but if your transfer is taking a while, use the link from this popup window to check in on the status of the transfer or look in the Activity tab. You will know that the transfer is complete when Globus sends you a confirmation email.

Figure 6

Figure 7

Figure 8

Figure 9