Submit Data with the Dataset API

Quick Start

Start submitting test datasets with ESS-DIVE's Dataset API right away using our Jupyter Notebook tutorials! No coding knowledge is necessary to run the notebooks.

(1) Visit our GitHub linked below to launch tutorial notebooks directly in your browser. (2) Select the Google Colab or Jupyter+R Binder button. We recommend Google Colab for a smoother experience (a Google account is required to use Colab). (3) Then, open the submit_data folder to access tutorials in R and Python.

LogoGitHub - ess-dive/essdive-tutorials: Tutorials for ESS-DIVEGitHub

About submissions with the Dataset API

Datasets are comprised of two components: (1) data files and (2) metadata which describes the data. To submit data files with the Dataset API, organize the data into one directory and enter the file path into our example code where instructed. To submit metadata, write dataset metadata that meet ESS-DIVE's Dataset Metadata Requirements and format the metadata into the JSON-LD schema in your script. Then, submit the data and metadata to ESS-DIVE using the appropriate HTTP method.

Continue reading this page for a more detailed recommendations on how to familiarize yourself with the process. In the code examples provided on the next page, we demonstrate how to format JSON-LD and submit requests in three different coding languages: R, Python, and Java.

Dataset metadata refers to the top level information that enables a dataset to be “discoverable” in search results. Examples of top-level metadata include the title, abstract, authors, variables and keywords.

JSON-LD is a schema to encode linked data using JSON.

1. Prepare Submission

  1. Review ESS-DIVE's Metadata Criteria for descriptions and expectations for each metadata field

  2. Review Metadata JSON-LD Formatting (cURL, HTTP) to learn the expected JSON-LD schema

    • The technical documentation linked above will help you get familiar with the interface, understand the available operations, possible errors and the structure of the JSON-LD produced. Hint: Our metadata criteria includes the JSON-LD equivalents for each field.

  3. Prepare Metadata JSON-LD in R, Python or Java using the JSON-LD code examples as reference.

2. Test Dataset Submission on Sandbox

After you have familiarized yourself with the Dataset API and JSON-LD, test your dataset submissions on Sandbox (https://api-sandbox.ess-dive.lbl.gov).

Two of the most common programming languages in the ESS space are Python and R. Either of these languages can be used to write scripts for submitting dataset metadata to be validated.

  • Python: dataset JSON-LD metadata can be submitted using the requests module

  • R: dataset metadata can be submitted using the httr and jsonlite packages

  • Java: datasets JSON-LD metadata can be submitted using Apache HTTPComponents

Be sure to use ESS-DIVE's test instance, Sandbox (https://api-sandbox.ess-dive.lbl.gov), while building scripts and making corrections to JSON_LD dataset submissions.

3. Complete Code and Submit Data to Production

Once you've familiarized yourself with ESS-DIVE's metadata and dataset API schema, use our production domain https://api.ess-dive.lbl.gov/ to submit datasets to ESS-DIVE for publishing and review.

Note that the https://data.ess-dive.lbl.gov/ domain is for directly interacting with the web user interface.

Only switch to ESS-DIVE's production instance (https://api.ess-dive.lbl.gov/) after your scripts are complete.

Learn More:

ESS-DIVE Dataset API

PreviousSubmit Data with Online FormNextCode Examples

Last updated