R Example
This page provides coding examples for retrieving a list of submitted datasets and submitting dataset metadata for validation using R.
While learning the expected schema for the metadata use https://api-sandbox.ess-dive.lbl.gov, as this is the domain shown in the examples in this documentation. Once you've familiarized yourself with ESS-DIVE's metadata and dataset schema, use our production domain https://api.ess-dive.lbl.gov/ to submit datasets to ESS-DIVE for publishing and review.
For additional information about the API, review the documentation at https://api-sandbox.ess-dive.lbl.gov.
ESS-DIVE Test API URL: https://api-sandbox.ess-dive.lbl.gov ESS-DIVE Production API URL: https://api.ess-dive.lbl.gov/ Help Desk: ess-dive-support@lbl.gov
Contents
Setup
Install the following R packages
Setup the Dataset API information
Check that your token is up-to-date; it expires after a few days
Submit a Dataset
After creating metadata, you have the option to submit a dataset with only metadata or to submit a dataset with both metadata and data files.
Create Metadata
The following lines of code validate JSON-LD metadata for a single dataset. The example provided is from the ESS-DIVE sandbox site. (See https://data-sandbox.ess-dive.lbl.gov/#view/doi:10.3334/CDIAC/spruce.001).
Due to R complex JSON-LD support limitations, you need to create a text file of your JSON-LD and add it’s directory in the following read_file function.
Here’s an example for a JSON-LD located on our ESS-DIVE package service examples github repository.
To make sure your file is properly saved in the JSON-LD format, consider using the Atom text editor (https://atom.io)
Metadata Only
Prepare and submit the metadata for validation
Make sure to enter your own file path.
E.g. C:\\User\\Files\API_Tutorial.json
for Windows & /Users/Files/API_Tutorial.json
for Mac
Review the results
Metadata and Data
To submit the metadata and data files, create a folder and add your data files to it then execute the following code:
Review the results
Search for Datasets
Anyone can search for public datasets on ESS-DIVE using the Dataset API. If you are registered to submit data, you can also search for your private datasets. Query your dataset searches by defining parameters.
Limited dataset metadata are returned in the response of this call. Additionally, this call cannot be used to download data files. To look up all dataset metadata and download data files, use the API to Download Dataset Metadata.
👉 Review parameters and responses for this call in the Dataset API technical documentation.
The following lines of code will get the list of dataset metadata that you have permissions to edit. This will return the most recent 25 records. If the results contain more than 25 packages, use the row_start
and page_size
query parameters to page through the results.
Transform the result into a data frame. (Ignore the warning)
Check the errors and view the data frame on success
Download Dataset Metadata
Anyone can search for individual public datasets on ESS-DIVE using the Dataset API. If you are registered to submit data, you can also download your private dataset metadata.
The response for this call will return all dataset metadata and attached data files. Metadata and data files can then be downloaded. If you'd like to look up the dataset upload date, last modified date, or dataset access status, use the API to Search for Datasets.
👉 Review responses for this call in the Dataset API technical documentation.
The following lines of code will get the metadata for a single dataset that you have permissions to edit.
Transform the result into a data frame. (Ignore the warning message)
Check for errors and view the data frame on success
Update a Dataset
It is possible to both update the metadata and data of an existing dataset. The following update scenarios are possible :
update metadata only
replace/add files only
both update metadata and replace/add files.
These examples will demonstrate both updating metadata and adding new files to the dataset created in previous sections.
Metadata Only
Use the PUT function to update the metadata of a dataset. This example updates the name of a dataset.
NOTE: The body argument is a string not a file.
Transform the result into a data frame. (Ignore the warning message)
Check the results for the changed metadata attribute
Metadata plus new data file
Use the PUT function to update a dataset. This example updates the date published to 2019 of a dataset and adds a new data file.
NOTE: The body argument is a string not a file.
Transform the result into a data frame. (Ignore the warning message)
Check the results for the changed metadata attribute and newly uploaded file
Check for errors and view the data frame on success
Troubleshooting
This error message indicates your token is either incorrect or expired. Please follow the instructions on the ESS-DIVE Dataset API page to retrieve a new token.
This error message indicates the file entered was not found. This could be because you are searching in the wrong directory or because you misrepresented the file name.
Last updated