# Code Examples

## Setup

The following code examples require installation of certain packages and authentication from your ESS-DIVE account. Follow the instructions for setting up the Dataset API for your preferred coding language before trying out the search code examples:

{% content-ref url="../../programmatic-tools/ess-dive-dataset-api/setup-and-troubleshoot" %}
[setup-and-troubleshoot](https://docs.ess-dive.lbl.gov/programmatic-tools/ess-dive-dataset-api/setup-and-troubleshoot)
{% endcontent-ref %}

***

## **Search for Datasets**

Anyone can search for public datasets on ESS-DIVE using the Dataset API. If you are registered to submit data, you can also search for your private datasets. Query your dataset searches by defining parameters. 

Limited dataset metadata are returned in the response of this call. Additionally, this call cannot be used to download data files. To look up all dataset metadata and download data files, use the API to [**Download Datasets**](#download-dataset)**.**

:point\_right: Review **parameters** and **responses** for this call in the [Dataset API technical documentation](https://api.ess-dive.lbl.gov/#/Dataset/listDatasets).

The following lines of code will return the most recent 25 records. If the results contain more than 25 packages, use the `rowStart` and `pageSize` query parameters to page through the results.

{% hint style="warning" %}
**For data users:** Must pass `isPublic=true` to search for <mark style="color:blue;">**public datasets**</mark>. This call searches for public datasets by default.&#x20;

**For data contributors:** Must pass `isPublic=false`  to search for private datasets.
{% endhint %}

{% tabs %}
{% tab title="Python" %}

## Python Search Example

:star:  **Make sure you have followed the Setup instructions before running this code**

Search for datasets using any of the available query parameters. The code example below demonstrates how to format the search query with all parameters. It also demonstrates how to search for one parameter, in this case `providerName` or the publishing project.

```python
# Enter parameters
creator= "creator/submitter of datasets"
providerName =  ""\"Next-Generation Ecosystem Experiments (NGEE) Arctic"\" # Use exact match formatting to return datasets from only one project
text= "any text"
datePublished = "[YYYY-MM-DD]"
keywords = "Partial Match Keywords" # OR ""\"Exact Match Keywords"\"
```

```python
# Contruct query URL
get_packages_url = "{}/{}?creator=\"{}\"&providerName=\"{}\"&text=\"{}\"&datePublished=\"{}\"&keywords=\"{}\"&isPublic=true".format(base,endpoint,creator,providerName,text,datePublished,keywords)

## Not interested in using all the parameters? Just remove them from the query like so
## This example only uses the "providerName" parameter:
# get_packages_url = "{}/{}?providerName=\"{}\"&isPublic=true".format(base,endpoint,providerName)
```

```python
# Call GET /packages
get_packages_response = requests.get(get_packages_url, 
    headers={"Authorization":header_authorization})

if get_packages_response.status_code == 200:
   #Success
   print(get_packages_response.json())
else:
   # There was an error
   print(get_packages_response.text)
```

{% endtab %}

{% tab title="R" %}

## R Search Example

:star:  **Make sure you have followed the Setup instructions before running this code**

Search for datasets using any of the available query parameters. At this time, this code example does **not** demonstrate how to format the query parameters. This example will return all public datasets on ESS-DIVE.

```r
# Construct query URL
call_get_packages <- paste(base,endpoint, sep="/")
```

```r
# Call GET /packages
get_packages = GET(call_get_packages,
       add_headers(Authorization=header_authorization))
```

```r
# Transform the result into a data frame. (Ignore the warning)
get_packages_text <- content(get_packages, "text")
get_packages_json <- fromJSON(get_packages_text)
get_packages_df <- as.data.frame(get_packages_json)
```

```r
# Check the errors and view the data frame on success

# Check for errors
if(!http_error(post_package) ){
  # Print the returned columns
  print(colnames(get_packages_df))

  # print the ESS-DIVE Ids
  print(get_packages_df['result.id'])

  # Iterator over the dataset column and print the data package name
  for ( d in get_packages_df['result.dataset']) { print(d['name']) }
}else {
  http_status(post_package)
}
```

{% endtab %}

{% tab title="Java" %}

## Java Search Example

:star:  **Make sure you have followed the Setup instructions before running this code**

Search for datasets using any of the available query parameters. At this time, this code example does **not** demonstrate how to format the query parameters. This example will return all public datasets on ESS-DIVE.

```java
  try{
    String url = base + endpoint;
    HttpGet request = new HttpGet(url);
    StringEntity params = new StringEntity(JSON_LD.toString()); //Setting the JSON-LD Object to the request params
    request.addHeader("content-type", "application/json");
    request.addHeader("Authorization", header_authorization);
    
    HttpResponse response;
    response = httpClient.execute(request);
    
    HttpEntity entity = response.getEntity();
    String responseString = EntityUtils.toString(entity, "UTF-8");
    
    if(response.getStatusLine().getStatusCode() == 200){
      System.out.println(response.toString());
      System.out.println(responseString);
    } else {
      System.out.println(response.getStatusLine().getReasonPhrase());
      System.out.println(response.toString());
      System.out.println(responseString);
    }
  } catch (Exception ex) {
    System.out.print(ex.getMessage().toString());
  }
```

{% endtab %}
{% endtabs %}

***

## Download Dataset

Anyone can search for individual public datasets on ESS-DIVE using the Dataset API. If you are registered to submit data, you can also download your private dataset metadata.

This call will look up one dataset and return complete dataset metadata and file details. Metadata and data files can then be downloaded using standard request packages.&#x20;

If you'd like to look up the dataset upload date, last modified date, or dataset access status, use the API to [**Search for Datasets**](#search-for-datasets).

:point\_right: Review **responses** for this call in the [Dataset API technical documentation](https://api.ess-dive.lbl.gov/#/Dataset/getDataset).

{% tabs %}
{% tab title="Python" %}

## Python Download Example

:star: **Make sure you have followed the Setup instructions before running this code**

```python
# ESS-DIVE Identifiers are in the format of: ess-dive-0f0348396e46261-20181022T131245032205
dataset_id = "<Enter an ESS-DIVE Identifier here>"
```

### **Download Metadata**

```python
# Send request
get_package_url = "{}{}/{}?&isPublic=true".format(base,endpoint, dataset_id)

get_package_response = requests.get(get_package_url, 
    headers={"Authorization":header_authorization})

if get_package_response.status_code == 200:
   #Success
   print(get_package_response.json())
else:
   # There was an error
   print(get_package_response.text)
```

### **Download Data Files**

```python
# Use the json message from get_package_response to grab dataset details 
dataset_detail = get_package_response.json()
dist = dataset_detail.get('distribution')
file_url = dist.get('contentUrl')
fn = dist.get('name')

# Define where you want to download the file locally
local_dir = "local_dir"
file_path = local_dir / fn

# Download the dataset locally
urlretrieve(file_url, file_path)

# Get the dataset citation
citation = dataset_detail.get('citation')
```

{% endtab %}

{% tab title="R" %}

## R Download Example

:star:  **Make sure you have followed the Setup instructions before running this code**

```r
# ESS-DIVE Identifiers are in the format of: ess-dive-0f0348396e46261-20181022T131245032205
id <- "<Place an ESS-DIVE identifier here>"
```

### **Download Metadata**

```r
# Send request
call_get_package <- paste(base,endpoint,id, sep="/")
get_package = GET(call_get_package,
    add_headers(Authorization=header_authorization))

# Transform the result into a data frame. (Ignore the warning message)
get_package_text <- content(get_package, "text")
get_package_json <- fromJSON(get_package_text)
```

```r
# Check for errors and view the data frame on success
# Check for errors
if(!http_error(post_package) ){
  print(get_package_json)
}else {
  http_status(post_package)
}
```

### Download Data Files

The Dataset API search result will provide direct URLs that point to the data files. To download data files, you will need to grab the `contentUrl` and download it locally using a standard requests package.

&#x20;and use a standard request or urlretrive package to download it locally. The data file URLs are stored in the JSON as such:

```r
{
"dataset": {
  "distribution": [
      {
          "contentSize": 8.958984375,
          "contentUrl": "https://data.ess-dive.lbl.gov/catalog/d1/mn/v2/object/ess-dive-b8f2258b6f49e86-20210428T053005824019",
          "encodingFormat": "eml://ecoinformatics.org/eml-2.1.1",
          "identifier": "ess-dive-b8f2258b6f49e86-20210428T053005824019",
          "name": "SPRUCE_S1_Bog_Environmental_Monitoring_Data_2010_2016.xml"
        }
      ]
    }
}
```

{% endtab %}

{% tab title="Java" %}

## Java Download Example

:star:  **Make sure you have followed the Setup instructions before running this code**

### **Download Metadata**

```java
  // ESS-DIVE Identifiers are in the format of: ess-dive-0f0348396e46261-20181022T131245032205
  
  try{
    // Replace `String id` with your dataset's ESS-DIVE Identifier
    String id = "<Enter an ESS-DIVE Identifier here>";
    
    String url = base + endpoint + File.separator + id;
    HttpGet request = new HttpGet(url);
    StringEntity params = new StringEntity(JSON_LD.toString()); //Setting the JSON-LD Object to the request params
    request.addHeader("content-type", "application/json");
    request.addHeader("Authorization", header_authorization);

    HttpResponse response;
    response = httpClient.execute(request);

    HttpEntity entity = response.getEntity();
    String responseString = EntityUtils.toString(entity, "UTF-8");

    if(response.getStatusLine().getStatusCode() == 200){
      System.out.println(response.toString());
      System.out.println(responseString);
    } else {
      System.out.println(response.getStatusLine().getReasonPhrase());
      System.out.println(response.toString());
      System.out.println(responseString);
    }
  } catch (Exception ex) {
    System.out.print(ex.getMessage().toString());
  }
```

### Download Data Files

The Dataset API search result will provide direct URLs that point to the data files. To download data files, you will need to grab the `contentUrl` and download it locally using a standard requests package.

&#x20;and use a standard request or urlretrive package to download it locally. The data file URLs are stored in the JSON as such:

```r
{
"dataset": {
  "distribution": [
      {
          "contentSize": 8.958984375,
          "contentUrl": "https://data.ess-dive.lbl.gov/catalog/d1/mn/v2/object/ess-dive-b8f2258b6f49e86-20210428T053005824019",
          "encodingFormat": "eml://ecoinformatics.org/eml-2.1.1",
          "identifier": "ess-dive-b8f2258b6f49e86-20210428T053005824019",
          "name": "SPRUCE_S1_Bog_Environmental_Monitoring_Data_2010_2016.xml"
        }
      ]
    }
}
```

At the end of the document, don’t forget to close the class and main function blocks by adding two curly braces if you hadn’t done that already.
{% endtab %}
{% endtabs %}