# Search with Deep Dive API

This page contains resources to get started with data search and discovery through the Deep Dive API, including an example search using the interactive documentation at [fusion.ess-dive.lbl.gov](https://fusion.ess-dive.lbl.gov/). 

{% hint style="success" %}
**See it in action:** Check out our [Jupyter Notebook tutorial ](https://github.com/ess-dive/essdive-tutorials/blob/main/search_data/Using_Data_with_Dataset_DeepDiveAPI_Python.ipynb)with ready-to-run code examples using the Deep Dive API. *(Tip: launch the tutorial directly in your browser with the* [*Google Collab button*](https://github.com/ess-dive/essdive-tutorials/tree/main)*)* 
{% endhint %}

## Search through files to find data

ESS-DIVE's Deep Dive API provides advanced search capabilities for standardized public data files and is separate from the [main ESS-DIVE search](https://docs.ess-dive.lbl.gov/searching-and-accessing-data/data-access) and [Dataset API](https://docs.ess-dive.lbl.gov/searching-and-accessing-data/search-with-dataset-api). Rather than searching across dataset-level metadata, the Deep Dive API allows for search **within data files** to efficiently find data relevant to your scientific research. 

Public datasets are indexed by the Fusion Database (Fusion DB), which parses datasets following the [File Level Metadata (FLMD)](https://ess-dive.gitbook.io/file-level-metadata-reporting-format) and [CSV Guidelines](https://ess-dive.gitbook.io/csv-file-structure-reporting-format) Reporting Formats. Datasets that use reporting formats are essential to enabling advanced search, and planned enhancements of the Fusion DB will leverage additional reporting formats where possible.

## What public data is available in DeepDive?

Not all data files on ESS-DIVE are available for search with the Deep Dive API. To be surfaced in this search tool, the **dataset metadata** and **data files** must meet the following criteria:

1. Dataset metadata includes ESS-DIVE’s standardized reporting format **keyword** for the File Level Metadata reporting format (ESS-DIVE File Level Metadata Reporting Format)
2. The dataset follows the [File Level Metadata reporting format](https://ess-dive.gitbook.io/file-level-metadata-reporting-format) by including both file level metadata and data dictionary files
3. The file level metadata and data dictionary files follow the respective file naming conventions
4. CSV files correctly follow the [CSV reporting format guidelines](https://ess-dive.gitbook.io/csv-file-structure-reporting-format) to ensure successful file parsing

For additional information on formatting files properly for validation and parsing, please refer to the [Reporting Format Requirements](https://docs.ess-dive.lbl.gov/publish-data/review-cycle-and-criteria/reporting-format-requirements) page. Documentation and instructions for the use of all data and metadata reporting formats can be found on the [ESS-DIVE Workspace GitHub](https://github.com/ess-dive-workspace).

{% hint style="info" %}
Find all datasets using reporting formats from [ESS-DIVE's Reporting Format data portal](https://data.ess-dive.lbl.gov/portals/reporting-formats). 
{% endhint %}

## Types of search endpoints

There are two search endpoints currently available through the Deep Dive API:

<table><thead><tr><th width="173">Name</th><th width="292">Endpoint</th><th>Usage</th></tr></thead><tbody><tr><td>Query-Data</td><td><code>deepdive</code></td><td>Searches within files on public datasets for data that matches your search criteria. </td></tr><tr><td>Get-Dataset-File</td><td><code>deepdive/{doi}:{file_path}</code></td><td>Retrieves a dataset file by its file path and summarizes all column/row headers in the file.</td></tr></tbody></table>

## Workflow for Data Discovery

In the next page, we'll walk through the intended use of the available search endpoints in the Deep Dive API for discovering data in ESS-DIVE.&#x20;

{% content-ref url="search-with-deep-dive-api/how-to-query-data" %}
[how-to-query-data](https://docs.ess-dive.lbl.gov/searching-and-accessing-data/search-with-deep-dive-api/how-to-query-data)
{% endcontent-ref %}