MINT Services and Software

Last updated: 2020-03-31

This document provides links to public MINT products (APIs, software, data and services) and a summary of the documentation and specifications available.

Each product is evolving, the release page contains announcements about new versions.

MINT User Interface

MINT assists an analyst to easily use sophisticated simulation models and data in order to explore the role of weather and climate in water on food availability in select regions of the world.

Relevant Pointers

Data Services

The MINT Data Catalog (see overview) provides access to a curated collection of a datasets in the MINT Data Catalog.

The MINT Data Catalog API documentation: https://data-catalog.mint.isi.edu/documentation

The MINT Data Catalog API performs the following tasks:

  • Create an entry for a dataset.
  • Create entries for the data files and the standard variable names in the data.
  • Get all the variables for a dataset.
  • Get all resources for a dataset.
  • Get all datasets for an area (given a bounding box).
  • Metadata about the dataset, standard variable names, and files.
  • Full-text search of a dataset.
  • Search datasets by name, id, or standard variable names.
  • Demo of registering a dataset using API
  • Demo of fetching for a dataset using API
  • Python notebook of registering a dataset using API
  • Python notebook of searching for a dataset using API

Upcoming and planned new features include:

  • Generate a pipeline to transform a dataset into a target format
  • Get all data transformations that use or generate a data format

Relevant Pointers

Model Services

The MINT Model Catalog provides detailed information about software models and metadata available in MINT. See an overview on the following link.

The models in the Model Catalog are executable through the Desktop Application for Model Execution (DAME).

Relevant Pointers:

The API relies on the Software Description Ontology for Models https://w3id.org/okn/o/sdm

The MINT Model Catalog API supports:

  • Get a list of all the resources (Model, ModelVersion, Parameters, Inputs, Variables and other classes).
    • For example: get all the models on MINT.
  • Get the detailed information of a resource.
    • For example: get the description of the model Topoflow
  • Create, updates and deletes resources on the Model Catalog.
    • For example: Create a new model, delete or update an existing model.

Transformation Services

The MINT Data Transformation System provides a list of supported transformations that can be used to transform datasets into different formats, which may be required by different models.

The main idea of the transformation system is that we use smaller components (we refer to them as adapters) which we ’concatenate’ to form a transformation flow (a pipeline). This modular design allows us to reuse existing modules and wrap ready-scripts to create a language-and-format-independent module and pipeline.

The current supported transformations contains:

  • Spatial Transformations: cropping, regirdding, resampling, etc.
  • Model-specific Transformations: PIHM, Cycles, Econ, etc.
  • General Transformations: joining, filtering, merging, etc.

Common Data Representation

To make the transformation adapters work with datasets in different formats and layouts (e.g., CSV, NetCDF, Spreadsheet), we developed a library named D-REPR that reads the datasets into a common data representation. We choose to represent the data in RDF. Given a D-REPR model of a dataset, the D-REPR library can either virtually map the data to RDF or material the data. The library and its documentation is available here

Relevant Pointers

Identifying and Transforming Units of Measurement in Scientific Data

The library, called CCUT, uses grammar tools to automatically parse the different components in a unit found in textual data and map them to elements of a standard ontology called QUDT to form a structured semantic output. The output depicts the different relationships, attributes and semantics of units and allows users to have a better understanding of their data. The system also enables data transformations such as unit conversions.

Relevant Pointers

Execution Management Services

Emulator APIs

We are creating two separate APIs for MINT Emulators

  • MINT Emulator Manager API: to support search and discovery of emulators that are available in MINT
  • MINT Emulator Data Services APIs: to support retrieval and processing of data in MINT emulators

We are building these two APIs based on an existing MINT Ingestion API that performs the following tasks:

  • Gather model ensemble execution results.
  • Store model ensemble results into a database.

Relevant Pointers

Ensemble Manager API

The Ensemble Manager executes a model ensemble for all possible parameters and data combinations in a MINT modeling thread. It performs the following tasks:

  • Submit model ensembles for execution (Options: Local execution or Scalable execution)
  • Get Model Ensemble execution status

Relevant Pointers

Scientific Variables

The Scientific Variables Ontology Framework is a methodology for uniformly representing scientific variables. The required elements are an upper ontology consisting of a set of domain-independent categories and a set of design patterns for modular creation of complex variable representations. The customizable lower ontology can be populated with any collection of variables as necessitated by an application.

Relevant Pointers

Remote Sensing

[Section to be completed]