MINT Project Releases

Last updated: 2023-06-15

Jun, 2023 #

  • Release of MIC Web: The Model Insertion web (MIC web) is an application to assist scientists in encapsulating their softwares
  • Release of MINT UI 7.1.0:
    • The selection of datasets and models in the model usage section has been improved.
    • Enabled the ability to send results via email and download them.
    • More changes can be found in the CHANGELOG
  • Model Catalog 1.8.1: FastAPI migration to improve performance and stability
  • MINT Installation for Kubernetes: MINT can be easily deployed on large Kubernetes clusters using Helm. Useful for production instances.
  • Ensemble Manager 4.1.0
    • Improve performance and stability
    • Limit the parallel execution of models
    • Multiplatform support (Linux, Mac)

May, 2021 #

  • Acquired and mosaicked MERIT DEMs for Horn of Africa
  • Acquired ISRIC Soil Grids 2017 1km data for globe
  • Set up DAKOTA on TACC (test runs, DataX Portal; Corral for storage)
  • Data Inventory
  • Release of MIC 2.1.0: Remove github dependency to store the MINT components.
  • New Data Transformation repository: A repository with Data Transformation ready to run using cwltool
  • New MINT installation repository: Contains docker-compose files to run MINT on cloud or servers.
  • Release of SPADE 1.0.0 (Newer version of B-Clean).
  • Release of Semantic Modeling 1.0.0: Build semantic models of tables.

April, 2021 #

December, 2020 #

  • Release of B-Clean v0.5 to support semi-automatic data cleaning
    • Better error detection performance
    • Full evaluation on different datasets
  • Release of Semantic Modeling API v0.3 to support mapping linked tables to Wikidata
  • Release of the Model Catalog 1.7.0 to support defining local SVO names
  • Release of the MINT UI 6.0.5 fixing usability bugs.

November, 2020 #

October, 2020 #

September, 2020 #

August, 2020 #

  • Release of B-Clean API v0.3 to support automatic data cleaning
    • Add deep learning model for few-shot data cleaning
  • Applications of the TopoFlow hydrologic model:
    • Comparisons of TopoFlow model output to remote sensing time series.
    • Comparisons of TopoFlow model output to daily discharge data at gauges.
    • Improvements to the TopoFlow model calibration module.
    • New model runs with 6-hourly CHIRPS rainfall data.
    • Improvements to TopoFlow Jupyter notebooks.
    • Testing of MIC with the TopoFlow model.
    • New script to auto-generate TopoFlow visualizations.
  • Release of MIC 1.3.3
    • Changelog
    • Bug fixes (errors in Windows executions, errors with double quotes in parameters)

July, 2020 #

  • Release of MIC 1.3.1
    • Changelog
    • Better capture of dependencies (starting from own image, committing image after user edits)
    • Auto-detection of parameters and inputs when encapsulating component.
    • Support for uploading data transformations.
    • Usability bugs and testing
    • Improved documentation
  • Release of DAME 5.3.1
    • Changelog
    • Improve errors in singularity image detection and minor bugs
    • Users can execute data transformations from DAME.
  • Release of Model Catalog API 1.5.0
    • Changelog
    • Model description fixes, improved model export functionality.
  • Release of CCUT-Wrapper 1.0.0
    • We are releasing a beta version of the wrapper UI for CCUT 1.0.0. This is an interactive web application that suggests semantic types for units of measurement, supports their transformations, and allows working with spreadsheet files.
  • Release of B-Clean API v0.2 to support automatic data cleaning
    • Increase training data: include ~1M web tables with ~7M attributes and ~200M cell values.
    • Support outlier detection based on n-gram uncommonness instead of whole string uncommonness
  • Release of Semantic Modeling API v0.2
    • The new model leverages available data on Wikidata and WebTables to improve performance on domains with little training data
  • Release of new Jupyter Notebook for the TopoFlow model, and updates to existing notebooks and underlying code:

June, 2020 #

May, 2020 #

April, 2020 #

  • Release of DAME 4.1.3
    • Changelog
    • Additional testing and bug fixes (Testing in OSX and Unix). DAME will ask for missing parameters and inputs, using defaults when provided.
    • Improvements to messages and logging in the UI. Now the singularity commands, inputs and Docker images are displayed, in case users want to execute models with their own means.
    • Improved documentation and examples
  • Initial release of MIC 0.2.0 - ALPHA
  • Release of Data Transformation service v1.0
    • Users can run the transformation pipeline through CLI, web service or Docker.
    • Release predefined pipelines in form of configuration files for:
      • Model-specific transformations: Topoflow
  • Release of MINT-Data-Sync system
    • Monitor when new GLDAS data files become available, upload them to MINT Data Server, and register them in MINT Data Catalog
  • Release of River Segment Surface Area Dataset version 1.0 for Ethiopia
    • Processed 8710 river segments (covering all of Ethiopia) using machine learning algorithms and satellite imagery to create surface area timeseries.
    • Uses Sentinel-2 imagery available from December 2015 until March 2020.
    • A csv with surface area timeseries for each river segment is available for download from the MINT Data Catalog.
  • Release of riverwidthEO version 1.0
    • A python package that processes river segments using machine learning algorithms and satellite imagery (Sentinel-2) to create surface area timeseries (delivered as a csv file).
    • Uses descarteslabs API to download data for any given segment.
    • Provides user with options to provide points on a river as input or just provide a region or country to select predefined points on the river. These predefined points are available for rivers (>100 meters in width) across the globe.
  • Release of a Jupyter Notebook for the TopoFlow model with an overview and introduction to new users

March, 2020 #

February, 2020 #

January, 2020 #

December, 2019 #

MINT Data Catalog #

MINT Model Catalog: #

Ensemble Manager API #

  • Release 1.0.0:
    • Execute models using singularity
    • Include parallelism option in config to do multiple model runs
    • Adding DELETE request to executions
    • Local delete for cache of execution

Ingestion API: #

  • Release 1.1.0:
    • Take model execution data and ingest into a database to enable interactive dashboards
    • Gather model ensemble execution results

November, 2019 #

MINT NetCDF Convention #

  • Release of MINT NetCDF convention
    • We propose a self-describing data format for structured gridded datasets for MINT data catalog and visualization based on the NetCDF and the CF convention. Check here for the lastest document. Please open new issues on GitHub or on Google doc for comments.
  • Release of MINT-GeoViz
    • We are releasing the v1 of MINT-GeoViz, an interactive visualization library for large geospatial datasets that follow MINT NetCDF convention. Check out this GitHub repo for more details.
    • Check out our full demo and notebook examples on how to use the library

October, 2019 #

Data Catalog #

  • Release of D-REPR: a library for reading heterogeneous datasets into common representations. Check its GitHub for more information.

May, 2019 #

Model Catalog #

  • New content: The MINT model catalog has been expanded to include a semantic representation of units, scientific variables and links to Wikidata. Check the release on GitHub for more information.
  • New APIs for adding content: We have expanded our APIs with a new client based on Open API that allows anyone to insert models in the model catalog. The API is accessible in the following link: https://api.models.mint.isi.edu/v0.0.2/ui/#/
  • New APIs for accessing content: 3 new methods have been added to our GRLC API. The first one, to obtain additional information of a Scientific Variable given its label (getStandardVariableMetadata). The other two (getInputCompatibleConfig) and (getOutputCompatibleConfig) retrieve those models or software compatible with a target model. This facilitates understanding which software has variables that may interoperate with other software.
  • A new client for accessing content: With our new Python client, now it is possible to access the content in the model catalog without having to issue SPARQL queries or API queries. Check it out here.
  • New examples: Check out our notebook for examples on how to use the client.
  • The Model Explorer has been updated to show the contents of models in a more user-friendly manner. Check here the latest version. Please open new issues on GitHub if you find bugs.