Last updated: 2021-06-02
May, 2021 #
- Acquired and mosaicked MERIT DEMs for Horn of Africa
- Acquired ISRIC Soil Grids 2017 1km data for globe
- Set up DAKOTA on TACC (test runs, DataX Portal; Corral for storage)
- Data Inventory
- Release of MIC 2.1.0: Remove github dependency to store the MINT components.
- New Data Transformation repository: A repository with Data Transformation ready to run using cwltool
- New MINT installation repository: Contains
docker-composefiles to run MINT on cloud or servers.
- Release of SPADE 1.0.0 (Newer version of B-Clean).
- Release of Semantic Modeling 1.0.0: Build semantic models of tables.
April, 2021 #
- Release of Model constraint checking 0.0.1
- Release of MIC 2.0.0: Convert Ipython notebooks to model components + improved documentation for modelers.
December, 2020 #
- Release of B-Clean v0.5 to support semi-automatic data cleaning
- Better error detection performance
- Full evaluation on different datasets
- Release of Semantic Modeling API v0.3 to support mapping linked tables to Wikidata
- Release of the Model Catalog 1.7.0 to support defining local SVO names
- Release of the MINT UI 6.0.5 fixing usability bugs.
November, 2020 #
- Release of the Model Catalog API 1.6.0
- Release of the Model Catalog Python client 6.0.0
- Release of the Model Catalog NPM client 6.0.0
- Release of MIC 1.3.7
- Update to new model catalog API
- Release of DAME 5.4.5
- Update to use the new model catalog API
- Release of new Jupyter Notebook for transforming daily discharge dataset
- Transforms irregularly-structured data into time-series format
- Provides example usage of T2WML and KGTK for transforming data
- New version of the Drought indices model (1.2.0)
- New forecasting model for drought indices, DroughtPredict 1.0.0
- Release the first version of an evaluation dataset for Semantic Modeling that contains 500 Wikipedia Tables manually mapped to Wikidata.
October, 2020 #
- Release of B-Clean API v0.4 to support automatic data cleaning
- Improve performance of deep learning model
- Support user curation in active learning fashion
- Provide datasets for system evaluation
- Release of the Model Catalog 1.6.0
September, 2020 #
- Release of Data Transformation service v1.2
- Now support CHIRPS dataset download with temporal and spatial cropping (example).
- Release a draft dataset of ~1M semantic models of Wikipedia tables to Wikidata and an UI to create and curate semantic models
- Release of MIC 1.3.4
- Bug fixes and tests (error in automated parameter detection, errors in hyphens in the main configuration, etc.)
- Release of the Software Description Ontology v1.8.0
- Support for additional properties for describing results of a transformation.
- Release of the Software Description Ontology for Models v1.7.0
- Suppot for model categories and description of coupled models.
August, 2020 #
- Release of B-Clean API v0.3 to support automatic data cleaning
- Add deep learning model for few-shot data cleaning
- Applications of the TopoFlow hydrologic model:
- Comparisons of TopoFlow model output to remote sensing time series.
- Comparisons of TopoFlow model output to daily discharge data at gauges.
- Improvements to the TopoFlow model calibration module.
- New model runs with 6-hourly CHIRPS rainfall data.
- Improvements to TopoFlow Jupyter notebooks.
- Testing of MIC with the TopoFlow model.
- New script to auto-generate TopoFlow visualizations.
- Release of MIC 1.3.3
- Bug fixes (errors in Windows executions, errors with double quotes in parameters)
July, 2020 #
- Release of MIC 1.3.1
- Better capture of dependencies (starting from own image, committing image after user edits)
- Auto-detection of parameters and inputs when encapsulating component.
- Support for uploading data transformations.
- Usability bugs and testing
- Improved documentation
- Release of DAME 5.3.1
- Improve errors in singularity image detection and minor bugs
- Users can execute data transformations from DAME.
- Release of Model Catalog API 1.5.0
- Model description fixes, improved model export functionality.
- Release of CCUT-Wrapper 1.0.0
- We are releasing a beta version of the wrapper UI for CCUT 1.0.0. This is an interactive web application that suggests semantic types for units of measurement, supports their transformations, and allows working with spreadsheet files.
- Release of B-Clean API v0.2 to support automatic data cleaning
- Increase training data: include ~1M web tables with ~7M attributes and ~200M cell values.
- Support outlier detection based on n-gram uncommonness instead of whole string uncommonness
- Release of Semantic Modeling API v0.2
- The new model leverages available data on Wikidata and WebTables to improve performance on domains with little training data
- Release of new Jupyter Notebook for the TopoFlow model, and updates to existing notebooks and underlying code:
June, 2020 #
- Release of B-Clean API v0.1 to support automatic data cleaning
- Release of Semantic Modeling API v0.1 to support automatic data modeling
- Release of DAME 5.2.0
- Added changes so users can specify their own parameter values instead of defaults.
- Release of MIC 1.0.0
- Now users can start from their model folder (no requirements for data organization)
- Automatic detection the framework/language used by the model component (python, conda, java, general)
- Automated extraction of dependencies according to the framework/language (pip is supported)
- Creation a Linux environment using Docker
- MIC can now trace commands and automatically detect configuration files, inputs and outputs
- Release of riverwidthEO version 1.2
- updated the classification model by adding ~3,000 labeled and 90,000 unlabeled images.
- Release of River Segment Surface Area Dataset version 1.2 for Ethiopia
- updated the results for all of Ethopia using the updated model.
- Release of two new Jupyter Notebooks for the TopoFlow model, and updates to several existing notebooks:
May, 2020 #
- Release of DSI v1.1.0
- Configuration of the model using a JSON file
- Add support to ECMWF
- Add support to run globally
- Improve compatibility with xarray (performance improvement)
- Release of DAME 5.1.1
- Run models using local data
- Configure with custom ModelCatalog or user
- Support Docker on Windows, Linux and macOS (Beta)
- Validation of URLs
- List and show model configurations and setups
- Integration of data transformations for TopoFlow Data (Weather)
- New release of MIC 0.4.1 - ALPHA
- End-to-end encapsulation of model components.
- Provides templates for creating components.
- Enables testing through Docker images (locally)
- Create a snapshot of a model component, saving code in GitHub and Docker image in DockerHub.
- After testing, pushes changes to model catalog.
- Release of Data Transformation service v1.1
- Support data streaming
- Add new pipeline/adapters to support GLDAS2Cycles transformation
- Add new pipeline/adapters to support variable aggregation in GLDAS data by both time and woreda
- Design a procedure to quickly import external transformation libraries (e.g. Topoflow transformation notebooks)
- Release of riverwidthEO version 1.1
- updated the methodology to detect cloudy pixels.
- updated the methodology to use clustering and classification together to handle hazy images (that are missed by cloud filters)
- updated the classification model by adding more training images.
- Release of River Segment Surface Area Dataset version 1.1 for Ethiopia
- updated the results for 5 basins in Ethopia using the updated model.
- Release of three new Jupyter Notebooks for the TopoFlow model, and three updates to existing notebooks:
- Release of notebook and Python tools for performing scientific variable exploration and grounding to Scientific Variables Ontology (SVO) variables and entries in the World Modelers Indicators (WMI) list:
April, 2020 #
- Release of DAME 4.1.3
- Additional testing and bug fixes (Testing in OSX and Unix). DAME will ask for missing parameters and inputs, using defaults when provided.
- Improvements to messages and logging in the UI. Now the singularity commands, inputs and Docker images are displayed, in case users want to execute models with their own means.
- Improved documentation and examples
- Initial release of MIC 0.2.0 - ALPHA
- Release of Data Transformation service v1.0
- Release of MINT-Data-Sync system
- Monitor when new GLDAS data files become available, upload them to MINT Data Server, and register them in MINT Data Catalog
- Release of River Segment Surface Area Dataset version 1.0 for Ethiopia
- Processed 8710 river segments (covering all of Ethiopia) using machine learning algorithms and satellite imagery to create surface area timeseries.
- Uses Sentinel-2 imagery available from December 2015 until March 2020.
- A csv with surface area timeseries for each river segment is available for download from the MINT Data Catalog.
- Release of riverwidthEO version 1.0
- A python package that processes river segments using machine learning algorithms and satellite imagery (Sentinel-2) to create surface area timeseries (delivered as a csv file).
- Uses descarteslabs API to download data for any given segment.
- Provides user with options to provide points on a river as input or just provide a region or country to select predefined points on the river. These predefined points are available for rivers (>100 meters in width) across the globe.
- Release of a Jupyter Notebook for the TopoFlow model with an overview and introduction to new users
March, 2020 #
- Release of MINT-UI 4.3.4
- Users can run their ModelConfigurations
- Release of DAME 3.3.0
- Execute models from MINT on Desktop/Server
- Release of Model Catalog API 1.4.0
- Users can insert their ModelConfigurations
February, 2020 #
- Release of MINT-UI 4.3.0 (Feb 26)
- Release of MINT-UI 4.2.1) (Feb 19)
- Release of MINT-UI 4.2.0) (Feb 14)
- Release of MINT-UI 4.1.0(https://mint.isi.edu/)
- Fixing bugs and usability improvements Release Release 4.1.0 · mintproject/mint-ui-lit
January, 2020 #
- Release of MINT-UI 4.0.0
- Bug fixes and usability improvements Release Release 4.0.0-0 · mintproject/mint-ui-lit
December, 2019 #
MINT Data Catalog #
- Releases of MINT Data Catalog UI
- MINT Data Catalog Github repository. Specifically,
MINT Model Catalog: #
Ensemble Manager API #
- Release 1.0.0:
- Execute models using singularity
- Include parallelism option in config to do multiple model runs
- Adding DELETE request to executions
- Local delete for cache of execution
Ingestion API: #
- Release 1.1.0:
- Take model execution data and ingest into a database to enable interactive dashboards
- Gather model ensemble execution results
November, 2019 #
MINT NetCDF Convention #
- Release of MINT NetCDF convention
- Release of MINT-GeoViz
October, 2019 #
Data Catalog #
- Release of D-REPR: a library for reading heterogeneous datasets into common representations. Check its GitHub for more information.
May, 2019 #
Model Catalog #
- New content: The MINT model catalog has been expanded to include a semantic representation of units, scientific variables and links to Wikidata. Check the release on GitHub for more information.
- New APIs for adding content: We have expanded our APIs with a new client based on Open API that allows anyone to insert models in the model catalog. The API is accessible in the following link: https://api.models.mint.isi.edu/v0.0.2/ui/#/
- New APIs for accessing content: 3 new methods have been added to our GRLC API. The first one, to obtain additional information of a Scientific Variable given its label (getStandardVariableMetadata). The other two (getInputCompatibleConfig) and (getOutputCompatibleConfig) retrieve those models or software compatible with a target model. This facilitates understanding which software has variables that may interoperate with other software.
- A new client for accessing content: With our new Python client, now it is possible to access the content in the model catalog without having to issue SPARQL queries or API queries. Check it out here.
- New examples: Check out our notebook for examples on how to use the client.
- The Model Explorer has been updated to show the contents of models in a more user-friendly manner. Check here the latest version. Please open new issues on GitHub if you find bugs.