data-profiling

The Open Context Layer for Data and AI , OpenMetadata is the open platform for building trusted data context and business semantics for humans, AI assistants, and agents.

1 Line of code data quality profiling & exploratory data analysis for Pandas and Spark DataFrames.

Always know what to expect from your data.

Cleanlab's open-source library is the standard data-centric AI package for data quality and machine learning with messy, real-world data and labels.

Visualize and compare datasets, target values and associations, with one line of code.

Data Contracts engine for the modern data stack. https://www.soda.io

🚚 Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark

First open-source data discovery and observability platform. We make a life for data practitioners easy so you can focus on your business.

Automatically find issues in image datasets and practice data-centric computer vision.

Know your data better！Datavines is Next-gen Data Observability Platform, support metadata manage and data quality.

Engine for AI/ML/Data tracking, visualization, explainability, drift detection, and dashboards for Polyaxon.

Monitor the stability of a Pandas or Spark dataframe ⚙︎

Desbordante is a high-performance data profiler that is capable of discovering many different patterns in data using various algorithms. It also allows to run data cleaning scenarios using these algorithms. Desbordante has a console version and an easy-to-use web application.

Code review for data in dbt

Databricks framework to validate Data Quality of pySpark DataFrames and Tables

Lineage metadata API, artifacts streams, sandbox, API, and spaces for Polyaxon

Data Quality and Observability platform for the whole data lifecycle, from profiling new data sources to full automation with Data Observability. Configure data quality checks from the UI or in YAML files, let DQOps run the data quality checks daily to detect data quality issues.

Installer and quickstart for DataKitchen's open-source DataOps TestGen and DataOps Observability. Installs TestGen with Docker Compose or pip, Observability with Docker Compose.

🚕 A spreadsheet-like data preparation web app that works over Optimus (Pandas, Dask, cuDF, Dask-cuDF, Spark and Vaex)

Papers about training data quality management for ML models.