# Deequ Data Quality

> Open-source Spark library from AWS Labs to define data quality constraints, profile columns, and get suggested checks on large datasets.

Source: https://toolsfordata.com/tools/deequ/ · Updated 2026-11-13

[GitHub](https://github.com/awslabs/deequ) (3.6k stars)

Best for: data engineers running Apache Spark at scale who want unit-test-style data checks in Scala or Python (PyDeequ).

## What it's used for

Deequ is used to check data quality on large datasets in Apache Spark: you declare constraints in Scala or Python (PyDeequ), and Deequ runs them as Spark jobs, suggests new constraints from a column profile and flags anomalies in stored metrics.

## Pricing

Deequ is free under Apache 2.0; you pay only for the Spark compute it runs on. AWS Glue Data Quality is the managed AWS service built on Deequ.

## Pricing

- Open source: OSS library
- Source: https://github.com/awslabs/deequ (checked 2026-11-02)

## Where it fits

- data testing (core): Open-source Spark library from AWS Labs where data tests are constraints in a VerificationSuite, written in Scala or Python with PyDeequ and run as Spark jobs, with suggested constraints and anomaly detection on stored metrics.
- open source (core): Open-source Spark library under Apache 2.0 from AWS Labs for data quality constraints, profiling, and anomaly detection; AWS Glue Data Quality is the managed AWS service built on it.
- data profiling (strong): Open-source Spark library from AWS Labs with a column profiler that computes completeness, distinct values, data types, and statistics on large datasets and suggests constraints from the data profile.

## Integrations

1 integrations, each sourced from the vendor docs.

- Data Transformation: Apache Spark

## Alternatives

- [Great Expectations](https://toolsfordata.com/tools/great-expectations/): Consider Great Expectations for checks beyond Spark: files, SQL databases and DataFrames.
- [Spark Expectations](https://toolsfordata.com/tools/spark-expectations/): Consider Spark Expectations if bad rows should be quarantined before they reach the target table.
- [DQX by Databricks](https://toolsfordata.com/tools/dqx/): Consider DQX on Databricks: PySpark checks generated from profiling, in YAML or Python.
- [SparkDQ](https://toolsfordata.com/tools/sparkdq/): Consider SparkDQ for a lighter, config-driven PySpark check library.
- [Soda Core](https://toolsfordata.com/tools/soda-core/): Consider Soda Core for YAML checks that run on warehouses as well as Spark.
- [AWS Glue Data Quality](https://toolsfordata.com/tools/aws-glue-data-quality/): The managed option on AWS: Glue Data Quality builds on Deequ, paid per use.
