Datafold Data Quality

Datafold data testing

Datafold's core feature is data diff: value-level, row-by-row comparison of two tables in the same or different databases. It also runs SQL tests and shows the rows that differ or fail.

Compare: data comparison tools · data testing tools

• SQL Table Tests: Data Test monitors validate data with business rules and surface specific records that fail (docs).

• Table Comparison Tests: Data Diff monitors compare row counts, aggregates, and values between any two datasets within or across databases (docs).

• Data Diff: Core feature — value-level row-by-row comparison between two datasets with primary key analysis and Match Score (docs).

• Inspect Failed Rows: Data Test monitors surface specific records that fail business rules, and results materialization shows row-level match/mismatch details (docs).

• DataFrame Tests: Datafold is a SaaS platform; no Pandas, Spark, or Polars DataFrame testing support mentioned (docs).

• Unstructured Data Tests: No LLM-based semantic validation of free-text or unstructured columns mentioned (docs).

• Data Contracts: No data contract format or producer-consumer schema enforcement mentioned (docs).

Datafold data observability

Datafold profiles columns and runs monitors for data diffs, schema changes, metrics with ML anomaly thresholds and custom SQL. Monitors are set up in the UI or as YAML.

Compare: data observability tools · data profiling tools

• Data Profiling: Column Profiles display aggregate statistics including averages, counts, ranges, and histogram charts (docs).

• Anomaly Detection: Metric monitors include anomaly detection thresholds using ML-driven monitoring for statistical deviations (docs).

• Freshness Monitors: Freshness is a standard metric type in Datafold's metric monitors (docs).

• Volume Monitors: Row count is a standard metric type in Datafold's metric monitors (docs).

• Schema Monitors: Schema Change monitors alert on unexpected table schema modifications (column add/remove/rename) (docs).

• Field Monitors: Cardinality and column-level metrics are tracked over time as standard metric monitor types (docs).

• Distribution Monitors: Column Profiles include histogram charts tracking value distributions; column-level metric monitors track distribution over time (docs).

• Custom SQL Monitors: Metric monitors support custom SQL expressions tracked as metrics over time (docs).

• Partition-Based Monitors: Metric monitor configs include a filter field (WHERE clause) that can scope evaluation to a recent data partition (e.g. WHERE updated_at > NOW() - INTERVAL) (docs).

• UI-Based Monitors: Monitors can be created and managed through the Datafold UI as well as via YAML code (docs).

• Monitors-as-Code: Monitors-as-Code supports managing all four monitor types via version-controlled YAML with cron/interval scheduling (docs).

• Segment Monitors: The metric monitor API schema has no GROUP BY or segment dimension field; per-segment breakdowns are not supported (docs).

• Window-Based Monitors: No tumbling/sliding time window aggregation parameters exist in the monitor schema; only schedule-based evaluation (docs).

• Pipeline Monitoring: Datafold has no orchestrator job-level monitoring (Airflow DAG durations, dbt run times, Spark job performance) (docs).

Datafold incident management

Datafold sends alerts and integrates with ticketing and on-call tools. It has no incident triage workflow or data quality dashboards.

• Alert Notifications: Monitors support email, Slack, Teams, PagerDuty, Opsgenie, webhooks, and Jira notifications on failure (docs).

• Ticketing Integrations: Jira is listed as a notification channel for monitor alerts (docs).

• On-Call Integrations: PagerDuty and Opsgenie are both listed as supported notification channels in monitor configuration (docs).

• Alert Feedback: The monitor API schema has no snooze, mute, or feedback mechanism for suppressing or retraining on specific anomalies (docs).

• Incident Triage: Not explicitly documented — no owner assignment, priority setting, or grouping workflow mentioned (docs).

• Incident Resolution: Not explicitly documented — no mark-as-resolved or false-positive logging workflow mentioned (docs).

Datafold data quality analytics

• Data Health Scores: No per-table quality score or health percentage mentioned; Match Score applies to data diffs only (docs).

• Incident Resolution Metrics: Datafold supports alerts via Slack/PagerDuty/email/webhook, but no MTTD/MTTR dashboard, incident volume trends, or resolution-rate-by-owner reporting is documented (docs).

• Data Quality Dashboards: No dedicated data quality dashboard with KPIs or test pass rates documented (docs).

Datafold data lineage

Datafold's Data Explorer builds column-level lineage from warehouse metadata, query history, dbt and BI tools, end to end, with impact analysis in pull requests.

Compare: data lineage tools

• Table-Level Lineage: Data Knowledge Graph automatically maps table-level lineage from dbt manifests and BI tool integrations (docs).

• Column-Level Lineage: Column-level lineage traces data field history, transformations, and dependencies with upstream/downstream views (docs).

• End-to-End Lineage: Lineage spans the full stack via integrations with dbt, Looker, Tableau, Mode Analytics, Power BI, and Hightouch (docs).

• Impact Analysis: Downstream impact analysis shows BI dashboards and data app dependencies affected by a data diff or anomaly (docs).

Datafold data governance

Datafold's Data Explorer is a searchable inventory of data assets with their lineage, business logic, usage and owners.

• Data Catalog: Data Explorer / Data Knowledge Graph provides a searchable inventory of data assets with lineage, business logic, usage, and ownership (docs).

• Data Catalog Integration: Datafold has its own catalog (Data Knowledge Graph); no integration to push quality scores into external catalogs documented (docs).

• Data Classification: No automated PII/PHI pattern matching or ML-based column classification mentioned (docs).

• Data Products: Data Explorer groups assets by connection, tags, data owner, and asset type, but there is no documented concept of grouping monitors/quality scores under a data product or domain (docs).

Datafold workflows

Datafold has a REST API, a Python SDK, webhooks, an MCP server and a dbt integration. It runs data diffs and impact analysis in pull requests on GitHub, GitLab, Azure DevOps and Bitbucket.

• Scheduling: Monitors support interval (hourly/daily) and cron expression scheduling (docs).

• CI/CD Integrations: Datafold integrates with GitHub, GitLab, Azure DevOps, and Bitbucket for deployment testing and PR-blocking quality checks (docs).

• Webhooks: Webhooks are a supported notification channel for monitor alerts and custom integrations (docs).

• REST / GraphQL API: Comprehensive REST API covers monitors, data diffs, data sources, CI runs, audit logs, BI integrations, and lineage exploration (docs).

• MCP Server: Public HTTP MCP server (app.datafold.com/mcp/) enables Claude, Cursor, VS Code, and other AI clients to manage diffs, monitors, and lineage (docs).

• Python SDK: Datafold SDK is pip-installable and supports programmatic diff creation, dbt artifact submission, and custom CI integration (docs).

• dbt Integration: Deep dbt integration: manifest.json import, dbt Cloud/Core CI, dbt metadata sync for model descriptions, tags, and primary keys (docs).