DQOps Data Observability

DQOps data testing

DQOps has hundreds of built-in table and column checks, configured in the UI or in YAML files, plus table comparisons and failing-row samples. A rule mining engine proposes checks from profiling.

Compare: data testing tools

• SQL Table Tests: Table and column-level checks span nulls (completeness), patterns and accepted values (validity), numeric ranges (accuracy), uniqueness, freshness, and Lookup Key integrity checks for referential consistency (docs).

• Table Comparison Tests: Table comparison checks for row count match and column count match between two data sources are supported (docs).

• Inspect Failed Rows: Error samples capture individual invalid rows for failed column-level checks, stored in the .data/error_samples folder (docs).

• DataFrame Tests: DQOps is a server-based platform that connects to databases via SQL; no Python DataFrame (Pandas/Spark) testing library is provided (docs).

• Unstructured Data Tests: No LLM-based semantic validation for unstructured text fields found; checks are SQL-based pattern and format validation (docs).

• Data Diff: No row-level data diff (comparing individual records between two datasets) found; table comparisons are aggregate-level (row/column counts only) (docs).

• Data Contracts: No data contract feature (machine-readable producer-consumer schema and quality SLA agreements) found in documentation; YAML check configurations define tests but do not formalize ownership or enforcement contracts (docs).

DQOps data observability

ML anomaly checks over 90-day rolling windows watch volume, freshness and numeric metrics, and every check configuration lives in YAML for version control.

Compare: data profiling tools · data observability tools

• Data Profiling: Profiling checks assess initial data quality by collecting statistics including row counts, value ranges, null counts, and sample values (docs).

• Anomaly Detection: ML-based anomaly detection using 90-day rolling windows to detect outliers in numeric metrics, volume, freshness, and distinct counts (docs).

• Freshness Monitors: Data freshness, staleness, ingestion delay, and reload lag checks monitor data timeliness with anomaly detection variants (docs).

• Volume Monitors: Table volume checks monitor row counts with daily and partition-level variants, detecting significant increases or decreases (docs).

• Schema Monitors: Schema checks detect column additions, deletions, reordering, and data type changes at both table and column level (docs).

• Field Monitors: Column-level checks monitor null percentages, distinct values, numeric statistics, and anomalies per individual field (docs).

• Distribution Monitors: Column anomaly checks monitor sum, mean, median, min, and max anomalies detecting distributional shifts (docs).

• Segment Monitors: Data grouping feature runs checks per distinct data stream, vendor, or department values, producing separate KPIs per group (docs).

• Custom SQL Monitors: Custom SQL checks at both table and column level allow user-defined SQL expressions for arbitrary validation logic (docs).

• Partition-Based Monitors: Partition checks apply GROUP BY time_period_column and WHERE filters to monitor specific partitions or date ranges independently (docs).

• Window-Based Monitors: Anomaly checks use a 90-day rolling window; change detection checks compare against 1-day, 7-day, and 30-day windows (docs).

• UI-Based Monitors: Full web UI for configuring, running, and reviewing data quality checks, incidents, and dashboards (docs).

• Monitors-as-Code: All check configurations are stored as YAML files in a DQOps User Home folder, enabling version control and code-first data quality management (docs).

• Pipeline Monitoring: No pipeline job monitoring features documented (docs).

DQOps incident management

DQOps sends alerts, tracks incidents through triage and resolution, and opens tickets.

• Alert Notifications: Slack webhook integration and generic webhook support for forwarding incident notifications to external systems (docs).

• Incident Triage: Incidents can be managed through status updates (open, acknowledged, resolved, muted) to track resolution progress (docs).

• Incident Resolution: Incidents support full lifecycle tracking including resolved status and links to external ticketing systems like Jira and ServiceNow (docs).

• Ticketing Integrations: Webhook notifications include custom issue URL fields linking to Jira, Azure DevOps, and ServiceNow tickets (docs).

• On-Call Integrations: No native PagerDuty or Opsgenie integration found; incident alerts are delivered via Slack or generic webhooks (docs).

• Alert Feedback: No explicit false positive marking or alert feedback mechanism found; severity thresholds can be adjusted but no ML feedback loop is documented (docs).

DQOps data quality analytics

DQOps' data quality KPI dashboards and health scores are its core strength.

• Data Quality Dashboards: 50+ built-in data quality dashboards via Looker Studio show KPIs, anomalies, incidents, and quality trends (docs).

• Data Health Scores: Data Quality KPIs measure the percentage of passed checks at table, column, database, domain, and vendor levels (docs).

• Incident Resolution Metrics: DQOps has incident grouping/workflow and notification features, but no MTTD/MTTR, resolution-rate, or owner/team performance analytics dashboard is documented (docs).

DQOps data lineage

DQOps shows table- and column-level lineage that you configure by hand or import from Marquez, the OpenLineage service.

Compare: data lineage tools

• Table-Level Lineage: OpenLineage/Marquez integration provides table-level lineage tracking (docs).

• Column-Level Lineage: DQOps supports both table-level and column-level data lineage, configured manually or imported from Marquez (docs).

• End-to-End Lineage: No end-to-end cross-system lineage (spanning BI tools, orchestrators, warehouses) documented (docs).

• Impact Analysis: No downstream impact analysis of data quality issues on dependent tables or BI reports documented (docs).

DQOps data governance

DQOps classifies data.

• Data Classification: DQOps includes a dedicated PII column checks category that detects and classifies sensitive patterns — US phone numbers, email addresses, ZIP codes, and IP addresses — in text columns (docs).

• Data Catalog: DQOps has no built-in data catalog for discovering and documenting data assets; it focuses on quality checks and observability (docs).

• Data Catalog Integration: No data catalog integrations (Alation, Atlan, Collibra, etc.) found in DQOps documentation (docs).

• Data Products: "Data domains" in DQOps is an Enterprise multi-tenancy/isolation feature, and "data grouping" segments rows within a single table by a discriminator column (location, business unit, vendor) — neither groups monitors/quality scores by data product or domain in a data-mesh sense (docs).

DQOps workflows

DQOps has a REST API, a Python client, webhooks and a dbt integration. An Airflow operator halts DAGs when DQOps finds fatal issues.

• Scheduling: DQOps supports configurable scheduling at multiple levels for monitoring checks, with daily and monthly check variants (docs).

• CI/CD Integrations: DqopsAssertTableStatusOperator halts Airflow DAGs on fatal issues; REST API and Python client let orchestrators and CI pipelines gate on quality status (docs).

• Webhooks: Generic webhook support delivers incident JSON payloads to external systems for custom notification routing (docs).

• REST / GraphQL API: REST API available for all platform operations; Python client package on PyPI provides programmatic access (docs).

• Python SDK: REST API Python Client on PyPI enables programmatic execution of checks, statistics collection, table imports, and job management (docs).

• dbt Integration: DQOps integrates with dbt Cloud and dbt Core to run data quality checks as part of dbt pipeline workflows (docs).

• MCP Server: No MCP server for AI client integration found in documentation (docs).

DQOps AI agents

DQOps proposes checks automatically from profiling with its rule mining engine. It has no AI chat, AI triage or MCP server.

• Automated Test Generation: Rule mining engine uses ML to automatically propose check configurations based on statistics, sample values, and profiling results (docs).

• AI Chat Assistant: DQOps has no in-product AI chat assistant; its rule miner derives check configurations automatically from data statistics without any user interaction (docs).

• Natural Language Test Generation: Checks are configured via the UI or YAML/Python (about 150 predefined checks); no AI chat or plain-English prompt-to-check generation interface is documented or listed on pricing (pricing).

• AI Issue Triaging: No AI-powered root cause analysis or automated issue triaging feature documented (docs).

• AI Issue Resolution: No AI-powered automated issue resolution feature documented (docs).