DataKitchen Data Observability

DataKitchen data testing

DataKitchen TestGen profiles your tables and auto-generates SQL tests across 49 types, from null and duplicate checks to custom queries.

Compare: data testing tools

• SQL Table Tests: 49 test types cover null/missing checks (completeness), duplicate detection (uniqueness), value range and format validation (accuracy/validity), timeliness assertions, and custom SQL — all executed in-database against SQL tables (docs).

• DataFrame Tests: TestGen operates directly on SQL databases; no Pandas, Spark, or PySpark DataFrame testing library is provided (docs).

• Unstructured Data Tests: No LLM-based semantic validation of free-text or unstructured warehouse columns documented in TestGen (docs).

• Data Diff: No row-by-row comparison between two datasets documented in TestGen (docs).

• Data Contracts: No machine-readable producer-consumer schema and quality SLA contracts documented in TestGen (docs).

• Inspect Failed Rows: The MCP server exposes data samples to external AI agents; the Data Catalog shows up to 100 generic sample rows per table (not rows that triggered a specific test failure); the API returns test run summaries. No UI feature surfacing the specific rows that caused a test to fail is documented (docs).

• Table Comparison Tests: Accuracy tests compare values 'against baselines, references, and expected distributions' but explicit cross-table aggregate comparison (row counts, sums between source and destination) is not documented as a named feature (docs).

DataKitchen data observability

TestGen profiles 55 characteristics per column and detects volume and metric anomalies with a SARIMAX time-series model. DataKitchen Observability tracks pipeline runs in Airflow, dbt, Azure Data Factory and Databricks.

Compare: data observability tools · data profiling tools

• Data Profiling: Scans tables to collect 55 column-level characteristics including null %, distinct count, min/max, standard deviation, value distributions, pattern analysis, and data type detection (docs).

• Anomaly Detection: Volume and metric monitors use a SARIMAX ML time-series prediction model to detect anomalies without hardcoded thresholds; freshness monitors use gap-based detection. Requires ~30 runs to train (docs).

• Freshness Monitors: Freshness monitors compute fingerprints of key columns to detect late or out-of-schedule table updates (docs).

• Volume Monitors: Volume monitors track row count changes over time and use ML anomaly detection to flag deviations from expected ranges (docs).

• Schema Monitors: Schema monitors detect table additions/drops and column additions or modifications (docs).

• Custom SQL Monitors: Metric monitors track user-defined SQL expressions that return a single numeric value over time, with ML anomaly detection (docs).

• UI-Based Monitors: All monitor types and test generation are configurable through the TestGen web UI without writing code (docs).

• Pipeline Monitoring: DataKitchen Observability monitors batch pipeline runs (Airflow, dbt, Azure Data Factory, Databricks) and streaming pipelines, tracking job execution status end-to-end across tools and teams (docs).

• Field Monitors: TestGen's four monitor types are freshness, volume, schema, and custom SQL metric — continuous per-column statistics (null rate, mean, stddev) tracked as time series are not a named monitor type (docs).

• Distribution Monitors: No distribution shift monitoring (histogram tracking, category frequency drift) documented in TestGen monitors (docs).

• Segment Monitors: No dimension-based segment monitoring (GROUP BY a category column) documented in TestGen (docs).

• Partition-Based Monitors: No partition-based or time-filter scoped scanning to reduce compute cost documented in TestGen monitors (docs).

• Window-Based Monitors: No tumbling or sliding time window evaluation documented in TestGen (docs).

• Monitors-as-Code: Tests can be exported and imported via the API for environment promotion; Enterprise pricing mentions git-based version control — but YAML/Python code-first authoring of monitors is not explicitly documented (pricing).

DataKitchen incident management

DataKitchen sends alerts and opens tickets. It has no incident triage workflow.

• Alert Notifications: TestGen sends email notifications for test failures, profiling changes, monitor anomalies, and quality score drops. Observability adds rule-based alerts (trigger-condition-action) delivered to email, Slack, Teams, Jira, and other tools (docs).

• Ticketing Integrations: DataKitchen Observability explicitly supports Jira integration via rule-based alerts; ServiceNow is not documented (docs).

• Incident Triage: No incident triage workflow (classify, assign, group) documented in TestGen (docs).

• Alert Feedback: No false positive marking, snoozing, or anomaly feedback mechanism documented in TestGen (docs).

• Incident Resolution: No incident resolution lifecycle documented in TestGen (docs).

• On-Call Integrations: No PagerDuty or Opsgenie integration documented in TestGen or Observability; alert channels cover email, Slack, Teams, and Jira (docs).

DataKitchen data quality analytics

DataKitchen shows dashboards and health scores for data quality.

• Data Quality Dashboards: Project dashboard displays quality scorecards with live scores and history sparklines, filterable by table group, quality dimension, domain, and stakeholder group (docs).

• Data Health Scores: Per-table and per-column quality scores combining profiling and test results with importance weighting; separate CDE score tracks quality of critical data elements (docs).

• Incident Resolution Metrics: TestGen surfaces failure triage (new vs. chronically flaky failures) and quality scorecards, but no MTTD/MTTR dashboards, incident volume trends, or resolution-rate-by-owner analytics are documented (website).

DataKitchen data governance

TestGen keeps a catalog of profiled tables and flags PII columns from names and data.

Compare: data classification tools

• Data Catalog: Searchable hierarchical catalog of tables and columns with profiling stats, quality scores, metadata tags (8 categories), descriptions, CDE/XDE flagging, and data preview up to 100 rows (docs).

• Data Classification: Automatically scans column names and data patterns to identify PII (emails, SSNs, credit card numbers, phone numbers, national IDs) using pattern matching; flagged columns are masked for users without PII access (docs).

• Data Catalog Integration: No integration to push quality metadata into external catalogs (Atlan, Alation, Collibra, Purview) documented in TestGen (docs).

• Data Products: TestGen scorecards roll up by table, project, DAMA category, Critical Data Element, or business goal — a configurable scoring rollup, but not a data-mesh "data product"/domain construct for grouping monitors and tests (website).

DataKitchen workflows

DataKitchen has a REST API built for CI/CD, to trigger profiling and test runs and poll for results, plus an MCP server and a dbt integration.

• Scheduling: Built-in scheduler supports preset frequencies (hourly, daily, weekly, specific days) and custom cron expressions for profiling runs, test runs, and monitor checks (docs).

• CI/CD Integrations: The REST API is designed for CI/CD pipeline integration — trigger profiling and test runs, poll for status, and retrieve results; CLI provides scriptable commands for pipeline automation (docs).

• REST / GraphQL API: REST API with OAuth 2.1 authentication — triggers profiling and test runs, polls job status, retrieves results, and imports/exports test definitions for environment promotion (docs).

• MCP Server: MCP server exposes tools (list test results, run tests, run profiling, retrieve source data samples), prompts (investigate failures, compare runs), and resources (test type catalog) to AI coding agents (docs).

• dbt Integration: DataKitchen Observability includes a pre-built dbt Core integration agent for monitoring dbt pipeline runs; dbt manifest import or test result ingestion into TestGen is not documented (docs).

• Webhooks: No webhook notification channel in TestGen; only email notifications are supported (docs).

• Python SDK: No pip-installable Python library for programmatic import; TestGen provides a CLI and REST API but not a native Python SDK (docs).

DataKitchen AI agents

DataKitchen generates tests automatically from profiling. An MCP server is also available.

• Automated Test Generation: Core feature: automatically generates tests across 49 types from profiling results using statistical baselines — no manual authoring required (docs).

• AI Chat Assistant: No built-in in-product conversational AI assistant in TestGen; AI capabilities are exposed externally via the MCP server to agents like Claude or Cursor (docs).

• Natural Language Test Generation: TestGen's MCP server lets Claude/Cursor/Copilot trigger commands like "generate tests and run them," but the tests themselves are still auto-written from data profiling (the existing Automated Test Generation feature) rather than derived from a user's plain-English description of a specific check (website).

• AI Issue Triaging: TestGen exposes investigation data via MCP to external AI agents but does not itself perform AI-powered root cause analysis — the analysis is done by the external AI, not TestGen (docs).

• AI Issue Resolution: No AI-suggested or automated remediation in TestGen (docs).