Validio Data Observability

Validio data testing

Validio checks SQL tables, compares tables, shows the rows that fail, and checks unstructured text by calling warehouse LLM functions such as Snowflake Cortex.

• SQL Table Tests: Validators cover null/empty string/distinct/duplicate checks (completeness/uniqueness), referential integrity (consistency), numeric and categorical distribution (accuracy/validity), freshness, and custom SQL assertions (docs).

• Unstructured Data Tests: Custom SQL Sources call warehouse-native LLM functions (Snowflake Cortex, BigQuery ML, Databricks AI Functions) to produce scored/flagged columns, which Validio then validates — enabling sentiment, PII, toxicity, and semantic checks on free-text data without sending rows to third-party LLMs (docs).

• Table Comparison Tests: Reconciliation validators check data consistency across sources — comparing aggregate metrics between two tables to validate replication fidelity or migration correctness (docs).

• Inspect Failed Rows: The Debug feature automatically generates SQL to reproduce the incident and provides a 'Load Samples' function to surface representative failing rows — users can copy and run the query in their preferred database tool (docs).

• DataFrame Tests: Validio connects to data warehouses and streams via SQL; no Python DataFrame (Pandas, Spark, Polars) in-memory validation library is provided (docs).

• Data Diff: No row-by-row dataset comparison (dev vs. prod, pre/post-migration diff surfacing inserted/deleted/changed records) found; Reconciliation validators compare aggregate metrics across sources (docs).

• Data Contracts: No machine-readable data contract feature (producer-consumer schema and quality SLA agreements) found in documentation (docs).

Validio data observability

ML thresholds learn baselines from history, validators segment by any dimension, and you manage validators in the UI or as Python code with Validio Code.

Compare: data observability tools · data profiling tools

• Data Profiling: The Schema & Profiling tab computes per-column null %, unique values, min, max, and mean; segment-level profiling is also supported (docs).

• Anomaly Detection: Dynamic thresholds use ML-based anomaly detection, automatically learning baselines from historical metric values without requiring hardcoded expectations (docs).

• Freshness Monitors: Freshness validators monitor data recency and detect when tables have not been updated within their expected ingestion window (docs).

• Volume Monitors: Row Count validators track record quantity over time and detect anomalous volume changes using dynamic or fixed thresholds (docs).

• Schema Monitors: Schema monitors detect metadata changes; schema detection runs hourly across all connected sources (docs).

• Field Monitors: Null, distinct, duplicate, empty string, and numeric/categorical distribution validators track column-level metrics as continuous time series with threshold-based alerting (docs).

• Distribution Monitors: Categorical Distribution and Numeric Distribution validator types track statistical distributions per column over time and detect abnormal shifts (docs).

• Segment Monitors: Segmentation validates metrics per distinct value of a dimension column (equivalent to GROUP BY), with support for multi-field segmentation combinations (docs).

• Custom SQL Monitors: Custom SQL validators allow user-defined SQL queries tracked as continuous monitors; AI-powered SQL generation assists with creating custom validator queries (docs).

• Partition-Based Monitors: SQL Filters apply WHERE clauses to scope validators to a specific data slice; combined with tumbling windows, this enables time-partitioned scanning (e.g. last 24h) to reduce compute cost on large tables (docs).

• Window-Based Monitors: Tumbling Windows define fixed-sized non-overlapping time intervals (e.g., hourly, daily) for metric evaluation; Global Windows perform full data loads per polling cycle (docs).

• UI-Based Monitors: All validator types can be created and configured through the Validio web UI without writing code (docs).

• Monitors-as-Code: Validio Code is a Python-based IaC solution for version-controlled validator management with a Plan/Apply workflow that integrates with CI/CD pipelines (docs).

• Pipeline Monitoring: Validio integrates with Airflow and dbt to trigger validations but does not track orchestrator job stats (run times, DAG durations, schedule adherence) (docs).

Validio incident management

Validio has a full incident workflow: alerts, triage, feedback, resolution, ticketing and on-call integrations.

• Alert Notifications: Notifications delivered via Slack, Microsoft Teams, email, Jira, ServiceNow, and webhooks, configurable through notification rules (docs).

• Incident Triage: Incidents can be reviewed on the Incidents page with severity classification (Low/Medium/High), commenting, and status changes; notification muting is also supported (docs).

• Alert Feedback: Incidents can be marked 'Not an Anomaly' to flag false positives, and notification muting is documented — providing feedback signals to the monitoring system (docs).

• Incident Resolution: Incidents can be marked as Resolved or Not an Anomaly (false positive), with full incident history and root cause analysis via the lineage graph (docs).

• Ticketing Integrations: Native Jira and ServiceNow integrations create tickets from incidents with bidirectional status synchronization (docs).

• On-Call Integrations: PagerDuty is listed as a notable integration on the Validio homepage for on-call alerting; the webhook notification channel can deliver JSON payloads to any HTTP endpoint including PagerDuty and Opsgenie APIs (website).

Validio data quality analytics

Validio's dashboards, health scores and resolution metrics track data quality over time.

• Data Quality Dashboards: The Overview Dashboard provides a summary of data quality KPIs, incident trends, and asset health across all monitored sources (docs).

• Data Health Scores: Data quality scoring reflects incident status per asset, providing a continuous health indicator that updates as incidents are detected and resolved (docs).

• Incident Resolution Metrics: The Incidents page includes a 'Time to resolution' graph showing resolution-time trends and daily incident volume over a selectable time range (calculated once incidents are marked Resolved/False Positive), though it lacks owner/team breakdown or an explicit MTTD metric (docs).

Validio data lineage

Validio builds table- and field-level lineage from query history, dbt manifests and OpenLineage events, with impact analysis.

Compare: data lineage tools

• Table-Level Lineage: Asset-level lineage tracks table-to-table connections, derived automatically from query logs, Information Schema metadata, and dbt manifests (docs).

• Column-Level Lineage: Field-level lineage tracks column-to-column dependencies, visualized by expanding nodes in the lineage graph (docs).

• End-to-End Lineage: Lineage spans data warehouses, dbt transformations, and BI tools (Looker, Tableau, Power BI, Sigma, Omni) for full-stack end-to-end tracing (docs).

• Impact Analysis: Root cause analysis traces incident propagation downstream through the lineage graph, surfacing affected dependent assets and field-level severity indicators (docs).

Validio data governance

Validio's catalog covers tables, streams, BI and dbt assets with a business glossary.

Compare: data catalog tools

• Data Catalog: Built-in Data Catalog discovers tables, streams, BI, and dbt assets from credentials, with a business glossary, domains, classifications, and optional four-eyes approval of catalog changes (docs).

• Data Catalog Integration: Atlan integration is documented for pushing data quality metadata into external catalog platforms (docs).

• Data Classification: Validio's Classifications feature is a manual governance tagging system (CDE, Regulatory Scope, Data Sensitivity labels) — not an automated scanner that detects PII/PHI/PCI using pattern matching or ML classifiers (docs).

• Data Products: Validio's 'Domains' feature organizes glossary terms and data assets by business area for lineage visualization, but does not group validators/monitors or roll up quality scores under a domain/data-product; a 'customer-facing data products' page is only a use-case description, not a grouping mechanism (docs).

Validio workflows

Validio has an API, a Python SDK, webhooks and a dbt integration. Validio Code deploys validators from CI/CD with a plan and apply workflow.

• Scheduling: Validators run on configurable polling cycles; global windows evaluate on clock time and tumbling windows define fixed-interval scan schedules (docs).

• CI/CD Integrations: Validio Code integrates with CI/CD pipelines for automated validator deployment via a Plan/Apply workflow; hard blocking of deployments when checks fail is not explicitly documented (docs).

• Webhooks: Webhook notification channel delivers JSON payloads to custom HTTP endpoints when incidents are detected (docs).

• REST / GraphQL API: GraphQL API enables full platform interaction — managing sources, validators, incidents, notifications, and catalog assets (docs).

• MCP Server: Validio MCP Server provides a natural language interface for configuring validators and managing incidents via AI assistants (docs).

• Python SDK: Python SDK provides a GraphQL client with full pydoc and Sphinx documentation for programmatic platform interaction (docs).

• dbt Integration: Both dbt Cloud and dbt Core are supported, with dbt Manifest files used for lineage and transformation pipeline validation (docs).

Validio AI agents

Validio writes custom validators from plain-language prompts and suggests validators automatically. An MCP server is also available.

• AI Chat Assistant: A built-in Generate SQL button in custom validators accepts a plain-language description and uses a configured LLM (OpenAI, Anthropic, Bedrock, Gemini) to produce the SQL check; the MCP server also accepts prompts like "Create a validator to ensure no duplicate emails in users" and creates the validator (docs).

• Natural Language Test Generation: AI SQL generation creates custom validator queries and filters from context using LLM providers (OpenAI, Anthropic, AWS Bedrock, Google Gemini) (docs).

• Automated Test Generation: Validator Recommendations automatically analyze data sources to suggest tailored validators, windows, filters, and segmentations using LLM or ML analysis — applied via a review-and-approve workflow (docs).

• AI Issue Triaging: Root Cause Analysis (RCA) traces incidents through lineage connections to identify upstream assets — lineage-based propagation, not AI-powered reasoning. No LLM or ML analysis of the failure is performed (docs).

• AI Issue Resolution: No AI-suggested or automated remediation for data quality incidents documented in Validio (docs).