Lightup Data Observability

Lightup data testing

Lightup checks SQL tables, compares tables, and runs row-by-row comparisons on join keys that report a match percentage. You can inspect the rows that fail.

Compare: data comparison tools

• SQL Table Tests: Conformity metrics check the percentage of rows meeting conditions (uniqueness, value ranges, multi-column rules); null percent, aggregation, and SQL metrics add further rule-based coverage (docs).

• Table Comparison Tests: Compare Aggregate Metrics tracks the degree to which two aggregate metrics (e.g., row counts or sums from different tables) align, surfacing divergence over time (docs).

• Data Diff: Row-by-Row Metrics compare a source and target table at the record level using configurable join keys, reporting a match % and providing CSV download of mismatched rows (docs).

• Inspect Failed Rows: A Failing Records Query (custom SQL) runs from the incident's Failing Records tab and exports mismatched or failing rows as CSV; default queries are provided for null % and conformity metrics (docs).

• DataFrame Tests: Platform is SQL pushdown-based; no Pandas, Spark, or Polars DataFrame validation is documented (docs).

• Data Contracts: No data contract format (YAML/JSON Schema producer-consumer SLA) is documented (docs).

• Unstructured Data Tests: Lightup lists 'Data Quality for Unstructured Data' on its homepage but no specific documentation on LLM-based or JSON schema validation for free-text columns was found (website).

Lightup data observability

Prebuilt ML models set dynamic thresholds with seasonality, slices train a model per segment, and you configure checks in the UI or with its Python SDK and CLI.

Compare: data observability tools · data profiling tools

• Data Profiling: Set-up-data-profiling page and a dedicated ProfilerClient in the Python SDK confirm table and column profiling configuration (docs).

• Anomaly Detection: Prebuilt ML models derive dynamic thresholds from historical metric values, with seasonality detection and no required hardcoded rules (docs).

• Freshness Monitors: Data Delay Metrics track lag in the arrival of expected data, detecting stale or delayed datasets against expected ingestion windows (docs).

• Volume Monitors: Data Volume Metrics (included in auto metrics) track row count changes and flag anomalies in data volume over time (docs).

• Schema Monitors: Auto Metrics include column activity metadata metrics that are event-based, detecting system events during schema scans such as column additions or removals (docs).

• Field Monitors: Null Percent Metrics and aggregation metrics track column-level statistics as continuous time series with ML anomaly detection (docs).

• Distribution Monitors: Distribution Metrics are an explicit metric type tracking the statistical spread of column values over time (docs).

• Segment Monitors: Slices enable GROUP BY dimension breakdowns for any metric, training independent anomaly models per segment and dynamically detecting new slice values (docs).

• Custom SQL Monitors: SQL Metrics let users define arbitrary SQL expressions returning a single numeric value per time interval, tracked continuously with anomaly detection (docs).

• Partition-Based Monitors: Partitions scope monitor scans to a recent time-filtered slice of a table, reducing compute cost by limiting the data queried to the relevant partition (docs).

• Window-Based Monitors: Aggregation Interval configures non-overlapping tumbling time windows (hourly, daily, or custom) over which metrics are bucketed and independently evaluated (docs).

• UI-Based Monitors: Monitors can be created and configured entirely through the Explorer GUI without writing code (docs).

• Monitors-as-Code: Python SDK (MetricClient, MonitorClient) and lightctl CLI enable programmatic creation, update, and deletion of metrics and monitors for code-driven deployments (docs).

• Pipeline Monitoring: No orchestrator-level job monitoring (Airflow DAG durations, dbt model run times, schedule adherence) is documented (docs).

Lightup incident management

Lightup sends alerts, tracks resolution, and integrates with ticketing and on-call tools.

• Alert Notifications: Alerts delivered via Slack, Microsoft Teams, email, PagerDuty, OpsGenie, and ServiceNow; supports immediate or digest (hourly/daily/weekly) formats (docs).

• Incident Resolution: Incident Revalidation validates whether a fix resolved the issue in production data; incidents can be marked resolved (docs).

• Ticketing Integrations: Native Jira and ServiceNow integrations are documented with dedicated docs pages (docs).

• On-Call Integrations: PagerDuty and OpsGenie are documented as native integrations with dedicated docs pages (docs).

• Incident Triage: Incident status can be changed and incidents bulk-managed, but no explicit owner assignment, priority setting, or formal triage workflow is documented (docs).

• Alert Feedback: No false-positive marking, snooze, or per-anomaly feedback mechanism to retrain detection models is documented (docs).

Lightup data quality analytics

Lightup's dashboards and health scores track data quality.

• Data Quality Dashboards: Built-in Data Quality and Incident dashboards with KPI charts, trend views, and customizable chart layouts (docs).

• Data Health Scores: Table Health tab displays per-table health percentage derived from monitor pass/fail results; lineage view surfaces health metrics per table (docs).

• Incident Resolution Metrics: "Analyze an incident" documents collaborative investigation via comments, tickets, and status changes, not MTTD/MTTR analytics, incident-volume trends, or resolution-rate-by-owner reporting (docs).

Lightup data lineage

Lightup shows table-level lineage; column-level lineage isn't documented.

• Table-Level Lineage: Lineage Explorer (Beta) visualizes table-level relationships and dependencies within a schema (docs).

• Column-Level Lineage: Lineage Explorer shows column metadata (types, monitor status) but field-to-field dependency derivation is not explicitly documented (docs).

• End-to-End Lineage: Native lineage is scoped to within-Explorer; Alation/Atlan catalog integrations may extend it across the full stack, but this is not explicitly documented as end-to-end lineage (docs).

• Impact Analysis: Incident indicators appear on downstream tables in the lineage view, but no explicit downstream impact analysis workflow is documented (docs).

Lightup data governance

Lightup syncs results to the Alation, Atlan and Collibra catalogs.

• Data Catalog Integration: Native integrations with Alation, Atlan, and Collibra (Beta) push quality scores and DQ metadata into external catalog platforms; gated to Enterprise tier (docs).

• Data Catalog: Explorer is a data hierarchy browser for monitored assets, not a searchable catalog with descriptions, tags, and ownership metadata (docs).

• Data Classification: No automated PII/PHI/PCI detection via pattern matching or ML classification is documented (docs).

• Data Products: Workspaces can be categorized by data domain/department, but they function as access-segmented, isolated environments (separate datasources/monitors/dashboards per workspace) rather than a mechanism to roll up quality status for a domain/data product within a shared view; Slices only segments a single table's rows by a column value (docs).

Lightup LLMOps

• AI Observability: No specific documentation on monitoring AI/LLM pipeline outputs or prompt/response quality was found despite the unstructured data quality product listing (website).

Lightup workflows

Lightup has a REST API, a Python SDK, the lightctl CLI and an MCP server. It has no dbt integration.

• Scheduling: Metrics and monitors run on configurable schedules (cron expressions, fixed intervals, or event-triggered) without requiring external orchestration (docs).

• REST / GraphQL API: Full REST API with 40+ endpoints covering metrics, monitors, incidents, datasources, users, and workspaces (docs).

• MCP Server: Lightup Agent (Beta) is tightly integrated as an agentic interface powered by Claude, Gemini, or Codex, enabling natural language control of the platform from AI coding agents (website).

• Python SDK: pip-installable lightctl library exposes MetricClient, MonitorClient, IncidentClient, SourceClient, and other clients for full programmatic control (docs).

• dbt Integration: No dbt manifest import, dbt test result surfacing, or dbt CI pipeline integration is documented (docs).

• CI/CD Integrations: API-triggered metric runs are supported, but no specific GitHub Actions, GitLab CI, or PR-blocking quality gate integration is documented (docs).

• Webhooks: No generic HTTP webhook callback capability is documented; notifications route through named integrations (Slack, PagerDuty, etc.) rather than a generic webhook endpoint (docs).

Lightup AI agents

Lightup has Genie, a chat interface that answers questions, finds docs and turns monitors on or off. It also suggests checks, and an MCP server connects it to AI assistants such as Claude and Gemini.

• AI Chat Assistant: Genie is an in-product conversational interface that answers questions, retrieves docs, and enables/disables checks in bulk via natural language commands (docs).

• Automated Test Generation: Auto Metrics applies predefined metric templates across data hierarchy (schema, table, column levels) in bulk without per-metric manual authoring (docs).

• Natural Language Test Generation: Genie (Lightup's GenAI assistant, Beta) can only enable/disable existing pre-built metrics via conversational commands and answer questions; the docs explicitly note it does not yet generate new custom checks from a plain-English description (docs).

• AI Issue Triaging: Time-correlated incidents are automatically identified, but this is statistical correlation by time overlap, not AI/LLM reasoning about root cause (docs).

• AI Issue Resolution: Lightup lists 'Data Remediation' as a product feature on its homepage, but no documentation on AI-proposed or automated fix suggestions was found (website).