What you will take away
- Define the dataset contract with the people producing and consuming the data.
- Separate availability, freshness and quality; a successful execution does not guarantee all three.
- Use lineage and owners to identify impact and resolve incidents.
- Let AI propose rules without delegating the definition of business meaning.
A green indicator does not guarantee useful data
A job can load every row and still duplicate orders, mix currencies or deliver yesterday's sales. For analytics and AI, those defects propagate into metrics, searches and answers. In October 2026, a practical priority is to operate data trust as part of the service: define expectations, measure them and know who acts when they are not met. Executable quality rules and lineage offer concrete mechanisms for doing this [1][2].
The contract starts with business questions
Before automating controls, agree on keys, types, field meaning, time zone, currency, frequency and ownership. Document which changes require a new version and who consumes the dataset. A contract must distinguish a technically valid field from a useful value: a numeric amount without a currency may be impossible to interpret. AI can help draft rules from metadata and examples; the data owner validates the conditions and exceptions.
| Agreement | Order example |
|---|---|
| Identity | The key combines the source and order identifier. |
| Semantics | Amount and currency are interpreted together; returns have their own rule. |
| Freshness | The cutoff time and time zone are documented. |
| Evolution | An incompatible change requires a new version and notice to consumers. |
Example: checking sales before the daily report
A company receives sales files and loads them into staging. The process validates required fields, duplicate keys, currency codes and dates within the allowed period. Questionable records go to a review area with the reason, batch and owner; errors affecting critical totals stop publication. After correcting the source, the batch is reprocessed without duplicating sales. This is an illustrative design, not a success story or a feature automatically attributed to Dova.
Do not silently discard incorrect rows to achieve a green result. Databricks documents options to retain, drop or stop on failed rules [1]. The choice depends on business impact and must remain visible to the consumer.
Distinct signals and explicit objectives
| Signal | Measurement and decision |
|---|---|
| Availability | Can the dataset be queried? Record access failures and their duration. |
| Freshness | Up to which source event is it current? Compare the watermark with the agreed cutoff. |
| Quality | Which rules pass and which rows fail? Separate critical rules from warnings. |
| Drift | Are volume or distributions changing? Compare equivalent periods and validate the cause. |
An SLO is a service objective agreed with the consumer. Define its window, denominator and response when it is missed. Avoid a global average that hides a failed critical rule. Seasonal variation may explain volume changes; a drift alert calls for investigation and does not, by itself, prove that data is wrong. Google Cloud documents quality rules and measurements managed as code [3].
From the symptom to the source with lineage
Lineage connects sources, transformations and outputs. OpenLineage defines interoperable metadata for jobs, runs and datasets [2]. Its operational value is answering which batch changed, which transformation was involved and which consumers may be affected. Coverage must be verified: a manual transformation or an uninstrumented process may leave a gap. Adding an owner and recovery procedure turns a technical map into a response tool.
Start with a dataset that matters
- Choose a dataset with an identified consumer and owner.
- Agree on critical rules, the publication schedule and exception handling.
- Version rules and test them against known failures before enabling them.
- Show results by batch and rule, including retained or excluded records.
- Rehearse an incident and measure detection, diagnosis and recovery through consumer validation.
Dova documents mappings, workflows, sessions, workers and monitoring for operating integrations. A quality program can be organized around these flows, but contracts, complete lineage and SLOs require specific definition and implementation. The product is not assumed to include all these components.
Questions worth asking
Does data quality mean having no null values?
No. It depends on purpose: keys, consistency, meaning, freshness and business rules also matter. Some fields can legitimately allow nulls.
Can AI define every rule?
It can suggest checks, but it does not automatically know business exceptions and commitments. The owner must validate and version the rules.
Must every dataset be controlled from the start?
No. Prioritize an important flow, demonstrate detection and recovery, and expand coverage using what you learn.
Where Dova fits
A governed integration can make responsibilities and execution visible. Review Dova's documentation for mappings, workflows and monitoring, and define the business quality rules your team needs to implement.
Sources & further reading
- Manage data quality with pipeline expectationsDatabricks · Accessed
Official documentation for executable quality constraints and retaining, dropping or failing records that do not meet expectations.
- About OpenLineageOpenLineage · Accessed
Official specification overview for interoperable lineage metadata describing jobs, runs and datasets.
- Auto data quality overviewGoogle Cloud · Accessed
Official quality scan documentation reached from the former Dataplex URL. Describes quality rules, measurement and managing rules as code; no October 2026 release claim is made.
Practical guidance for the October 2026 edition. Sources support the technical principles; examples are illustrative, not product announcements or customer results.