Coinbase Shifts Data Quality Checks Into a Shared Platform
Coinbase developed its Data Quality Observability Platform (DQOP) after identifying a gap in how its internal teams managed data reliability. By late 2024, the company already had thousands of data quality checks embedded in pipelines, directed acyclic graphs (DAGs), and service code. Those checks had helped prevent potential financial losses, but incidents continued because the checks were difficult to discover, extend, and connect to clear responsibility.
The core problem was not simply a lack of detection. Teams consuming a dataset often could not see which validations had been applied upstream. Adding checks was largely restricted to the teams producing the data, even when analysts and finance partners understood the business rules that mattered. In addition, a failed check did not necessarily identify an accountable owner or trigger a defined response.
Coinbase launched DQOP in the first quarter of 2025 as a control plane embedded in its internal data catalog. Anyone with access to a table can view its checks, owners, failure history, and quality score, as well as add new checks. The approach is designed to make data quality easier to manage before an incident exposes a weakness.
DQOP Separates Check Configuration From Execution
Rather than replacing existing data processing systems, DQOP manages how checks are created, configured, scheduled, and routed. It delegates execution to existing engines, allowing thousands of established checks to be brought under the platform without rewriting them.
Each check is stored as a versioned YAML document instead of being maintained only as code in a repository. This configuration can be displayed in the user interface, generated through an API or an agent, and reviewed through version comparisons. Immutable versions also provide an audit history and allow previous configurations to be restored.
The platform distinguishes between a check and a data contract. Anyone can create a check to measure quality, but a contract represents an agreement accepted by the data-producing team, including a service-level commitment and a paging policy. DQOP rejects contracts without a paging destination. This allows the number of checks to increase without automatically creating more overnight alerts for teams.
Four Services Manage the Quality Check Lifecycle
DQOP uses four services to manage execution, evaluation, storage, and alerting. The process passes execution, evaluation, and persistence events through Kafka, allowing individual stages to be redeployed or experience backlogs without losing results.
Profiling-service: Written in Go, it manages check configurations and runs SQL checks asynchronously against Snowflake or Databricks through the shared Query Builder Service.
TDWMS: Compiles schedules into Temporal, a durable execution engine that can resume work when a worker fails during a check.
ThirdEye: Written in Python, it evaluates whether measured results meet expectations. It loads configuration and up to 180 days of metric history, evaluates separate dimension groups, and produces verdicts with sample violating rows.
Mercury: A PostgreSQL-backed state store that retains configurations, execution states, results, and alert history. It routes notifications to Slack, PagerDuty, and the metrics pipeline.
DQOP also distinguishes three different non-passing outcomes. Configuration problems, such as invalid SQL or a renamed column, are sent to the check author. System failures, such as a warehouse timeout, are retried before the platform team is paged. Only a failed expectation indicates that the data itself violated a rule. Checks that could not run are excluded from pass-rate calculations so execution problems do not silently distort quality scores.
Separate Configurations Make Checks Easier to Maintain
Each check has two configurations: one defines what to measure, and the other determines what result is acceptable. The profiling configuration specifies SQL, metrics, dimensions, and scheduling. The ThirdEye configuration defines expectation functions, thresholds, and notifications.
This separation means a single measurement can support multiple quality rules. For example, one query can provide data for a volume check, a regional minimum, and a completeness check. Teams can adjust thresholds without changing the underlying SQL.
DQOP also retains metric history so historical comparisons do not always require rescanning large warehouse tables. A check comparing today's volume with yesterday's can use a stored row count, while a 30-day trailing average can read previously collected values from PostgreSQL. This design makes the cost of warehouse queries depend more on the number of collected metrics than on the number of comparisons performed.
The platform supports different levels of measurement detail. Partition metadata is inexpensive to collect but may not reveal incorrect values in a partition that arrived on time. Record-level comparisons are more precise but can produce too many differences to investigate efficiently. Coinbase recommends starting with metrics grouped across roughly four to ten dimensions, which can help teams identify where a regression occurred without overwhelming them with results.
Table and Event Checks Cover Different Data Risks
DQOP supports two asset categories. Table quality checks validate stored data, covering freshness, volume, completeness, and business rules. One example is a rule requiring that settlement never precede execution. These checks are accessible through a Quality tab on Snowflake and Databricks tables, and a dry run shows which rows would be flagged before a check can trigger production alerts.
Event quality checks focus on streaming data. Instead of validating a table after data has been stored, consumers specify the requirements they have for individual event fields. Rules can check whether fields are present, correctly typed, non-empty, within an acceptable range, or experiencing an unusual change in null rates. The platform compiles these declarations into a query and an alert that identifies the affected fields.
Each asset family also receives a quality score. For tables, the score combines Health, which covers pipeline reliability and check pass rates; Enrichment, which reflects metadata and check coverage; and Importance, which considers downstream usage. A table without checks receives a poor score rather than being treated as neutral, and scores can be aggregated by schema or tag to provide a broader view of a data product.
Why a Composite Score Cannot Replace Specific Alerts
One key lesson from DQOP is that a composite score can summarize overall health while failing to detect an isolated but serious data problem. Coinbase illustrated this with an event field whose empty-string rate rose from 0.01% to more than 30% of records.
The Event Quality Score ranges from zero to 100 across seven dimensions: metadata, freshness, and volume stability each contribute 20 points, while delivery latency, duplicate rate, null density, and adoption each contribute 10 points. Alert thresholds depend on criticality: 80 for Tier 0, 60 for Tier 1, and 40 for Tiers 2 and 3.
For a Tier 0 event, a complete failure of the 10-point null-density dimension would leave the total score at 90, still above the alert threshold of 80. The score also changes only when null density crosses defined bands, and its aggregate calculation can hide a problem affecting a single field. In the example, the null-density rate moved from 32.7% to 39.6%, remaining within the same scoring band.
Coinbase's solution is to use field-level expectations to detect specific failures and retain composite scores for summarizing overall health. The distinction is practical: direct checks can identify a broken field, while a single score communicates broader condition and can flag wider degradation.
Coinbase Plans More Automated Check Creation
Coinbase is developing additional DQOP capabilities, including a prompt-based interface that would let users describe the data behavior they care about. The platform could then propose checks supported by profiling evidence, run them in a dry run, and present them for human approval before activation.
The company is also working on lineage-aware alert routing, assisted incident triage, and adaptive thresholds. Its profiling system already retains 90 days of per-column history for most tables, providing evidence that can help guide check creation.
Another planned use is a dataset certification program. Tables would earn a catalog badge based on ownership, documentation, and data quality criteria. The program would include freshness, uniqueness, completeness, and volume checks, initially running in an advisory mode so Coinbase can measure real pass rates before making them prerequisites for certification.
The broader lesson from DQOP is that reliable data requires more than detecting anomalies. Checks need to be easy to create and inspect, while formal contracts must establish who is responsible for responding. Composite scores help communicate quality, but specific checks are needed to detect individual failures before they become larger problems.
