Quality of Data

Operator Horizonte wants to avoid duplicated invoices and responses from AI based on outdated contracts. "quality Data" requires a definition linked to each use: a data can serve an exploration and be insufficient to issue an invoice.

Key concepts

Quality expresses whether data serve a specific purpose. Before measuring, identify the population, period and authorized evidence: a result acceptable for exploration may be insufficient for invoicing.

Completeness asks whether the required value is present; validity, whether it follows the allowed format or domain; accuracy, whether it matches the reality it represents. These are different checks: a start date of 31 AUG may be present and use an allowed format, yet be wrong if the verified contract states 01 SEP.

Consistency checks that related values do not contradict each other; uniqueness, that a business identity is not repeated under the agreed rule; timeliness, that data are available within the time needed for their use. A date check does not establish any of these three dimensions. Each requires its own rule, population and evidence.

The verified contract starts on 01 SEP, but 31 AUG is captured. The same value undergoes three checks: it is present and passes completeness; it uses an allowed format and passes validity; it does not match the contract and fails accuracy. It must be corrected against authorized evidence. This case does not establish consistency across sources, identity uniqueness or timely arrival: each dimension needs its own rule, population and evidence for the intended use. ↗

Rules, Thresholds and Retirement

The rule begins with a necessity: "Do not issue twice the same invoice." The Filling owner defines the corporate key, tolerance and effect of a non-compliance. The steward translates that decision into an understandable rule. The engineer implements the check and records result, period and version.

A threshold should indicate numerator, denominator, exclusion treatment and comparison operator. "At least 99 %" includes 99 %; "more than 99 %" No. Tolerance depends on risk: some checks prevent them from publishing and others allow them to continue with an authorized exception. The mean of many rules should not cover up a critical defect.

The business need is to avoid issuing the same invoice twice. The owner agrees the issuer-plus-number key, September cutoff, exclusions and 100 % uniqueness threshold; the steward documents R-02 and the engineer preserves its version and result. Measurement finds 198 distinct keys in 200 rows: 99 % is below 100 %, so publication is blocked and D-17 is opened. An owner, priority and target date are assigned, the cause is investigated and corrected at the source. R-02 is then repeated on the corrected period: close only with evidence of compliance; if it fails, the incident remains open. ↗

Remediation includes owner, priority, cause, target date and later monitoring. Cleansing a copy can be useful for a specific consumption but does not remove the problem at capture. Horizon distinguishes corrected data and rejected data and an accepted exception. retains the reason to explain the published result.

Quality with Oracle AI Data Platform and Oracle Autonomous AI Lakehouse

Oracle AI Data Platform allows data to be prepared and transformed using notebooks and workflows. The computer can express checks with SQL or code and preserve results. Oracle Autonomous AI Lakehouse and Data Studio from Oracle Autonomous AI Database provide statistics, analysis and preparation. Table AI Assist propose SQL and recipes that a person checks before applying them. It doesn't prove that the information's true.

Implement acceptance rules with notebooks and coordinate your execution with Oracle AI Data Platform workflows. To Horizonte, check uniqueness's invoice and client's existence, record results and separate non-compliance with their motif. The owner approves the rule and action. The flow publishes the accepted data and re-measures the corrected records. Thus you turn a quality dimension into a repeatable monitoring.

A valid rule is evaluated using Oracle AI Data Platform notebooks and workflows. The consistent data continue to an authorized publication at Oracle Autonomous AI Lakehouse. Failed ones have motif, owner and state. they are tipped and re-measured. The publication path needs to accept the rule and other controls for use. ↗

Oracle Data Transforms allows to organize transformations and executions. Statistics show useful patterns for modelling but an unexpected distribution requires an interpretation of the business. The technical detection does not decide only if the closure is blocked. Also, the quality validation of an Analytical View has a specific scope: it's not an integral certification from all sources.

Documents every rule in a tab with purpose, population, logic, threshold, owner, result and action. Oracle AI Data Platform and Oracle Autonomous AI Lakehouse allow its implementation. The tab remains valid if the engines or data location change.

Explanated example

Horizonte receives 1 000 contracts. Ten have no starting date and another 20 have an outside date. For the presence rule, 990 1 000 fulfill: 99 %. If the two defective stocks do not overlap, the combined presence and period rule has 970 concurrently: 97 %.

The grid represents 1 000 contracts, ten per square, with defects without overlap. The endowment reached 990/1 000 = 99 % and exceeded its threshold. The current and within date rule reaches 970/1 000 = 97 % and misses the minimum 98 %. Distinguish population, defects and thresholds from each rule before combining or comparing percentages. ↗

The owner had agreed at least 99 % for presence and 98 % for the combined rule. The first goes and the second goes wrong. The pipeline should not replace both with an average that looks acceptable. The steward assigns the cases, identifies if the capture allows impossible dates and coordinates the correction of the form or production process.

It is then re-measured about an identified population. A stated improvement with no period and no denominator can compare results and can simply cover up that difficult cases have been excluded.

Common

• "Without noses means exact." The presence doesn't check the value against reality.
• "A cleanup resolves the cause." The error can continue to come through the same process.
• "The mean quality allows to publish." A critical malfunction can demand blockade even if other rules are perfect.

Decision exercise

A load of 200 invoices contains two surplus records depending on the agreed key and counting criterion: there are 198 unique keys. The policy requires 100 % from uniqueness to publish invoices. Another rule of completeness reached 100 %. The team propose to mean both and continue. He decides what to do and who to figure out.