Data freshness in Data 360: why your unified profile is older than you think, and how to monitor the whole chain

Most Data 360 problems that reach a customer aren’t loud. Nothing throws an error. The dashboards are green. And yet an agent tells a caller their balance is $0 when it isn’t, a segment activates to people who churned last week, or a “high-value customer” flag fires on the wrong currency. The data was ingested fine. It just wasn’t current, or it was current in one stage and stale in the next.

This is the failure class teams underinvest in because it’s invisible until it’s embarrassing. Everyone monitors whether a job ran. Almost nobody monitors whether the unified answer an agent reads is fresh, because that answer is assembled from several stages that each run on their own schedule, and the freshness of the whole is only as good as the least-recently-run link. Once you’re grounding an agent on this data, stale is no longer a reporting nuisance. It’s a confidently wrong statement to a real person. Here’s where staleness hides, and how to watch for it.

Freshness is a chain, not a number

The instinct is to look at a data stream, see “last refreshed 2 minutes ago,” and call the data fresh. But the number on the data stream describes only the first link. A fact travels through roughly five stages before an agent or a segment ever sees it, and each has its own clock:

Ingestion, the data stream lands source data into a Data Lake Object (DLO), on a streaming/CDC feed (near real-time) or a batch schedule (every N minutes/hours).
Mapping, the DLO is mapped into a Data Model Object (DMO), the standardized shape everything downstream reads.
Identity resolution: match-and-reconcile runs on its own schedule to fold records into unified individuals. Until it reruns, a brand-new record isn’t part of the unified profile yet.
Derivation: calculated insights and streaming insights recompute the metrics (lifetime value, last-order date, risk scores) that people act on.
Consumption: segments publish and retrievers ground the agent on whatever the stages above produced.

Here’s the trap: a fact is only as fresh as the slowest stage it has passed through. You can stream orders into a DLO every few seconds, but if identity resolution runs on a schedule and the calculated insight for “lifetime value” recomputes on another, the “live” order won’t move the number an agent quotes until both of those later stages have run. The data stream is honest. It refreshed two minutes ago. The unified profile is hours behind, and nothing told you.

Fresh ingestion is necessary and nowhere near sufficient. The number that matters is the freshness of the last stage before consumption, not the first.

The silent failures that never throw

An outright job failure is the easy case. It shows up in refresh history with a red status, and you go fix it. The dangerous failures succeed loudly and are wrong quietly:

A stalled schedule that still looks “on.” A batch data stream configured to refresh hourly can silently stop advancing (an expired credential, a changed source schema, a paused connector) while the last-successful timestamp keeps receding. If you only check “did the last run error,” you miss “the last successful run was 14 hours ago.”
Fresh source, stale derivation. The DLO is current, but the calculated insight that rolls it up hasn’t recomputed. Every number derived from that insight is stale even though the raw data isn’t. This is the single most common “the data’s right but the answer’s wrong” bug.
Fresh data, stale identity graph. New records landed, but identity resolution hasn’t rerun, so they’re not attached to the unified profile. The agent, reading the profile, behaves as if they don’t exist.
Over-merge and split. Match rules that are too loose fold two people into one; too strict, and one person splits across several profiles. Both produce confidently wrong unified answers with zero error anywhere in the pipeline.
Stale consent. A preference or consent change lands in the source but hasn’t propagated to the stage activation reads, so you reach someone who opted out. That’s not an inconvenience; in a regulated context it’s a violation.
The wrong-window / wrong-unit calculation. A metric computed over the wrong date range or a mismatched currency, the classic example being a “top spender” score that summed across currencies without normalizing. Perfectly fresh, perfectly wrong.

None of these trip an alarm, because from the platform’s point of view every job did exactly what it was told. Monitoring “did it run” catches none of them. You have to monitor what the data says and when each stage last advanced.

What Data 360 gives you natively

Start with what’s built in before you build anything.

Refresh history. Each data stream and DLO exposes a refresh history, when it last ran, whether it succeeded, and how much it processed. This is your first-line check for the stalled-but-green case: don’t ask “did the last run fail,” ask “how long ago was the last successful run, and is that within the tolerance this data needs.” A full refresh, note, tears the data down and rebuilds it from the latest source values, which is your reset button when a stream has drifted, but it’s heavier and it resets your incremental baseline, so it’s a repair tool, not a monitoring one.

Refresh dependencies. Some refreshes are deliberately gated, a stream or downstream step configured to advance only after identity resolution completes, so you’re not activating on a half-resolved graph. Knowing which of your stages are gated on which others is half of understanding your true end-to-end latency. Map it once; it’s rarely what people assume.

Data Explorer and the query surface. The most under-used monitoring tool is simply querying your own data. Data Explorer, the Query API, or SOQL/SQL from Apex let you ask the data directly: what’s the newest timestamp in this object, how many rows, how many nulls in the field the agent depends on. That’s the difference between trusting a green checkmark and verifying the number.

Build a freshness check you can alert on

The job-ran signal lives in refresh history. The data-is-current signal you build yourself, and it’s cheap. The pattern is a small set of validation queries per critical object, run on a schedule, that assert two things: recency (the newest record is recent enough) and volume (the row count is in a sane range, so a source that stopped sending shows up as a flat line).

A recency-and-volume probe against a DMO is a few lines of Data 360 SQL:

-- Freshness + volume probe for a single model object.
-- Run on a schedule; alert when max_event_ts falls outside tolerance
-- or row_count drops far below the trailing norm.
SELECT
    MAX(event_timestamp__c)                       AS max_event_ts,
    COUNT(*)                                       AS row_count,
    SUM(CASE WHEN customer_id__c IS NULL THEN 1
             ELSE 0 END)                           AS missing_key_count
FROM order__dlm
WHERE event_timestamp__c >= DATEADD(hour, -24, CURRENT_TIMESTAMP);

Three columns, three different failures caught: max_event_ts sliding out of tolerance catches the stalled-but-green stream; row_count collapsing catches a source that silently stopped; missing_key_count climbing catches a schema or mapping change that’s about to break identity resolution downstream. Field and object names vary with your model, the shape is what transfers.

The same probe belongs on the derived layer, not just ingestion, because that’s where the common bug lives. Query the calculated insight or the DMO an agent grounds on and assert that its newest value is recent, not just that the raw DLO feeding it is. If your source is minutes fresh but the insight is a day old, this is the query that tells you, and nothing else will.

To turn a probe into an alert, wrap it: a scheduled Flow or Apex query that runs the check and raises a platform event, a Slack message, or a case when recency or volume breaches tolerance. Set the tolerance per object from the data’s real cadence, a payments feed might allow 15 minutes of lag, a nightly product catalog 26 hours. A single global threshold either cries wolf on the slow feeds or sleeps through a stalled fast one.

Finally, reconcile against the source periodically. A daily count of rows in Data 360 versus the system of record catches slow leaks, the connector dropping 2% of records for a week, that a recency check sails right past because the newest record is always fresh even while older ones go missing.

The freshness-versus-cost tension nobody warns you about

The obvious fix to staleness is “refresh everything more often,” and it’s a trap, because in Data 360 recomputation is what you pay for. More frequent identity resolution, more frequent calculated-insight recompute, more frequent segment publishes: each is a credit-consuming operation, and cranking all of them to maximum frequency inflates the bill without necessarily improving any decision. The discipline is to match cadence to how fast the underlying decision changes: a fraud-risk score grounding a real-time agent needs minutes; a “customer segment tier” that shifts quarterly does not need to recompute hourly just because it can.

So freshness monitoring isn’t only a quality control. It’s a cost control. When you can see the true end-to-end latency of each pipeline, you can lower the refresh frequency of the stages where staleness is harmless and spend that budget on the one or two stages where it reaches a customer. The teams that both trust their data and control their bill are the ones who measured freshness per stage instead of guessing globally in either direction.

Tie every number back to a source

One more capability closes the loop when a stale or wrong number does slip through: data lineage. When an agent quotes a figure and someone asks where it came from, lineage traces it back through the derivation, the mapping, and the stream to its source, which is how you diagnose which link went stale instead of guessing. Freshness monitoring tells you something is wrong; lineage tells you where. You want both wired before an agent is answering customers, not after the first “why did it say that.”

The takeaway

Freshness in Data 360 is not the timestamp on a data stream. It’s the freshness of the slowest stage in a five-link chain, and staleness hides in the later links where nothing throws an error. Watch the whole chain: use native refresh history to catch stalled-but-green streams, understand which refreshes are gated on identity resolution, and build cheap recency-plus-volume probes on the derived objects your agents and segments read, not just on ingestion. Alert per object at a tolerance set from the data’s real cadence, reconcile against the source to catch slow leaks, and use that same visibility to cut refresh frequency where staleness is harmless, funding real-time where it reaches a customer. Do that and you close the gap between “the job ran” and “the answer is true”, which, once an agent is grounded on this data, is the only gap that matters.

Understanding the basics

Why is my Data 360 data stale if the data stream refreshed recently?

Because ingestion is only the first of several stages. After a data stream lands data into a Data Lake Object, that data still has to be mapped to a Data Model Object, folded into the unified profile by identity resolution, and rolled up by calculated insights before a segment or an agent reads it, and each of those runs on its own schedule. The data stream can be minutes fresh while the calculated insight an agent quotes is a day old. The freshness that matters is the last stage before consumption, not the first.

How do I monitor data freshness in Data 360?

Combine two signals. Native refresh history on each data stream and DLO tells you whether the last run succeeded and how long ago the last successful run was. Check the latter, not just the former. Then build your own validation queries with Data Explorer, the Query API, or SOQL/SQL from Apex that assert recency (newest record is recent enough) and volume (row count is in a sane range) on the derived objects your agents and segments read. Wrap those in a scheduled Flow or Apex job that alerts when a tolerance is breached, and periodically reconcile row counts against the source system.

What are the silent data-quality failures to watch for?

The ones that succeed without throwing an error: a batch stream that stopped advancing while still showing green, a fresh source whose calculated insight hasn’t recomputed, new records that identity resolution hasn’t attached to the profile yet, match rules that over-merge two people or split one across profiles, consent changes that haven’t propagated before activation reads them, and metrics computed over the wrong window or across mismatched currencies. None of these trip a job-failure alarm, which is why monitoring “did it run” misses all of them. You have to monitor what the data says and when each stage last advanced.

Does refreshing more often fix staleness?

Only partially, and it has a cost. In Data 360 the recomputation stages (identity resolution, calculated insights, segment publishes) consume credits, so raising every refresh frequency inflates the bill without necessarily improving a single decision. The better approach is to measure true end-to-end latency per pipeline, then match each stage’s cadence to how fast the underlying decision changes: minutes for a real-time risk score, far less often for a slowly shifting segment tier. Good freshness monitoring is what lets you spend the refresh budget where staleness reaches a customer and save it where it doesn’t.

Grounding an agent on Data 360 and not sure whether the profile it reads is current? Talk to us. Getting the freshness chain observable, and matching refresh cost to where staleness hurts, is exactly the unglamorous work that keeps an agent from confidently quoting yesterday.