Problem Records in Data 360: why records vanish on ingestion, and how to get them back

Here’s a failure mode that costs more trust than an outright error ever does: the data stream runs, the status turns green, the job reports success, and the row count in your data lake object is lower than the row count at the source. Nothing failed. Nothing paged anyone. A segment is built on incomplete data, an agent grounds on a profile that’s missing its most recent orders, and the first anyone hears of it is a business user asking why a customer they know exists isn’t in the audience.

The rows didn’t vanish. Data 360 does something deliberate and, if you don’t know about it, invisible: when it can’t process a record cleanly, it doesn’t fail the whole batch. It quarantines the bad record and lets the good ones through, isolate-and-continue, availability over strict correctness. The quarantined rows land in an auto-created object with a PR_ prefix that most teams have never opened. This post is about that object: why records end up there, how to read the error taxonomy, and how to fix and reprocess them without creating duplicates.

If your problem is that the profile is stale rather than incomplete, fresh data that’s arriving late, that’s a different chain, and we mapped it in data freshness in Data 360. This post is about correctness and completeness: rows that should be there and aren’t.

What a Problem Record is

When a data stream ingests, Data 360 evaluates each record against the target data lake object’s schema and constraints. A record that can’t be written cleanly is not dropped and is not allowed to fail its siblings. Instead, the platform writes it to a Problem Records data lake object, a DLO whose name is prefixed with PR_, created automatically the first time a stream produces one.

That design choice is the whole story. The batch reports success because, from the pipeline’s point of view, it succeeded: every processable row was processed, and every unprocessable one was safely set aside for review. The completeness gap only exists if you never look at the PR_ object. This is the trade every ingestion platform makes somewhere; Data 360 makes it here, and the operational tax is that watching the Problem Records count is your job, not the platform’s alert.

The PR_ object shows up wherever you browse data lake objects, and you can query it directly in Data Explorer or the Query Editor like any other DLO. That’s the first move when a count looks wrong:

-- Count and categorize what got quarantined for a given stream's target DLO
SELECT error_code__c, COUNT(*) AS problem_count
FROM PR_MyOrders_Home__dll
GROUP BY error_code__c
ORDER BY problem_count DESC

The exact column and object names depend on your source object; the point is that the quarantined rows are sitting in a queryable table, tagged with why they were rejected. You don’t have to guess.

The error taxonomy: the four reasons a row gets quarantined

Data 360 tags each Problem Record with an error category, and knowing the four you’ll hit turns a vague “records are missing” into a specific fix. The recurring ones:

• NOT_NULL_CONSTRAINT_VIOLATION, the record was rejected because a required field was null, most commonly the primary key. A source row with a blank ID, or a mapping where the primary-key field didn’t line up, produces this. It’s the single most common cause of “my count is lower than the source.”
• DUPLICATE_ROW, two or more records in the same batch share the same primary key. Data 360 can’t decide which one is authoritative within the batch, so it quarantines the collision. (Note the scope: this is same-batch duplication, distinct from how upsert resolves a key that already exists in the DLO, more on that below.)
• TRANSFORM_EVALUATION_ERROR, the record was partially rejected because a field transformation failed evaluating it. A formula field, a type cast, or a mapping transform that chokes on a specific value lands here. This one is sneaky because it’s value-dependent: 99% of rows transform fine and the 1% with an unexpected value get quarantined.
• UNHANDLED_ERROR, a catch-all for an error during row processing that doesn’t fit the categories above. When you see these, the row-level detail in the PR_ object is where you start.

Two of these, the null-key and the duplicate-key cases, trace back to the same root cause: your primary key isn’t doing its job. Which is the thing to get right before anything else.

The primary-key problem underneath most of it

Ingestion into Data 360 is fundamentally keyed. Every record needs a primary key that is present and unique, and the two ways that goes wrong map exactly onto two of the four error codes above.

If your source doesn’t have a single field that’s reliably unique, the answer is a composite key: concatenate the fields that are jointly unique into one key field, either in the source, in an ingestion transform, or as a formula. A classic shape:

-- Composite key from fields that are jointly unique
CONCAT(Region__c, '-', StoreNumber__c, '-', TransactionId__c)

But composite keys come with a downstream cost worth naming: the key you invent for ingestion uniqueness is also the identity Data 360 reasons about, and an over-engineered composite key can fragment identity resolution later, the same customer arriving through two sources with two different composite keys won’t match. Design the key for both jobs: unique enough to ingest, stable enough to resolve on.

There’s also a resolution rule you have to know because it silently decides which version of a record you keep. When a record arrives whose primary key already exists in the DLO (an upsert), the record with the more recent Record Modified timestamp wins. If your source doesn’t populate a trustworthy modified timestamp, you can lose the update you expected to keep, not to a Problem Record, but to a resolution you didn’t intend. Bad or missing timestamps are a data-quality issue that never shows up in the PR_ count, which is exactly why they’re easy to miss. This is upstream of everything, and it’s the reason getting the data AI-ready starts before the stream, not after.

The one that isn’t a problem: the “multiple threads” error

Before you treat every red line as data loss, know the false alarm. A data stream can surface a “multiple threads” error when an ingestion job and a routine table-maintenance job try to write the same DLO at the same time. It looks alarming, and teams often panic and start re-sending data.

Don’t. In this specific case no data is lost, the contended records simply ingest on the next run. Re-sending in a panic is how you turn a non-event into an actual DUPLICATE_ROW problem. Confirm it’s the threads error (the message is specific), let the next scheduled run clear it, and move on. Knowing which errors are self-healing is half of not making them worse.

Diagnosing, in order

A repeatable triage beats poking around. When a count is wrong or a downstream consumer reports missing data:

1. Confirm the gap is real. Compare the source row count to the DLO row count for the same window. A difference is your signal; a match means the problem is elsewhere (mapping to the data model object, a data space partition, or a segment filter, not ingestion).
2. Open the PR_ object for that stream and group by error code, as in the query above. The distribution tells you which of the four causes dominates.
3. Read row-level detail for the top category. A NOT_NULL_CONSTRAINT_VIOLATION clustered on one source system points at a mapping or an extract problem there; a TRANSFORM_EVALUATION_ERROR on scattered rows points at specific values your transform didn’t anticipate.
4. Check the boring stuff. Ingested dates default to UTC: a date-format misconfiguration on the stream shifts or rejects timestamps, and “the dates are all a day off” is usually this, not a bug.
5. Fix at the source of the category, not row by row. Null keys get a mapping or composite-key fix; transform errors get a guarded transform or a cleaned value; duplicates get a better key.

Track the Problem Records metrics as an ongoing signal, not a one-time hunt. Salesforce shipped Problem Record metrics so data-quality monitoring is a dashboard you watch, not an archaeology dig you launch after someone complains. A rising PR_ count on a stream that used to be clean is an early warning that something changed upstream (a source schema drift, a new bad-data pattern) before it becomes a business-visible gap.

Reprocessing without duplicating

Once you’ve fixed the root cause, you need the quarantined rows back, and the natural fear is that re-sending corrected data will double the good rows that already made it in. Two facts make this safe.

First, Data 360 ingestion is idempotent by design. Reprocessing the same file does not create duplicate records: the platform computes a checksum (file fingerprinting) to identify a file it has already processed and skips the redundant work. So re-sending a corrected extract doesn’t re-ingest the rows that were already fine.

Second, Problem Records are retained for 30 days. That’s your window to diagnose and remediate. Long enough for a real investigation, short enough that you can’t treat the PR_ object as permanent storage. After 30 days the quarantined rows age out, so a gap you never noticed becomes a gap you can no longer recover from the quarantine table. You’d have to re-extract from the source. This retention window is the quiet argument for monitoring rather than reacting: a Problem Records count you check weekly stays inside the recovery window; one you discover after a quarterly audit may not.

The clean remediation loop, then: fix the root cause at the source (the key, the transform, the mapping), re-send the corrected data, and let idempotency ensure only the previously-quarantined rows land. Verify by re-running the count query and watching the PR_ category you targeted drop toward zero.

Why this matters more in the agent era

Incomplete ingestion used to degrade a report. Now it degrades an agent’s answer. When an Agentforce agent grounds on Data 360 through a retriever, the completeness of the underlying DLOs is the ceiling on the agent’s accuracy. A customer whose latest three orders sit in a PR_ object because of a null key isn’t a slightly-stale profile. It’s an agent confidently telling that customer they have no recent orders. The quarantine that used to cause a quiet reporting gap now causes a wrong, authoritative-sounding answer to a real person.

That raises the stakes on watching the PR_ count from “good hygiene” to “a grounding correctness control.” An agent can only be as truthful as the data it retrieves, and a Problem Record is, precisely, a true fact the agent can no longer see.

The takeaway

When a Data 360 count looks wrong and nothing errored, the rows are almost always in a PR_ Problem Records object, quarantined so the batch could succeed without them. Open it, group by error code, and you’ll usually find a null primary key, a same-batch duplicate key, or a value-dependent transform failure: most of which trace back to a key that isn’t present, unique, and stable. Fix the cause at the source, re-send, and lean on idempotency so you don’t duplicate what already landed. And do it inside the 30-day retention window, because the quarantine table is a recovery buffer, not an archive. The teams that get burned aren’t the ones whose streams produce Problem Records, every stream does. They’re the ones who never look.

Understanding the basics

Why is my Data 360 data stream showing fewer records than the source?

Because Data 360 uses an isolate-and-continue model: when it can’t process a record cleanly, it quarantines that record into an auto-created PR_ Problem Records data lake object and lets the good rows through, so the batch reports success while the count comes up short. Query the PR_ object for that stream and group by error code to see why: the common causes are a null primary key (NOT_NULL_CONSTRAINT_VIOLATION), a duplicate key within the batch (DUPLICATE_ROW), and value-dependent transform failures (TRANSFORM_EVALUATION_ERROR). Fix the root cause at the source and re-send; you have a 30-day window before quarantined rows age out.

What is a PR_ object in Salesforce Data Cloud?

A PR_ object is a Problem Records data lake object that Data 360 creates automatically to hold records it couldn’t ingest cleanly from a data stream. Rather than failing the entire batch when one record is malformed, the platform writes the bad record to the PR_ DLO, tagged with an error category, and continues processing the rest. It’s queryable in Data Explorer and the Query Editor, retained for 30 days, and it’s the first place to look when an ingested count is lower than expected.

How do I reprocess failed records in Data 360?

Fix the underlying cause first (the missing or non-unique primary key, the failing transform, the mapping mismatch) then re-send the corrected data through the same stream. Data 360 ingestion is idempotent: it computes a file checksum (file fingerprinting) and skips files it has already processed, so re-sending a corrected extract won’t duplicate the rows that already ingested successfully; only the previously quarantined rows land. Do this within the 30-day Problem Records retention window, and verify by re-running a count query against the PR_ object and watching the targeted error category drop.

What causes DUPLICATE_ROW errors in Data 360 ingestion?

DUPLICATE_ROW means two or more records in the same ingestion batch share the same primary key, so Data 360 can’t determine which is authoritative and quarantines the collision. It’s distinct from an upsert against a key that already exists in the DLO, that case resolves by keeping the record with the more recent Record Modified timestamp. The fix for DUPLICATE_ROW is a better key: if no single field is reliably unique, build a composite key by concatenating the fields that are jointly unique, while keeping it stable enough not to fragment identity resolution downstream.

---

Chasing a unified profile that’s missing rows, or an agent grounding on data that’s incomplete? Talk to us: building an ingestion layer you can trust, with the monitoring to catch a Problem Records spike before a business user does, is exactly the foundation work that decides whether everything downstream is right.