Ingesting unstructured data into Data 360: the UDLO and UDMO pipeline underneath Data Library

Point an Agentforce Data Library at a folder of PDFs and it just works: the agent starts answering from your documents and you never see a pipeline. That’s the point of the abstraction, and it’s the right starting move for most teams. But the day the answers get subtly wrong, or you need to ground an agent on documents and the structured records they relate to, or finance asks why your storage bill moved, the abstraction stops being enough. You have to open the box.

Inside the box are two objects you probably haven’t configured by hand: the Unstructured Data Lake Object (UDLO) and the Unstructured Data Model Object (UDMO). Every Data Library, every Intelligent Context setup, every hand-built RAG pipeline in Data 360 rests on them. Understanding what they are, and specifically what they reference versus what they copy, is the difference between an agent whose grounding you can reason about and one that’s a black box you hope keeps working. This post is the plumbing underneath the abstractions: the ingestion pipeline for unstructured data, end to end.

The one fact that changes how you think about it

Start with the mechanic that surprises almost everyone: Data 360 does not import your unstructured data. When you ingest a folder of documents, the UDMO references the files where they live, in the external blob store, rather than pulling copies into the platform. The bytes of your PDFs stay in Google Cloud Storage, Amazon S3, Azure, Box, or Google Drive. Data 360 builds the model, the chunks, and the search index that point at them.

This is the same architectural instinct as zero copy for structured data: keep the data where it is, model it where you need it. It has real consequences you should design around from day one:

• Your source of truth stays in the blob store, so lifecycle and access there still matter. Data 360 references files; it doesn’t take custody of them.
• What you’re billing and indexing is the derived layer (chunks, embeddings, the search index) not a second copy of every file.
• If a file moves or its permissions change at the source, the reference is what has to stay valid. The pipeline is only as reliable as the store it points at.

Data 360 doesn’t swallow your documents. It builds a searchable model that references them where they live. Treat the blob store as part of your grounding architecture, not a place files came from once.

UDLO → UDMO: the two objects and how they map

The ingestion pipeline has a fixed shape, and it mirrors the structured world closely enough that if you understand Data Lake Objects versus Data Model Objects, you already have the pattern.

You begin by creating a UDLO, an Unstructured Data Lake Object that references the unstructured data at an external source. Creating it prompts Data 360 to automatically create a data stream that ingests from that source on a schedule. The UDLO is the raw, source-shaped landing object: it knows where your files are and keeps the reference current.

The UDLO then maps to a UDMO: the Unstructured Data Model Object, the harmonised model your retrievers and agents work against. The mapping has one property worth committing to memory: each UDLO maps to exactly one UDMO, but multiple UDLOs can map to the same UDMO. That many-to-one is not a footnote. It’s the whole reason the model exists. When the same kind of content arrives from different sources (support articles from a Google Drive folder and from a Box folder, say) you point both UDLOs at one UDMO and the agent treats them as a single logical dataset instead of two disconnected piles. Get this mapping right and “search the knowledge” means one coherent thing; get it wrong and you’ve split your corpus without noticing.

The connectors that can feed a UDLO cover the places unstructured content accumulates: Google Cloud Storage, Amazon S3, Azure Storage, Google Drive, Box, and web content via a crawler or sitemap, plus source-specific connectors for systems like GitHub, Jira, Guru, and Helpjuice. File-format support depends on the connector (the web content connectors, for instance, handle HTML and PDF alongside image formats like .jpg, .jpeg, and .png) so before you commit to a source, check its entry in the current Unstructured Data File Formats and Connectors documentation for the exact formats and any size limits, rather than assuming your file type is covered.

Chunking: the decision the defaults make for you

A document is useless to a retriever whole. It has to be split into chunks small and coherent enough that a semantic search returns the right passage instead of a whole 40-page contract. Chunking is where retrieval quality is won or lost, and Data 360 gives you two strategies plus a default that chooses for you.

• Semantic chunking splits on the document’s own structure, headings and sections, so each chunk is a meaningful unit that respects how the document was written. This is usually what you want for structured documents: manuals, knowledge articles, anything with a real outline.
• Window-based chunking works at the paragraph or sentence level, sliding a window across the text. It’s the better fit for prose without strong structure, or when you need finer-grained passages than the headings give you.

Here’s the part to be deliberate about: when you create a UDMO, there’s a checkbox, left selected by default, that creates a search index configuration using system defaults, automatically selecting the text fields and applying a chunking strategy for each one. That’s convenient and it’s exactly what a Data Library does under the hood. It’s also a decision made for you. For a first pass, take it. But the moment retrieval quality matters, the agent is returning the confidently-wrong chunk, the chunking strategy is one of the first knobs to revisit, because it’s upstream of everything the model sees. If the declarative config can’t express what you need, this is the point where teams drop into the Code Extension to write a custom chunking strategy in Python. And once the index exists, which index (keyword, vector, or hybrid) is its own consequential choice we covered in choosing a search index that grounds your agent.

From UDMO to a grounded answer: the retriever

The UDMO and its search index are the corpus. What turns them into an agent’s answer is a retriever: the component that sits on top of the index and acts, in Salesforce’s own framing, as the AI’s librarian. A retriever is called from inside a prompt template: it takes the user’s question, retrieves the relevant chunks from the search index, and augments the prompt with that retrieved context before the model ever generates a word. That’s retrieval-augmented generation, assembled from Data 360 primitives instead of a bespoke vector-database stack.

One detail that pays off in practice: a retriever can return additional fields alongside the chunk it retrieved, drawn from the chunk’s own DMO or from the original DMO. So the passage doesn’t arrive naked. It can carry the document title, the source URL, an effective date, a product code. That context is what lets an agent cite where an answer came from instead of asserting it, which is the same discipline we argue for in data lineage: an answer you can trace beats an answer you have to trust.

The move that most teams skip: linking unstructured to structured

Here’s where this stops being “RAG on PDFs” and becomes something more useful. Unstructured content rarely stands alone. A contract belongs to an account. A support PDF belongs to a product. A transcript belongs to a case. The structured record is context the unstructured chunk desperately needs, and Data 360 lets you wire them together.

You connect the UDMO and a structured DMO by foreign keys, and the result is a unified semantic graph: a model where an agent can retrieve a passage from a document and traverse to the structured record it relates to, giving it multimodal context for a single grounded answer. The agent isn’t choosing between “answer from the knowledge base” and “answer from the CRM record”. It’s grounding on both, joined.

This is the architecture that separates a document Q&A toy from an agent that can do a job. “What’s the renewal clause on this customer’s contract, and when does their term end?” needs the clause (unstructured, from the UDMO) and the account’s dates (structured, from the DMO) in the same reasoning step. Without the foreign-key link, you get half an answer twice. With it, you get one answer grounded end to end, and it’s the same principle behind grounding agents on a semantic layer so they don’t invent your metrics, extended to the documents your structured data can’t hold.

Where the pipeline goes wrong

Four failure patterns show up repeatedly, and every one is cheaper to design against than to debug in production.

• Splitting a corpus across UDMOs by accident. Because each UDLO maps to one UDMO, it’s easy to create a UDMO per source and end up with the same logical knowledge fragmented. Decide what’s one dataset first, then map multiple UDLOs into it. The many-to-one exists precisely so “all our support content” can be one searchable thing.
• Taking the default chunking as final. System defaults get you running; they don’t get you accurate. When an agent retrieves fluent, confident, wrong passages, look at chunking before you touch the prompt, the model can only be as good as the chunks it was handed.
• Forgetting the reference is a reference. Because Data 360 references files rather than importing them, source-side permission changes, moves, and deletions matter. This is also why right-to-be-forgotten and governance need thinking through, the bytes live somewhere you don’t fully control from the platform.
• Grounding on documents alone when the answer needs a record. If your agent keeps giving half-answers, check whether you ever built the foreign-key link to the structured DMO. A lot of “the agent doesn’t know enough” is really “we never joined the unstructured model to the structured one.”

The takeaway

A Data Library is the right way to start grounding an agent on your documents, but it’s an abstraction over a real pipeline, and when grounding gets hard, you need to see the pipeline. Unstructured ingestion in Data 360 is a UDLO that references your files in the external blob store (it doesn’t import them), an auto-created data stream that keeps that reference fresh, a UDMO that models the content with a chunking strategy and search index, and, in the move most teams skip, a foreign-key link from that UDMO to a structured DMO that builds the semantic graph a multimodal agent needs. Get the UDLO-to-UDMO mapping right so your corpus is one coherent thing, be deliberate about chunking instead of inheriting the default, and join the unstructured to the structured, and your agent’s grounding becomes something you can reason about, not just hope about.

Understanding the basics

What is the difference between a UDLO and a UDMO in Data 360?

A UDLO (Unstructured Data Lake Object) references unstructured data at an external source and, when created, prompts Data 360 to automatically create a data stream to ingest from that source. A UDMO (Unstructured Data Model Object) is the harmonised model your retrievers and agents work against. Each UDLO maps to exactly one UDMO, but multiple UDLOs can map to the same UDMO, which is how content of the same kind from different sources becomes one logical dataset. Importantly, Data 360 references the files in the external blob store rather than importing copies of them.

How does chunking work for unstructured data in Data 360?

Chunking splits a document into smaller, meaningful passages a retriever can search. Data 360 supports semantic chunking, which splits on the document’s structure such as headings, and window-based chunking, which works at the paragraph or sentence level. When you create a UDMO, a checkbox (selected by default) creates a search index configuration using system defaults that auto-selects text fields and applies a chunking strategy. You can accept those defaults to start and customise the strategy later, or write a custom strategy in the Data 360 Code Extension when the declarative options aren’t enough.

How do you ground an agent on both documents and structured records?

Connect the UDMO to a structured Data Model Object (DMO) by foreign keys. The result is a unified semantic graph that gives an agent multimodal context. It can retrieve a passage from a document and traverse to the related structured record in the same reasoning step. A retriever, called from inside a prompt template, pulls relevant chunks from the search index and can also return additional fields from the chunk’s DMO or the original DMO, so answers arrive with the context needed to cite and trust them.

---

Trying to move past a Data Library’s defaults to a grounding pipeline you can reason about, or join your documents to the CRM records they belong to? Talk to us. Getting the unstructured foundation right before an agent reads from it is exactly the work we do.