Data Transforms in Data 360: batch vs. streaming, and shaping data before it hits your model

Data almost never lands in the shape your model wants. Phone numbers arrive in three columns when your model wants one normalized object. Orders come in per-line when you need them per-customer. A source system stamps its own status codes that mean nothing to the unified profile downstream. Between “ingested” and “usable,” something has to reshape the raw data — join it, clean it, aggregate it, standardize it — and in Data 360 (the platform formerly known as Data Cloud, renamed at Dreamforce in October 2025) that something is a Data Transform.

Transforms are the least-glamorous, most-load-bearing part of a Data 360 build, and they come in two flavors — batch and streaming — that are not two speeds of the same tool. They differ in what they can do, not just how fast they do it, and the difference has a sharp edge most teams find the hard way: a streaming transform can’t join, and its output isn’t ready for segmentation or activation until you take another step. This post is what a Data Transform actually is, where it sits in the pipeline, the concrete difference between batch and streaming with the SQL and the node model, and the decision rule that keeps you from building the wrong one.

Where a transform sits — and what it is not

Data 360 ingests through data streams into data lake objects (DLOs) — raw storage containers, one per source feed. The unified model you actually segment and activate on is made of data model objects (DMOs), and the platform’s model is normalized, so raw source data has to be shaped to fit before it maps across. The pipeline reads:

data stream → source DLO → Data Transform → target DLO → mapping to DMO → identity resolution, segmentation, activation, insights.

A Data Transform takes one or more source DLOs, reshapes their data, and writes the result to a target DLO. That’s the whole job: change the shape of the data on disk so that when you map it to the model, the fields line up and the profile makes sense.

Two boundaries matter, because conflating them is where designs go wrong.

First, a transform is not mapping. Mapping connects a DLO field to a DMO field — it’s the wiring step. A transform prepares the data before that wiring, so the fields are worth connecting. They’re sequential and distinct: transform, then map.

Second — and this one costs people real money — a transform is not an insight. A calculated or streaming insight computes a metric (lifetime value, average order value, page views in a window) across your unified estate. A transform changes the shape of data and produces a new DLO. If you find yourself reaching for a transform to calculate “revenue per customer,” you want an insight. If you’re reaching for an insight to flatten three phone columns into one object, you want a transform. Same SQL editor in places, completely different job.

A transform reshapes data on disk; an insight computes a number from it. Get that boundary wrong and you’ll either build a metric that can’t be re-aggregated, or a DLO that quietly costs you every time it recomputes something a metric should have handled once.

Batch transforms: the visual builder that can join

A batch data transform is the powerful one, and it’s entirely visual — a drag-and-drop node builder, no SQL. You wire source DLOs into a graph of transformation nodes and out to one or more target DLOs. Because it operates on data at rest, it can do the things streaming can’t:

• Join across multiple source DLOs. The Join node supports the four you’d expect — inner, left outer, right outer, and full outer — combining rows on related fields.
• Aggregate — group by a key and sum, average, or count, including hierarchical aggregation.
• Filter, append, and apply formula columns (a calculated field like a lifetime-value sum, a flag, a derived key).
• Take multiple source DLOs in and write multiple target DLOs out in a single transform.

A representative batch flow: source the merchandise-purchase DLO and the ticket-purchase DLO, Join them on customer id, use a Transform/formula node to compute lifetime value as the sum of both and flag anyone over a threshold as VIP, Filter to just the VIPs, Aggregate to a per-industry rollup, and Output to a target DLO. Every step is a node; nothing is hand-written SQL.

Batch transforms run on a schedule — scheduled automation has been part of the product since Winter ‘24 — and you can also trigger one manually with Run Now, or from automation via the Flow action “Run a Batch Data Transform in Data Cloud.” A couple of structural rules to design around: you can’t use the same data object as both an input and an output of a transform, and when a transform has multiple output nodes they must share a single write mode. Output supports an upsert-style mode (update existing, insert new) and a replace-style mode that behaves incrementally, loading only what changed since the last run — worth choosing deliberately, because it affects both correctness and how much work each run does.

Streaming transforms: one object, one SQL statement, no joins

A streaming data transform is a different animal, launched back in Summer ‘23. It cleans and enriches data in near real time as it arrives, running continuously and picking up new and changed records. And unlike batch, it’s defined as a single SQL statement — no visual builder.

The constraint that defines it, and surprises everyone migrating ETL logic in: a streaming transform is limited to a single data object. No joins. You get basic SQL — SELECT, WHERE, CASE WHEN, string and date functions, and UNION to stack derived selects from the same source — but the moment you need to join two objects or aggregate across rows, you’re describing a batch transform, not a streaming one.

Here’s the canonical use: normalizing contact data on arrival. Say a source lands mobile, home, and work phone as three columns and your model wants one normalized phone object with a type and a composite key. A streaming transform flattens it:

SELECT CONCAT(Contact_core.CustomerId, "_Mobile") AS PhoneId,
       Contact_core.CustomerId                    AS CustomerId,
       Contact_core.MobilePhone                   AS PhoneNumber,
       "Mobile"                                    AS PhoneType
FROM Contact_core
WHERE ISNOTNULL(Contact_core.MobilePhone)
  AND Contact_core.MobilePhone <> ""

UNION a near-identical select for home and work, and three columns become one clean, typed object as each record streams in. Note the dialect details that trip up people expecting ANSI SQL: string literals are double-quoted, and null checks use ISNOTNULL(), not IS NOT NULL. (Object and field names above are illustrative — yours depend on your DLOs. The authoritative list of supported functions lives in the Salesforce Help docs; don’t assume a function exists because standard SQL has it.)

Streaming carries a set of setup rules that are non-negotiable and easy to miss:

• Every streaming SQL statement requires a primary key. No PK, no transform.
• If the target object is categorized as Engagement, you must map the Event DateTime field.
• The target DLO must be created in the UI and Active — you can’t point a streaming transform at a DLO that a data stream created, and while a data stream is connected to a DLO you can’t write to it from a streaming transform.
• You can’t edit a streaming transform after saving it. Changing the logic means deleting and recreating. Treat the SQL as immutable once live and version it outside the platform.

The gotcha that locks your output out of segmentation

Here’s the streaming edge that generates the most “why can’t I use this?” tickets. A streaming transform writes to a DLO — and a raw DLO is not, by itself, usable in segmentation or activation. Only fields properly mapped to a DMO, with valid object relationships, are available to the segmentation and activation surfaces. So streaming output is not activation-ready until you map it into the model.

This isn’t a permanent barrier — it’s a missing step. The mental model that keeps you out of trouble: a streaming transform’s job ends at “clean, near-real-time data in a target DLO.” Making that data segmentable is the separate mapping/harmonization step that follows. Teams who assume the transform is the finish line ship a beautifully clean DLO and then can’t figure out why it won’t appear in the Segment Builder. Map it to a DMO, and it will.

The decision rule

Strip away the detail and the choice is simple, because it’s dictated by capability before speed:

• Need to join, aggregate across rows, combine multiple sources, or do genuinely complex shaping? That’s a batch transform. Streaming physically can’t do it. Accept the schedule latency; it’s the price of the join.
• Need to clean, standardize, or enrich a single object’s data the moment it arrives — normalize a date, build a composite key, flatten columns, drop the obvious junk — and freshness matters? That’s a streaming transform, with the mapping step remembered.

And when the requirement is really “compute a metric,” step out of transforms entirely and into insights. Transforms shape; insights measure. A serious Data 360 build uses both, in sequence: transform the raw feeds into clean, model-shaped DLOs; map them into DMOs; resolve identities; then compute insights and build segments on top. Getting the transform layer right is what makes everything above it trustworthy — it’s the same principle as data quality for AI, applied one layer earlier in the pipeline. Clean the data before it becomes a profile, not after an agent grounds on the mess.

Where the credits go

Data Transforms consume Data 360 credits, metered against the volume of rows they process — and they’re charged every time rows run through, not once at setup. Two consequences follow.

First, streaming costs materially more per row than batch. A streaming transform is a standing, always-on process aggregating a live feed; a batch transform does a scheduled pass and stops. Reaching for streaming when the freshness isn’t consumed downstream is the same expensive reflex we flag for insights — real-time is a capability you pay for continuously whether or not anyone reads the freshness. If a source only needs cleaning once a day before an overnight segment runs, a scheduled batch transform does it for a fraction of the standing cost.

Second, an over-frequent batch schedule bills for work nobody asked for. A transform over a source that changes daily gains nothing from a more frequent run except a bigger line on the invoice. Match the schedule to how often the data actually changes. These are exactly the knobs our Data 360 credit optimization playbook exists to catch, and transforms are one of the easiest places to leave one turned to “expensive.” If you’re still sizing a first commitment, the mechanics of how credits accrue are in the pricing and credits guide.

(Exact per-row credit rates are published on Salesforce’s current rate card and shift between releases — check the live rate sheet before you forecast, rather than trusting a number from a blog, including this one.)

What to actually do

Start from the shape your model needs and work backwards to the feed. If the gap between raw and usable requires a join, an aggregation, or more than one source — it’s a batch transform, built in the visual node builder, scheduled no faster than the data changes. If it’s single-object cleaning that has to happen the instant data lands, it’s a streaming transform, written as one SQL statement in Data 360’s dialect, with a primary key, an Event DateTime if the target is Engagement, and a UI-created active target DLO — and remember it’s immutable once saved and not segmentable until you map it to a DMO. Keep transforms and insights in separate boxes: transforms reshape data, insights compute metrics, and the pipeline runs transform → map → resolve → measure.

Do that and the data reaching your unified profiles is already the right shape, which is what makes identity resolution match well, segments count correctly, and agents ground on records that make sense. Skip it, and every layer above inherits the mess — an over-merged profile, a segment that’s wrong, an agent confidently reading garbage. The transform layer is quiet, but it’s where the trustworthiness of the whole estate is decided. If you’re building the ingestion-to-model pipeline under a Data 360 program and want the batch-versus-streaming calls and the credit exposure made deliberately, talk to us — getting the data foundation right so it grounds AI without bill shock is exactly the work we do.

Understanding the basics

What is a Data Transform in Data 360?

A Data Transform reshapes, cleans, and enriches data as it moves from a source data lake object (DLO) to a target DLO, so that ingested data fits the normalized data model before it’s mapped to data model objects (DMOs). It sits between ingestion and mapping in the pipeline. There are two types: a batch transform (a visual, node-based builder that can join, aggregate, filter, and combine multiple sources on a schedule) and a streaming transform (a single SQL statement that cleans and enriches one object’s data in near real time). A transform is distinct from mapping (which connects fields) and from an insight (which computes a metric).

What’s the difference between a batch and a streaming data transform?

Capability first, speed second. A batch transform is a visual builder that can join across multiple DLOs, aggregate, filter, and write multiple outputs, running on a schedule over data at rest. A streaming transform is defined as a single SQL statement, runs continuously in near real time, and is limited to a single object with no joins allowed. Use batch when you need joins, aggregations, or multi-source shaping; use streaming for near-real-time single-object cleaning where freshness matters. Streaming also can’t be edited after saving (you delete and recreate), requires a primary key, and needs its target DLO created in the UI and mapped to a DMO before the data is usable in segmentation or activation.

Why can’t I segment on my streaming transform’s output?

Because a streaming transform writes to a raw DLO, and a DLO isn’t directly usable in segmentation or activation — only fields mapped to a DMO with valid relationships are. The streaming transform’s job ends at producing clean, near-real-time data in a target DLO; making that data segmentable is the separate mapping and harmonization step that follows. Map the target DLO into a DMO and it becomes available in the Segment Builder. Teams often assume the transform is the finish line and get stuck here; it isn’t — the mapping step is.

---

Building the pipeline from raw feeds to unified profiles and unsure whether a reshape should be a batch transform, a streaming transform, or an insight — or why the transform layer is costing more than you expected? Talk to us — designing the Data 360 foundation so data arrives in the right shape and grounds AI cleanly is exactly the work we do.