When real-time marketing data is worth the extra cost

Real-time capability needs a decision-level business case

Paying for faster marketing data makes sense when speed changes a customer decision. When data already arrives before that decision must be made, buying more speed adds cost without improving the outcome. A marketing technology, or martech, purchase should therefore start with the deadline for using the data.

That buying decision has become harder to separate from a long-standing promise: real-time data would solve customer engagement and experience problems. A martech enthusiast traces the term to computing, particularly operating systems and data processing. In the commentator’s account, customer relationship management, or CRM, and web analytics vendors adopted it in the 1990s to distinguish immediate system updates from overnight batch processing, which handles accumulated data together. Martech vendors later made real-time processing a premium capability, giving them a financial interest in selling greater speed.

Yet having that capability available leaves buyers with two questions: can the organization support it, and which decisions require it? The commentator argues that most organizations lack the infrastructure and systems to support real-time processing at scale. Infrastructure readiness and business need are separate constraints, so buyers must assess both against the deadline for the customer interaction.

Define freshness by when a decision starts to degrade

The decision deadline turns “fresh enough” into an engineering requirement. For each data item, ask which decision it enables and how old it can become before that decision gets worse. Different decisions can tolerate different ages of data even when they share infrastructure, so a platform-wide preference for real-time processing cannot establish the right requirement.

To match processing speed to those needs, the commentator proposes real-time handling for the small amount of data that genuinely requires it, while arranging for most data to arrive before use. This split reflects differences in how quickly customer data changes and how often an organization collects it. Those differences should guide pipeline design: how data moves from its origin to the systems that use it.

Making these requirements explicit applies decision intelligence: examining how decisions are made and using feedback to improve their inputs and outcomes. Gartner defines it as “a practical discipline that advances decision-making by explicitly understanding and engineering how decisions are made, and how outcomes are evaluated, managed, and improved via feedback.” Gartner also has a commercial interest here: as a research and advisory company, it benefits from demand for guidance on this discipline. Applied to freshness, the approach means establishing a decision’s time window and urgency before choosing how often its inputs should update.

Choosing that interval requires weighing the consequences in both directions. Updating too often consumes time, money, and resources without a corresponding benefit; updating too slowly can damage operational processes, customer experiences, and brand equity and health. The commentator argues that delivery speed becomes a vanity metric when it has no effect on whether an input remains useful at the moment of decision.

Account for processing costs and reconciliation

Once a decision has an explicit freshness requirement, the cost of exceeding it becomes easier to examine. The commentator acknowledges the technical appeal of making every data stream real-time, including designs that aim for delivery in less than a millisecond. But that ambition brings construction, operation, and maintenance obligations that remain even when a scheduled update would serve the decision equally well.

Those obligations begin with streaming infrastructure, which processes data as it arrives, and the always-on pipelines that keep it moving. Both need continuing operation and maintenance, so building them is only part of the expense. When the relevant marketing work can run periodically, the organization still pays to keep those systems active between decisions. Batch processing can instead group the work within the time available.

Keeping pipelines active also creates recurring work inside them. Incoming data often needs transformation into a form another system can use, so continuous processing means repeating those transformations as updates arrive. Identity resolution, which connects data to the relevant customer identity, adds engineering and operating demands when it must happen with very little delay. Storage adds another expense, making an unnecessary speed requirement an ongoing budget commitment across these systems.

The commentator also links faster ingestion, the arrival of data into a system, with greater difficulty maintaining consistency and reconciliation, the work of making records agree. As data arrives faster, the system must maintain that agreement at a faster pace. A decision may therefore receive a newer input while the organization faces more work establishing whether the combined view is dependable.

Where no immediate decision is waiting, the commentator argues that allowing time for reconciliation can help maintain a reliable shared view of the data. Reliability therefore belongs in the business case alongside infrastructure spending. Some decisions do lose value if processing waits, so the requirement must specify dependable data by the deadline. Faster processing is justified when meeting that deadline requires it.

Give each marketing decision its own refresh interval

The commentator’s examples show how these deadlines differ within marketing. Cart abandonment data, fraud detection signals, and personalization during a web or app session need to be available within seconds. For an interaction already under way, a later update may arrive after the opportunity to act has passed, so delay can directly change the outcome.

Other work allows a wider window: the commentator places retargeting audiences, lifecycle segmentation, and preparation of weekly campaigns in the category that can use hourly or daily updates. A weekly campaign describes the work being prepared; its inputs may still need to refresh more frequently than once a week. These intervals express the commentator’s use-case guidance, with the decision’s actual time window determining the requirement.

Campaign preparation also depends on inputs with much tighter deadlines. The commentator calls for nearly immediate updates to consent and suppression data so that opt-outs and compliance issues affect whether a customer can be contacted. An audience refresh may tolerate an hourly schedule while permission to contact someone changes almost immediately. The preparation schedule therefore cannot determine the refresh interval for every input.

These different deadlines can coexist within the same workflow. Engineering audience preparation and permission checks separately preserves urgent handling while allowing other inputs to follow a slower schedule. This gives each decision a specific requirement rather than a blanket instruction to process everything in batches.

Translate deadlines into SLAs and data access choices

To establish those requirements, the commentator recommends starting at the customer-facing channel or touchpoint. Identify the interaction and the decision, determine how often the data must be accessed or refreshed, and then trace it backward to its origin. This sequence establishes acceptable latency, meaning the delay before data becomes available for use. Starting with the infrastructure’s maximum speed risks making its capability the default requirement.

The fraud-detection example shows why the interaction must come first. Fraud checks within a digital app may need to happen much more frequently than checks for updates to a credit score, even though both assess a customer in a digital setting. Each check needs its own freshness requirement because the interaction determines how often the relevant information must be reconsidered.

Those requirements can then become service-level agreements, or SLAs, that specify the service a system must deliver for each use case. Each SLA should state the tolerable age of the data and the decision deadline clearly enough to guide engineering choices. This avoids applying the “fastest speed available” across the infrastructure while keeping tighter requirements for urgent inputs within a slower workflow.

An explicit deadline also helps separate quick access from the decision to copy data into another system. Modern architecture vendors offer hybrid or federated approaches that combine access methods or retrieve data where it already resides. These vendors have a commercial interest in selling those approaches; the relevant capability is that a consuming system can obtain information from its existing location. For tightly timed decisions, that can provide quick access without requiring every input to be copied first.

Where the deadline permits, the commentator recommends batch processing or change data capture, which tracks changes to data at its origin. The choice depends on the decision’s timing and how the data must reach the system using it. Retrieval, duplication, ingestion, and continuous processing are separate architectural choices, so a requirement for one need not dictate all the others.

These choices give procurement a concrete acceptance criterion: can the proposed system deliver the relevant insight in time for the intended decision? The commentator presents decision intelligence as the next competitive focus for martech, beyond storing data, advertising real-time capability, and trying to respond immediately to every customer event. Buyers can put that focus into practice by requiring each proposed speed premium to serve an explicit use-case SLA.

Key executive takeaways

• Tie speed spending to business value: Martech buyers need evidence that faster data changes a customer decision before paying a real-time premium. Assess infrastructure readiness separately from the business need for speed.
• Define freshness for each decision: Business owners and engineering teams need to establish how old each input can become before the decision deteriorates. Use that threshold to set processing intervals and evaluate outcomes.
• Budget for ongoing processing costs: Architecture teams need to account for always-on pipelines, transformations, identity resolution, storage, and reconciliation. Batch processing can reduce these demands where the decision deadline allows it.
• Separate urgent inputs from scheduled work: Marketing operations needs nearly immediate consent and suppression updates, while audience preparation may tolerate hourly or daily refreshes. Assign each input its own deadline, even within the same workflow.
• Turn deadlines into procurement requirements: Use-case SLAs give engineering and procurement teams clear criteria for acceptable data age and delivery time. Evaluate streaming, batch processing, change data capture, and direct retrieval against those requirements.