Managed data feed

Overview

The Managed data feed turns a WFX pipeline into a secure web address that another system reads from on its own schedule. Instead of us pushing your data somewhere, the system on the other end pulls the current product data — and only what has changed since it last looked — whenever it is ready for it.

It is the right choice whenever the receiving system can call a web address and read JSON: a data lake, a data warehouse (Snowflake, BigQuery, Databricks), or a bespoke integration.

Is this right for you?

Choose the Managed data feed when:

• The receiving system ingests on its own schedule — a nightly load, an hourly job, an on-demand refresh — and would rather pull than be pushed to.
• You want incremental updates ("just what changed"), not the whole catalogue every time.
• You are feeding a store that should hold its own history — a data lake or warehouse — while we hold nothing.

Look elsewhere when:

• You are connecting Power BI or Fabric specifically — use the Power BI feed, which is shaped for Get Data → Web.
• You need us to push each change to an endpoint you run, the moment it happens — use the Webhook or Custom API endpoint connector instead.

How it works

1. You build a normal pipeline: WFX in, Managed data feed out, and you map the fields you want to expose.
2. Saving the feed gives you a web address and a consumer key — the password the other system presents to read it.
3. The receiving system does a one-time snapshot to load everything, then polls the changes endpoint on its own cadence to stay current. Each poll returns only the records that changed, each tagged as an update or a removal.

You choose, per feed, where the data lives:

• Live (zero retention) — the default. Every read triggers a fresh look at WFX right then and returns the current, changed records. We store none of your product data — only enough fingerprints to know what has already been sent. Best when data custody matters: the receiving system keeps the history.
• Cached — we keep an encrypted copy of just your chosen fields, refreshed on the pipeline's schedule. Reads are instant and unlimited, with a replayable position marker. Best for very large catalogues or many simultaneous readers.

Setting it up

The exact fields are in the reference below. In short: create the pipeline, choose Live or Cached, map your fields, save, then copy the web address and consumer key from the Feed tab.

Then hand your integration team the step-by-step: Consuming a Managed data feed — it walks through the snapshot-then-changes pattern, authentication, polling cadence and the record shape, with examples.

FAQ

Is my WFX login exposed? No. The consumer key only unlocks this feed's chosen fields. It never grants access to WFX or anything else.

How often can it be read? In Live mode, once per interval you set (default every 15 minutes) — a read triggers a real WFX scan, so there is a short cool-down between reads. In Cached mode, as often as you like.

What if the catalogue is huge? Live mode is built to fail loudly rather than hand back a half-complete table — if a feed is too big to read live in one pass, either narrow the pipeline (fewer fields, or a product-category scope) or switch it to Cached.

Can it carry full version history? Yes — set the feed's record key to the WFX record id (rather than the product code) and each version becomes its own row. The guide explains it.