Data Pipelines

Data pipelines let you transform telemetry and optionally send it outbound before it is stored and shown in the UI.

Roles (keep these separate)

Layer	Job	Where
Data / Attribute Pipelines	Transform + optional webhook outbound	These pages
Device Profile Alarms	Threshold / no-update detection	Device Profiles → Alarms
Blank pipeline	Passthrough — data flows unchanged	Default when none selected

Order on ingest: pipeline transform (+ webhook side-effect) → persist → realtime → profile alarms.

Webhook failures are fail-open (ingest continues). Alarms are not configured inside pipelines.

When to use

• Rename keys across firmware versions (temp_c → temperature)
• Drop invalid telemetry (range checks / conditional drop)
• Normalize payloads (flatten, coerce types, defaults, map values)
• Forward cleaned telemetry to an external system (webhook)

How it works

• Optional on a device profile
• No pipeline selected → telemetry stored as received
• Runs before DB write, realtime, and profile alarms
• Fail-open: if a step throws, ingest continues with original telemetry

UI

• Data Pipelines — /data-pipelines
• Data Pipeline Editor — /data-pipelines/:id