Data Pipelines (Developer)

Data pipelines run on telemetry before persistence, realtime, and profile alarm evaluation.

Product boundaries

• Transform + webhook outbound = data/attribute pipelines (linear)
• Alarms = device profile alarm rules only
• Graph / rule-engine canvas is not in the product surface (code may exist but is not exposed in nav)

API endpoints

• GET /api/data-pipelines — list
• GET /api/data-pipelines/:id — details
• POST /api/data-pipelines — create
• PATCH /api/data-pipelines/:id — update
• DELETE /api/data-pipelines/:id — delete
• POST /api/data-pipelines/:id/start — set status active
• POST /api/data-pipelines/:id/pause — set status paused
• POST /api/data-pipelines/preview — preview transform (no webhooks sent)

Preview response extras

• webhookWarnings — human notes
• webhookPreviews — { url, method, headers, body, note } for each webhook step

Step types

rename_keys, change_scope, filter_range, pick_keys, drop_keys, scale_keys, derive_key, flatten_object, clamp_range, set_default, coerce_types, copy_key, round_keys, remove_empty, map_values, string_transform, conditional_drop, webhook_forward

Webhook body

Default (includeContext !== false):

{
  "ts": 1710000000000,
  "deviceId": "…",
  "tenantId": "…",
  "profileId": "…",
  "telemetry": { "temperature": 22.5 }
}

Rollback flag

• DATA_PIPELINES_ENABLED=false — disables execution (passthrough)