DataHub

Architecture Overview

How DataHub services connect, how telemetry flows, and where data is stored.

DataHub is built as a local, offline-first platform. One docker compose up starts the full stack — no cloud dependency at runtime.

High-level data flow

adapters / simulator ──► NATS JetStream (datahub.telemetry.>)

              ┌───────────────┼───────────────┐
              ▼                               ▼
      telemetry-worker                  rule-worker ──► BullMQ ──► action-worker
      history → PostgreSQL              alarms → PostgreSQL       Slack / Email / Telegram / HTTP
      latest  → Redis

   Next.js console ──► NestJS API ──────────┘ (rule.changed cache reload)

Services

ServiceResponsibility
apiREST API: auth, hierarchy CRUD, rules, alarms, dashboards, telemetry queries
adapter-orchestratorReconciles adapter processes against the device registry
telemetry-workerPersists telemetry history to PostgreSQL (batched); latest values and liveness to Redis; skips inactive tags
rule-workerEvaluates rules in real time; creates alarms; enqueues action jobs
action-workerDelivers notifications via configured channels
frontendOperator console and public documentation
postgresSource of truth: hierarchy, users, rules, alarms, dashboards, history
redisLatest-value cache, device liveness, rate metrics, BullMQ queue
natsJetStream telemetry stream + control subjects

All backend services ship in one Docker image with different entry commands.

Data model

Hierarchy follows ISA-95 alignment:

Plant (Site) ── Machine (Work Center) ── Device ── Tag (signal)
Plant (Site) ── Warehouse ── Location ── Item / SparePart / Stock

Every entity has a URL-safe code. A tag's identity is its tagPath:

plantCode/machineCode/deviceCode/tagCode

Example: gebze/press-line-01/s7-1500-01/motor.current

Telemetry contract

Adapters publish JSON events to NATS subject datahub.telemetry.<plantCode>.<deviceCode>. See Adapter contract for the full payload schema.

Live vs history

  • Live console views (Operations Center timeline, UNS, device latest values) read the Redis cache for active tags only
  • History queries read PostgreSQL; inactive tags do not receive new rows after deactivation
  • Backlog drain or batch replay must not be mistaken for live plant state — see Telemetry pipeline

Control events

The API publishes control messages when configuration changes:

  • datahub.rule.changed — rule-worker reloads its in-memory cache
  • datahub.device.changed — orchestrator reconciles adapter processes

Security model

  • JWT authentication for all console and API access (except documented public share routes)
  • Resource + action RBAC — not role-name gates
  • Plant-scoped data access for multi-site deployments

Read RBAC for the permission model.

Next steps