Components Overview
Apache and sibling trees live as git submodules under components/.
ASF forks track rch/devenv. Signals pins SHAs from that line.
| Component | Purpose |
|---|---|
| signals-protocol | Shared zndx.engine.v1 / scheduler.v1 (trunk) |
| Atlas | Governance + OpenLineage REST; AGE on :21010 |
| Marquez | OpenLineage UI; proxies Atlas /api/v1 (:21011) |
| Ranger | Tag-based authz from Atlas classifications |
| Kudu | Hot-tier columnar store |
| Impala | SQL across Kudu and Iceberg |
| Impala FDW | PostgreSQL → Impala HS2 / kudu_scan |
| Iceberg | Warm tier on RustFS |
| Polaris | Iceberg REST catalog (rch/asf-polaris) |
| YuniKorn | Application admission and queues |
| Airflow | Coordination Activities and Metaflow DAGs |
| Metaflow | Platform metadata + @kubernetes tasks |
| Hermes Agent | Agent peer pin (zndx/oss-hermes-agent) |
| MiNiFi C++ | Sentinels (C2 + OTel) on Knative |
| Signals Control UI | Primary backplane, :9889 (weathership/signals-ui) |
| NiFi | Flow routing (planned) |
git clone --recurse-submodules git@github.com:weathership/signals.git
git submodule update --init --depth 1
impala_fdw is weathership/impala_fdw (trunk). Marquez is
zndx/oss-marquez. signals-protocol is zndx/signals-protocol
(trunk). Hermes is zndx/oss-hermes-agent. MiNiFi is
weathership/oss-minifi-cpp.
C++ builds (Kudu, Impala) need cmake, ninja, gcc, protobuf,
flatbuffers, cyrus_sasl, openssl — all in devenv. The FDW needs
PostgreSQL 16 headers (pg_config).