Lead Contagion Platform — Architecture

# Lead Contagion Platform — Architecture

_Lyréco B2B funnel-intelligence & lead-generation platform · architecture as of 2026-05-27_

---

1. What it is

The Lead Contagion platform is a behavioural-intelligence graph for Lyréco's B2B e-commerce funnel. It unifies CRM records, web/email/order behaviour, and a relationship graph into one model that answers two questions for two teams:

Web-funnel conversion — where do customers drop, why, and what moves them?
Lead generation — which prospects look like our best customers, how warm are

they, and what should we offer them?

The organising idea (adapted from the OMEGA swarm work) is contagion: influence and behaviour propagate through a network — champions warm their neighbours, communities reveal segments the CRM can't see, and "fronts" of about-to-convert accounts can be identified before they convert.

---

2. Architecture at a glance

` EXTERNAL SOURCES Oracle Exadata (Halo DWH) Eloqua GA4 Companies House / BvD FMP/FRED/Massive │ (GlobalProtect VPN) │ ▼ ▼ ┌───────────────────────────────────────────────────────────────────┐ │ INGESTION — scripts/daily_refresh.py + workers/oracle_*_pull │ │ windowed pulls (current/week/predict), dedup, UTF-8, ANALYZE │ └───────────────────────────────────────────────────────────────────┘ │ ▼ ┌──────────────── 3-TIER DATA LAYER (Docker) ───────────────────────┐ │ PostgreSQL :5433 │ TimescaleDB :5434 │ Neo4j :7688 │ │ CRM / firmographic │ events / time-series│ influence graph │ │ 44 GB │ 120 GB │ ~1.15M edges │ └───────────────────────────────────────────────────────────────────┘ │ ▼ ┌──────────────── INTELLIGENCE LAYER (workers/) ────────────────────┐ │ journeys · path-mining · RFM/churn · communities · product │ │ appetence · segmentation CUBES (shrinkage) · contagion graph │ └───────────────────────────────────────────────────────────────────┘ │ ▼ ┌──────────────── APPLICATION LAYER ────────────────────────────────┐ │ FastAPI + HTMX dashboard (NSSM service, dual-stack, Tailscale) │ │ 30+ panels · Ask-Claude analyst · floating Chat widget │ └───────────────────────────────────────────────────────────────────┘ │ │ operators over Tailscale Claude agent layer (claude -p) + Swarph mesh (peer DMs) `

---

Data sovereignty — where the LLM sits in the stack

A deliberate trust boundary runs through the platform. Lyréco's sovereign data is ingested and crunched by deterministic Python; the LLM only ever works the aggregated, derived intelligence that pipeline produces.

` ┌──────────────────────────── LLM (Claude) ─────────────────────────────┐ │ interprets AGGREGATED intelligence → reports · recommendations · chat │ │ (communities · segmentation cubes · profiles · shrunk deviations) │ └────────────────────────────────────────────────────────────────────────┘ ═══════════════ trust boundary · PII-extraction approval gate ═══════════ ┌──────────────────── deterministic Python · no LLM ────────────────────┐ │ INGEST Oracle / Eloqua / GA4 pulls (scripts/, workers/oracle_*) │ │ CRUNCH communities · appetence · cubes · RFM/churn (workers/) │ │ raw, record-level Lyréco data — handled only by code │ └────────────────────────────────────────────────────────────────────────┘ `

Below the line — Python only. Every raw Lyréco record is pulled and

transformed by plain, deterministic Python. No LLM participates in ingestion or record-level computation; the pipeline is reproducible and auditable.

Above the line — the LLM on aggregates. Claude operates on the *finished

product* — communities, segment cubes, profiles, shrunk deviations — interpreting it to build reports, recommendations and chat answers. It reads the derived intelligence, not the raw record store.

A guarded boundary. A PII-extraction approval gate intercepts any

raw-record-shaped request for commander sign-off; the analyst runs on the operator's own Claude subscription; secret + PII stores stay access-gated. The model is positioned as an interpreter of aggregates, not a handler of sovereign records.

---

3. Data layer (3-tier, Dockerised)

Three databases, each chosen for its access pattern. Cross-database joins are not done in one query — bridges are always two-step (pull IDs from one, query the other with ANY(%s)).

| Store | Port | Role | Holds | Size | |---|---|---|---|---| | PostgreSQL | 5433 | CRM / firmographic / derived | accounts (56k), users (317k), products (1.47M), communities, segmentation cubes, prospect tables | 44 GB | | TimescaleDB | 5434 | event & time-series | customer_timeline (~173M unified events), GA4 (action/product/search/page/sessions ≈ 335M rows), ecom_order_lines (~25.5M), ecom_invoice_lines (~24M), email, stock history | 120 GB | | Neo4j | 7688 | influence graph | :BOUGHT_TOGETHER (~1.15M edges), referral/co-occurrence, community structure | — |

Storage discipline: Timescale chunks are compressed (GA4 ~8×); a compression policy auto-compacts prospect_scores past 7 days. The Docker WSL2 disk is periodically TRIM'd + compacted to return freed space to the host.

---

4. Ingestion

Source of record: Oracle Exadata "Halo" data warehouse, reached over the

PaloAlto GlobalProtect VPN; ORACLE_TNS_ADMIN → repo oracle/.

Pipeline: scripts/daily_refresh.py orchestrates windowed pulls —

GA4 ×5 + ANALYZE, Eloqua activity/results, ecom_order_lines, RFM/churn, customer_timeline rebuild, HubSpot funnel — typically ~30–50 min.

Scope rule: narrow windows by default (current ≈ latest day, week,

predict ≈ 180d on-demand) — the source tables are hundreds of millions of rows.

Bridges: identity is stitched across systems — `ecom_users.contact_id ↔

account_number ↔ gravitee_user_id ↔ ga4_sessions.last_sso_uuid`; HubSpot account_number LPAD-to-10; Eloqua via lower(contact_email); BvD via payer_number = account_number; Companies House via cro_number.

---

5. Intelligence layer (`workers/`)

The analytical engines that turn raw data into decisions:

Journeys & paths — 7 canonical journeys + journey_alignment (~33M rows);

path_mining_results (~1,350 patterns) over pre-aggregated visitor sequences.

RFM / churn — recency-frequency-monetary + Chandelier-Exit churn scoring.
Communities — firmographic communities (KMeans, soft cosine, k=60 per

country) + emergent behavioural communities; prospects soft-match into them.

Product appetence — segment_family_appetence: per (segment × family)

online/offline lean, co-purchase lift, view→cart.

Segmentation cubes (the "sauce") — the load-bearing pattern, sourced from

the Swarph mesh (lab-ovh + gridiron): a fact → rollup → volume-shrinkage pipeline. Sparse fit × engagement × channel × product cells are shrunk toward their parent marginal by volume (estimate = cell·n/(n+K) + parent·K/(n+K), K set inversely to signal persistence) so deep cells become trustworthy and the combinatorial blow-up stops mattering. Powers:

Prospect segmentation → prospect_profile (fit × engagement quadrant) +

community_product_offer (the 5-SKU pack a prospect inherits).

Customer segmentation → segment_cube_cell + the cross-sell / whitespace /

channel-migration / churn lenses (one cube, four views).

Contagion graph — Neo4j community detection, patient-zero / contagion-front

identification, peer influence.

---

6. Application layer

Dashboard: FastAPI + HTMX, ~30 panels across tabs (Ask-Claude, Data

Intelligence, Reports, SWARPH DMs, Customers, Prospects, Journey, Campaigns, Friction, Market, Contagion). Server-rendered HTML fragments, htmx-loaded.

Runtime: Windows NSSM service LeadContagionDashboard →

start_dashboard_dualstack.py pre-binds a dual-stack socket (IPv4 + IPv6, required for Tailscale MagicDNS). Reachable by colleagues over Tailscale at workstation.tail288b3b.ts.net:8001.

Analyst agent (Ask-Claude / Chat): prompts run claude -p headless against

the repo + DBs, returning analyst deliverables. Two front-ends: a templated Ask-Claude panel and a global floating Chat widget with persistent per-user memory (carry-over + ~1M-token compaction) and a superadmin oversight surface.

---

7. Cross-cutting: the Claude-agent & Swarph-mesh layer

The platform is itself operated and extended by a Claude agent (this workstation = the workstation-lc Swarph mesh peer):

claude -p runner — subscription-billed, session-resume threading,

per-user timeout floors, approval gate for PII-extraction-shaped prompts.

Swarph mesh — peer-to-peer DMs to lab-ovh (orchestrator), gridiron,

droplet, etc., over a gateway; architectural patterns (e.g. the shrinkage cube) are sourced cross-peer. Tooling: scripts/send_mesh_dm.py, watch_mesh_dms.py; a per-day Claude-activity digest.

---

8. Security & access

AuthN: HTTP Basic auth (lyreco/lucas/darshan); role tiers —

superadmin (commander) > Lucas > Darshan.

Perimeter: Tailnet + private repos = "within network"; mesh peers inside the

perimeter. No public exposure yet (a PHP-proxy public path is a gated future step).

Guardrails: PII-extraction prompts hit a commander-approval gate (DM

with approve/deny) unless the user is trusted-bypass; chat oversight edits are audit-logged (who-changed-whose-memory); the Claude harness auto-mode classifier blocks self-installing persistence.

Secrets: API tokens in env / .dashboard_extras.env (gitignored);

imports/ (PII) and exports/ gitignored.

---

9. Deployment & ops

DBs: Docker (docker-compose), volumes on the host WSL2 disk.
Dashboard: NSSM service; nssm restart LeadContagionDashboard to reload.
Data: daily_refresh on VPN; cube/appetence recomputed after refresh.
Observability: Claude-activity watcher (scheduled at logon) digests what the

agent shipped; refresh/compute logs under logs/.

---

10. Design principles

1. Shrinkage over bigger cubes — never trust a thin cell; shrink toward the parent until it earns its own signal. Solves the sparse-segmentation blow-up. 2. Coverage-aware — an unobserved channel is unknown, not cold; scores carry confidence. 3. Canonical dimensions first — lock + normalize taxonomies before cubing. 4. Two-step cross-DB bridges — never join across Postgres/Timescale in one query. 5. Narrow data windows — pull what you need; the warehouse is huge. 6. Reuse the engine — prospect & customer cubes share one shrinkage core.

---

_See PLATFORM.md for the full platform overview (capabilities, data assets, deliverables, roadmap). Design specs + implementation plans live under docs/superpowers/._