Deployment Blueprint: One Inference Tier Powers Video, Security, and Robotics

This blueprint is a representative reference architecture — anonymized and generalized from the deployment patterns we design. No client is named, and every claim below is marked as either a cited, proven deployment or architecture we propose.

The situation

A firm running physical sites — a construction group with multiple active projects, a facilities operator, a security-conscious industrial site — has three AI workloads that never used to belong on the same rack: reviewing camera archives after an incident, triaging security alerts and matching badges and vehicles against its own records, and running voice interaction and policy control on site robots. Each workload today runs on a different cloud API, at different latency, with footage, log data, and robot telemetry crossing a different vendor boundary. Every one has the same complaint: the API is fast enough to demo and too slow to trust unsupervised. A guard reviewing yesterday's footage can tolerate a ten-second delay; a SOC analyst confirming a badge match at the gate cannot; a robot waiting on a language model to decide what to say next cannot wait at all.

The constraint

• Data residency and third-party exposure. Site video, access logs, and vehicle records are exactly the data classes firms are least willing to hand a hosted API — under PIPEDA in Canada and state-level video- and biometric-privacy statutes in the US (the Illinois BIPA pattern being the strictest), sending identifiable footage or access data to a third party creates a disclosure and vendor-liability chain the firm would rather not create at all.
• No path to biometric overreach. Any identification pipeline has to be scoped deliberately: matching against the firm's own badge and fleet records, not building or querying an external facial-recognition database. That's a design constraint as much as a legal one.
• Latency is a safety property, not a convenience. A robot control loop or a live access decision that waits multiple seconds on a round-trip to a shared cloud endpoint is not a system a firm can put in front of a moving vehicle gate or a human coworker.
• Export-control and vendor-policy risk. The mid-2026 tightening of foreign access to frontier US models is a live reminder that a workload built entirely on someone else's managed API can be repriced, rate-limited, or cut off by a policy change the firm didn't make.

The requirement, in the client's own words on every engagement like this: the AI has to be fast enough to sit inside the decision, not report on it afterward — and it has to run on infrastructure the firm controls.

Why latency changes the product

This isn't a marginal speed-up. Hosted frontier APIs typically serve 50–200 tokens/sec; the fast-inference tier this blueprint uses runs one to two orders of magnitude faster:

• Cerebras CS-4, launched August 19, 2026 on the overclocked WSE-3 Turbo chip (2.8GHz versus the prior generation's 1.4GHz, same 900,000 cores), delivers 750 PFLOPS sparse FP16 per rack and serves models like Qwen3-Coder-480B and gpt-oss-120B at roughly 2,000–3,000 tokens/sec.
• Groq and SambaStack hold the 70B-class tier at roughly 250–460 tokens/sec — still 2–5x a typical hosted API, at a different cost profile.
• NVIDIA's GB300 NVL72 rack posts the highest raw aggregate numbers tested here — around 6,000 tokens/sec per GPU on DeepSeek-R1, over a million aggregate per rack in MLPerf submissions — by aggregating general-purpose silicon rather than a single specialized chip.

The proven result of this class of speed: OpenAI's GPT-5.3-Codex-Spark, its first production model not hosted on NVIDIA GPUs, serves real-time interactive coding at 1,000+ tokens/sec on Cerebras hardware. And the Cerebras/Hugging Face voice pipeline — Parakeet ASR, a language model on Cerebras, Qwen3TTS — hits sub-100ms round-trip and already runs in production inside more than 9,000 Reachy Mini robots. That's the order of magnitude this blueprint borrows: fast enough that a verification pass, a human-in-the-loop check, or a live voice turn finishes inside the interaction instead of after it.

Vendor comparison

	Cerebras CS-4 (WSE-3 Turbo)	Groq (LPU)	SambaStack (SN40L)	NVIDIA GB300 NVL72
70B-class throughput	~1,500–1,800 tok/s	~250–315 tok/s	~461 tok/s	~6,000 tok/s per GPU, rack aggregate into the millions
Deployment model	On-prem CS system or dedicated cloud endpoint	GroqRack colocation or hosted API	Turnkey rack appliance, ~90-day stand-up	Rack sold via OEMs/hyperscalers
Sovereign track on record	Cerebras for Nations (UAE, India, US DOE)	Bell Canada's AI Fabric; Saudi/HUMAIN	stc Saudi Arabia, OVHcloud Europe, SoftBank APAC, US DOE labs	Standard path for regulated/sovereign buyers (Azure, CoreWeave)
Best fit in this blueprint	Bursty reasoning: SOC triage, voice turns	Sustained throughput at lower cost per token	Fastest turnkey on-prem stand-up	Broadest model support; native path to Cosmos/Nemotron/NIM for the video pipeline

The architecture

One firm-owned gateway sits in front of the fast-inference tier and routes three pipelines to it. The tier is sized to the workload: Cerebras CS-4 or Groq/SambaStack dedicated capacity for SOC-triage and robotics-voice, where per-turn latency is the whole point, and a GB300 NVL72 rack where the workload needs NVIDIA's NIM-native Cosmos and Nemotron microservices for video.

flowchart TD
    subgraph Inputs
        A[CCTV / site cameras]
        B[SOC logs, badge & vehicle records]
        C[Robot mic array & sensors]
    end
    A --> G[Firm-owned inference gateway]
    B --> G
    C --> G
    G --> T["Dedicated inference tier<br/>Cerebras CS-4 / Groq LPU / SambaStack<br/>or GB300 NVL72 rack"]
    T --> P1["Video search & summarization<br/>VLM + LLM + RAG (VSS pattern)"]
    T --> P2["SOC triage & identification<br/>Morpheus pattern"]
    T --> P3["Robotics voice & policy<br/>ASR -> LLM -> TTS, Jetson Thor edge"]
    P1 --> R1[Security lead reviews flagged alerts]
    P2 --> R2[SOC analyst confirms every match]
    P3 --> R3[Robotics engineer validates in digital twin first]

Pipeline detail

Pipeline	Models	Hardware tier	Latency target	Who reviews
Video search & analysis	Vision-language model (Cosmos-class) + LLM (Nemotron-class) + RAG over the archive, indexing assisted by GLM-5.3-Flash's native multimodal path	GB300 NVL72 rack or multi-GPU NIM-native node	Sub-minute summarization of an hour of footage; flagged alerts surfaced within seconds	Site security lead reviews every flagged alert before any action is taken
Identification & SOC	Morpheus log-triage and phishing/anomaly pipelines, plus an open-weight LLM (GLM-5.3-Flash or gpt-oss-120B class) on the fast-inference tier for badge and vehicle matching against internal records	Cerebras CS-4 or Groq/SambaStack dedicated endpoint	Alert triage in 1–2 seconds; badge/vehicle match under 500ms	SOC analyst confirms every identification match; no autonomous access action
Robotics & real-time voice	ASR → LLM → TTS pipeline (the Reachy Mini pattern), Cosmos Edge running on Jetson Thor onboard, GR00T-style manipulation policies	Jetson Thor edge module on-robot, with the fast-inference tier as a cloud-assisted reasoning fallback	Sub-100ms voice round-trip; on-device policy inference in real time	Robotics engineer validates every policy in an Omniverse/Isaac digital twin before field deployment

The video and identification pipelines lean on NVIDIA's NIM microservices — Cosmos, Nemotron, and Morpheus ship that way, not as raw open weights. The robotics pipeline is closer to the Cerebras/Groq pattern: a fast open-weight ASR→LLM→TTS chain plus Cosmos Edge, NVIDIA's 4B-parameter world model built for on-device robot-policy inference on Jetson Thor. Every GR00T-style manipulation policy is trained and validated inside an Omniverse or Isaac Lab digital twin before it runs near a person — the same "prove it in simulation first" discipline NVIDIA's Mega blueprint applies to industrial robot fleets.

Licensing note

The video and identification pipelines carry a real licensing obligation the robotics pipeline mostly doesn't. NVIDIA's Developer Program permits free self-hosting of NIM containers — Cosmos, Nemotron, Morpheus — on up to 16 GPUs, but only for development and evaluation; production deployment needs either the 90-day NVIDIA AI Enterprise trial or a paid subscription. A narrower carve-out exists for certain NIMs on RTX workstation GPUs used non-commercially, which doesn't apply here. Budget the AI Enterprise subscription into the video and SOC tiers from day one; the Cerebras/Groq/SambaStack tier serving open-weight models under permissive licenses (MIT, Apache 2.0) carries no equivalent fee.

Sovereignty rationale

Every vendor here is actively building a sovereign or dedicated-capacity track, which is what makes this architecture defensible rather than exotic: Cerebras for Nations already runs UAE's JAIS2 and holds a US Department of Energy MOU under the Genesis Mission; Groq is the exclusive inference provider for Bell Canada's hydro-powered AI Fabric; SambaStack has stood up turnkey racks for Saudi Arabia's stc, OVHcloud in Europe, SoftBank in APAC, and multiple US DOE national labs. The pattern these programs share — open-weight models served at extreme speed on hardware inside the customer's own jurisdiction — is what this blueprint borrows at firm scale: video archive, security logs, and robot telemetry never leave a gateway the firm owns, on a tier it can own or lease as dedicated capacity, running models it can inspect and keep running regardless of any single vendor's pricing or access decisions.

Rollout

Phase	Scope
Assessment	Map camera coverage, SOC log volume, and robot fleet inventory; size the inference tier to peak concurrent load across all three pipelines
Pilot	Stand up video search and SOC triage first — highest immediate value, no on-robot hardware risk; run identification matches in shadow mode against a human-reviewed baseline
Robotics validation	Train and test voice and manipulation policies in the Omniverse/Isaac digital twin until they clear the review bar, before any on-site deployment
Production	Deploy Jetson Thor edge modules; cut the identification pipeline over from shadow mode to reviewed-live once match accuracy is proven against the firm's own data
Ongoing	Monthly model refresh cadence on the fast-inference tier; quarterly digital-twin re-validation for any updated robot policy

Kept honest

Two pieces of this blueprint are proven, cited deployments, not proposals: the Cerebras/Hugging Face sub-100ms voice pipeline already runs in production inside more than 9,000 Reachy Mini robots, and OpenAI's GPT-5.3-Codex-Spark serves real-time coding at 1,000+ tokens/sec on Cerebras silicon. NVIDIA's own blueprint numbers are also real and cited: the VSS pattern summarizes an hour of video in under a minute, and Morpheus-based phishing detection runs at Best Buy with 96% accuracy and a false-positive rate under 20%. The sovereign programs — Cerebras for Nations, Groq's Bell Canada partnership, SambaStack's turnkey deployments — are likewise real and operating today. What is not a cited deployment is the specific combination this blueprint proposes: running NVIDIA's video-search and Morpheus-style SOC patterns on Cerebras-, Groq-, or SambaStack-class silicon rather than NVIDIA's own GPU stack. That pairing is architecturally sound — it follows the published patterns and hardware numbers directly — but no vendor has published a case study running that way. We present it as the architecture we'd build and validate in a pilot, not as a reference someone else has already shipped.

Sizing guidance for the fast-inference tier itself is in our optimized-inference deployment guide; isolation patterns for the identification pipeline's access records are covered in the air-gapped AI service. See our construction and defense practices for site-specific variants of this pattern, and the GLM-5.3-Flash model page for the multimodal model this blueprint uses on the video-indexing path.