Edge AI Inference Box Product

Overview

An edge AI inference box runs trained neural networks where the data is produced — at a factory line, a traffic intersection, a retail ceiling, inside a vehicle — instead of streaming raw video to a data center. The economics are straightforward: a handful of 1080p camera streams saturate an uplink and cost real money to haul to the cloud, while the inference results (a count, a defect flag, a license plate) are a few bytes. Latency and privacy push the same direction: a safety interlock cannot wait on a round trip, and many sites are not permitted to export video at all.

The box is built from a merchant AI Compute Module — a system-on-module in the style of NVIDIA's Jetson or comparable NPU modules — mated to an application-specific Carrier Board, all sealed inside a Heatsink Chassis that cools the silicon without a fan. The Power Input Stage stage lets it wire directly into vehicle or cabinet DC, and the Wireless Connectivity radios cover sites without wired backhaul.

How it works

Cameras connect through the RJ45 Port pair, with the PoE Controller powering each camera over its data cable so a deployment needs no separate camera supplies. Compressed H.265 streams land in the SoC's hardware video decoders, which can unpack a dozen or more 1080p feeds in parallel without touching the CPU. Decoded frames flow through a preprocessing stage (resize, color conversion, normalization) and into the accelerator, where the network — typically an object detector or segmentation model quantized to INT8 — executes across hundreds of parallel multiply-accumulate units.

Throughput is usually bounded by memory, not arithmetic. Every layer of the network streams weights and activations through LPDDR5 Memory, so the module's ~200 GB/s memory bandwidth determines how many camera streams sustain real-time rates; the headline TOPS figure is reached only by models that fit the on-chip caches well. The Module PMIC enforces selectable power modes — the same module might run at 25 W on a vehicle budget or 60 W on mains — trading clock speed for thermal headroom.

Results leave the box small: detection metadata to an MQTT broker or REST endpoint over the uplink, short annotated clips to the NVMe SSD as a rolling buffer for audits. The supervisor Microcontroller on the carrier runs a hardware watchdog and reboots the module if the inference pipeline stalls, since these boxes operate unattended for years.

Thermal design

The accelerator dissipates a server-class heat flux in a fist-sized enclosure with no airflow. The die couples through Thermal Interface into a copper Heat Spreader, which conducts into the wall of the Finned Shell; the entire extrusion becomes a radiator running warm to the touch. At 60 W and +60 °C ambient the design margin is thin, so the firmware throttles clocks against junction temperature, and the SSD Thermal Pad keeps even the SSD controller coupled to the case — an NVMe drive sealed in still air will otherwise throttle before the SoC does. Eliminating the fan removes the dominant field-failure mode and lets the box sit in dusty, oily, or washdown-adjacent locations with no filter maintenance.

Deployment interfaces

Vehicle installs wire the DC Input Connector to switched battery power; the Ignition Logic delays boot until voltage stabilizes after cranking and gives the OS a clean shutdown window when the ignition drops, protecting the filesystem. The Surge Stage absorbs ISO 7637 load-dump transients that would destroy a desktop supply. On a factory wall the DIN Bracket snaps the box onto a DIN rail beside the PLC it feeds, and the CAN Transceiver or isolated DIO lines on the terminal blocks let an inference result trip a reject gate within milliseconds. The Display Output and USB Port bank exist mostly for commissioning — once provisioned, the box is headless, managed over the network through the Wi-Fi Module or Cellular Module, with fleet tooling pushing model updates to the microSD Slot-recoverable system image.