Edge AI hardware is the compute that runs AI inference on-device, without a cloud connection. An edge AI compute unit mounted inside a tram, bus, or industrial vehicle faces constraints that no benchmark sheet captures: it must hold a power budget drawn from the vehicle's auxiliary supply, shed heat passively inside a sealed and vibration-exposed enclosure, survive years of operation across temperature extremes, and communicate with the vehicle controller over a bus protocol the vehicle OEM specifies. Peak TOPS is one input to the selection decision. Power budget, thermal design, MTBF, and integration interface are the inputs that decide whether the system actually ships.
Key Takeaways
Edge AI runs AI algorithms locally on the device rather than sending data to a cloud server for processing. The core benefit is latency: local processing means the result is available in milliseconds, not hundreds of milliseconds. For real time processing in a safety system, that latency difference is not a performance metric; it is the design constraint that determines whether the system is viable at all.
The contrast with cloud AI is direct. Cloud AI sends data to a remote data center, waits for the inference result, and returns it to the device. That round-trip adds 50–500+ ms of variable latency depending on network conditions. Edge AI eliminates the round-trip. AI inference happens on the edge AI compute unit inside the vehicle, with zero-to-low latency and no dependency on connectivity. Edge AI also enhances privacy by keeping data on local devices: sensitive video or sensor data never crosses an external network.
For a full technical comparison of edge AI and cloud AI architectures, see edge AI vs cloud AI.
The model families used in edge computer vision deployments are driven by the constraint that matters on every platform: the ratio of inference accuracy to compute cost. One-stage detectors (YOLO family, EfficientDet, MobileNet-SSD) dominate real-time edge deployments because they complete a forward pass in a single network evaluation, making them compatible with the frame-rate requirements of live video (30–60 FPS) on constrained hardware.
Deep neural networks designed for cloud GPUs are not directly deployable on edge hardware. They must be optimized through model conversion for the target runtime (TensorRT, OpenVINO, TFLite, ONNX Runtime), quantization from floating-point to INT8 or INT4 (which reduces model size and increases inference speed at a small accuracy cost), and pruning to remove low-weight connections. Modern edge hardware supports model optimization and quantization for efficiency natively: NVIDIA Jetson uses TensorRT, Intel's OpenVINO targets its own NPU and CPU stacks, and Hailo's Dataflow Compiler handles the compilation and quantization pipeline for its accelerators. Machine learning models that bypass these pipelines and run in generic frameworks typically achieve a fraction of the throughput that the hardware is capable of.
See model quantization for edge deployment for the optimization mechanics and accuracy trade-offs.
Edge computing moves computation from a centralized data center to hardware located at or near the point where data is generated. In an in-vehicle context, that means the AI engine sits inside the vehicle cab, processing sensor and camera data frame by frame without any cloud call.
The AI engine on a modern SoC (system on chip) is a dedicated processing unit separate from the CPU and GPU: a DLA (Deep Learning Accelerator) on NVIDIA Jetson platforms, a Neural Engine on Apple SoCs, or a dedicated NPU on Hailo, Qualcomm, and Rockchip platforms. This single chip or dedicated block handles the matrix multiplications that dominate neural network inference, running them at far higher throughput and lower power draw than a general-purpose CPU. Localized processing lets models run directly on the device at the latency required for frame-by-frame safety decisions, and specialized hardware accelerators improve inference speed by 10–100x over CPU-only execution at equivalent power levels.
The AI engine's role in an edge AI system is to receive preprocessed sensor data (a camera frame, a point cloud slice, a fused multi-sensor input), run the forward pass through the model, and return structured outputs (bounding boxes, class labels, confidence scores) to the application layer, all within the frame budget. For 30 FPS video, that budget is 33 ms end-to-end.
Three patterns cover the majority of in-vehicle edge AI deployments:
The AI engine is integrated on the same silicon as the CPU, GPU, memory controller, and I/O subsystems. NVIDIA Jetson Orin is the canonical example: a single SoM containing Ampere GPU cores, Arm CPU cores, DLA accelerators, and ISP, all sharing a unified memory architecture. This pattern minimizes latency between sensing and inference, simplifies thermal design (one TDP budget), and reduces board complexity. It is the right choice when the inference workload is well matched to the SoC's AI engine and when the software ecosystem (CUDA, TensorRT, DeepStream) provides the tooling needed.
A host CPU or SoC handles application logic and pre/post-processing while a discrete AI accelerator (Hailo-8 via M.2 or PCIe, Google Edge TPU, Intel Movidius) handles the inference-heavy model forward pass. This pattern suits deployments where existing edge hardware needs to be augmented with AI capability, or where the AI workload is well-defined and the host processor handles everything else. The Hailo-8 delivers 26 TOPS at 2.5W typical power (Hailo, 2024), making it a practical addition to power-constrained in-vehicle systems where the host platform cannot be replaced.
The edge device handles latency-sensitive, safety-critical inference locally while offloading computationally heavier but non-time-critical tasks (retraining data preparation, long-horizon analytics, model evaluation) to the cloud during connectivity windows. This is the standard production architecture for managed fleet deployments: the key components at the edge handle the safety function; the cloud handles the model lifecycle.
Accelerator comparison
NVIDIA's Jetson AGX Orin tops the GPU lineup, delivering 275 TOPS at 15–60 W. It's the most flexible option on this list, backed by a rich SDK and the ability to run multiple models at once, all within the mature CUDA and TensorRT ecosystem.
The Jetson Orin NX sits a tier below, at 100 TOPS and 10–25 W. It's the pragmatic middle ground for teams that need real throughput without committing to the AGX Orin's power budget or price tag, and it runs on the same CUDA and TensorRT stack.
The Jetson Orin Nano rounds out NVIDIA's lineup as the entry point: 40–67 TOPS at 7–25 W. It's the board most teams reach for when validating a pilot before scaling to production hardware, again within CUDA and TensorRT.
On the NPU side, Hailo's chips take a different approach entirely. The Hailo-8 delivers 26 TOPS at a typical 2.5 W, fanless, and built for power efficiency rather than raw flexibility. It runs on Hailo's own SDK alongside TFLite and ONNX support. The Hailo-8L pushes further into low-power territory: 13 TOPS at just 1–2.5 W, aimed squarely at compact, battery-sensitive deployments.
For teams that need deterministic latency and custom logic rather than general-purpose flexibility, AMD/Xilinx's Kria KV260 FPGA is the go-to. Power draw stays under 10 W, and the tradeoff for that determinism is a steeper development curve using Vitis AI and Vivado.
Google's Coral USB Accelerator sits in its own category. At roughly 4 TOPS and 2 to 4 W, it's not built for production, it's a bench prototyping tool, useful for early testing on the TFLite stack before committing to a production-grade accelerator.
GPU strengths lie in flexibility and ecosystem maturity. GPU computing on Jetson Orin runs any model that TensorRT can optimize, handles multiple concurrent AI applications on the same hardware, and benefits from NVIDIA's extensive tooling (DeepStream for video analytics, Isaac ROS for robotics). The trade-off is power: 15–60 W for AGX Orin requires active thermal management in most enclosure designs.
NPU (neural processing unit) strengths lie in power efficiency. The Hailo-8 delivers 26 TOPS at 2.5 W typical, enabling passive cooling even in compact sealed enclosures. Its dataflow architecture eliminates external DRAM, reducing both cost and supply chain complexity. The trade-off is flexibility: Hailo chips run quantized models through the Hailo Dataflow Compiler; they do not support arbitrary GPU workloads.
FPGA strengths lie in deterministic latency and reconfigurability. An FPGA pipeline can be designed to deliver a guaranteed inference result within a fixed clock cycle count, with no jitter from OS scheduling or memory contention. AMD/Xilinx Kria modules processed four camera streams at under 20 ms latency at under 10 W in industrial inspection deployments (Promwad, 2025). The trade-off is development time: FPGA expertise is scarce, and the toolchain (Vitis AI, Vivado) requires specialist knowledge.
Software support significantly impacts edge AI deployment performance regardless of the accelerator chosen. A model that runs at 60 FPS with TensorRT on Jetson may run at 8 FPS without it. The ecosystem ties matter: NVIDIA Jetson supports CUDA and TensorRT natively; Google Edge TPU requires TFLite models compiled for the Edge TPU; Intel Movidius uses the OpenVINO toolkit; AMD/Xilinx Kria requires FPGA expertise and the Vitis AI framework (vendor ecosystems, 2024).
A board is not a deployable system. This is the section that chip vendor datasheets skip and that distinguishes a system integrator with in-vehicle experience from one without it.
Power budget. A vehicle's auxiliary power supply is not a server rack. A tram or bus typically provides 24V or 110V DC auxiliary power, with a current limit defined by the vehicle's electrical system design. An AI compute unit that draws 60 W peak may trip a breaker, flatten an auxiliary battery during idle periods, or violate the vehicle OEM's electrical integration requirements. Low power consumption is not a preference in an in-vehicle system; it is a specification. The platform must be sized to stay within the power budget at sustained inference load, not just at idle.
Thermal management. Edge AI hardware is designed for compact form factors and passive thermal management because active cooling (fans) is unreliable in automotive and rail environments: fans accumulate dust and debris, fail from vibration, and add acoustic noise in passenger spaces. A sealed fanless enclosure is the target for in-vehicle deployments. That means the chip's TDP must be dissipated through the enclosure walls into the surrounding air, with no forced airflow. An AGX Orin at 60 W requires a substantial heatsink and enclosure thermal design to avoid throttling in a hot cab; a Hailo-8 at 2.5 W can passively dissipate its heat in almost any enclosure geometry. The thermal budget assessment must be done for the worst-case ambient temperature the vehicle will experience, not the lab's 22°C.
Vibration and shock. A tram car subjects onboard hardware to continuous vibration from the track, plus sharp shock loads at rail joints and during braking. Connectors loosen, solder joints crack, and thermal interface material compresses under sustained vibration. Hardware must be rated to EN 50155 (railway electronic equipment) or an equivalent automotive standard (ISO 16750). Consumer-grade development boards are not rated for these conditions. Industrial-grade modules with conformal-coated PCBs, ruggedized connectors, and vibration-tested mounting are the minimum specification for a production deployment.
MTBF and reliability. An in-vehicle AI compute unit must operate continuously for the service life of the vehicle, which may be ten to twenty years for rail. MTBF (mean time between failures) is the reliability metric that procurement teams and vehicle OEMs require. A claimed MTBF of 100,000 hours at 40°C ambient is not the same as 100,000 hours at 70°C; the Arrhenius relationship means reliability degrades rapidly with temperature. Hardware that does not provide a temperature-dependent MTBF figure is not specifying reliability at all.
Vehicle integration. The AI compute unit does not operate in isolation. It receives power from the vehicle's auxiliary supply (voltage tolerance, connector type, and circuit protection must match the vehicle specification), receives sensor data from cameras and LiDAR (camera interface: MIPI CSI-2, GigE, USB; LiDAR: Ethernet or serial), and sends outputs to the vehicle controller over an integration interface. That interface is typically UDP over a vehicle Ethernet network, a CAN bus message to the vehicle control unit, or a dry-contact relay output wired into a safety circuit. The choice is determined by what the vehicle OEM permits and what the safety case requires. Mounting must account for shock isolation and cable strain relief.
Edge AI hardware requires a fundamentally different architecture from cloud AI hardware: the design brief is not "maximize throughput" but "maximize reliable inference within the constraints of a moving vehicle." See our deep-dive on AI collision avoidance for trams and light rail for a real-world example of these constraints applied to a safety-critical system.
The most common computer vision tasks running on edge AI hardware in industrial and transport deployments are:
Object detection is the foundation application: detect and localize objects in a camera frame, classify them by type (person, vehicle, obstacle), and return bounding boxes with confidence scores. Security cameras leverage edge AI for real-time object detection without transmitting video to a cloud server. Industrial smart cameras use on-device detection for defect classification and quality control at line speed.
Multi-camera video analytics runs multiple detection pipelines simultaneously on a single edge AI compute unit. An AGX Orin processing four 1080p camera feeds at 30 FPS is a common configuration for onboard transit systems covering front, rear, and cabin views.
Image processing and preprocessing runs on the edge before the model: lens dewarping for fisheye cameras, resizing to the model's input resolution, normalization, and color-space conversion. These operations can consume significant CPU time if not offloaded to the ISP or GPU; on Jetson platforms, NVIDIA's VPI (Vision Programming Interface) handles this efficiently.
Machine vision for industrial IoT includes surface inspection, dimensional measurement, and assembly verification. Edge AI enables real-time analytics at production line speed (hundreds of parts per minute) with inference running on embedded hardware in the factory environment.
On-device preprocessing recommendations: run dewarping and resize on the hardware ISP or GPU, not the CPU; target model input resolutions of 640x640 or lower for detection tasks at 30 FPS on constrained platforms; validate the end-to-end pipeline (capture to output) at the target frame rate on the target hardware before committing to the architecture.
Benchmark claims on datasheet are measured under controlled conditions that rarely match the deployment environment. Run benchmarks on the target hardware, under the target configuration, before selecting a platform.
Run YOLOv8n or YOLOv8s at 640x640 input resolution, single-batch inference, INT8 precision, at 30 FPS sustained for 30 minutes. Measure: mean inference latency, 99th-percentile latency (to identify thermal throttling events), throughput in FPS, CPU and GPU utilization, and power draw at the wall.
Run the same model across 4 simultaneous camera streams at 30 FPS each. Measure: total throughput (FPS across all streams), per-stream latency, memory bandwidth utilization, and temperature after 30 minutes of sustained operation. A platform that handles 4x30 FPS in a cool lab may drop to 4x22 FPS after thermal throttling begins at 65°C ambient. This is the AI performance figure that determines whether the system is deployable in the target environment, not the peak TOPS figure from the marketing sheet.
The core comparison for in-vehicle applications reduces to one question: can the application tolerate network-dependent, variable-latency inference? For real-time safety decisions, the answer is no.
Edge AI delivers reliable performance without internet connectivity. A vehicle in a tunnel has no network coverage; a vehicle in a cellular dead zone has intermittent connectivity; a vehicle during a network outage has none. An AI system with a cloud dependency fails in all three scenarios. An edge AI system running local inference is unaffected by any of them.
Edge AI lowers bandwidth usage by processing data locally and sending only essential insights to the cloud. A camera generating 4 Mbps of continuous video that is processed on-device sends a few kilobytes of structured event data per detection. Across a fleet of 100 vehicles, the bandwidth saving is the difference between a feasible back-end architecture and an unworkable one.
Cloud AI remains the right choice for model training, fleet monitoring, and batch analytics where latency above seconds is acceptable. The production architecture for most managed fleet deployments is hybrid: infer at the edge with low power consumption and local processing, retrain in the cloud with seamless connectivity to the model update pipeline. See edge AI vs cloud AI for the full comparison.
Autonomous vehicles represent the most demanding compute requirement for in-vehicle edge AI: perception (cameras, LiDAR, radar, ultrasonic), prediction (trajectory estimation for all detected objects), and planning (path selection) all running simultaneously at high frame rates, with functional safety requirements that mandate redundancy.
Sensor fusion combines inputs from multiple sensor types into a unified environmental model. A camera provides high-resolution color data for classification; LiDAR provides precise depth; radar provides velocity and all-weather range. Fusing these inputs requires synchronization (all sensors timestamped to a common clock), calibration (spatial transforms between sensor coordinate frames), and a fusion model that weights each input based on confidence. See sensor fusion at the edge for the architecture.
Redundancy levels for safety-critical real time decision making: a safety system should not have a single point of failure in its perception pipeline. Automotive-grade platforms (ISO 26262 ASIL-D) require redundant processing paths. Rail has its own regime under the CENELEC EN 5012x framework with SIL-based requirements. In both cases, industrial applications that use edge AI for safety functions must document the failure modes of the hardware, including the compute unit, and demonstrate that the system fails safely.
A SoM integrates CPU, GPU/NPU, memory, and storage on a small standardized module that plugs into a carrier board. The carrier board provides I/O, power conditioning, and integration interfaces specific to the application. SoM advantages: the compute and software are developed once; the carrier board is customized per deployment without re-qualifying the module. NVIDIA Jetson, Hailo, and Rockchip all offer SoM products designed for this pattern.
Discrete AI accelerators in M.2 (B+M key, A+E key) or PCIe form factors add AI capability to an existing edge compute platform without replacing it. This suits retrofit deployments where the host compute is already installed and validated. The Hailo-8 M.2 module adds 26 TOPS at 2.5 W to any platform with an M.2 slot (Hailo, 2024).
USB-connected AI accelerators (Google Coral USB, Intel Neural Compute Stick) are suitable for bench prototyping and model development. They are not appropriate for in-vehicle deployment: USB connectors are not rated for vehicle vibration, USB host controllers are not designed for sustained throughput under vibration, and the USB power delivery is insufficient for production inference workloads.
Select edge devices for I/O and ruggedization, not just compute. The questions that determine deployability: does the compute unit have the camera interfaces the vision system requires (MIPI CSI-2 for embedded cameras, GigE for industrial cameras, USB 3.x for certain sensors)? Does it provide the appropriate hardware output interface for the vehicle controller? Does external memory use ruggedized connectors? Is the operating temperature range sufficient for the deployment environment?
Edge hardware typically consists of compact devices suitable for harsh environments when industrial-grade versions are selected. Verify that the rated temperature range covers the deployment worst case, not just the nominal ambient.
For early prototyping and model evaluation, evaluation kits reduce time to first inference significantly:
Jetson Orin Nano Developer Kit ($249) is the recommended starting point for teams new to NVIDIA's ecosystem. It runs the full JetPack SDK and TensorRT pipeline and supports Super Mode (as of JetPack 6.2, January 2025) for up to 67 TOPS. It emulates Orin NX and AGX Orin modules for upward-scaling evaluation (NVIDIA, 2025).
Jetson AGX Orin Developer Kit targets high-throughput multi-camera evaluation (up to 275 TOPS). Useful for benchmarking the upper bound of the Jetson family before selecting the production module size.
AMD/Xilinx Kria KV260 Vision AI Starter Kit is the evaluation path for FPGA-based vision pipelines. Supports Vitis AI for model deployment and Vivado for custom hardware logic. The evaluation path for teams considering FPGA for deterministic low-latency pipelines.
Hailo-8 M.2 module on compatible carrier board is the evaluation path for ultra-low-power NPU inference. Run on any Linux host with M.2 B+M key to evaluate the Hailo SDK and model zoo.
For low-end reference: a Raspberry Pi 5 paired with a Hailo-8 M.2 HAT+ provides a low-cost environment for model development. This is a development tool, not a production platform.
Baseline benchmarks to run on any evaluation kit: YOLOv8n at 640x640 INT8 for throughput; the same model at 1280x1280 for small-object accuracy; and a 4-stream concurrency test for multi-camera assessment. Compare results against the datasheet TOPS figure to understand utilization efficiency.
The software ecosystem around an AI accelerator determines how much engineering effort a deployment actually requires, often more than the hardware specs suggest.
Mature, well-documented, with extensive community support and model zoo. TensorRT optimizes and compiles models for Jetson's DLA and GPU. DeepStream provides the video analytics pipeline layer. Edge Impulse supports Jetson for no-code model training and deployment. The learning curve is real (DeepStream is complex), but resources are abundant.
Converts and quantizes models from TensorFlow, PyTorch, ONNX, and Keras to Hailo's native format. HailoRT is the runtime for inference. Model conversion is a distinct step that requires the Dataflow Compiler to be run offline before deployment; models cannot be loaded dynamically in the field without a prior compilation step.
Targets Intel CPUs, integrated GPUs, and Movidius VPUs. Handles model conversion from most major frameworks. Strong for deployments on Intel-based edge platforms with an existing x86 software stack.
Required for Xilinx FPGA-based AI deployments. Significant toolchain complexity; plan for a specialist FPGA engineer on the team.
For model deployment to a fleet, a CI/CD pipeline for model updates is essential: version-controlled models, automated benchmark regression tests before release, staged rollout to a subset of the fleet before full deployment, and rollback on detected performance regression. See MLOps for deployed edge devices. Software support significantly impacts edge AI deployment performance: a model that achieves 90% of peak TOPS with the right runtime may achieve 20% without it.
TOPS (tera-operations per second) measures peak throughput on integer matrix operations. It is a useful proxy for comparing accelerators within the same architecture family. It is a poor proxy for comparing across architectures, because utilization efficiency varies widely: a 26 TOPS NPU that achieves 90% utilization on a quantized detection model delivers more effective throughput than a 100 TOPS GPU achieving 20% utilization on the same model.
Performance-per-watt matters more than peak TOPS for power-limited in-vehicle systems. The Hailo-8 delivers approximately 10 TOPS per watt (26 TOPS at 2.5 W). The Jetson AGX Orin delivers approximately 4.6 TOPS per watt (275 TOPS at 60 W). For a vehicle with a 10 W power budget for the AI compute unit, the Hailo-8 is deployable; the AGX Orin is not, regardless of its higher absolute TOPS figure. Edge devices operate with strict power efficiency suitable for battery use and vehicle auxiliary power supply constraints. Edge AI supports low-power consumption for battery-operated devices precisely because the inference is done on specialized silicon, not on a general-purpose server.
Hardware comparison (indicative, vendor-sourced figures)
The Jetson AGX Orin 64GB sits at the top of NVIDIA's lineup, delivering 275 TOPS at 15–60 W of typical power, which works out to 4.6 TOPS per watt. It's built for multi-camera, multi-model deployments where throughput matters more than power budget.
Step down to the Jetson Orin NX 16GB and you get 100 TOPS at 10–25 W, translating to 4 to 10 TOPS per watt depending on workload. This is the board most teams land on for mid-range vision systems running four cameras or so.
The Jetson Orin Nano (Super) is NVIDIA's entry point: 67 TOPS at 7–25 W, or 2.7 to 9.6 TOPS per watt. It's the right choice for single-camera setups and teams getting their first edge AI deployment off the ground.
Hailo's M.2 discrete module, the Hailo-8, delivers 26 TOPS at a typical 2.5 W, which puts it at 10.4 TOPS per watt, among the most efficient numbers on this list. It's a natural fit for power-constrained, fanless retrofits where you can't add cooling or draw much current.
The Hailo-8L pushes efficiency further still: 13 TOPS at just 1 to 2.5 W, yielding 5.2 to 13 TOPS per watt. This is the chip for ultra-low-power IoT and wearable applications where every milliwatt counts.
The Kria KV260 FPGA from AMD/Xilinx runs a custom pipeline rather than a fixed TOPS figure, drawing under 10 W. Its TOPS-per-watt figure depends entirely on the workload you compile onto it, but the payoff is deterministic latency and the flexibility to build custom logic pipelines that a fixed-architecture chip can't match.
Sources: NVIDIA (2025), Hailo (2024), Promwad (2025). All figures are vendor-stated or third-party benchmarked; actual performance depends on model architecture, quantization, and runtime implementation.
Energy efficiency at the system level also depends on how long the device runs between reboots, battery life considerations for non-wired deployments, and thermal design that keeps the chip within its rated temperature envelope without external cooling.
Assess the thermal budget before selecting a platform. The sequence: define the maximum ambient temperature the enclosure will experience (worst case, not average); estimate the enclosure's thermal resistance from chip to ambient; multiply TDP by thermal resistance to get junction temperature; verify that junction temperature stays below the chip's rated maximum across all operating modes. A Hailo-8 at 2.5 W in a 0.5°C/W enclosure rises 1.25°C above ambient, which is manageable at 85°C ambient. An AGX Orin at 60 W in the same enclosure rises 30°C, and thermal throttling begins before the ambient limit is reached.
The AI compute unit is a physical endpoint accessible to anyone with access to the vehicle. Processed locally means the model weights, inference logs, and any buffered sensitive data are stored on the device. Secure boot ensures only signed firmware and model artifacts can run. Device attestation confirms to the back-end update server that it is communicating with a legitimate device before pushing a model update.
In-vehicle and rail deployments have service lives of ten to twenty years. NVIDIA has committed to Jetson Orin availability until 2032 (NVIDIA, 2025). Hailo's industrial-grade chips are rated to -40°C to 85°C with long-term availability commitments. Verify the manufacturer's product lifecycle commitment before selecting a platform for a program with a ten-year horizon. A chip discontinued in year three forces a mid-life hardware redesign.
Edge devices can operate independently in locations with poor connectivity: tunnels, underground stations, remote industrial sites. Design the system to function fully without network access. Cloud connectivity is for model updates and telemetry, not for inference.
Use this checklist when evaluating edge AI hardware for an in-vehicle or safety-critical deployment.
Selection criteria
TOPS, or inference throughput, needs to be sufficient for the target model running at the target frame rate. The only way to confirm this is to benchmark the actual production model on the actual device, not on a reference board running a demo model.
Power consumption has to stay within the vehicle's auxiliary power budget. That number should be measured under sustained load, not at idle, since idle draw tells you almost nothing about what the chip pulls once it's running inference continuously.
Thermal design needs to hold up with passive cooling at the worst-case ambient temperature the vehicle will see. Verify this with a thermal simulation first, then confirm it with a field test, since simulations miss real airflow conditions inside an enclosure.
Operating temperature range must cover the full worst case of the deployment environment, not just a typical range. Check the datasheet, then validate it in the field, because datasheet figures are often measured under lab conditions that don't match a tram cabin in a Polish winter.
Vibration and shock rating should meet EN 50155 for rail applications or ISO 16750 for automotive ones. This is verified through certification documentation, not a vendor's word.
MTBF target should be consistent with the vehicle's expected service life. Ask for a temperature-derated MTBF figure from the vendor, since a standard MTBF number calculated at room temperature won't reflect real-world failure rates.
Form factor has to fit the mechanical envelope available. Build a physical mock-up before finalizing board layout, since a compute unit that looks fine on a spec sheet can still fail to fit once cabling and mounting brackets are accounted for.
Camera interface needs to match the sensor type in use, whether that's MIPI CSI-2, GigE, or USB 3.x. Confirm interface count and type directly against the datasheet.
Vehicle integration interface must support UDP, CAN, or whatever protocol the vehicle controller requires. This is confirmed through the hardware interface documentation, not assumed from a general product page.
SDK and toolchain need to compile the model and run it at the required throughput. The only reliable way to check this is a pilot evaluation on an actual dev kit, since SDK claims on marketing pages rarely translate directly into real throughput numbers.
Long-term availability matters more than most teams budget for. The product lifecycle commitment needs to cover the full program life, and that should come as a vendor statement or distributor confirmation in writing, not an assumption based on the chip being currently in stock.
Secure boot needs to be supported and documented. This gets checked as part of a security architecture review, since "supported" can mean very different things depending on how it's implemented.
Appropriate hardware selection for a safety-critical deployment is not a datasheet exercise. The checklist above identifies what to specify; field validation under real conditions confirms that the specification is met.
InTechHouse case study: In-vehicle edge AI compute unit for rail safety
InTechHouse designed and integrated the onboard edge AI compute unit for a tram collision avoidance and CCTV behavior detection system developed in partnership with PESA, one of Europe's leading rail vehicle manufacturers. The accelerator selection was driven by the power budget available from the vehicle's 24V auxiliary supply, the requirement for passive cooling inside a sealed enclosure (the tram cab does not permit fan-cooled hardware), and the MTBF target consistent with the vehicle's planned service life.
The compute unit processed front-facing camera and LiDAR data for collision avoidance and multiple cabin camera feeds for behavior detection simultaneously, with an end-to-end latency under 30 ms from sensor input to brake command output. Integration with the vehicle controller used a hardware interface agreed with PESA's engineering team, with the brake command delivered over a protocol the vehicle's braking system could accept directly.
Thermal performance was validated across the full seasonal temperature range encountered in the deployment environment. The system has operated continuously across the vehicle's service schedule without performance degradation due to thermal throttling.
Define the target model and FPS requirement. Run the model on two or three candidate platforms using the benchmark protocol from the Computer Vision Benchmarking section. Measure throughput, latency, power draw, and thermal behavior at sustained load. Select the platform that meets all requirements at the lowest power consumption.
Mount the selected compute unit in a representative enclosure. Validate passive thermal performance at worst-case ambient temperature. Connect production camera and sensor hardware. Validate the vehicle controller integration interface with a hardware-in-the-loop test. Measure end-to-end latency from sensor input to output delivery.
Install the prototype in a target vehicle. Collect model deployment data from real operating conditions: lighting, vibration, temperature cycles, connectivity dropouts. Measure real time processing performance in the field. Identify model drift or hardware issues. Collect labeled data for model retraining.
Finalize hardware design for production (connector specification, conformal coating, vibration isolation, mounting). Validate OTA model update delivery to the production unit. Establish continuous monitoring telemetry from deployed units. Certify to applicable functional safety standard before operational deployment. Scale to fleet.
For in-vehicle edge AI vision, the platform selection sequence is: define the power budget first; select accelerators that operate within it at sustained inference load; validate thermal performance at worst-case ambient; verify mechanical ratings for the vehicle environment; confirm the integration interface with the vehicle OEM; then benchmark model performance on the shortlisted hardware. Peak TOPS is the last criterion to optimize, not the first.
For appropriate hardware selection for a specific in-vehicle application, including MTBF specification, power budget analysis, and integration with vehicle control systems, InTechHouse can help scope the requirements and recommend a deployment-ready edge AI compute unit. Talk to our team about your in-vehicle edge AI project.
What hardware do you need for edge AI?
The minimum components are an AI accelerator (NPU, GPU-equipped SoC, or FPGA), sufficient memory for the model and inference buffers, camera or sensor interfaces matching the input hardware, and a power supply compatible with the deployment environment. For in-vehicle deployments, the hardware must also meet vibration and temperature ratings for the vehicle class, operate within the vehicle's auxiliary power budget, and interface with the vehicle controller over a supported protocol (CAN, UDP, or equivalent).
What is an edge AI compute unit?
An edge AI compute unit is a self-contained embedded computer that runs AI inference locally, without a cloud connection. It typically comprises an AI accelerator (NPU, GPU, or FPGA), a host CPU for application logic, memory, storage, and the I/O interfaces required for the deployment environment. In an in-vehicle context, it processes sensor and camera data in real time and delivers inference outputs (alerts, classifications, brake commands) to the vehicle's systems without any network dependency.
GPU vs NPU vs FPGA for edge AI, which is best?
There is no universal answer. Use a GPU-based platform (NVIDIA Jetson Orin) when the workload is complex, multi-model, or benefits from CUDA's ecosystem; plan for 15–60 W power consumption and active or substantial passive cooling. Use an NPU (Hailo-8) when power efficiency is the primary constraint: 26 TOPS at 2.5 W enables passive cooling in compact sealed enclosures, making it the right choice for power-limited in-vehicle systems. Use an FPGA (Kria) when deterministic latency and custom signal processing logic are required alongside AI inference. Most industrial applications need 10–20 TOPS; chasing the highest TOPS adds cost and power for no benefit.
How do you run AI vision hardware inside a vehicle?
Select hardware rated for the vehicle's vibration, shock, and temperature profile (EN 50155 for rail, ISO 16750 for automotive). Design for passive cooling within a sealed enclosure. Size the power draw to fit within the vehicle's auxiliary power supply capacity. Integrate sensor inputs (cameras via MIPI CSI-2 or GigE, LiDAR via Ethernet) and vehicle controller outputs (CAN bus, UDP, or relay) as specified by the vehicle OEM. Mount with shock isolation, use ruggedized connectors with strain relief, and validate thermal performance at worst-case ambient before deployment.
What makes edge AI hardware reliable (MTBF)?
MTBF for edge AI hardware is determined by four factors: component quality (industrial-grade components rated for the temperature range); thermal design (keeping junction temperatures well below rated maximums extends component life significantly via the Arrhenius relationship); mechanical design (vibration isolation, ruggedized connectors, conformal coating to prevent moisture ingress); and software stability (secure boot, watchdog timers, and health monitoring to detect and recover from software faults without requiring physical intervention). For a ten-year vehicle service life, require a temperature-derated MTBF figure from the vendor at the expected ambient temperature, not just the datasheet figure at 25°C.
An academic lecturer at the Bydgoszcz University of Science and Technology. He has experience in advanced technologies, with a particular focus on UAV systems and related solutions.
In his academic work, he is actively involved in educating future specialists in the UAV domain, combining theoretical knowledge with practical experience gained from real-world projects.
This initial conversation is focused on understanding your product, technical challenges, and constraints.
No sales pitch - just a practical discussion with experienced engineers.
Share a few details about your product and context. We’ll review the information and suggest the most appropriate next step.
