Computer vision in transportation uses cameras and AI to detect, classify, and track what is happening on and around a vehicle or in a station. In rail and public transport, that capability is moving onboard: onto the tram, onto the train, onto the bus, where it operates in real time, independently of any control-room connection. This page maps the use cases across rail and public transit, explains what makes onboard deployment genuinely hard, and outlines how an operator can get started.
Key Takeaways
Rail and public transport operators are under pressure on two fronts: safety incidents that damage public trust and drive regulatory scrutiny, and operational inefficiencies (overcrowding, delays, maintenance failures) that erode service quality. Computer vision technology addresses both. It enhances safety and efficiency in transportation by giving operators continuous, automated awareness of what is happening across their fleet and infrastructure, without requiring staff to watch every camera feed.
The priority use cases for a transit operator starting out are collision avoidance and obstacle detection (highest safety impact, clearest ROI), threat and behavior detection in CCTV (regulatory pressure, passenger confidence), and passenger counting (immediate operational value, low deployment complexity). Infrastructure inspection and platform intrusion monitoring follow as natural expansions once the core hardware and data pipeline are in place.
The global rail transit safety monitoring system market was valued at USD 24.48 billion in 2024 and is projected to reach USD 54.82 billion by 2032, at a CAGR of 12.3% (Intel Market Research, 2024). The smart railways market is valued at USD 149 billion in 2025 and is projected to reach USD 1,124.5 billion by 2035, growing at a CAGR of 22.4% (Future Market Insights, 2025). Operators who move now are building the data and operational capability that will define service quality for the next decade.
Three forces are driving the shift: safety liability, automation pressure, and passenger experience expectations.
On safety, video analytics in transit networks helps ensure safety by detecting dangerous behaviors (aggression, trespassing, obstacles on the track) in time to act rather than in time to report. The critical word is "in time." A control-room operator reviewing a live feed reacts in seconds. An onboard vision system running inference at 30 frames per second reacts in milliseconds.
On automation, regulators in Europe and Asia are pressing operators toward higher grades of automation (GoA2 through GoA4). Computer vision is a prerequisite for that progression: you cannot automate a vehicle that cannot see.
On passenger experience, real time data processing onboard enables services that riders notice directly: accurate crowding information, faster emergency response, and visible safety technology that builds confidence in the network.
The shift from control-room to onboard processing is fundamentally a shift in where inference happens. For a deeper explanation of why that matters technically, see what is edge AI and the edge AI vs cloud AI comparison.
The core capabilities that transit operators need from a computer vision system are: object detection (people, vehicles, obstacles), multi-object tracking across frames, video analytics for behavior and anomaly analysis (aggression, trespassing, crowding), and alert generation with explainable outputs.
Delivering those capabilities onboard requires choices at every layer of the stack.
Processing: edge vs. cloud. For onboard safety applications, the answer is edge. A cloud-based inference call adds 50–500 ms of network latency; onboard inference runs in 10–30 ms. In a tunnel with no cellular coverage, the cloud option ceases to function entirely. For station-based analytics where latency and connectivity are less constrained, cloud or hybrid architectures are viable.
Camera and sensor selection. Fixed focal-length cameras suit forward-facing collision detection. Wide-angle and fisheye lenses cover cabin interiors but require dewarping before standard models can process the footage reliably. Thermal cameras extend detection capability in low light and adverse weather. Stereo camera pairs enable depth estimation for more accurate obstacle sizing and distance measurement. Sensor fusion, combining cameras with radar or LiDAR, improves reliability in rain, fog, and direct sunlight, which are the conditions that cause purely vision-based systems to degrade. See our guide on sensor fusion at the edge.
Model training and deployment. Models are trained on labeled datasets in the cloud, optimized for the target hardware (quantized to INT8, pruned, converted to a runtime format such as TensorRT), then deployed to edge devices onboard the vehicle. Adverse environmental conditions (rain, glare, night, motion blur at speed) affect accuracy significantly if the training data does not represent them. A model trained on daytime footage from a European summer will degrade in winter. The labeling and retraining pipeline is not a one-time project; it is an ongoing operational requirement.
For hardware specifics relevant to in-vehicle deployments, see edge AI hardware for in-vehicle systems.
Use-case map
Collision avoidance and obstacle detection is deployed onboard, using front-facing cameras, with the primary benefit of preventing collisions and protecting life.
Threat and behavior detection (CCTV) is deployed both onboard and at stations, primarily serving security and incident response needs.
Passenger counting and crowd density monitoring is likewise deployed both onboard and at stations, supporting capacity management and scheduling.
Platform intrusion and level-crossing monitoring is deployed trackside and at stations, with the primary benefit of preventing track incursions.
Asset and infrastructure inspection is deployed trackside and at depots, primarily enabling predictive maintenance.
Each use case below is a two-to-three sentence summary. Follow the linked cluster article for full technical depth.
Onboard computer vision for trams and light rail processes feeds from forward-facing cameras to detect pedestrians, vehicles, and obstacles in the vehicle's path, triggering driver alerts or automatic emergency braking when a hazard is confirmed within the stopping distance. The system must operate in real time, independently of any network connection, because a cellular dropout in a tunnel or a 200 ms network spike cannot be permitted to disable a safety-critical function. A system deployed by Cognitive Pilot across more than 350 trams detected obstacles up to 30 meters ahead; when a hazard appeared within 5 meters without driver response, the tram initiated an automatic stop. The technology helped prevent over 2,450 traffic incidents as of August 2025.
For detailed implementation guidance including hardware selection and certification considerations, see AI collision avoidance for trams and light rail.
AI-powered cameras in transit hubs and onboard vehicles monitor for safety hazards including aggression between passengers, weapon presence, and abandoned objects that may indicate a security threat. AI can detect unusual activities and notify staff in real time, reducing the response window from minutes (a human reviewing footage after the fact) to seconds (an automated alert to the nearest member of staff or control room). On a major metropolitan rail network, AI analysis across a large CCTV estate has helped staff detect tunnel trespassers in real time and intervene before anyone is harmed. That is a capability that would require one operator per camera feed to replicate manually.
For the full technical breakdown of onboard and station threat detection, see edge AI CCTV threat detection.
Computer vision systems count passengers boarding and alighting at each stop, building a real-time occupancy model for every vehicle in the fleet. AI systems can predict crowding thresholds at transit stations and adjust train dispatch patterns accordingly, reducing the frequency of overcrowded services, improving use of available space per vehicle, and cutting the dwell time penalty that comes from passengers unable to board. Crowd-density heatmaps generated from station cameras let operators identify chronic pinch points (stairwells, platform ends, fare gates) and redesign them based on measured data rather than staff observation.
See passenger counting with edge AI for implementation specifics.
Platform edge detection systems use computer vision to identify when a person or object has crossed into the gap between the platform and the track, or onto the track itself, triggering alerts to train operators and platform staff before a train arrives. Level-crossing monitoring extends the same logic to road-rail intersections: cameras classify whether the crossing is clear before a train is given proceed authority, providing a redundant check on gate and barrier systems. Obstacle and intrusion detection improves safety across both settings by catching edge cases that fixed-point sensors miss, such as a person who has fallen onto the track after the train has entered the station zone.
Platform screen doors, where installed, reduce intrusion risk mechanically, but most light rail and tram infrastructure operates with open platforms where only a detection system provides the warning. Computer vision fills that gap cost-effectively: a camera covering a 50-meter platform section can replace multiple point sensors and handles edge cases (partially fallen objects, small children near the edge) that beam-based systems miss. Integration with the platform announcement system allows an automated warning message to play within seconds of a detection, before any staff member needs to act.
These use cases share infrastructure with collision avoidance and benefit from the same onboard and trackside hardware.
Computer vision cameras mounted on inspection vehicles or trains in service capture continuous footage of track, overhead line equipment, tunnel walls, and platform infrastructure. Infrastructure maintenance can be enhanced by computer vision detecting structural anomalies (cracks, corrosion, ballast displacement) that are invisible to periodic manual inspections and only become apparent in high-resolution images analyzed frame by frame. Cameras can detect surface wear on rail heads and wheel flanges and generate repair priorities automatically, shifting maintenance from time-based schedules to condition-based interventions and reducing the risk of in-service failures.
The economic case is strong. A track geometry defect caught by a vision inspection pass costs a fraction of the same defect caught after an in-service incident. Vision-based inspection also enables inspection frequency that manual teams cannot match: a camera mounted on a revenue-service train inspects the route on every trip, while a manual inspection team covers the same track once a week at best. Early adopter networks have reported reductions in track incidents of 30–40% after deploying real-time condition monitoring (Intel Market Research, 2024).
This use case connects naturally to predictive maintenance platforms that aggregate inspection data with sensor readings from the rolling stock itself.
A computer vision model that achieves strong accuracy on a benchmark dataset is not a deployable rolling stock AI system. The gap between the two is where most projects stall.
Vibration and shock. Rail vehicles subject hardware to continuous vibration across a wide frequency spectrum, plus sharp shock loads at rail joints and switches. Camera mounts, cable connectors, and compute units must be rated for these conditions. Consumer-grade hardware fails. Industrial-grade hardware rated to EN 50155 (railway electronic equipment) or equivalent is the baseline.
Temperature range. A tram left overnight in a Polish winter reaches -25°C. The same vehicle on a summer afternoon in a glass-sided depot exceeds +70°C in the cabin. Compute units and cameras must operate across this range without thermal throttling that degrades inference performance.
Power budget. Onboard systems draw from the vehicle's auxiliary power supply, which has strict limits. A compute unit that requires 150W may not be installable on board a legacy tram with a 12V auxiliary circuit designed for lighting. Power draw must be scoped before hardware selection, not after.
Retrofit vs. new rolling stock. New rolling stock can be designed with computer vision integration in mind: conduits for cabling, mounting points, power allocation. Retrofit onto existing fleets is significantly more complex than new-build integration: it requires working around existing wiring, structural constraints, and legacy systems, which raises installation time and cost substantially. The wiring routing, connector types, and EMC shielding required in a vehicle cab are not the same as those in a server room.
Camera placement and lens choice. Forward-facing collision detection needs a specific field of view and focal length to detect a person at 30 meters while moving at 50 km/h. Fisheye cameras covering the cabin interior require dewarping before standard object detection models work reliably on the feed. Getting placement wrong means rerunning cable and remounting hardware after installation.
Certification. Safety-critical onboard systems must demonstrate functional safety compliance (typically IEC 62280 for railway communication security, EN 50126/50128/50129 for RAMS). That process requires documented failure mode analysis, validation test evidence, and often independent assessment. It takes months, not weeks, and cannot be bolted on after development.
For a detailed treatment of hardware constraints and in-vehicle deployment specifics, see edge AI hardware for in-vehicle systems, fisheye dewarping for CV, and explainable AI for safety alerts.
The data pipeline for an onboard vision system has five stages: capture, preprocessing, inference, result transmission, and storage.
Capture. Cameras generate continuous video streams. At 1080p and 30 FPS, a single camera produces roughly 4–8 GB per hour of uncompressed data. The onboard system must buffer, compress, and manage that volume within available storage constraints.
Preprocessing. Raw frames are resized, normalized, and (for fisheye lenses) dewarped before being passed to the model. This processing happens on the edge device in the inference pipeline.
Inference. The model runs on the edge accelerator. Results are structured outputs: object class, bounding box coordinates, confidence score, timestamp, GPS position. That result record is kilobytes, not megabytes.
Transmission. Inference results, aggregated metrics, and flagged event clips are transmitted to the back-end system when connectivity is available, typically at depot, over Wi-Fi, or during surface running with cellular coverage. Processing onboard reduces what must be transmitted and stored: instead of uploading hours of raw footage, the system sends structured event logs and short clips around detected incidents.
Storage and retention. Raw footage is stored locally on the edge device with a rolling retention window (typically 24–72 hours, depending on storage capacity). Flagged clips are retained longer and synced to central storage. Retention policies must comply with applicable data protection rules governing CCTV footage in the operating jurisdiction.
Integration APIs connect the vision system to the operator's back-end: fleet management systems, incident management platforms, driver alert displays, and passenger information systems. Standard protocols (REST, MQTT for IoT-style telemetry) are the norm; custom integrations with legacy systems are common and should be scoped early.
Onboard cameras in public transport record passengers. That is a significant data responsibility, and operators must manage it explicitly.
Anonymization. For use cases that do not require individual identification (passenger counting, crowd density, anomaly detection based on behavior patterns), video feeds can be processed to blur or mask faces before any data leaves the device. Some platforms process entirely on-device and never store raw video at all, transmitting only aggregate counts or behavioral flags. This approach significantly reduces GDPR exposure.
Human-in-the-loop review. Automated threat detection alerts should route to a human reviewer before any action is taken against a specific individual. The system flags; a person decides. That architecture helps protect both passengers and the operator from the consequences of false positives, and demonstrates to regulators that footage is handled responsibly.
Transparency and communication. Passengers have a right to know that cameras are in use and what data is collected. Clear signage, published data protection notices, and accessible opt-out or complaint mechanisms are both legal requirements (under GDPR and similar frameworks) and trust-building measures. The on-device privacy advantage is worth communicating publicly: "footage never leaves the vehicle" is a meaningful statement that addresses a common passenger concern.
Specific regulatory requirements vary by jurisdiction and change over time. Verify current applicable rules with legal counsel before finalizing a deployment architecture.
Hardware costs. AI dash camera systems for fleet vehicles typically cost $25–$60 per vehicle per month for cloud-connected telematics systems (Samsara, Lytx, Motive tier; source: BusCMMS, 2026). Hardware-only entry-level dash cameras with ADAS functionality are available in the $200–$500 per vehicle range. Full ADAS installation including hardware, wiring, calibration, and software setup on commercial vehicles has been estimated at $1,200–$2,500 per vehicle (source: US DOT ITS Deployment Evaluation, 2024). For transit-grade, certified onboard vision systems covering collision avoidance and CCTV, per-vehicle costs are higher and depend on camera count, compute specification, and integration complexity.
Recurring operational costs. Software subscriptions, model update delivery, cloud storage for synced clips and telemetry, and maintenance contracts for onboard hardware. These costs are ongoing and compound across large fleets. They should be modeled over a three-to-five year period, not just year one.
ROI drivers. Computer vision can reduce operating costs and insurance claims through better monitoring. A single exonerated not-at-fault insurance claim can save $5,000–$25,000 in legal and claims costs; for a fleet of 50 vehicles, even one avoided claim per month covers a significant portion of the system's annual cost. Samsara's 2025 Fleet Safety Report, based on 2,600 fleets over 30 months, found that AI cameras with driver coaching achieved a 73% crash rate reduction. Coach USA, a major bus operator, reported 92% fewer preventable accidents after implementation.
Cybersecurity risks. Onboard connected hardware is an attack surface. Firmware must be updatable securely (signed OTA delivery). The communication channel between onboard systems and back-end must be encrypted and authenticated. Physical access to hardware in a depot or at a terminus must be controlled. The US TSA's November 2024 notice of proposed rulemaking includes sweeping cybersecurity requirements for rail operators and their vendor ecosystems. Operators procuring CV systems should require explicit security documentation from vendors.
Days 1–30: Pilot scoping
Select one use case. Collision avoidance on a small tram fleet (3–5 vehicles) or passenger counting on a high-volume bus route are both strong starting points. Define the success metrics before procurement. Adoption decisions made without clear measurable targets produce pilots that nobody can evaluate: what does 'working' look like in numbers? Shortlist two or three system vendors and request a structured proof-of-concept proposal, not a demo on their own hardware.
Days 31–60: Trial and data collection
Install hardware on the pilot vehicles. Run the system in monitoring-only mode initially: alerts are generated but no automatic actions are taken. Use the trial period to collect labeled data from the real operating environment (weather, lighting, passenger behavior, route geometry). That data is the foundation for model fine-tuning. Measure against the pre-defined metrics.
Days 61–90: Evaluate and plan scale
Review trial data against success metrics. Identify the failure modes that matter (missed detections, false alerts, hardware issues) and the model or hardware changes needed to address them. Prepare a business case for scale deployment: per-vehicle cost, expected savings, integration requirements, training needs for operations staff. Commit to a phased rollout plan tied to fleet maintenance cycles to minimize service disruption.
Transit operators and agencies worldwide are moving from trials to operational deployments. Cognitive Pilot deployed collision avoidance across more than 350 trams by August 2025, with the system credited with preventing over 2,450 traffic incidents. The Moscow Central Circle launched the world's first GoA3 operation on an open railway system in January 2024, with a computer vision system capable of recognizing objects at up to 1 km ahead. The Massachusetts Bay Transportation Authority (MBTA) uses AI to optimize bus routes and schedules, improving service reliability and reducing operational costs. These examples show a consistent pattern: agencies that commit to a specific use case and measure it rigorously build the internal capability to scale.
McKinsey research projects that AI-optimized traffic signal management can reduce urban travel times by up to 20%. Applied to transit scheduling and dispatch, the same principle (AI acting on real-time density and flow data from onboard vision) enables on-time performance improvements of comparable magnitude.
Benefits to look for in a partner track record: demonstrated deployment on revenue-service vehicles (not lab conditions), evidence of certification compliance for the relevant safety standards, a structured approach to model maintenance and retraining, and a data protection architecture that keeps raw footage on-device.
KPIs
On safety KPIs, the target obstacle detection rate is above 98% on the test set, the false positive alert rate should stay under 2 per vehicle per day, and platform intrusion response time should be under 3 seconds from detection to alert.
On operational KPIs, passenger count accuracy should exceed 95% compared to manual counts, dwell time reduction should be measurable within 6 months, and infrastructure defect detection rate should exceed 90% compared to manual inspection.
On privacy and compliance KPIs, the target is zero raw footage transmitted off-device for anonymized use cases, and 100% compliance with the defined data retention policy window.
InTechHouse case study: Onboard vision for tram safety
InTechHouse developed and deployed an onboard computer vision system for a tram operator in partnership with PESA, one of Europe's leading rail vehicle manufacturers. The brief covered two functions: forward-facing collision avoidance and onboard CCTV behavior detection for passenger safety.
Both ran onboard in real time on embedded hardware inside the tram. The latency requirement for collision avoidance was non-negotiable: the system had to detect and classify an obstacle within the tram's physical stopping window, which at urban operating speeds gives a budget measured in tens of milliseconds. A cloud round-trip was not an architectural option. The CCTV behavior detection function carried an equally hard constraint: passenger footage could not leave the vehicle under the applicable data protection rules, so all inference ran on-device with only structured alert metadata transmitted to the control room.
The final system operates in tunnels, in areas with no cellular coverage, and across the full seasonal temperature range without performance degradation. The cloud is used for model versioning, aggregated telemetry, and retraining. It is never used for inference.
Safety KPIs measure whether the system is catching what it is supposed to catch. The primary metrics are detection rate (what percentage of real events are detected), false positive rate (how many alerts per shift are non-events), and response time from detection to alert delivery. For intrusion detection, a sub-3-second window from event to alert is a reasonable operational target. Reductions in recorded platform intrusion incidents and near-miss events are the lagging indicators that matter for regulators and insurers.
Operational KPIs measure service quality impact. Passenger count accuracy above 95% is the threshold at which count data is reliable enough to drive dispatch decisions. Dwell time at stops can be reduced when loading data informs driver coaching and timetable revision. AI-optimized dispatch patterns informed by real-time density data can improve on-time arrivals by around 10% and reduce passenger wait times by approximately 15%, based on published figures from AI public transport implementations (McKinsey/XenonStack, 2025).
Privacy and compliance KPIs measure whether the data governance architecture is working. The key metric for on-device processing deployments is the volume of raw identifiable footage transmitted off-device. For anonymized use cases, the target is zero. Retention policy compliance (footage deleted within the defined window, access logs auditable) must be verifiable, not assumed.
Start with the use case that has the highest safety impact and the clearest success metric. For most rail and tram operators, that is collision avoidance; for bus operators, it is often CCTV-based behavior detection or passenger counting. A single well-executed pilot on 3–5 vehicles produces more useful data than a broad deployment that lacks the operational focus to iterate.
Budget for the full system, not just hardware. Model maintenance, data labeling, certification documentation, integration engineering, and operations staff training are all cost line items that projects routinely underestimate.
Assign clear ownership. The technical team integrating the hardware and the operational team using the alerts need a shared definition of success, a shared escalation path for false positives, and a shared commitment to the retraining cycle and ongoing vendor support that keeps the system accurate over time.
To discuss scoping an onboard vision system for your fleet or drafting an implementation roadmap, talk to the InTechHouse team. For safety-critical industrial deployments, see our industrial safety practice.
Computer vision on trains and trams can detect obstacles on the track and trigger automatic braking, monitor the cabin for aggressive behavior or security threats, count passengers boarding and alighting, inspect track and infrastructure for defects, and detect platform intrusions. All of these functions can run onboard in real time without a cloud connection.
Yes. Onboard edge AI systems run inference entirely on embedded hardware inside the vehicle. The cloud is used for model training, updates, and back-end analytics, but the real-time detection that drives safety alerts and automatic actions happens on-device. This is a technical requirement for safety-critical applications: cloud latency and connectivity loss are not acceptable failure modes for a collision avoidance system.
The main use cases are collision avoidance and obstacle detection, threat and behavior detection in CCTV footage, passenger counting and crowd density management, platform intrusion and level-crossing monitoring, and asset and infrastructure inspection for predictive maintenance. Each use case has different latency, accuracy, and hardware requirements that determine whether it runs onboard or in a station/trackside configuration.
Collision avoidance on a tram requires forward-facing cameras with an appropriate field of view and focal length, an onboard AI compute unit (an embedded NPU or GPU rated for the vehicle's vibration and temperature profile), a trained object detection model optimized for the target hardware, and integration with the vehicle's braking system or driver alert system. The system must be calibrated for the specific tram geometry and validated in the operating environment (including low light, rain, and winter conditions) before going into service. Certification to applicable functional safety standards is required for automatic braking functions.
Typical onboard CV hardware includes industrial-grade cameras (fixed focal, wide-angle, or fisheye depending on the use case), an AI accelerator module (NVIDIA Jetson, Hailo-8, or equivalent NPU rated for the operating environment), local storage for video buffering, and a communication interface for transmitting results when connectivity is available. Hardware must be rated to EN 50155 for railway use, designed for the vehicle's power supply voltage and current limits, and mounted to withstand continuous vibration and the operating temperature range. See edge AI hardware for in-vehicle systems for selection guidance.
An expert in Artificial Intelligence, professor and researcher, who has authored numerous scientific publications and led international projects focused on AI, machine learning, and data-driven systems.
His work connects academic research with industrial applications, applying advanced AI models to practical challenges across sectors such as defense, telecommunications, smart industry, and cybersecurity. He has extensive experience in designing and implementing intelligent systems in complex, high-demand environments.
In addition to his technical work, Prof. Andrysiak shares insights on AI trends and applications as a speaker, mentor, and author, contributing to discussions on the role of AI in modern technology and digital transformation.
This initial conversation is focused on understanding your product, technical challenges, and constraints.
No sales pitch - just a practical discussion with experienced engineers.
Share a few details about your product and context. We’ll review the information and suggest the most appropriate next step.
