RTOS in Cybersecurity: Securing IoT and Embedded Systems Infrastructure

Table of Contents

Ready to :innovate: together?

RTOS in Cybersecurity: Enhancing IoT And Embedded System Security

An RTOS (Real-Time Operating System) is an operating system specifically designed to handle real-time applications where precise response time is crucial. According to a report by MarketsandMarkets, the RTOS market was valued at USD 5.4 billion in 2022, and it is projected to grow to USD 7.5 billion by 2027, representing a growth rate of 6.7% CAGR. Unlike more widely known operating systems such as Linux or Windows, which may process tasks in a non-deterministic manner, an RTOS must ensure that every task is completed within a strictly defined time frame. Examples of RTOS applications include industrial control systems, medical devices, and a wide range of IoT devices that are popularly used nowadays. H. Kopetz noted in his book: “Real-time operating systems require rigorous timing and scheduling guarantees, but security is often sacrificed to achieve these”. That is why the challenges related to their security are becoming increasingly complex.

In this article, we will examine the most common attacks on RTOS, and demonstrate their mechanisms, impacts, and preventive measures.

How Denial of Service (DoS) Attacks Impact OS Scheduling in RTOS

In DoS attacks, the goal is to disrupt the system’s operation by overloading resources such as the processor or memory. In the case of an RTOS, which operates under strict time constraints, this overload can lead to delays in task execution or even a complete halt of critical processes. According to research conducted by NCC Group, 98% of IoT devices using RTOS are vulnerable to various types of attacks, including 58% specifically to denial-of-service attacks.

Explanation: 

An attacker may flood the RTOS task queue, causing the system to stop functioning in real-time and become unable to perform its core functions.

Example:
One of the most well-known attacks of this type is the Mirai botnet attack in 2016. It was a botnet primarily composed of IoT devices, such as IP cameras, DVRs, and other embedded devices. Mirai exploited IoT devices with poorly secured interfaces, frequently using default login credentials, such as factory-set passwords. The Mirai attack overwhelmed victim servers with massive network traffic, overloading their resources and causing them to become unresponsive. One of the most notable instances of the Mirai attack was the assault on Dyn, a DNS service provider. This attack disrupted the operation of many popular online services, including Twitter, Netflix, and Amazon. As a result many websites and online services were interrupted for several hours.
Solutions:
  • Sacurely updating firmware provided by the manufacturer, for example, with Secure Boot and digital signatures,
  • If an IoT device does not require remote access, disabling it or securing it properly with a firewall, VPN, or other access control methods,
  • Instead of using simple, unencrypted protocols, implementing secure communication protocols such as TLS/DTLS to protect data transmission,
  • Implementing traffic filtering mechanisms, for example, through firewalls or more advanced methods like Access Control Lists (ACL) and DDoS filters that identify and block invalid or malicious requests,
  • Implementing rate limiting, which restricts the number of requests that can be sent to a server or application in a short time period,
  • Filtering volumetric attacks, using tools that block large-scale traffic spikes aimed at overwhelming the system,
  • Using traffic monitoring tools, such as Wireshark or NetFlow, to analyze device behavior and detect unusual activity patterns, which could indicate an attack attempt,
  • Utilizing SIEM (Security Information and Event Management) systems, which combine data from various sources, such as firewalls, network devices, and embedded systems,
  • Implementing network segmentation, which involves isolating IoT devices and embedded systems from critical parts of the network. This can be achieved by creating VLANs (Virtual Local Area Networks) or DMZ zones, ensuring that if one device is compromised, the attack does not spread to the rest of the infrastructure,
  • Introducing secure Over-The-Air (OTA) updates to ensure the safe and reliable updating of device software.

Cybersecurity RTOS: Mitigating Man-in-the-Middle and Spoofing Threats

It should be noted that IoT and embedded systems using RTOS often communicate via lightweight protocols (e.g., MQTT, CoAP), which can be vulnerable to Man-in-the-Middle (MitM) and spoofing attacks. These attacks involve intercepting or falsifying communication between devices, potentially leading to the takeover of the device or the theft of data.

Example:

Check Point Research discovered that vulnerabilities in the communication between Philips Hue smart bulbs and their central control unit (bridge) allowed for a Man-in-the-Middle (MitM) attack in 2020. In this attack, an attacker could manipulate the traffic between the device and the controller, intercepting data and even altering the behavior of the lighting system.

The attacker could take control of the smart bulb and through MitM communication, send malicious commands to the central bridge, which is responsible for managing the lighting network. The bridge would recognize these commands as legitimate, creating the possibility of expanding the attack to other devices connected to the same network, or even to the user’s home network. This demonstrated that even small devices, such as smart bulbs, can serve as an initial attack vector for more advanced actions, such as taking control of an entire network.

Solutions:
  • Utilizing intrusion detection and prevention systems (IDS/IPS): IDS (Intrusion Detection Systems) can help identify potential threats,
  • Employing DDoS protection services, such as Cloudflare, Akamai, or AWS Shield, which can monitor and filter network traffic in real-time, minimizing the impact of attacks on infrastructure,
  • Using Content Delivery Networks (CDN) and distributed DNS servers, which can spread traffic across multiple geographic locations, reducing the risk of overloading a single location,
  • For communication based on the HTTP protocol, implementing HSTS ensures that browsers and devices will always use a secure HTTPS connection, reducing the risk of session hijacking,
  • If possible, it is advisable to implement two-factor authentication (2FA),
  • Every system component should use digital certificates to verify its identity,
  • Using a VPN (Virtual Private Network) to create a secure communication tunnel can protect data transmitted between IoT devices and servers, especially when communicating over public or untrusted networks.

Real-Time Operating System And Buffer Overflow Attacks

Improper memory handling is one of the most common attack vectors on embedded systems with RTOS. In Buffer Overflow attacks, an attacker attempts to input more data into a buffer than it can handle. This can lead to system memory being overwritten and malicious code being executed, allowing the attacker to take control of the device.

Explanation: 

An RTOS may handle input from a sensor or network device. If the data is not properly validated, a buffer overflow could occur, resulting in the execution of malicious software.

Example:

One of the more recent and well-known Buffer Overflow-type attacks was the Heartbleed attack in 2014. Although it is not a classic example of a Buffer Overflow, the exploit taking advantage of the Heartbleed vulnerability is closely related to buffer overflow mechanisms and memory manipulation.

Heartbleed was a serious vulnerability in the implementation of the OpenSSL protocol (specifically in the “Heartbeat” mechanism). The bug in the code allowed attackers to read portions of the server’s memory, including sensitive data such as private keys, login credentials, passwords, and even encrypted user data.

In the Heartbeat mechanism, which was designed to maintain a connection between the client and server, there was a flaw that failed to properly validate the length of the transmitted packet. An attacker could send a specially crafted packet with a falsely declared length, causing the server to return more data than it should. Heartbleed affected millions of servers worldwide because OpenSSL was widely used in various applications and services, such as banking services, social media platforms, and e-commerce systems. As a result of this vulnerability, many companies and institutions had to immediately update their systems and replace cryptographic keys that might have been stolen.

Solutions:

  • Using functions that control buffer size, such as fgets() instead of gets(), to avoid exceeding available memory,
  • Verifying that input data fits within the allocated buffer before writing it to memory,
  • Applying memory management protection techniques such as DEP (Data Execution Prevention) or ASLR (Address Space Layout Randomization),
  • Compilers like GCC or MSVC offer buffer overflow protection mechanisms, such as stack canaries, which detect stack overwrites before malicious code can be executed,
  • Utilizing modern libraries that automatically manage buffer size and dynamic memory can help eliminate such errors.

More about IoT hardware development you can find out here:

https://intechhouse.com/blog/navigating-iot-hardware-development-a-comprehensive-guide/

Security Future of Embedded Systems And Real-Time Systems

  • Hardware-Based Security: Strengthening security at the hardware level will become a key aspect of protecting IoT devices and RTOS. Trusted Platform Modules (TPM) assist in this. These are specialized security chips that can store cryptographic keys and manage encryption processes, ensuring secure system boot (Secure Boot) and protecting against software tampering. Processor-level security technologies like Intel SGX and ARM TrustZone enable the isolation of critical data and processes in so-called trusted enclaves, which are more difficult to attack even if the operating system is compromised.
  • Blockchain in IoT Security: Blockchain can be used to create immutable records of IoT device identities, enabling verification of the authenticity of each device on the network. This makes it harder to carry out spoofing attacks, where an attacker impersonates another device. Blockchain can also ensure transparency and integrity in the firmware update process for IoT systems. Each update could be recorded in the blockchain, preventing the introduction of malicious modifications to the firmware.
  • IoT Zero Trust Architecture (ZTA): ZTA assumes that no part of the system—device, user, or application—should automatically be considered trusted, even if it is part of the internal network. Therefore, each IoT device on the network will need to be authenticated, and its communication will be monitored for compliance with established rules. Even if the device is authenticated, access to data or services will be granted based on its identity, security level, and adherence to security policies.
  • Microprocess Isolation and Sandboxing: To protect against attacks on RTOS systems, microprocess isolation and sandboxing techniques will be essential. They allow critical processes to run in isolated environments, making it harder for attackers to unauthorized access the system’s main resources. The future of RTOS may focus on even more minimized operating system kernels (nano-/microkernels), which have a reduced attack surface and enable the isolation of tasks with different priorities. Implementing these solutions will help prevent increasingly sophisticated cyberattacks, ensuring greater security in the IoT and RTOS ecosystem.

InTechHouse: Trusted Partner in Data Security of RTOS, IoT and Embedded Devices

Securing RTOS in the context of IoT and embedded systems is a challenge that requires an understanding of the specific nature of these systems and the application of appropriate protective technologies. The rapid development of IoT and the increasing number of cyberattacks on critical infrastructure mean that IT specialists must continuously adapt their approach to new threats. Implementing multi-layered protection strategies, monitoring network activity, and using appropriate cryptographic tools are just some of the steps that can effectively safeguard RTOS infrastructure against modern cyber threats.

InTechHouse offers much more than expertise in IoT, RTOS, and embedded systems. We are your comprehensive technology partner, ready to support a wide range of business needs. From full-stack development to cybersecurity and data analytics, InTechHouse ensures that your business stays ahead in the rapidly evolving tech landscape. Schedule a free consultation and let us help you innovate, streamline operations, and achieve your strategic goals across various industries.

FAQ

What is Task Hijacking and how does it affect RTOS?
Task Hijacking is an attack in which the attacker takes over or alters the priorities of tasks in an RTOS system. The attacker can manipulate the task scheduling, block critical operations, or force the system to execute malicious code. In real-time systems like RTOS, such manipulations can lead to serious disruptions, including downtime, system crashes, or loss of control over the device.

Why is RTOS more vulnerable to attacks compared to traditional operating systems?
RTOS, due to its design for use in resource-constrained devices, often lacks the advanced security mechanisms that are standard in traditional operating systems like Linux or Windows. Additionally, RTOS is often used in critical applications, making it an attractive target for attackers. The lack of regular updates and complex infrastructure also increases the risk of attacks.

Can RTOS attacks affect home devices like smart speakers?
Yes, home devices such as smart speakers, lighting systems, or electronic locks often run on RTOS and can be vulnerable to various attacks, including Denial of Service, memory attacks (Buffer Overflow), or task hijacking. If not properly secured, they can serve as gateways for attacks on the home network or stored data.

What is ASLR and how does it affect RTOS security?
ASLR (Address Space Layout Randomization) is a technique that randomly arranges memory protection areas in an operating system. In RTOS, ASLR makes it more difficult for attackers to predict the location of key data or code, significantly complicating Buffer Overflow attacks. ASLR is an effective tool in mitigating memory-based exploits.