Many performance problems first look like “the CPU is too slow”: a button responds late, UART handling misses data, network packets pile up, the UI stalls, or sensor data reaches the application thread late.
But CPU load is only one layer. What often decides responsiveness is when the code gets to run.
Code existing in memory does not mean it can run at any moment. It may not be ready yet. It may be waiting for a lock, queue, or I/O. It may be blocked behind a higher-priority task. It may have just been woken by an interrupt but not yet selected by the scheduler.
The safest first model is this: a scheduler is not a tool that makes code faster. It is the mechanism that chooses which runnable execution flow uses the CPU next. Responsiveness depends on the whole path from event occurrence to readiness, scheduling, and actual execution.
Event occurs
-> Interrupt or kernel path handles the event
-> A task/thread becomes ready
-> Scheduler chooses the next runner
-> Target task actually gets CPU time
-> Task runs until it blocks, yields, is preempted, or uses up its time slice
If any part of that path is slow, the application sees a slow response.
What the Scheduler Actually Schedules
A scheduler does not schedule “functions.” It schedules execution entities that can occupy the CPU.
In Linux user space, engineers often talk about processes and threads, but the scheduler effectively works with thread-like execution entities. Different threads in one multithreaded process can separately be running, ready, or blocked.
In an RTOS, the scheduler usually schedules tasks directly. Each task has its own stack, priority, context, and state.
Whatever the name, the scheduler answers similar questions:
- who is running now
- who is ready and able to run
- who is blocked and cannot run yet
- if several execution flows can run, who gets the CPU first
- when the current runner should stop running
So when debugging “why did this task not run,” the first step is not whether the function exists. It is which state the execution flow is in.
Running: currently using the CPU
Ready: able to run, but not selected yet
Blocked: waiting for a condition
Sleeping: waiting for time or a timeout
Exited: execution flow has ended
Ready and blocked are very different. A ready task that does not run points to scheduling competition. A blocked task that does not run points to a missing condition.
Slow Response Does Not Mean Slow CPU
One peripheral event can pass through several layers before application code reacts.
For example, when a network card receives a packet:
NIC receives a packet
-> Interrupt fires
-> Driver acknowledges and takes data
-> Kernel network stack processes it
-> Thread waiting on the socket is woken
-> Scheduler chooses the thread
-> Thread returns from read/recv
-> Application code starts processing
If the application says “packet handling is slow,” the cause may be many things:
- interrupt latency is high
- driver handling is slow
- softirq or kernel queues are backed up
- target thread priority is too low
- target thread is blocked on a lock
- CPU is occupied by other work
- the application thread wakes up and immediately waits for another resource
Looking only at CPU frequency or one function’s runtime can miss the real waiting segment.
Responsiveness is not just “how long did this function execute.” It is “how much waiting and preemption happened between the event and the target execution flow completing the important action.”
Priority Only Chooses Among Ready Work
Priority is one of the most commonly misunderstood scheduler concepts.
High priority does not mean “finishes first.” It usually means: when multiple execution flows are ready, the scheduler prefers the higher-priority one.
A high-priority task may still fail to run because:
- it is waiting for a queue message
- it is waiting for a mutex
- it is waiting for I/O completion
- interrupts were disabled too long, so the wakeup event is delayed
- even higher-priority interrupts or tasks keep using the CPU
That is why real-time systems cannot be understood from the priority table alone. Priority only matters after the task is ready. A high-priority task that is not ready cannot run by magic.
Preemptive priority scheduling in an RTOS can be understood as:
If a higher-priority task becomes ready
-> the current lower-priority task may be preempted
-> CPU switches to the higher-priority task
This can provide good responsiveness, but only if the high-priority task is short, its waiting relationships are clear, and it does not hold the CPU for too long.
Time Slices Decide How Peers Share CPU
If multiple tasks have the same priority, or the system uses time-sharing scheduling, another question appears: several execution flows can run, so how should they share the CPU?
A time slice is a common answer. After a thread runs for a short period, if it has not blocked or exited, the scheduler can pause it and give the CPU to another suitable thread.
That explains how desktop systems can appear to run many programs at once. They are not all occupying the same CPU core at exactly the same time; they run in turns at a short time scale.
But time slicing has costs:
- a long time slice can hurt interactivity
- a short time slice increases context-switch overhead
- CPU-heavy tasks can still require priority and wakeup tuning for I/O threads
- multicore systems also care about load balancing and cache locality
Fairness is not the only goal. A system trades off throughput, responsiveness, real-time behavior, and overhead.
Blocking Is More Common Than Running
Many programs spend most of their time waiting, not running.
Common waits include:
- network data
- disk or flash I/O
- mutexes
- condition variables
- queue messages
- timer expiration
- peripheral or DMA completion
This is central to scheduling. The scheduler can only choose from the ready queue. It cannot directly run a task that is waiting for a condition.
For example, a thread blocked in read() is not being ignored by the scheduler. It may be blocked until data arrives, a timeout occurs, a signal interrupts it, or the file descriptor state changes.
The same applies in an RTOS. A task waiting for a queue message is not in the ready queue. After an ISR or another task posts a message, it may become ready and then participate in scheduling.
So debugging must first separate:
- the task is ready but not scheduled
- the task has not become ready
- the task ran but immediately blocked again
- the task ran too long and delayed others
Those four cases lead to different fixes.
Preemption Improves Response and Complicates Concurrency
The value of preemptive scheduling is that urgent work does not have to wait for the current task to voluntarily give up the CPU.
For example, a low-priority task is doing background computation. A UART receive task is woken by an interrupt. If it has higher priority, the scheduler can preempt the background task and handle UART data first.
That improves response time, but it creates shared-state problems.
The preempted task may have been modifying a data structure. If another task accesses the same structure at the same time, it may see an inconsistent state. Any mutable data shared by multiple execution flows needs locks, atomics, critical sections, queues, or other synchronization.
So preemption is not a free upgrade. It reduces waiting for voluntary yielding, but it increases concurrency-control complexity.
In an RTOS, if a high-priority task runs for a long time without blocking, lower-priority tasks may starve. Real-time systems aim for predictable critical paths, not natural fairness for every task.
Scheduling Latency Is Not Interrupt Latency
When a device responds slowly, the cause is often described as “interrupts are slow.” In practice, at least two intervals must be separated:
- interrupt latency: event occurrence to ISR start
- scheduling latency: ISR wakes a task to that task actually running
The interrupt can arrive quickly while the task still runs late.
Possible causes include:
- ISR only wakes a task; real work is deferred
- a higher-priority task is still running
- the target task is waiting for a lock
- long critical sections or interrupt-disabled time affect scheduling
- system load is high and the ready queue is competitive
The reverse is also true. Fast scheduling does not prove low interrupt latency. If an event happens while interrupts are disabled, the ISR itself can start late.
For response analysis, split the timestamps:
Hardware event time
ISR start time
ISR end or task wakeup time
Target task start time
Application work completion time
With those points, the delay can be assigned to interrupt handling, scheduling, locks, I/O, or application logic.
More Tasks Do Not Automatically Make the System Faster
Multiple threads or tasks can improve structure and throughput for waiting-heavy workloads, but they do not create extra CPU cores.
Too many small tasks can introduce:
- more context switches
- more locks and queues
- more stack memory use
- less predictable scheduling relationships
- more copying or synchronization between tasks
On RTOS devices, task count also directly affects RAM. Each task needs a stack. Too small causes overflow; too large wastes memory.
A better division is not “one task per feature.” Ask:
- does this work need independent response time
- can it block for a long time
- does it need a different priority
- should complex logic be isolated
- can a queue clearly express the data flow
Task boundaries should serve responsiveness and resource management, not the source-code directory layout.
What to Check First During Scheduling Bugs
When a system stalls, has latency, or a task does not run, do not start by changing priorities.
A better order is:
First, check the target task state. Is it ready but not running, or blocked on a lock, queue, I/O, or timer?
Second, check who owns the CPU. Is a high-priority task running too long, are interrupts too frequent, or is background computation failing to yield?
Third, check the wakeup path. After the event occurs, does the system correctly wake the target task? Does it enter the ready queue?
Fourth, check shared resources. Is the target task waiting for a lock? Which task holds it? Does the holder block while holding it or get preempted by medium-priority work?
Fifth, check timestamps. Record event time, ISR time, task wakeup, task start, and application completion, not just the final log.
These questions are more reliable than “raise the priority a little.” A wrong priority change can simply move the problem from one task to another.
What to Remember in Practice
The scheduler decides which runnable execution flow uses the CPU next.
System responsiveness is not a single CPU-speed problem. It is determined by this path:
when the event occurs
when the task becomes ready
when the scheduler chooses it
whether it blocks again after running
whether it is preempted by higher-priority work
Priority only affects the choice among ready tasks. Time slices affect fairness among peers or time-sharing workloads. Blocking decides whether a task is eligible for scheduling. Preemption decides whether urgent work can interrupt the current runner.
Once these layers are separated, “the system is slow,” “the task did not run,” and “the interrupt was not timely” become measurable runtime paths instead of vague symptoms.