Process isolation solves an important problem: if one process corrupts its own memory, it usually does not directly corrupt another process.
But isolation creates another problem: two processes cannot share ordinary variables like two threads can. A pointer inside one process usually means nothing inside another process. Once a system is split into multiple processes, IPC, or inter-process communication, becomes necessary.
IPC is often understood as “send data from A to B.” That is only part of it.
The safest first model is this: IPC is the mechanism for exchanging data, synchronizing state, and passing control across process isolation boundaries. It is not only about how bytes move; it is also about who owns the data, who waits for whom, what happens when buffers are full, and what happens when the peer dies.
Process A
-> cross isolation boundary through kernel objects or shared mappings
-> Process B
If you only ask whether data can be transferred, you miss the parts that decide system behavior: synchronization, backpressure, lifetime, and failure boundaries.
Why Processes Cannot Communicate With Ordinary Pointers
Threads can directly share variables because they are inside the same process address space.
Processes cannot do that by default. Each process has its own virtual address space. The same numeric address in two processes may map to different physical pages, or may be invalid in one process.
Process A: pointer 0x1000 -> some memory in A
Process B: pointer 0x1000 -> different memory in B, or invalid
So Process A cannot send an ordinary pointer to Process B and expect B to read the same object.
The first job of IPC is to create a controlled exchange channel across that isolation boundary.
That channel may copy data, map the same physical pages into both processes, or only deliver an event or handle. In all cases, the kernel or runtime must know where the boundary is.
Pipes and Sockets Solve Byte Streams
Pipes and sockets can be understood as communication channels maintained by the kernel.
Process A writes data, the kernel puts it into a buffer, and Process B reads it from the other end.
Process A write
-> kernel buffer
-> Process B read
Their boundaries are clear:
- processes do not share ordinary memory
- data flows through read/write interfaces
- peer close can be observed
- blocking, nonblocking, and timeout behavior can be expressed through system APIs
The cost is that data usually passes through kernel buffering and may be copied. Many small messages add syscall and scheduling overhead. Large data adds copying and buffer pressure.
More importantly, a byte stream does not automatically preserve business message boundaries. Three writes do not guarantee three reads. The peer may read half a message or several messages at once.
So stream-based IPC usually needs framing:
- fixed length
- length field
- delimiter
- TLV
- higher-level protocol header
Otherwise “the data arrived” does not mean “the message boundary is clear.”
Message Queues Solve Discrete Messages and Waiting
Message queues emphasize separate messages rather than a raw byte stream.
A sender posts a message, and the receiver takes messages in queue order. This model is natural for RTOS tasks and multiprocess services:
Producer
-> message queue
-> consumer
The value of a message queue is not only data transfer. It also defines synchronization.
The consumer can block when there is no message. After the producer posts one, the system wakes the consumer. If the queue is full, the producer may block, fail, drop the new message, drop old messages, or overwrite according to policy.
These choices directly change system behavior:
- a short queue drops bursts easily
- a long queue can hide latency for a long time
- blocking producers can back up upstream work
- low-priority consumers allow queues to accumulate
- messages containing pointers must still handle lifetime and address-space meaning
So a message queue is not a fancier array. It defines waiting, backpressure, and drop policy between producers and consumers.
Shared Memory Reduces Copying, Not Synchronization
Shared memory is often considered the fastest IPC. It is fast because two processes can map the same physical pages and avoid copying large data from A to B each time.
Process A: virtual address VA1 -> physical page P
Process B: virtual address VA2 -> physical page P
The two virtual addresses may differ, but the physical memory behind them is the same.
Shared memory fits large, frequent, or low-latency data paths, such as image frames, audio buffers, capture data, and inter-process ring buffers.
But shared memory only solves “both sides see the same data.” It does not solve:
- who is writing
- who may read
- when the data is complete
- when a buffer can be reused
- who cleans up if the peer crashes
- multicore cache coherency and memory ordering
So shared memory is usually paired with synchronization:
- mutexes
- semaphores
- futexes
- eventfd
- condition variables
- atomics
- ring-buffer protocols
Shared memory lowers data copying cost, but exposes the synchronization protocol to the developer. That tradeoff is explicit.
Signals and Events Are Control Notifications
Some IPC mechanisms do not mainly carry data. They carry control events.
For example, signals can notify a process about:
- termination
- configuration reload
- child process exit
- timer expiration
- an exceptional condition
Event objects, eventfd, and semaphores are also often used to express “something happened” or “a counter changed.”
The key is not data volume. The key is wakeup and control semantics:
- who may send
- who is woken
- whether events accumulate
- whether events can merge
- what happens if the receiver is not ready
If an event notification is treated as a reliable message queue, state can be lost. Many notification mechanisms only say “go check the state”; they do not preserve the full history of every state change.
Backpressure Is More Important Than Transfer
If the sender can be faster than the receiver, backpressure is unavoidable.
Backpressure answers: what should happen when the receiver cannot keep up?
Common strategies include:
- block the sender
- return an error
- drop new messages
- drop old messages
- overwrite with latest state
- increase buffer size
- reduce upstream production rate
Different strategies fit different business requirements.
Logs may be dropped under pressure. Control commands usually should not be dropped casually. Latest sensor values may overwrite old ones. If a file-writing task blocks, it may stall the whole business path.
Many system stalls are not because IPC cannot transfer data, but because backpressure was not designed. When a queue is full, who blocks, for how long, whether a lock is held while blocking, and whether a high-priority path is backed up are more important than the IPC mechanism name.
IPC Must Handle Ownership and Lifetime
When data crosses processes, ownership must be clear.
For a buffer:
- can the sender modify it after sending
- who frees it after the receiver reads it
- who reclaims it if the receiver crashes
- can the receiver keep reading if the sender exits
- can data be reused while multiple receivers read
Copy-based IPC is simpler. After data is copied into the kernel or receiver, the sender can manage its original memory independently.
Shared memory is more complex. Both sides see the same data, so ownership and lifetime must be defined by protocol. Otherwise:
- sender overwrites data the receiver has not read
- receiver reads half-written state
- both sides assume the other will release resources
- shared state remains stuck after peer crash
The most easily missed part of IPC design is not transfer. It is “who owns this object until when.”
How to Choose an IPC Mechanism
Choosing IPC should not start with “which one is fastest.”
Better questions are:
- is the data a byte stream, discrete messages, or shared state
- how large and frequent is the data
- must message boundaries be preserved
- can the sender block
- what happens when the receiver is slow
- how are peer crash and cleanup handled
- does it cross machine, container, or permission boundaries
- are there multiple producers or consumers
- can the system tolerate shared-memory synchronization complexity
A simplified tradeoff is:
Pipe/socket: clear boundary, general-purpose, possible copying and framing cost
Message queue: clear message semantics, natural waiting and queue policy
Shared memory: fast data path, complex synchronization and lifetime
Signal/event: good for notification, not a replacement for a message protocol
The thing that decides stability is usually not the IPC name, but its waiting, backpressure, synchronization, and failure semantics.
What to Check During IPC Bugs
When inter-process communication hangs, drops messages, has high latency, or corrupts data, separate these layers.
First, did data enter the channel? Did the send succeed? What did the syscall return? Is the buffer full?
Second, was the receiver woken? Is it blocked waiting, or ready but not scheduled?
Third, are message boundaries correct? Is a byte stream split or coalesced? Is the length field parsed correctly?
Fourth, what is the backpressure path? When the queue is full, who blocks? Is it blocking while holding a lock? Does it delay a high-priority path?
Fifth, is shared-memory synchronization correct? Are read/write pointers, visibility, memory ordering, and buffer reuse clear?
Sixth, is peer lifetime handled? After peer exit, crash, or restart, are handles, shared memory, and queue states cleaned up?
These questions turn “why did the data not get there” into a runtime path that can be checked.
What to Remember in Practice
IPC is not only data transfer.
It handles these issues across process isolation boundaries:
- how data crosses
- how message boundaries are expressed
- who waits for whom
- what happens when the receiver is slow
- who owns buffers
- what happens when the peer dies
- how shared state is synchronized
Copy-based IPC gives clear boundaries at the cost of copying and syscall paths. Shared memory gives a faster data path at the cost of synchronization, ownership, and lifetime complexity. Event notifications are good for wakeup, not for replacing a message protocol.
Once these semantics are clear, IPC design moves beyond “can the data get through” and starts working under load, failure, and concurrency.