Asynchronous I/O on Linux
In systems programming, the I/O (input/output) model determines how you interact with data, and it has a critical impact on overall system performance. The I/O model directly affects several major areas:
- Throughput
- Machine resource utilization, e.g. CPU usage
- System scalability
- Programming complexity
In network programming, the I/O model is the foundation of a high-performance server. If it's poorly designed, no matter how elegant the upper layers are, everything stalls at the I/O layer and becomes the performance bottleneck.
This post focuses on two core I/O models on Linux—synchronous and asynchronous I/O—and explains them alongside the relevant socket functions.
Part 1: Synchronous I/O—the familiar "blocking" and "non-blocking"
In plain terms, synchronous I/O means: after the caller initiates an I/O operation, it must wait for the data to be ready before it can continue.
Core sub-model 1: Blocking I/O
In Linux, a socket created with socket() is in blocking mode by default. In this mode, when you call recv(), read(), or accept() and the data isn't ready, the process is suspended into a blocked state until the data becomes ready, at which point the process resumes.
"Process suspension" here means thread suspension in a multithreaded scenario.
This approach has pros and cons. The upside is that it's simple to program—the logic is intuitive and easy to understand. The downside is low efficiency: especially under high concurrency, a single thread can handle only one connection, which fails to make full use of the CPU.
Core sub-model 2: Non-blocking I/O
On Linux, non-blocking I/O is enabled by using fcntl() to put the socket into non-blocking mode. Once that takes effect, if the data isn't ready the caller immediately gets an error return (usually EAGAIN or EWOULDBLOCK); the calling thread isn't suspended and can go do other work.
The advantage here is obvious: the thread doesn't block, so it can handle many more tasks and make much better use of the CPU. But the downside is equally clear: it needs some kind of continuous polling to check whether the I/O data is ready, which also burns CPU.
A summary of the two modes
The essence of synchronous I/O is that after issuing an I/O request, the caller must wait for the data copy to complete. Blocking and non-blocking are just different strategies for the stage of waiting for the data to become ready.
Part 2: Asynchronous I/O—let the OS handle I/O for you
In Part 1 we saw that blocking I/O has low CPU utilization, while non-blocking I/O is complex to control and its mechanism also consumes a lot of CPU. To solve these problems more effectively, OS-level asynchronous I/O was introduced.
The flow is as follows: after the caller initiates the I/O operation, it can return immediately and continue with other tasks. When the I/O operation completes, the OS notifies the caller through some mechanism (such as a callback). It's like ordering takeout—you can keep working, and you get a notification when it arrives.
This way, the OS solves the control complexity and CPU consumption of non-blocking I/O for us.
Here we introduce Linux's asynchronous I/O mechanisms. Note that they are meant for reading disk block devices, not for network I/O; the goal is to illustrate the idea of asynchronous I/O on Linux. Asynchronous I/O breaks down into the following steps:
Step 1: Issue the asynchronous I/O request
The flow for issuing asynchronous I/O:
- The application calls a special asynchronous I/O request function (e.g.
aio_read()orio_submit()). - This call immediately submits the I/O request (including the file descriptor to read, the buffer address, the data size, and so on) to the kernel.
- This is the key step. The submission function returns immediately rather than waiting for the data read/write to complete. This means the application can keep doing other tasks without having to stop.
LibAIO vs. Linux native asynchronous I/O functions
LibAIO is POSIX-compliant and has better cross-platform support, but it actually simulates asynchrony using a thread pool, so its performance is not ideal. Typical functions:
aio_read()aio_write()aio_fsync()aio_error()aio_return()aio_suspend()aio_cancel()aio_listio()Linux's native asynchronous I/O functions are a more direct and efficient way to interact with the kernel. They submit I/O requests directly into the kernel's event queue instead of relying on thread simulation, achieving true kernel-level asynchrony. Their main drawback is that they only run on Linux and are not portable. Typical functions:
io_setup()io_getevents()io_submit()io_cancel()io_destroy()
Step 2: The kernel takes over and processes the I/O
Once the request is submitted, the rest of the work is done by the kernel, and the application hardly needs to worry about it.
- Kernel scheduling: After receiving the request, the kernel puts it in a queue and schedules an appropriate time to handle it.
- Hardware interaction: The kernel interacts with the hardware (such as the disk controller) to start the data transfer. This happens entirely in the background and usually requires no CPU involvement (via DMA, Direct Memory Access).
- Data copy: Data is copied directly from the hardware device into the memory buffer specified by the application. The copy is handled entirely by the kernel, with no application involvement.
Step 3: The I/O operation completes and notifies
Once the copy is done, the kernel needs to notify the application in some way to tell it "your I/O task is complete."
Notification mechanism: This is the heart of the asynchronous I/O model. There are several ways to notify:
- Signal: The kernel can send the application a signal (e.g.
SIGIO) to indicate the I/O task is complete. The application must register a signal handler in advance to catch it. - Callback function: This is the more common way. When issuing the request, the application provides the address of a callback. When the I/O completes, the kernel calls it automatically, passing the result (such as success or failure) as an argument.
Step 4: The application handles the result
Once the application receives the completion notification, it can safely process the data.
- Check the result: After receiving the notification, the application checks the result of the operation, such as the number of bytes read or whether an error occurred.
- Continue processing: The buffer now holds the complete data, so the application can start parsing, processing, or responding to it.
Part 3: I/O multiplexing—the dawn of asynchrony in a synchronous world
The two parts above introduced non-blocking I/O and asynchronous I/O programming. We saw that non-blocking is inefficient while asynchronous I/O programming is fairly complex. Is there a middle ground? Yes—I/O multiplexing.
I/O multiplexing means a single thread can simultaneously watch multiple socket file descriptors; when an event (data readable or writable) occurs on one of them, the kernel notifies the application layer.
Here we cover just a few core APIs:
select(): The earliest API. Its advantage is cross-platform support; its drawbacks are that performance degrades linearly with the number ofsockets and there is a limit on the number of file descriptors.poll(): An improved version ofselect()that removes the file-descriptor count limit.epoll(): A high-performance API unique to Linux. It has two modes—edge-triggered (ET) and level-triggered (LT)—and it excels at large-scale concurrent connections because it returns only the descriptors that are already ready, without polling all of them.
I/O multiplexing is still fundamentally synchronous I/O (because after the kernel notifies it, the application layer still has to issue its own read() or write() to copy the data). But by waiting for multiple file descriptors to become ready, it solves both the efficiency problem of blocking I/O and the polling problem of non-blocking I/O, achieving the efficient "one thread handling many connections" pattern.
Part 4: Summary and choices
Choosing among blocking I/O, non-blocking I/O, I/O multiplexing, and asynchronous I/O requires weighing several factors—programming complexity, performance, applicable scenarios, and so on—to pick the approach that fits your business.
General recommendations:
- Blocking I/O: Suitable for client applications and servers with few connections and modest performance requirements.
- I/O multiplexing (especially
epoll): The first choice for high-concurrency servers, such as web servers and chat rooms. It strikes the best balance between programming complexity and performance. - Asynchronous I/O: Suitable when you need extreme I/O performance and can accept the complexity of the programming model, such as databases and high-performance storage systems.
I/O model technology keeps advancing; a notable example is the arrival of io_uring on Linux, which can support all types of I/O, including network I/O.