Async Rust in Three Parts
[DRAFT]
- Introduction (you are here)
- Part One: Futures
- Part Two: Tasks
- Part Three: IO
Async/await, or "async IO", is a new-ishRust has had async/await since 2019. For comparison, C# added async/await in 2012, Python added it in 2015, JS in 2017, and C++ in 2020. language feature that lets our programs do more than one thing at a time. It's sort of an alternative to multithreading,Throughout this series we'll compare examples using threads to examples that accomplish the same thing using async. If this is your first time using threads in any language, though, this approach might be more confusing than helpful, because you'll have to learn two things at once. For an introduction to threads, see Chapter 16 and Chapter 20 of The Book. though Rust programs often use both. Async is popular with websites and network services that handle many connections at once,"Many" here conventionally means ten thousand or more. This is sometimes called the "C10K problem", short for 10k clients or connections. because running lots of "futures" or "tasks" is more efficient than running lots of threads. This series will be an introduction to futures, tasks, and async IO.
If we think of threads as asking our OS and our hardware to do things in parallel for us, then we can think of async/await as reorganizing our own code to do that ourselves. This requires both new high-level concepts and also new low-level machinery, and that combination can be overwhelming. This series will mostly skip the concepts and jump straight into the machinery.For example, we're not going to talk about "parallelism" vs "concurrency" at all. We'll start by translating ("desugaring") async examples into ordinary Rust that we can run and understand, and gradually we'll build our own async "runtime".For now, a "runtime" is a library or framework that we use to write async programs. Building our own futures, tasks, and IO will gradually make it clear what a runtime does for us. The async examples in this introduction and in Part One will use the Tokio runtime. There are several async runtimes available in Rust, but the differences between them aren't important for this series. Tokio is the most popular and the most widely supported.
#/media/File:Parallelism_vs_concurrency.png){kind=link}
In Rust maybe more than in other languages, async/await pulls together all the tools in the language toolbox. In Part One alone we'll need enums, traits, generics, closures, iterators, and smart pointers. I'll assume that you've written some Rust before and that you've read The Rust Programming Language ("The Book") or similar.Again, the multithreaded web server project in Chapter 20 is especially relevant. If not, you might want to refer to Rust By Example whenever you see something new.If you're the sort of programmer who doesn't like learning languages from books, consider this advice from Bryan Cantrill, who's just like you: "With Rust, you need to learn it…buy the book, sit down, read the book in a quiet place…Rust rewards that."
Let's get started by doing more than one thing at a time with threads. This will go smoothly at first, but then we'll run into trouble.
Threads
Here's an example function foo that takes a second to run:
fn foo(n: u64) {
println!("start {n}");
thread::sleep(Duration::from_secs(1));
println!("end {n}");
}
If we want to make several calls to foo at the same time, we can spawn a
thread for each one. Click on the Playground button to see that this takes one
second instead of ten:You'll probably also see the "start" and "end" prints appear
out of order. One of the tricky things about threads is that we don't which
one will finish first.
fn main() {
let mut thread_handles = Vec::new();
for n in 1..=10 {
thread_handles.push(thread::spawn(move || foo(n)));
}
for handle in thread_handles {
handle.join().unwrap();
}
}
Note that join here means "wait for the thread to finish". Threads start
running in the background as soon as we call spawn, so all of them are making
progress while we wait on the first one, and the rest of the calls to join
return quickly.
We can bump this example up to a hundred threads, and it works just fine. But if we try to run a thousand threads,On my Linux laptop I can spawn almost 19k threads before I hit this crash, but the Playground has tighter resource limits. it doesn't work anymore:
thread 'main' panicked at /rustc/3f5fd8dd41153bc5fdca9427e9e05...
failed to spawn thread: Os { code: 11, kind: WouldBlock, message:
"Resource temporarily unavailable" }
Each thread uses a lot of memory,In particular, each thread allocates space for its "stack", which is 8 MiB by default on Linux. The OS uses fancy tricks to allocate this space "lazily", but it's still a lot if we spawn thousands of threads. so there's a limit on how many threads we can spawn. It's harder to see on the Playground, but we can also cause performance problems by switching between lots of threads at once.This is a demo of passing "basketballs" back and forth among many threads, to show how thread switching overhead affects performance as the number of threads grows. It's longer and more complicated than the other examples here, and it's ok to skip it. TODO: Is this version still blocked? Threads are a fine way to run a few jobs in parallel, or even a few hundred, but for various reasons they don't scale well beyond that.A thread pool can be a good approach for CPU-intensive work, but when each jobs spends most of its time blocked on IO, the pool quickly runs out of worker threads, and there's not enough parallelism to go around. If we want to run thousands of jobs at once, we need something different.
Async
Let's try the same thing with async/await. Part Two will go into all the
details, but for now I just want to type it out and run it on the Playground.
Our async foo function looks like this:
async fn foo(n: u64) {
println!("start {n}");
tokio::time::sleep(Duration::from_secs(1)).await;
println!("end {n}");
}
Making a few calls to foo one at a time looks like this:In Parts Two and Three of this series, we'll implement a lot of
what #[tokio::main] is doing. Until then we can just take it on faith
that it's "the thing we put before main when we use Tokio."
#[tokio::main]
async fn main() {
foo(1).await;
foo(2).await;
foo(3).await;
}
Making several calls at the same time looks like this:Unlike the version with threads above, you'll always see this version print its start messages in order, and you'll usually see it print the end messages in order too. However, it's possible for the end messages to appear out of order, because Tokio's timer implementation is complicated.
#[tokio::main]
async fn main() {
let mut futures = Vec::new();
for n in 1..=10 {
futures.push(foo(n));
}
let joined_future = future::join_all(futures);
joined_future.await;
}
Despite its name, join_all is doing something very different from the
join method we used with threads. There joining meant waiting on something,
but here it means combining multiple "futures" together. We'll get to the
details in Part One, but for now we can add some more prints to
see that can see join_all doesn't take any time, and none of foos start
running until we .await the joined future.
Unlike the threads example above, this works even if we bump it up to a thousand jobs. In fact, if we comment out the prints and build in release mode, we can run a million jobs at once.For me this takes about two seconds, so it's spending about as much time working as it is sleeping. And remember this is on the Playground, with tight resource limits. The tasks version of the basketball demo above is also much more efficient than the threads version, but it requires lots of concepts we haven't explained yet, so I don't want to focus on it.
Important Mistakes
We can get some hints about how async works if we start making some mistakes.
First let's try using thread::sleep instead of tokio::time::sleep
in our async function:
async fn foo(n: u64) {
println!("start {n}");
thread::sleep(Duration::from_secs(1)); // Oops!
println!("end {n}");
}
Oh no! Everything is running one-at-a-time again! It's an easy mistake to make, unfortunately.There have been attempts to automatically detect and handle blocking in async functions, but that's led to performance problems, and it hasn't been possible to handle all cases. But we can learn a lot about how a system works by watching it fail, and what we're learning here is that all of the jobs running "at the same time" in the async examples above were actually running on a single thread. That's the magic of async. In Part One, we'll dive into all the nitty gritty details of how exactly this works.
TODO: This also doesn't work:
#[tokio::main]
async fn main() {
let mut futures = Vec::new();
for n in 1..=10 {
futures.push(foo(n));
}
for future in futures {
future.await; // Oops!
}
}