Transcript
Intro
0:00 · What would your pitch be on why it's worth checking out Rust or working with Rust? You need a language that's reliable that's going to have as few bugs as possible. That's the idea of Rust. What is the ownership model?
0:11 · The idea that if you have a variable containing some object, then the object is only there. So, it's kind of exclusive. We talked about memory safety, but there is a keyword in Rust called unsafe. Generally, when unsafe is used, it's to add a new features to the language. As long as you design your API right, you can add new language features by using unsafe. What are additions?
0:32 · Additions are basically the way that Rust makes breaking changes without breaking people because they might change the syntax of the language. Do you use any of the AI tools for your day-to-day work contributing to Rust projects? So, I have been trying to use them. Honestly, Rust is quietly spreading as a language of choice to build reliable and performant applications. But what makes it different? Ellis Real is a software engineer working on Google's Android Rust team, a core maintainer of Tokyo, the de facto async runtime for Rust, and is a Rust language team adviser. In today's conversation, we cover the pitch on why Rust is worth to consider whether you're using TypeScript or C++ today.
1:15 · how concepts like ownership, the borrow checker, and the unsafe keyword work, and what are things that trip up newcomers to Rust, how the language is governed without a benevolent dictator, and how RFC's and additions work, and many more. If you want to understand what makes Rust different and why so many engineers say once it compiles, it works, this episode is for you. This episode is presented by Antithesis.
1:38 · Verify your systems correctness without human review or traditional integration test and avoid bugs or outages. This episode is brought to you by Sentry.
1:46 · Centry is application monitoring software built by developers for developers. The first time I used Sentry was 10 years ago back at Uber where Sentry helped keep us honest on when and where our services were breaking. I also use Sentry today to help me understand if the services and APIs I built for the pragmatic engineer are healthy or not.
2:04 · Sentry shows you the full context on issues, stack traces, user actions, environment details, and even the exact line of code that caused the issue. It supports pretty much every modern tech stack. TypeScript, JavaScript, Python, Go, and others. It works on back and front and mobile, you name it. One new feature Sentry launch is Seir, their AI debugging agent. Let me show you. I open the Seir agent and ask about what are some repeated errors happening on my back end. Seir figures out that a repeated issue is a network call failure. I can then ask for more details and debug more efficiently with this AI agent integrated neatly in this entry.
2:38 · Sierra is a neat tool to fix the hard issues, the ones that are just hard to debug. Check out Sentry at centry.io/pragmatic and start monitoring today. Alice, welcome to the podcast.
2:49 · Thank you for having me.
2:50 · It It's really nice to have you here.
2:52 · How did you get into software engineering?
2:54 · It actually all started with Minecraft.
2:56 · No way.
2:57 · I wanted to write my own mod for Minecraft, so I learned Java. I didn't get very far with the Minecraft modding, but that's where it started.
3:06 · How did you continue? Did you go to university? This was just before I started in high school. And then after high school, I had a year where I worked uh full-time as a software engineer. And then I moved on to starting my bachelor. Did that for 3 years and then I did a masters for 2 years.
3:22 · How did you end up at Google? Was that straight out of university?
3:26 · Actually started part-time at the same time as when I started my masters and then I switched to full-time after I finished.
3:33 · How did you get involved with the Rust community? Was was it at Google? Was it before Google?
3:37 · Oh, way before.
3:38 · Way before.
3:39 · I've been doing Rust for a long time.
3:41 · When I was in school, I spent a lot of time on the what's called the Rust users forum. Well, I was answering questions really. I have maybe 10,000 posts on there or something. At some point, I also started being active in some chat servers, the Discord server for something called Tokyo. I kept doing that, answering questions. When I saw common questions, I would uh fix the documentation. That's how I got into Tokyo, which I'm now one of the maintainers of.
4:08 · And then for those of us not as familiar with Rust and and Tokyo, uh what is Tokyo inside of Rust and why is it important?
Tokio: an overview
4:16 · So Tokyo is well, it's an asynchronous runtime for Rust. You can think of it as the standard library for Rust when you're using async. I mean, if you compare with something like JavaScript in the browser, you might compare Tokyo with the browser itself. For example, in JavaScript, you have this loop, this event loop which has all the tasks that are able to run and then they get executed one after the other. And especially if you use the await stuff, then you can have tasks pause and then another task start running on the same thread. And Tokyo does something similar. It has a queue of things that are able to run and then it will run them. So unlike JavaScript, Tokyo can be multi-threaded. So you can have multiple cues running in parallel.
5:05 · This seems like a pretty core part of of Rust as a language, as a as a ecosystem.
What Alice likes about Rust
5:11 · How did you gravitate towards this? Cuz it sounded like you were, if I understand, you were lurking on the Rust forums. You were helping out here and there. What drew you to this part of the language or or the e ecosystem, should I say? I think part of what I liked about Rust is this feeling that as you write the code when it compiles it works. I mean this has to be in quotes right because obviously it's possible that there are bugs but this is something a lot of people say about Rust and there's a reason people say it even though it's not necessarily literally true.
5:45 · How do some other languages compare to begin with? I think to have a language that feels this way you have to have a type system. That's where it all starts. Yeah, I do think that even compared to other languages with type systems, I think Rust does a better job than many languages, even others with type systems. I mean, the the classic example is Java's null. It was Tony hair who invented uh the null and he called it his billiond dollar mistake because it's so easy to have I mean every time you call a function you might have a crash in your program and in Rust I think they're really good at making sure that when you call a function there's no chance that it might be null right that problem just doesn't exist so you can't have that kind of crash and so you have to explicitly say this object might be null and then the compiler will force you to check for null before you use it. So you can't forget like you can in Java.
6:45 · Can we talk about Rust as a whole? This was a language that was created in in 20 years ago in 2006 and it feels like it's become a lot more popular over time. Do you know of like the scale that Rust is at in terms of usage or popularity or or things that among the Rust Rust community numbers that are known there?
7:06 · One thing I found kind of funny, I I checked here all the day and uh we have actually overtaken PHP and Go on the Tyobi index.
7:15 · Oh, this that that index tried to estimate the the usage of languages, right? Usage or popularity or something like that.
7:23 · Wow. Cuz the feeling I have again in general is that Rust is popping up in more and more places. More companies are building on Rust. Oxide is a good example. they decided to go all in on Rust and like there's a sense of of Rust is is becoming a popular place to build I guess high performance applications even increasingly contributing to to the kernel as an engineer who you know might know other languages from like Typescript Java etc. What would your pitch be on why it's worth checking out Rust or working with Rust?
7:56 · That depends a lot on which language you're coming from. Oh, let's come from Typescript. It's it's one of the most popular languages right now. The pit from TypeScript would be a lot different than the pit from C++. But okay, so to begin with, I think when it comes to TypeScript, Rust fits in as the backend language. That's where I would put it. I wouldn't use it in the front end. I think it's a pretty good fit for backends, API servers. One way to put it is you don't want to be waking off at night because there are problems with your web server. You need a language that's reliable, that's going to have as few bucks as possible. I mean, obviously, it would be nice if it had zero bucks, but you know, it's going to be hard to get there, but as few bucks as possible. That's the idea of Rust. I mean, I already mentioned null, but it does a lot of stuff like that. So there's no null checking and and this is done through you know it has this enum type which I mean I think Typescript can do something similar. TypeScript is actually kind of good on that front but but okay the other thing I think is quite good is uh error handling. So on one hand Russ doesn't really use exceptions. So it actually returns the error as a value.
9:11 · So you return a value that's either using an enum either the result or the error. And the way this is done is that there's an operator question mark which says so you write my function and then question mark at the end. And this means if this function fails return the error.
9:32 · So it's really easy to handle errors but it's not zero characters like like it is with exceptions right? So it's explicit on the other hand and if you forget to put the question mark that's a combination error.
9:43 · Wonderful.
9:44 · So you have to check it.
9:45 · Yeah.
9:45 · Right.
9:45 · And of course you can also handle that manually. But but the point is it's this idea of there are these things where you write some code and there's some implicit error condition you didn't think of and now you just you know took down your server or something. Another thing I would I quite like is how it handles documentation. How does it for one when you have a comment you make it into a documentation comment by having three slashes instead of two. Now the thing is you can of course write examples in your documentation and Rust makes all examples into tests.
10:24 · This means that if you change the underlying code now your test fails. And so this means that you can't even you can't forget to update your examples in your documentation. Yeah. And I guess there's also things like you can you forget to initialize a variable in Rust?
10:42 · I mean, no, it doesn't allow you, right? The comp you have to set a value before you use it for the first time.
10:48 · Yeah. And I guess the things with uh checking the format of incoming JSON, it also forces you to do that.
10:53 · Yeah.
10:53 · So there's a pretty cool library called Surya in Rust where you you have your strct so your type you know with fields and then you can say I want to be able to pass this from for example JSON and then SA will it's a macro it will generate code which checks the JSON that it's in the same format it has all the fields you need and they have the right types and it generates native code that's specific to that particular shape of JSON. So the charge are really efficient and again yeah so it helps you avoid errors and there's this thing which again I I learned just very recently thanks to thanks to you as we're talking ahead of this is the switch statement right so in almost every language you have a you have a switch for an enum and you handle cases and then you might have a default or you know everything else and sometimes you forget one of them it's not a big deal or maybe it's a big deal but in Rust you cannot do that either right that's right so in rust it's not called pitch it's called match but it's the same idea you can match on your enum and then you can have a branch for each possibility at rank B and if you are missing one that's a compiler error of course you can have a catch all case if you want to but most of the time you would just list all the cases and then in the future when you add a new variant the compiler will tell you oh you need to update your code here here and here and I think this is kind of leads to another way that Rust really helps with reliability, which is that if you're refactoring, I think Rust is really good at telling you all the places you need to update.
12:30 · I've done this sometimes where I would refactor something. I change the code. I change the return type or whatever it is, and then I just fix the compiler errors and until the compiler stops shouting. And then once I've done that, I've updated every place I need to update. I I get a sense that the language designers have thought really hard of what are ways that typically go wrong in a lot of other programming languages and they just try to fix it through the compiler. Like the documentation example is is the one where I'm still like wow like it happens all the time like you you have a comment example or not and then it gets out of sync and we always complain about this and we don't know how to fix it. I think we've been I I know I've been complaining or like for you know like a decade plus this dross is the first language where I hear a like an actual solution even if it's not a perfect one and I really think this pattern just every where you look you have this kind of thing again and again that oh if you messed up they you know either it won't compile or at the very least there's a lint for it they just catch a lot of cases at compile Now, we've had the pitch from Typescript or similar languages. What about the pitch from C++? So, here I actually think it's even stronger. The thing with C++ is that if you make a mistake, right, in in JavaScript, maybe you take down your server, which is already bad enough, but in C++ when you make a mistake there, now it's actually a security vulnerability most of the time.
Rust for TypeScript engineers
Moving from C++ to Rust
14:08 · Mhm.
14:08 · If you do something as trivial as you did an off by one in your IRA or whatever it might be, that's a security vulnerability and that this just keeps happening.
14:18 · Small mistakes become security vulnerabilities. And in Rust, so Rust is memory safe, right? I mean, we talked a bunch about different ways that Rust is more reliable. And we didn't even touch memory safety.
14:29 · Let's talk about memory safety. Memory safety is this idea that no matter how stupid the code you write is, it's not going to have a certain class of bugs.
Memory safety
14:40 · And this is the, you know, the kind of bug that usually leads into security vulnerabilities. You know, the kind of thing where you read past the array and you just look at random memory or you destroyed an object and then you used it afterwards. So now you actually touched the memory of some other random object.
14:59 · Yep.
14:59 · And then attacker who figures this out could populate something there eventually get that code somehow executed or configuration read and then boom like the classic example in the kernel is let's say you have some object and you manage to make it so that the object that's actually there right because the original object is gone so the memory got reused and now it has a task strct it's called and that's basically your process and it has a field called user ID. And it's pretty common for code to write zero zeros to memory. But if you write a zero to the user ID, now you're root.
15:41 · That's a root user.
15:42 · Yeah, that that's a really classic way of exploiting this kind of vulnerability.
15:48 · And then once an attacker manages to do that, they can take over the whatever your server or whatever is running. And then of course from there on it can just spiral out of control, right? Once you're rooe, you're lost.
16:00 · Yeah.
16:00 · Do I understand that a very strong pitch of rust, especially coming from C++, is memory safety eliminates this whole class of bugs, which which are which can turn into security vulnerabilities, which are one of the most serious threats any any software can have.
16:16 · Lost all of the reliability ones I mentioned in the beginning.
16:19 · That that is a pretty good pitch. We just talked about how Rust offers several safety features like memory safety, error handling, and a type system. This is because reliable is not about one thing but several things at once. This is where I need to mention our presenting sponsor antithesis. Like the designers of Rust, antithesis also believes there's no one silver bullets for reliability. You need many different tools and approaches. This is why they've released Hegel Rust, a free open- source property based testing library for Rust built by the team behind Hypothesis. Rust compiler is brilliant at catching whole categories of bugs at compile time, but at the edge cases, the weird input combinations, the assumptions that turn out not to hold, those runtime bugs need a different tool. Hegel provides powerful ergonomic property based testing for fast local development. It'll check edge cases you never thought of and catch unknown unknowns before they bring down production. And if you try Hegel and like it, your Hegel test will run an antithesis as written, so you can easily add determinism and the world's most thorough runtime verification to your reliability arsenal. Go to hegel.dev to learn more. I'd also like to mention two conferences this year where I'll be talking and where we can also meet. On the 4th of June, I'll be doing a keynote at craft conference in Budapest, Hungary. This conference is one of the very best ones in Europe focused on software craftsmanship and one where I'm a returning speaker. For more details, check out craft--comf.com.
17:46 · And on the 15th and 16th of September, I'll be doing a keynote at ldx3 in New York. This is the festival for modern engineering leadership. Last year, I was at LDX3 in London. And so, I'm excited to be back this time in New York. For more details about this conference, head to leadub.com.
18:03 · If you'll be at either of them, I'll see you there. And with this, let's get back to Rust and to Alice. I wanted to ask why Rust is getting so popular, but I think we're starting to answer this question, right?
Garbage collection tradeoffs
18:15 · I think where Rust is really unique is in the combination of things. So on one hand, it it doesn't have a garbage collector and it it's usable in low-level context like the Linux kernel or firmware or whatever.
18:30 · Why is it a good or a bad thing to have a garbage collector? Java has a garbage collector. C# has a garbage collector.
18:36 · So, a garbage collector says once you've done using your objects, there's going to be a little piece of code that checks all of your co your objects and says this is not used anymore and then it cleans it up. Whereas in languages like Rust or C++, the variable is cleaned up at the end of the scope when it goes out of scope. And in the other one, they have to detect afterwards. And this kind of little piece of code that runs every so often to check all your objects. For embedded use cases, this might simply be not possible or unacceptable.
19:08 · The the performance overhead the the fact that you cannot you will not be able to control the memory as much.
19:12 · Even for back end, it can be a problem because if you have a request incoming right when it checks all of your objects like you have some sort of latency spike where you it takes much longer to to reply. So that's one of the reasons it can be helpful in in back end as well.
19:27 · For someone who has not yet seen Rust code, how would you describe it? how it compares to for example TypeScript.
19:34 · I mean in many cases Rust code is similar to many of the other languages. It has braces. It's not like Python where you know you use indentation. It has braces. It has semicolons and so on. So so it's pretty easy to read even if you don't if if if you know some similar languages you can look at it and you can get a rough idea.
19:53 · Yeah, I think I think you'll figure it out. It's not that it's not that foreign. Rust has a a learning curve on the side of the more difficult languages to learn. What do you see devs typically struggle with who are new to Rust and and what are the things that just makes it click for them? I think the most tricky thing people run into is how should you put together your data structures? I mean for your code you can mostly do the same things as you can in other languages. I mean that's not entirely true but the big thing is really the data structures. So if you have I don't know what would be a good example let's say some sort okay let's just say you have a object for a book and it has an array of pages and then you have an object for each page if I was writing typescript I would probably have a book field in the page that references the book and then the book of course also references the page right so it's cyclic so the book goes to the page goes back to the book and in Rust you usually have to design your objects so that they're not cyclic. It has to be a tree or a deck if you know what that mean.
21:00 · Yeah.
21:00 · Yeah. The di as graph. So if you go out and actually start to use reference counting, it becomes possible to have cyclic objects with some cabbage like you mentioned. But what people end up doing when they're langu learning the language is that they try to make cyclic objects without anything like that. and then it just doesn't work and they're going to run into all sorts of problems in their code while they're constructing. They're maybe creating their thing and then they get a compiler error and then they try to change the code and they get a compiler error again. But the the thing that a lot of people struggle with is that they keep trying to change the code when the solution is to change the strct.
21:43 · I can see how that's uh frustrating.
Ownership, references, and borrowing
21:46 · Rust has another model that might be new for especially for for people coming from language types is Rust's ownership model. What what is the ownership model? The ownership is just the idea that if you have a variable containing some object then the object is only there. It's kind of exclusive.
22:06 · So for example, if you have a variable let A equals your string or whatever and then you do let B equal to A, then this is a move. And so this means that using A afterwards is now a compiler error because its contents have been moved away. And of course this is important when we don't have a garbage collector because then when B goes out of scope, it has to clean up the string. Now if A was also valid then when they both went out of scope it would clean up the string twice which is not legal. In most other languages the garbage collector takes care of this. A and B just go out of scope do nothing and then later it cleans up the string. But here we actually have to do it when we go out of scope. And the ownership model allows you to move from A to B and then A becomes inactive or unusable and does not get cleaned up because it doesn't have a value to clean up anymore.
23:01 · And how does reference counting relate to all all of this?
23:04 · In Rust, we have a bunch of different types which are essentially different kind of pointers. And one of the pointers is it's called arc. It's and it's reference counter. And so the idea is you have some object somewhere and then you have an arc to the object and what you can do is you can call clone on the arc and this increments a counter and now you have two arcs to the same underlying memory. So the object might be really big but you have two arcs that share the same memory and when they go out of scope the counter is just decremented and when the counter reaches zero the object is cleaned up. So this is a way of saying this object actually needs to be in multiple places. There's no one place that owns it. And so this way you can use a counter to know how many owners are there. And then when the last owner goes away, it gets cleaned up.
23:56 · And on Rust also is another thing that was new to me, the borrow checker.
24:00 · So another pointer type we have in Rust is called the reference. And a reference is basically all it is is that it it's a pointer to the object and that's it. So we do no checking at runtime. So of course this means at compile time we have to make sure that the last use of the reference has to be before the object goes out of scope. And creating a reference is called borrowing. Right? The owner owns the value. And now we have a borrow of the value. the the reference and the borrow checkout checks that the reference is like the last use of the reference is before the object goes out of scope. So if you have say a let's say you have an immutable reference you're reading then if you change the object let's say it's in vector. So I have a mut an immutable reference I'm reading element five and then I change the vector I call clear.
25:00 · Then the borrow checker also ensures that you can't use the reference to object five afterwards.
25:06 · Yes.
25:06 · Because now it has been as soon as you change it it it went out of scope. It's it's now cleared. Right. Yeah, the the the way it works is that if there are mutable borrows, it ensures that the mutable borrow uh ends or starts before or after the previous borrows have ended basically.
25:28 · So they don't overlap. So you can only have one writer at a time or you can have any number of readers. It checks that on the scope like in a function.
25:37 · This concept is is is new to me as someone who's not used ROS for sure.
25:41 · It's very interesting. One thing I've read is on forums people complaining about fighting so-called fighting the borrow checker. What can make it challenging? When you write a some code that uses the object in a way that doesn't follow the rules I mentioned, that's a compiler error. And fighting the borrow checker is when you, you know, can't get out of those compiler errors. I guess I think a lot of the time this has to do with the strruct.
26:08 · One common mistake I see is that if you have a strct with a reference in it, Rust kind of assumes that it can check the scope of that reference like by just looking at a single function. But if you have your strct and you're passing it over functions that it might not be possible to make that analysis and so you just get a compiler error. And so in this case the solution is maybe to use a different pointer type. For example, the reference counterp pointer type often solves this kind of bug, right? So the solution is again to change the data structure.
26:41 · Yeah, I'm I'm starting to understand why you mentioned that data structures were a a a place of learning coming from other languages.
26:49 · Often times the solution seems to be just think about your data structures, understand them and do it in in the Rust way, right? We talked about memory safety, but there is a keyword in in Rust called unsafe. What does this do and in what cases do people typically use? When does it make sense to to to use and why does it even exist? That's just a naive question from me.
Unsafe in Rust
27:09 · So let let me begin with the what and let's take the why afterwards. So the what is unsafe is the escape hatch essentially. So I explained before how there are certain bugs where if your program has one of those bugs that's usually a security vulnerability. What Rust ensures is that if you have no use of unsafe, then no matter how stupid your code is, you will never have one of those bugs. Now, if you do use unsafe, then there are still some guarantees, but it's a bit weaker because each unsafe operation that you can perform has a list of rules. And if you violate these rules, then you might end up with one of these bad bugs. But of course, if you don't, then you it's okay. And it's interesting to point out here that unsafe does not disable the borrow checker or anything like that. It just gives you a few more operations you can perform that are not safe in general.
28:08 · And so you have to check yourself. Yeah, this is actually okay in this particular case to I mean let's take the vector example again. Normally when you index index five it will say oh let me check the length. So if the length is at least six yeah because it starts zero then it's okay otherwise you get a crash but maybe you're writing some super high performance code and you want to avoid this if you want to avoid this check or if you're in a loop every time. So so un unsafe will just avoid checks. uh may may that be is it runtime or compile time checks or both?
28:48 · What I would say here is that vector has two different ways of getting a value. There's the you know normal bracket operator we'll use for other languages which will do the check and crash your program if you get it wrong. But there's another function called get unchecked.
29:05 · And so if you do write veget get unchecked five then it will give you element five without checking the length. And this function like in the function signature it says unsafe fn. That's the function signature.
29:19 · And so you can only call this function from an unsafe block.
29:22 · Got it.
29:23 · And so all that the unsafe block let you do is call functions marked unsafe.
29:28 · And then in practice for sensible use cases of unsafe is it usually to do with high performance code typically or have you se like what what cases have you seen which are like legitimate? This is a great use case for for unsafe.
29:42 · So usually it will never show up in say a backend server. You would have zero uses of unsafe there. Generally when unsafe is used, it's to add a new feature to the language. Let's say the language didn't have a vector. The language still has a function to allocate memory, a function to free memory, a function to read at this pointer address. Then you could write strct ve the pointer is here and this is a raw pointer so it's unsafe to use. The length is this. The capacity is this. So I might and then you can write your little API. Of course the fields are private so you can't access them from outside the module. I mean if I could do ve.length equal 20 just access the field that would be pretty bad. And so you can write your own vector API and using field privacy and good API design, you can add a vector to the language if it doesn't already exist. and unsafe if encapsulated properly in this kind of API that doesn't permit you to mess it up. Then you can have a safe vector that you can use from safe code and no matter how stupid the thing you do with that vector is it's not going to do something bad.
31:00 · It's just going to crash or whatever check the length properly. As long as you design your API right, you can add new language features by using unsafe. The other classic example would be to call into a C library. You can write a library that calls into C and you can even have an safe API and then you can enforce that you only call the C API in the right way.
31:20 · Can we talk about the the Rust ecosystem, the broader ecosystem and when it comes to this the first thing that people come across include myself is the crate uh ecosystem. Rust is package man manager. What is crate and h how does it compare to other package managers in like places like npm or or pip and in python? So create is just the Rust word for a package. So it's just a package right of of your code.
Crates and Cargo
31:46 · So the way it works is that Rust has a tool called cargo and cargo is kind of um it does a lot of stuff. It's it's actually pretty neat tool that's kind of all in one. So it will do it's both what you use to run your code. You do cargo run and it will compile your code. You have this thing called cargo tunnel where you specify your dependencies and other information about your crate. And when you do cargo run, it will download the dependencies to something called a registry, which is basically just a directory with all the crates you've ever downloaded.
32:17 · Yeah.
32:18 · And you can also do cargo test. It will run all your tests. Cargo doc, it will generate your doc. Cargo test will also run your doc tests. There's even benchmarks and examples. So, so it's it it it does all the whole suite, so to say. So you don't need different tools to fetch dependencies and run your code and generate docs. I think the biggest difference from something like pip would be that they're not installed to your machine, right?
32:46 · With pip you either install things globally or you have these virtual environments. So so with cargo it's all local. You just run cargo run and it's only local to this. The only thing that's shared is this registry which is just to avoid downloading the same thing twice. You told me a funny anecdote about Linux tour worlds and his his reaction or what he told you about Rust and car and cargo the first time I was at the Linux plumbers conference at the very end. So Linux wasn't anywhere to to be seen throughout the entire conference but the at the very end of the social event he showed up and I went over to say hi. I mentioned I worked on Rust and immediately he started telling me about how he didn't like cargo and so the reason for that is that cargo downloads code from the internet and runs it like any package manager.
33:38 · Yep.
33:38 · And he doesn't want any any code on his computer to do that other than his uh distributions package manager. He doesn't trust anybody else to do that. In the node world, we're seeing more problems with vulnerabilities being injected into packages. Um, just a bad actor overtaking packages, you know, that they they put in whatever code might be from crypto, which is I guess the better part to security vulnerabilities.
34:05 · Does Cargo have this problem as well?
34:08 · Just like any package manager that is on the internet?
34:12 · I mean, in principle, yeah. I mean, ultimately, if you somehow got a hold of, you know, my keys to upload new Tokyo versions, you could upload a malicious one. I don't think we've had much problems with it compared to something like npm, but I don't really know if we I mean, it's a hard problem, right?
34:33 · It's a hard problem. somebody can impersonate the maintainer of a library then I mean I think they do do stuff like delete uh people scratting malicious crates and stuff like that.
34:43 · Yeah. Where where do you see the Rust ecosystem being the most mature and the less m and the least mature right now?
34:50 · My opinion is that the least mature area is front end.
34:55 · There have been some attempts to compile Rust to Web Assembly and then run it on the web as a front end as a replacement for TypeScript. But if I was writing a web server, I would totally use Rust for the back end and TypeScript for the front end. I would not really go the web assembly route. On the other hand, for for back end, I actually think it's a pretty great fit. And there's also stuff like command line tools. I think it's really really great fit for that.
35:29 · And then of course we have we are expanding into Linux and we are also expanding into a lot of embedded projects and there are even projects that are like C code bases that are starting to say hey it's went into Linux maybe we should start using it too.
35:46 · Maybe we should have a memory sake language too. I know that git is talking about it. I know that CPython is talking about it. There are probably others. Qo maybe.
Language design and RFCs
35:55 · Can we talk about how the language is is built? Who builds Rust? What the process for for for doing it? How does it compare to a project like Linux? I know it's not a language, but but it's still a large open source project. I mean, it's really an open source project. I mean, you may know that MOS that it originally originates from Masilla, but it's not a Masilla project anymore at all. So today there's the rust project which has a bunch of different teams.
36:24 · There's the language team there's the which you know deals with the language.
36:28 · There's the library API team which deals which decides on the API of the standard library. And these teams kind of govern it. So it's the people who are these teams that run the language. And one interesting difference compared to some other popular languages like like Python or projects like like Linux is they have a benevolent dictator for life and Rust does not. How is this working and and how are decisions made when especially when they're contagious or when it could help for you know some just someone to just make a decision? Honestly, if the team doesn't sign off on it, it doesn't happen. I mean if something is really contentious contentious or really big it will probably end up being discussed at a conference for example. So there's a conference called Rust week where they have an event called the Rust all hands where they're basically taking all of the Rust developers and putting them in one place. So if there was a problem like that they would probably discuss it there.
37:27 · And how are these teams structured? So like you said there's a compiler language library and and dev tools at at the very least. How do they define the boundaries? Is it just a team kind of roughly defining them and then you just kind of agree? Because as I'm thinking at a corporate level like as as a company would run there's typically teams are are founded often top down leadership doesn't mandate this will be this team that team there's bottoms up happening but often times it's top down just because it's easier but here as I understand there is no top down it's it's people self-organizing yeah really when it comes to the design of the language the teams are really that's really it teams have been pretty good as far as I've seen at delegating So I've seen them sometimes say we think this is a lips API decision and we will go with whatever lips API decides.
38:18 · One thing that is interesting to talk about is the RFC process. So there's requests for for process. RFC request for comment is the way the ROS project makes big decisions but you know big ones is important to say. Basically the way it works is let's say you have a language feature that's kind of big. For example, I had one called derive smart pointer which basically makes some types in the standard library less special. So I mean sometimes people begin implementation already but the idea is that you write the RFC first.
38:54 · By the way that that's no different inside of like at U we had RFC process and same thing usually you follow it but sometimes you kind of start to build it and then just the idea is you write the the RC first. Yeah, I I know. It's just funny how it happens everywhere where there's engineers. I think it's is to build.
39:11 · You write this doc and it has a template and I think it's actually a pretty good template. The idea is the first section is I think motivation. Well, the first one is summary, but the first important one is motivation. So, you explain why this feature and then I think they have two really interesting sections. They have the guide level explanation and the reference level explanation. And what this is is that in the guide level explanation, you explain your feature as if you were writing a guide as if the feature already existed. And in the reference level explanation, you explain your feature again as if it already existed, but as if it would be in the language reference instead of a tutorial. This is really interesting.
39:54 · You know, Amazon does just related to this. When they ask people to build a feature, they have a press release. like imagine if you announced this. I just love how like both this and and what you're saying it forces you to just think from a very different perspective right from from how people will use this feature and then and then other sections there's uh I think you mentioned uh rationale alternatives prior art future possibilities yeah so that you know you get to explain so rational and alternatives I think is a pretty important section because you get to answer all of the questions before they get asked and you get to explain why did you pick this design and then some other design and I think that's usually a pretty big part of an IFC and then of course it's good to look at what did C++ do and what can we do in the future as well and once you have the RFC you send it out what happens then well somehow people you get people to look at it there is an RFC's repository and you can open a pull request there with your markdown document with RFC and people can discuss it you might also write up your doc somewhere else then we call it a prefc if it's not in the IFC directory but it's kind of the same thing. Let's say that the language team like it's a language team IFC and they're happy with it. So I said the IFC was how the language makes big decisions but then they actually use a process that comes up for all decisions essentially called the final comment period. And so the idea is they tell the bot to please start the approval process and then it will create a comment on GitHub with a checkbox for every team member. And then team members check their own checkbox. And once everybody from the team except for at most two have checked their checkbox then the final comment period will begin which is 2 weeks.
41:52 · Oh. So when it's only two check boxes left then two weeks starts. What what's the rational for this? This is very interesting. It is a historic fact more more like I mean you know let's say that everybody checks that box immediately and then the two other people didn't get to a chance to look yet. So it's to make sure that everybody has a chance to see it.
42:15 · Yes, I understand. And also to keep things moving at a sensible pace. So there's another part of the process which is concerns. Team members can file a concern which basically pauses the process until it's resolved and then once the two weeks pass it's accepted.
42:32 · So that's essentially how the team makes decision and this is not just IFC's. If you have I don't know a pull request changing something in the reference then the language team might say oh we need to make sure that everybody is on board with this change. they'll tell the bot to start an FCP on this random PR or some random issue or what wherever it might be. The same process applies and then once let's say it says lang language feature okay the RFC is accepted we know what we're going to build then the feature is just built you start to open your pull requests and get into a language pretty much but you you put the feature behind a feature flag in Rust we have this thing called nightly features which basically means that you can't use it normally but if you use the nightly build of the compiler You can once you have your you know your feature I mean you might begin but you know once you have your accepted RFC you start submitting pull requests uh you implement your feature it gets merged and people start using it experimentally on a nightly build. Yeah. And then at some point you might say okay I think this feature is ready and this is kind of a recent invention but we have come up with this idea of a stabilization report in the pull request that change that removes the feature flag. We write up a little report saying for example how it's been used and explaining like oh here are the dangerous edge cases here are the tests for each of the dangerous ed. So that kind of stuff and then they use the FCP process again to agree to stabilize it and then it's merged and so now you have a feature without a feature flag.
Building new features
44:18 · Yeah.
44:18 · In the main branch.
44:20 · Yes.
44:21 · Between 0 days and 6 weeks from that there will be a beta release of the compiler and it will be and it will be in it.
44:28 · Yeah. It will be in it and then 6 weeks later the beta release becomes the next stable release.
44:35 · Okay.
44:35 · So, so like Rust has a strict six week release cycle unlike with Linux. Well, Linux also has a a rough emo, but here you just take the state stable bash. You're not doing any special.
44:46 · Yeah. And so we don't really have a concept of beta features that are only available a available on beta.
44:52 · You you have the nightly pretty much nightly or people can get on to the latest branch which is not yet cut.
44:58 · Yeah.
44:58 · The main purpose of beta is to test like you get six week to test the stable release. So hopefully if there's any problems we can fix them before it goes out into a stable release.
45:09 · And you mentioned that RFC's are only for large decisions that need to be made and you've you haven't written that many either just because they're large. What about smaller decisions that you also want to get consensus on? So here there are a few different ways to do it. So let's say you want to add a new function in the standard library or a new type.
45:29 · Then the library team actually has this thing called an ACP an API change proposal which is basically just an issue on GitHub. So you open your issue, you describe your API, describe what the interface is and explain why you should have it. uh and it's much smaller than an RFC would be and then the library team can say this is okay with us and then you can implement it on unstable or nightly and then later you can then stabilize it and similarly the compiler team has it's kind of what major change proposals you you need you need to say how how it's called MCPs what is going to confuse a lot of people outside of we're not inside of Rust yeah it is what it is which for the small smaller features Yeah, even though it's called major, right? It's it's big enough that they need you to put up an issue, but not big enough to need an RFC. And for the compiler team, the classic example would be a new compiler flag. Now, Rust has new new builds every 6 weeks, right? But it also has additions. There's one in 2015, 2018, 2021, 2024.
Editions vs. versions
46:38 · What are additions? So the thing about additions that is different from versions is that if you're using version 1.90 of the compiler, everything is using that version. But different crates can use different editions. And so I might have a crate using the 2025 edition of the language and I can keep using that forever because Rust has a really really high backwards compatibility guarantee. So you write all of your code and the guarantee is that it's going to keep working forever.
47:14 · That's the idea anyway. Additions are basically the way that Rust makes breaking changes without breaking people because they might change the syntax of the language. For example, async a weight was added in addition and so code using the old addition can't use async await there. You could have a variable called async if you want. lead as evil 5 and then in the new edition you can't but you can still mix and match code written in different editions so they work together.
47:46 · Oh interesting.
47:47 · So I might have a library written in the 2021 edition and you can write your library in 2024 edition or your binary project and then you can still use my library. Why do you think Russ decided on additions which feels a bit more confusing to me as a developer as opposed to versions? Like when I look at Java's versions or PHP's versions or even Ruby on Rails versions, you know, there's always a m there's a major number. And I might be wrong, but it sounds to me that an addition is almost an equivalent of a of a major version at other languages. Is that putting it correctly or am I missing some details here? Really the big idea is to say we want the old code to continue working but we still want to change the language. And so the the difference from other kinds of versions of languages I mean Python 2 version Python 3 comes to mind is that you can totally mix and match in any way you want.
48:45 · You are a language team adviser uh or you recently became one. What is a language team adviser and and what do you do as part of this role? We have of course the language team which you know they meet they have meetings every week and they do a lot of stuff and so on and so forth. The language team advisor role is a way to be part of the language team light in some sense. So you're someone that they've said, "Okay, we trust this person's opinion, but you don't necessarily go to every single meeting." Like there's no checkbox for you on an FCP, though you can still file a concern. So yeah, it's really a way to participate in or help the language team without full membership. And full membership might entail a lot of things, right? For example, for me, it would entail going to meetings every Wednesday evening instead of like when I would normally have dinner. from your observation how what is the corporate influence like on on Rust in the sense that when I talk with with Greg KH he was telling me that in practice about 80% of the frequent Linux contributors are typically employed by a company which sees value in um adding their their features or or maintaining their features in in Linux which which turns out to be like a nice kind of I guess symbiosis in some ways for for Rust the the people working on it? Are they usually paid by their employers like you are? I I'm I'm sure there's people who are just every now and then contributing, but people who are spending more time on this. Do you see a pattern of corporations actually supporting this or is there a foundation that also supports people to work on this full-time or close to full-time? I would say here that the Linux kernel is truly unique in this particular aspect in that they have thousands of contributors doing it on work time. I actually think that the Rust project is doing pretty well at having people do it and get paid for it like by their employer, but it's nowhere near the to the same extent as as the Linux kernel. The Rust project also has a few other interesting things.
Getting paid to work on Rust
50:58 · So the Rust Foundation have some grants that allow I mean for example if you're a student and you want to work on something you might be able to get a a grant some money from the Rust Foundation. I think that's a super cool thing that the foundation is doing especially I think getting people involved students involved or or you know the people who would find it harder or more daunting to to get involved in a project like this. Speaking of which, for a software engineer who would be interested to contribute to the Rust ecosystem, what would your suggestion be in terms of both resources or or ways to get started? I mean, obviously you can go to the issue tracker and look for issues that interest you. I think that often it's I mean, the best way to contribute something is if you have something you want to change about it. I think that's often a pretty good starting point. So if you wanted to contribute to the Rust language itself, Rust does a lot of its stuff on uh something called Sulip which is a chat server where people talk and you you could go there and talk to people. So the Rust for Linux project has a pretty nice contributing page.
Contributing to Rust
52:12 · So, rustforlin.com and there's a contributing page there with for one some of there are a few different um drivers you might be able to contribute to and they have links to the issue tracker there. Another thing that's really cool is you can contribute to the Linux kernel without contributing to the Linux kernel, right? because you might take a Rust language feature we need and implement it and now you've contributed to the Linux kernel indirectly or maybe you want to work on the Rust in GCC project and help move that forward and so I think there are a lot of different projects other than the Linux kernel that would help Rust in the Linux kernel a lot to contribute there. Can we talk a little bit about Rust in the Linux kernel on on how that has evolved?
Rust in the Linux kernel
53:05 · How you got involved in in writing Rust in in the Linux kernel? And have you seen the approach of of especially Linux kernel devs uh change or soften towards Rust? Because I I know it was a very heated topic earlier on. Linux has this conference called Linux plumbers which you know as part of my work on rest for the Linux kernel I've been going to for the past few years and one thing I think has been really obvious is every time I've gone you can say things have totally changed since last year. I've experienced this like four times in a row.
53:41 · That's kind of pretty cool.
53:43 · Yeah.
53:43 · I think there's really a lot of stuff that's happening. I mean, one year I might go and people think, "Oh, that's some nice little thing you have there, right?" And then the next year now they have Rust code in their subsystem. It it kind of keeps going.
53:58 · The the the most recent Linux plumbers from December 2025 was there. The big news were that at the Linux kernel maintainer summit, we agreed that Rust is no longer experimental in the kernel. That was really big for us compared to the previous year.
54:18 · Yeah.
54:18 · So, so now that means that it's official like like Rust has the same status as C, which is the lang the language that the kernel is written in, right?
54:28 · I guess. Well, not the same status, but but not not being experimental. It it clearly marks that it's more stable.
54:35 · Experimental sounds like it's it's not as stable. Basically, we've proved this is going to work. I mean based on what you've laid out with just with memory safety and of course being able to use unsafe when you need to that already like shows a lot of I guess promise plus there's there's some regulation as well right there's I think the US Department of Defense passed some regulations saying that they will not allow their agencies to use non-memory safe languages for the concern of security vulnerability. ilities and you know this would practically mean that Rust and other memory safe languages could be used but C, C++ or maybe one day systems that are built in this one might see larger scrutiny. Yeah, there have been different stuff like that from multiple different governments, right?
55:27 · These governments are saying, "You guys are using C or C++ in this project and it's causing an unacceptable amount of memory safety vulnerabilities that simply don't happen if you just don't. So what if you didn't do that?" in in this conversation, we managed to go for a very long time without even mentioning AI once, which uh given where the world right now is and and how these tools are are spreading. That that that's pretty impressive.
AI use cases for Rust
55:57 · I I was interested, do you use any of the AI tools for your day-to-day work building Tokyo, uh contribute contributing to Rust projects? Have you found a need for them at all? So, I have uh been trying to use them. Honestly, I'm still learning how to use these tools, but I have used them. I I quite like the Gemini command line interface.
56:22 · I think it's uh pretty neat. These tools are pretty proficient at outputting code or, you know, giving a prompt and it it putting out code. But in your day-to-day work, how how much code do you actually type as opposed to reviewing code, thinking about what to do, making a plan? Does it sound like as we're talking about the RFC, for example, the RFC process, my sense was most of that was thinking about what to build, getting a consensus, making sure it's right, and it almost sounded like the actual code itself would be the I don't know, lesser. Of course, you know, there's a lot of work involved, but you see what I mean. the the actual time spent will probably be less than everything else around it.
57:02 · I think it's still an important part of the the process. I mean, a lot of people talk that about that when you use AI, it's really important to write tests and that kind of thing because then the AI is actually able to tell if it did it right. I explained this story from before about how I was refactoring something. I just told did what the compiler told me. I think the same kind of principle applies with agents in that they can talk to the compiler. It will tell them what to fix. So I guess this could be a case. We we'll see. But Rust could be a pretty promising candidate for to use for agents because they can get more feedback and it's just hard it's harder to to ship certain type of bugs or maybe impossible to have certain type of bugs. Yeah, I think there is a aspect of that for places like the Linux kernel where I mean correctness is extremely important, reliability is very important. There's already multiple layers of human review which which is is in place. Do you see potential for AI generated code to prove helpful somewhere like actually truly helpful like again assuming that these things can generate you know as per specification or as per what what you need or or could could this be a place where it might be one of the places where we might not need that much because it it already feels a kernel to me it feels like a place where there's a lot of people coming in contributing and and there's there's a reason that change can propagate slower when When I was at the Linux kernel maintainer summit, one of the topics that came up was using AI for code review and there were some people there who had for example set up bots that would say when you send an email to the mailing list, it would feed it into an AI agent which would leave a review.
58:50 · That kind of stuff. And for example, Linos and others were talking about how these reviews were actually really impressive for kernel code. At least what's being discussed in the kernel community, that kind of use case seems like something people are excited about.
59:11 · And it sounds like this would not be a replacement obviously, but just one more way to get feedback, maybe doing it quicker and and just an additional safety net, if you will, right? I mean, that's the thing with memory safety, right? You make some sort of trivial mistake. It it's not some complex thing.
59:29 · It's just you make some trivial mistake and there's a bajillion places you could make it. So even if even if you're a really good programmer and only make it.1% of the time or whatever, it's still going to happen, right?
59:43 · And so if the AI can catch a lot of those, then you know that's pretty good.
59:47 · Well, and I guess some creative use cases could be for example there tests are are a big thing, but AI might be able to help figure out if there's edge cases missing again if if the language itself does not support it. There could be what I understand and there's there could be a lot of potential to add more safety nets that do not exist today.
1:00:06 · I think that's right. I like this approach. So this is this is one of the first conversations I'm having where we're talking about AI where we're talking about how to increase quality wi with AI because in again in startups or whoever is building there's a lot of talk about how it can make it faster faster restoration and there's usually a quality trade-off that we usually do not talk about but I I love that we're actually talking about the things that can increase quality or or at least keep it at the same level.
1:00:31 · I mean I have actually written a few patches with AI myself. It worked okay some I mean it definitely required my review before sending it out to the list. really did sometimes. But you know, another place I remember using it was I had this commit that ma made binder faster in some way and one of the review comments I had received was, "Hey, can you run a benchmark?" I said, "Oh my god, now I have to write a benchmark." But then I got the AI to benchmark it for me. And what it did, what it actually went and found an existing benchmark and modified it a little bit in a quite clever way. and then it ran it for me and then it wrote a Python script to analyze the text file with the numbers and presented it in a nice table.
1:01:17 · So it sounds like this was toil work that you could have done but yeah of course you might have not done as good of a work because you might have not found that utility you might not not figure out to do that.
1:01:27 · I mean at least in that instance it saved me time if nothing else. for a senior engineer who would like to learn Rust or you know just build with it what would your suggestion to be to get started of of course for AI might be one of the things which makes it a lot easier just to to get started but if if you would like to learn what do you recommend for for people who actually want to become proficient at the language. So for one, I would say that the Rust book which is so it's on the official Rust website is actually really good, right? And you don't have to it's freely available online. Rust has this idea of calling tutorials for books for some even though they're online which is kind of funny. But anyways, I think the Rust book is really good. Other than that, I think honestly the way you learn a language is you have you have a project. You you implement some sort of project with it. I mean maybe some sort of web server or something.
Learning Rust
1:02:21 · And one more thing on AI. So it's easy to get started with languages these days because you can just prompt the AI to like write this or that. In the case of Rust, do you think that could create a false sense of understanding especially we talked about the importance of understanding structures of understanding compiler issues which are there for a reason. You know tools could just like generate through this and create code that that runs. Yeah, it it's totally a danger. I recently tried using it for something where I guess I was not writing Rust code. I was writing make files. I wanted to add some support in the Linux kernel build system for for some feature and it went into the make file and added the Rust flags necessary to do it. But then I looked at the code and it was it had added the necessary build flags but to the C side it was passing a few more flags which were not required per se but they were there for a reason and it had just ignored them.
1:03:15 · Any human looking at this would be like why did you add Rust versions of all the flags? What is your favorite Rust feature if there is one? I'm currently working on a new feature which I think is really exciting. It's something that that we ran into in the Linux kernel where we needed in place initialization, the ability to construct values while knowing where they're being constructed so they don't get moved afterwards. I'm pretty excited about our work to put that into the language, but it's very much ongoing. I I think that's pretty cool stuff. And as as closing, what are what are books you would recommend that you've enjoyed and and why? I already mentioned the the official Rust book. I also think that so John Ganget has another book that I think is really good and this book is kind of aimed at the intermediate Rust developer. The the Rust developer who has gotten some Rust experience but wants to go further. I think that's also a pretty good book for for that audience. Some other resources which are not books but which I think are also really good is there's this thing called rustlings and the idea is that they give you some unfinished rust code and your task is to finish it.
Book recommendation
1:04:32 · Oh, nice.
1:04:33 · And I think that's a pretty cool way to learn the language as well.
1:04:37 · Awesome, Alice. Thank you. Thank you.
1:04:39 · Thank you so much for this. This was really interesting and I've learned a bunch.
1:04:44 · Thank you for having me on. I hope you enjoyed getting into the details about Russ as much as I did. One thing I appreciated talking about is how deliberately Rust seems to have been designed around the question, what mistakes do programmers actually keep making and how do we eliminate those mistakes. A few examples include memory safety, not having a null, and the errors being values that you cannot ignore. The documentation example is also an interesting one. Documentation tests or doc tests are automatically run as tests. If your example breaks, then your build breaks. This is the first time I've heard this clever idea implemented in a way that's easy to use in a language. Finally, the question on why Rust is becoming so popular might be because it's in a category that did not exist before, a language that's reliable and also performant at the same time.
1:05:28 · This makes it usable for high performance computing cases like kernel work while also being a good alternative for higher level back-end languages. Do check out the show notes for related the pragmatic engineer deep dives that go even deeper into back-end topics and stories of building languages like Cotlin and Swift. If you've enjoyed this podcast, please do subscribe on your favorite podcast platform and on YouTube. A special thank you if you also leave a rating on the show.