blob: b4bbb34cd51a481cd4172232d8282840a0171242 [file] [log] [blame] [view]
Nigel Tao187a4792023-09-28 22:30:441# What’s Up With Pointers
2
3This is a transcript of [What's Up With
4That](https://www.youtube.com/playlist?list=PL9ioqAuyl6ULIdZQys3fwRxi3G3ns39Hq)
5Episode 1, a 2022 video discussion between [Sharon ([email protected])
6and Dana ([email protected])](https://www.youtube.com/watch?v=MpwbWSEDfjM).
7
8The transcript was automatically generated by speech-to-text software. It may
9contain minor errors.
10
11---
12
13Welcome to the first episode of What’s Up With That, all about pointers! Our
14special guest is C++ expert Dana. This talk covers smart pointer types we have
15in Chrome, how to use them, and what can go wrong.
16
17Notes:
Daniel Verkamp3527a272023-10-03 22:42:5818
Nigel Tao187a4792023-09-28 22:30:4419- https://docs.google.com/document/d/1VRevv8JhlP4I8fIlvf87IrW2IRjE0PbkSfIcI6-UbJo/edit
20
21Links:
Daniel Verkamp3527a272023-10-03 22:42:5822
23- [Life of a Vulnerability]
24- [MiraclePtr]
Nigel Tao187a4792023-09-28 22:30:4425
26---
27
280:00 SHARON: Hi, everyone, and welcome to the first installment of "What's Up
29With That", the series that demystifies all things Chrome. I'm your host,
30Sharon, and today's inaugural episode will be all about pointers. There are so
31many types of types - which one should I use? What can possibly go wrong? Our
32guest today is Dana, who is one of our Base and C++ OWNERS and is currently
33working on introducing Rust to Chromium. Previously, she was part of bringing
Daniel Verkamp3527a272023-10-03 22:42:5834C++11 support to the Android NDK and then to Chrome. Today, she'll be telling
35us what's up with pointers. Welcome, Dana!
Nigel Tao187a4792023-09-28 22:30:4436
3700:31 DANA: Thank you, Sharon. It's super exciting to be here. Thank you for
38letting me be on your podcast thingy.
39
4000:36 SHARON: Yeah, thanks for being the first episode. So let's just jump
41right in. So when you use pointers wrong, what can go wrong? What are the
42problems? What can happen?
43
4400:48 DANA: So pointers are a big cause in security problems for Chrome, and
45that's what we mostly think about when things go wrong with pointers. So you
46have a pointer to some thing, like you've pointed to a goat. And then you
47delete the goat, and you allocate some new thing - a cow. And it gets stuck in
48the same spot. Your pointer didn't change. It's still pointing to what it
49thinks is a goat, but there's now a cow there. And so when you go to use that
50pointer, you use something different. And this is a tool that malicious actors
51use to exploit software, like Chrome, in order to gain access to your system,
52your information, et cetera.
53
5401:39 SHARON: And we want to avoid those. So what's that general type of attack
55called?
56
5701:39 DANA: That's a Use-After-Free because you have freed the goat and
58replaced it with a cow. And you're using your pointer, but the thing it pointed
59to was freed. There are other kinds of pointer badness that can happen. If you
60take a pointer and you add to it some number, or you go to an offset off the
61pointer, and you have an array of five things, and you go and read 20, or minus
622, or something, now you're reading out of bounds of that memory allocation.
63And that's not good. these are both memory safety bugs that occur a lot with
64pointers.
65
6602:23 SHARON: Today, we'll be mostly looking at the Use-After-Free kind of
67bugs. We definitely see a lot of those. And if you want to see an example of
Daniel Verkamp3527a272023-10-03 22:42:5868one being used, Dana has previously done a talk called, "[Life of a
69Vulnerability]." It'll be linked below. You can check that out. So that being
Nigel Tao187a4792023-09-28 22:30:4470said, should we ever be using just a regular raw pointer in C++ in Chrome?
71
7202:41 DANA: First of all, let's call them native pointers. You will see them
73called raw pointers a lot in literature and stuff. But later on, we'll see why
74that could be a bit ambiguous in this context. So we'll call them a native
75pointer. So should you use a native pointer? If you don't want to
76Use-After-Free, if you don't want a problem like that, no. However, there is a
77performance implication with using smart pointers, and so the answer is yes.
78The style guide that we have right now takes this pragmatic approach of saying
79you should use raw pointers for giving access to an object. So if you're
80passing them as a function parameter, you can share it as a pointer or a
81reference, which is like a pointer with slightly different rules. But you
82should not store native pointers as fields and objects because that is a place
83where they go wrong a lot. And you should not use a native pointer to express
Daniel Verkamp3527a272023-10-03 22:42:5884ownership. So before C++11, you would just say, this is my pointer, use a
Nigel Tao187a4792023-09-28 22:30:4485comment, say this one is owning it. And then if you wanted to pass the
86ownership, you just pass this native pointer over to something else as an
87argument, and put a comment and say this is passing ownership. And you just
88kind of hope it works out. But then it's very difficult. It requires the
89programmer to understand the whole system to do it correctly. There is no help.
Daniel Verkamp3527a272023-10-03 22:42:5890So in C++11, the type called `std::optional_ptr` - or sorry, `std::unique_ptr` -
91was introduced. And this is expressing unique ownership. That's why it's
Nigel Tao187a4792023-09-28 22:30:4492called `unique_ptr`. And it's just going to hold your pointer, and when it goes
93out of scope, it gets deleted. It can't be copied because it's unique
94ownership. But it can be moved around. And so if you're going to express
95ownership to an object in the heap, you should use a `unique_ptr`.
96
9704:48 SHARON: That makes sense. And that sounds good. So you mentioned smart
98pointers before. You want to tell us a bit more about what those are? It sounds
99like `unique_ptr` is one of those.
100
10104:55 DANA: Yes, so a smart pointer, which can also be referred to as a
102pointer-like object, perhaps as a subset of them, is a class that holds inside
Daniel Verkamp3527a272023-10-03 22:42:58103of it a pointer and mediates access to it in some way. So `unique_ptr`
Nigel Tao187a4792023-09-28 22:30:44104mediates access by saying I own this pointer, I will delete this pointer when I
105go away, but I'll give you access to it. So you can use the arrow operator or
106the star operator to get at the underlying pointer. And you can construct them
107out of native pointers as well. So that's an example of a smart pointer.
108There's a whole bunch of smart pointers, but that's the general idea. I'm going
109to add something to what a native pointer is, while giving you access to it in
110some way.
111
11205:40 SHARON: That makes sense. That's kind of what our main thing is going to
Daniel Verkamp3527a272023-10-03 22:42:58113be about today because you look around in Chrome, you'll see a lot of these
Nigel Tao187a4792023-09-28 22:30:44114wrapper types. It'll be a `unique_ptr` and then a type. And you'll see so many
115types of these, and talking to other people, myself, I find this all very
116confusing. So we'll cover some of the more common types today. We just talked
117about unique pointers. Next, talk about `absl::optional`. So why don't you tell
118us about that.
119
Daniel Verkamp3527a272023-10-03 22:42:5812006:10 DANA: So that's actually a really good example of a pointer-like object
Nigel Tao187a4792023-09-28 22:30:44121that's not actually holding a pointer, so it's not a smart pointer. But it
122looks like one. So this is this distinction. So `absl::optional`, also known as
123`std::optional`, if you're not working in Chromium, and at some point, we will
124hopefully migrate to it, `std::optional` and `absl::optional` hold an object
125inside of it by value instead of by pointer. This means that the object is held
126in that space allocated for the `optional`. So the size of the `optional` is
127the size of the thing it's holding, plus some space for a presence flag.
128Whereas a `unique_ptr` holds only a pointer. And its size is the size of a
129pointer. And then the actual object lives elsewhere. So that's the difference
130in how you can think about them. But otherwise, they do look quite similar. An
131`optional` is a unique ownership because it's literally holding the object
132inside of it. However, an `optional` is copyable if the object inside is
133copyable, for instance. So it doesn't have quite the same semantics. And it
Daniel Verkamp3527a272023-10-03 22:42:58134doesn't require a heap allocation, the way `unique_ptr` does because it's
Nigel Tao187a4792023-09-28 22:30:44135storing the memory in place. So if you have an `optional` on the stack, the
136object inside is also right there on the stack. That's good or bad, depending
137what you want. If you're worried about your object sizes, not so good. If
138you're worried about the cost of memory allocation and free, good. So this is
139the trade-off between the two.
140
14107:51 SHARON: Can you give any examples of when you might want to use one
142versus the other? Like you mentioned some kind of general trade-offs, but any
143specific examples? Because I've definitely seen use cases where `unique_ptr` is
Daniel Verkamp3527a272023-10-03 22:42:58144used when maybe an `optional` makes more sense or vice versa. Maybe it's just
Nigel Tao187a4792023-09-28 22:30:44145because someone didn't know about it or it was chosen that way. Do you have any
146specific examples?
147
14808:14 DANA: So one place where you might use a `unique_ptr`, even though
149`optional` is maybe the better choice, is because of forward declarations. So
150because an `optional` holds the type inside of it, it needs to know the type
151size, which means it needs to know the full declaration of that type, or the
152whole definition of that type. And a `unique_ptr` doesn't because it's just
153holding a pointer, so it only needs to know the size of a pointer. And so if
154you have a header file, and you don't want to include another header file, and
155you just want to forward declare the types, you can't stick an optional of that
156type right there because you don't know how big it's supposed to be. So that
157might be a case where it's maybe not the right choice, but for other
158constraining reasons, you choose to use a `unique_ptr` here. And you pay the
159cost of a heap allocation and free as a result. But when would you use an
160`optional`? So `optional` is fantastic for returning a value sometimes. I want
161to do this thing, and I want to give you back a result, but I might fail. Or
162sometimes there's no value to give you back. Typically, before C++ - what are
163we on now, was it came in 14? I'm going to say it wrong. That's OK. Before we
164had `absl::optional`, you would have to do different tricks. So you would pass
165in a native pointer as a parameter and return a bool as the return value to say
166did I populate the pointer. And yes, that works. But it's easy to mess it up.
167It also generates less optimal code. Pointers cause the optimizer to have
168troubles. And it doesn't express as nicely what your intention is. A return,
169this thing, sometimes. And so in place of using this pointer plus bool, you can
170put that into a single type, return an `optional`. Similar for holding
Daniel Verkamp3527a272023-10-03 22:42:58171something as a field, where you want it to be held inline in your class, but
Nigel Tao187a4792023-09-28 22:30:44172you don't always have it present, you can do that with an `optional` now, where
173you would have probably used a pointer before. Or a `union` or something, but
174that gets even more tricky. And then another place you might use it as a
175function argument. However, that's usually not the right choice for a function
176argument. Why? Because the `optional` holds the value inside of it.
177Constructing an `optional` requires constructing the whole object inside of it.
178And so that's not free. It can be arbitrarily expensive, depending on what your
179type is. And if your caller to your function doesn't have already an
180`optional`, they have to go and construct it to pass it to you. And that's a
181copy or move of that inner type. So generally, if you're going to receive a
182parameter, maybe sometimes, the right way to spell that is just to pass it as a
Daniel Verkamp3527a272023-10-03 22:42:58183native pointer, which can be null, when it's not present.
Nigel Tao187a4792023-09-28 22:30:44184
18511:29 SHARON: Hopefully that clarifies some things for people who are trying to
186decide which one best suits their use case. So moving on from that, some people
187might remember from a couple of years ago that instead of being called
188`absl::optional`, it used to be called `base::optional`. And do you want to
189quickly mention why we switched from `base` to `absl`? And you mentioned even
190switching to `std::optional`. Why this transition?
191
19211:53 DANA: Yeah, absolutely. So as the C++ standards come out, we want to use
Daniel Verkamp3527a272023-10-03 22:42:58193them, but we can't until our toolchain is ready. What's our toolchain? It's our
194compiler, our standard library - and unfortunately, we have more than one
Nigel Tao187a4792023-09-28 22:30:44195compiler that we need to worry about. So we have the NaCl compiler. Luckily, we
196just have Clang for the compiler choice we really have to worry about. But we
197do have to wait for these things to be ready, and for a code base to be ready
198to turn on the new standard because sometimes there are some non-backwards
199compatible changes. But we can forward port stuff out of the standard library
Daniel Verkamp3527a272023-10-03 22:42:58200into base. And so we've done that. We have a bunch of C++20 backports in base
201now. We had 17 backports before. We turned on 17, now they should hopefully be
Nigel Tao187a4792023-09-28 22:30:44202gone. And so `base::optional` was an example of a backport, while `optional`
203was still considered experimental in the standard library. We adopted use of
204`absl` since then, and `absl` had also, essentially, a backport of the
Daniel Verkamp3527a272023-10-03 22:42:58205`optional` type inside of it for presumably the same reasons. And so why have
Nigel Tao187a4792023-09-28 22:30:44206two when you can have one? That's a pretty good rule. And so we deprecated the
Daniel Verkamp3527a272023-10-03 22:42:58207`base` one, removed it, and moved everything to the `absl` one. One thing to
Nigel Tao187a4792023-09-28 22:30:44208note here, possibly interest, is we often add security hardening to things in
209`base`. And so sometimes there is available in the standard library something.
210But we choose not to use it and use something in `base` or `absl`, but we use
211it in `base` instead, because we have extra hardening checks. And so part of
212the process of removing `base::optional` and moving to `absl::optional` was
213ensuring those same security hardening checks are present in `absl`. And we're
214going to have to do the same thing to stop using `absl` and start using the
215standard one. And that's currently a work in progress.
216
21713:48 SHARON: So let's go through some of the `base` types because that's
218definitely where the most of these kind of wrapper types live. So let's just
219start with one that I learned about recently, and that's a `scoped_refptr`.
220What's that? When should we use it?
221
22213:59 DANA: So `scoped_refptr` is kind of your Chromium equivalent to
223`shared_ptr` in the standard library. So if you're familiar with that, it's
224quite similar, but it has some slight differences. So what is `scoped_refptr`?
Daniel Verkamp3527a272023-10-03 22:42:58225It gives you shared ownership of the underlying object. And it's a smart
226pointer. It holds a pointer to an object that's allocated in the heap. When all
Nigel Tao187a4792023-09-28 22:30:44227`scoped_refptr` that point to the same object are gone, it'll be deleted. So
228it's like `unique_ptr`, except it can be copied to add to your ref count,
229basically. And when all of them are gone, it's destroyed. And it gives access
Daniel Verkamp3527a272023-10-03 22:42:58230to the underlying pointer in exactly the same ways. Oh, but why is it different
Nigel Tao187a4792023-09-28 22:30:44231than `shared_ptr`? I did say it is. `scoped_refptr` requires the type that is
232held inside of it to inherit from `RefCounted` or `RefCountedThreadSafe`.
233`shared_ptr` doesn't require this. Why? So `shared_ptr` sticks an allocation
234beside your object and then puts your object here. So the ref count is
235externalized to your object being stored and owned by the shared pointer.
236Chromium took this position to be doing intrusive ref counting. So because we
237inherit from a known type, we stick the ref count in that base class,
238`RefCounted` or `RefCountedThreadSafe`. And so that is enforced by the
239compiler. You must inherit from one of these two in order to be stored and
Daniel Verkamp3527a272023-10-03 22:42:58240owned in a `scoped_refptr`. What's the difference? `RefCounted` is the default
Nigel Tao187a4792023-09-28 22:30:44241choice, but it's not thread safe. So the ref counting is cheap. It's the more
242performant one, but if you have a `scoped_refptr` on two different threads
243owning the same object, their ref counting will race, can be wrong, you can end
244up with a double free - which is another way that pointers can go wrong, two
Daniel Verkamp3527a272023-10-03 22:42:58245things freeing the same thing - or you could end up with potentially not
Nigel Tao187a4792023-09-28 22:30:44246freeing it at all, probably. I guess I've never checked if that's possible. But
Daniel Verkamp3527a272023-10-03 22:42:58247they can race, and then bad things happen. Whereas, `RefCountedThreadSafe`
Nigel Tao187a4792023-09-28 22:30:44248gives you atomic ref counting. So atomic means that across all threads, they're
249all going to have the same view of the state. And so it can be used across
250threads and be owned across threads. And the tricky part there is the last
251thread that owns that object is where it's going to be destroyed. So if your
Daniel Verkamp3527a272023-10-03 22:42:58252object's destructor does things that you expect to happen on a specific thread,
Nigel Tao187a4792023-09-28 22:30:44253you have to be super careful that you synchronize which thread that last
Daniel Verkamp3527a272023-10-03 22:42:58254reference is going away on, or it could explode in a really flaky way.
Nigel Tao187a4792023-09-28 22:30:44255
25617:02 SHARON: This sounds useful in other ways. What are some kind of more
Daniel Verkamp3527a272023-10-03 22:42:58257design things to consider, in terms of when a `scoped_refptr` is useful and
Nigel Tao187a4792023-09-28 22:30:44258does help enforce things that you want to enforce, like relative lifetimes of
259certain objects?
260
26117:15 DANA: Generally, we recommend that you don't use ref counting if you can
262help it. And that's because it's hard to understand when it's going to be
263destroyed, like I kind of alluded to with the thread situation. Even in a
264single thread situation, how do you know which one is the last reference? And
265is this object going to outlive that other object? Maybe sometimes. It's not
266super obvious. It's a little more clear with a `unique_ptr`, at least local to
267where that `unique_ptr`'s destruction is. But there's usually no
268`scoped_refptr`. You can say this is the last one. So I know it's gone after
Daniel Verkamp3527a272023-10-03 22:42:58269this thing is gone. Maybe it is, maybe it's not, often. So it's a bit tricky.
Nigel Tao187a4792023-09-28 22:30:44270However, there are scenarios when you truly want a bunch of things to have
271access to a piece of data. And you want that data to go away when nobody needs
272it anymore. And so that is your use case for a `scoped_refptr`. It is nicer
273when that thing being with shared ownership is not doing a lot of interesting
274things, especially in its destructor because of the complexity that's involved
275in shared ownership. But you're welcome to shoot yourself in the foot with this
276one if you need to.
277
27818:33 SHARON: We're hoping to help people not shoot themselves in the foot. So
279use `scoped_refptr` carefully, is the lesson there. So you mentioned
280`shared_ptr`. Is that something we see much of in Chrome, or is that something
281that we generally try to avoid in terms of things from the standard library?
282
28318:51 DANA: That is something that is banned in Chrome. And that's just
284basically because we already have `scoped_refptr`, and we don't want two of the
285same thing. There's been various times where people have brought up why do we
286need to have both? Can we just use `shared_ptr` now? And nobody's ever done the
287kind of analysis needed to make that kind of decision. And so we stay with what
288we're at.
289
29019:18 SHARON: If you want to do that, there's someone that'll tell you what to
291do. So something that when I was using `scoped_refptr`, I came across that you
292need a WeakPtrFactory to create such a pointer. So weak pointers and WeakPtr
293factories are one of those things that you see a lot in Chrome and one of these
294base things. So tell us a bit about weak pointers and their factories.
295
29619:42 DANA: So WeakPtr and WeakPtrFactory have a bit of an interesting history.
297Their major purpose is for asynchronous work. Chrome is basically a large
298asynchronous machine, and what does that mean? It means that we break all of
299the work of Chrome up into small pieces of work. And every time you've done a
300piece, you go and say, OK, I'm done. And when the next piece is ready, run this
301thing. And maybe that next thing is like a user input event, maybe that's a
302reply from the network, whatever it might be. And there's just a ton of steps
303in things that happen in Chrome. Like, a navigation has a request, a response,
304maybe another request - some redirects, whatever. That's an example of tons of
Daniel Verkamp3527a272023-10-03 22:42:58305smaller asynchronous tasks that all happen independently. So what goes wrong with
Nigel Tao187a4792023-09-28 22:30:44306asynchronous tasks? You don't have a continuous stack frame. What does that
307mean? So if you're just running some synchronous code, you make a variable, you
Daniel Verkamp3527a272023-10-03 22:42:58308go off and you do some things, you come back. Your variable is still here,
309right? You're in this stack frame and you can keep using it. You have
Nigel Tao187a4792023-09-28 22:30:44310asynchronous tasks. You make a variable, you go and do some work, and you are
Daniel Verkamp3527a272023-10-03 22:42:58311done your task. Boop, your stack's gone. You come back later, you're going to
Nigel Tao187a4792023-09-28 22:30:44312continue. You don't have your variable anymore. So any state that you want to
313keep across your various tasks has to be stored and what we call bound in with
314that task. If that's a pointer, that's especially risky. So we talked earlier
315about Use-After-Frees. Well, you can, I hope, imagine how easy it is to stick a
316pointer into your state. This pointer is valid, I'm using it. I go away, I come
317back when? I don't know, sometime in the future. And I'm going to go use this
318pointer. Is it still around? I don't own it. I didn't use a `unique_ptr`. So
Daniel Verkamp3527a272023-10-03 22:42:58319who owns it? How do they know that I have a task waiting to use it? Well,
Nigel Tao187a4792023-09-28 22:30:44320unless we have some side channel communicating that, they don't. And how do I
321know if they've destroyed it if we don't have some side channel communicating
Daniel Verkamp3527a272023-10-03 22:42:58322that? I don't know. And so I'm just going to use this pointer and bad things
Nigel Tao187a4792023-09-28 22:30:44323happen. Your bank account is gone.
324
32522:06 SHARON: No! My bank account!
326
32722:06 DANA: I know. So what's the side channel? The side channel that we have
328is WeakPtr. So a WeakPtr and WeakPtrFactory provide this communication
329mechanism where WeakPtrFactory watches an object, and when the object gets
Daniel Verkamp3527a272023-10-03 22:42:58330destroyed, the WeakPtrFactory inside of it is destroyed. And that sets this
Nigel Tao187a4792023-09-28 22:30:44331little bit that says, I'm gone. And then when your asynchronous task comes back
332with its pointer, but it's a WeakPtr inside of it and tries to run, it can be
333like, am I still here? If the WeakPtrFactory was destroyed, no, I'm not. And
334then you have a choice of what to do at that point. Typically, we're like,
335abandon ship. Don't do anything here. This whole task is aborted. But maybe you
336do something more subtle. That's totally possible.
337
33822:59 SHARON: I think the example I actually meant to say that uses a
339WeakPtrFactory is a SafeRef, which is another base type. So tell us a bit about
340SafeRefs.
341
34223:13 DANA: WeakPtr is cool because of the side channel that you can examine.
343So you can say are you still alive, dear object? And it can tell you, no, it's
344gone. Or yeah, it's here. And then you can use it. The problem with this is
345that in places where you as the code author want to believe that this object is
346actually always there, but you don't want a security bug if you're wrong. And
347it doesn't mean that you're wrong now, even. Sometime later, someone can change
348code, unrelated to where this is, where the ownership happens, and break you.
Daniel Verkamp3527a272023-10-03 22:42:58349And maybe they don't know all the users of a given object and changing its
Nigel Tao187a4792023-09-28 22:30:44350lifetime in some subtle way, maybe not even realizing they are. Suddenly you're
351eventually seeing security bugs. And so that's why native pointers can be
352pretty scary. And so SafeRef is something we can use instead of a native
353pointer to protect you against this type of bug. It's built on top of WeakPtr
354and WeakPtrFactory. That's its relationship, but its purpose is not the same.
355so what SafeRef does is it says - SafePtr?
356
35724:31 SHARON: SafeRef.
358
35924:31 DANA: SafeRef.
360
36124:31 SHARON: I think there's also a safe pointer, but there -
362
36324:38 DANA: We were going to add it. I'm not sure if it's there yet. But so two
364differences between SafeRef and WeakPtr then, ref versus ptr, it can't be null.
365So it's like a reference wrapper. But the other difference is you can't observe
366whether the object is actually alive or not. So it has the side channel, but it
367doesn't show it to you. Why would you want that? If the information is there
368anyway, why wouldn't you want to expose it? And the reason is because you are
369documenting that you as the author understand and expect that this pointer is
370always valid at this time. It turns out it's not valid. What do you do? If it's
371a WeakPtr, people tend to say, we don't know if it's valid. It's a WeakPtr.
372Let's check. Am I valid? And if I'm not, return. And what does that result in?
373It results in adding a branch to your code. You do that over, and over, and
374over, and over, and static analysis, which is what we as humans have to do -
375we're not running the program, we're reading the code - can't really tell what
376will happen because there's so many things that could happen. We could exit
377here, we could exit there, we could exit here. Who knows. And that makes it
Daniel Verkamp3527a272023-10-03 22:42:58378increasingly hard to maintain and refactor the code. So SafeRef gives you the
379option to say this is always going to be valid. You can't check it. So if it's
380not valid, go fix that bug somewhere else. It should be valid here.
Nigel Tao187a4792023-09-28 22:30:44381
38226:16 SHARON: So what kind of -
383
38426:16 DANA: The assumptions are broken.
385
38626:16 SHARON: So what kind of errors happen when that assumption is broken? Is
387that a crash? Is that a DCHECK kind of thing?
388
38926:22 DANA: For SafeRef and for WeakPtr, if you try to use it without checking
390it, or write it incorrectly, they will crash. And crashing in this case means a
391safe crash. It's not going to lead to a security bug. It's literally just
392terminating the program.
393
39426:41 SHARON: Does that also mean you get a sad tab as a user? Like when the
395little sad file comes up?
396
39726:47 DANA: Yep. It would. If you're in the render process, you take it down.
398It's a sad tab. So that's not great. It's better than a security bug. Because
399your options here are don't write bugs. Ideal. I love that idea, but we know
400that bugs happen. Use a native pointer, security problem. Use a WeakPtr, that
Daniel Verkamp3527a272023-10-03 22:42:58401makes sense if you want it to sometimes not be there. But if you want it to
Nigel Tao187a4792023-09-28 22:30:44402always be there - because you have to make a choice now of what you're supposed
403to do if it's not, and it makes the code very hard to understand. And you're
404only going to find out it can't be there through a crash anyhow. Or use a
405SafeRef. And it's going to just give you the option to crash. You're going to
406figure out what's wrong and make it no longer do that.
407
40827:38 SHARON: I think wanting to guarantee the lifetime of some other things
409seems like a pretty common thing that you might come across. So I'm sure there
410are many cases for many people to be adding SafeRefs to make their code a bit
411safer, and also ensure that if something does go wrong, it's not leading to a
412memory bug that could be exploited in who knows how long. Because we don't
413always hear about those. If it crashes, and they can reliably crash, at least
414you know it's there. You can fix it. If it's not, we're hoping that one of our
415VRP vulnerability researchers find it and report it, but that doesn't always
416happen. So if we can know about these things, that's good. So another new type
417in base that people might have been seeing recently is a `raw_ptr` which is
418maybe why earlier we were saying let's call them native pointers, not raw
419pointers. Because the difference between `raw_ptr` and raw pointer, very easy
420to mix those up. So why don't you tell us a bit about `raw_ptr`s?
421
42228:40 DANA: So `raw_ptr` is really cool. It's a non-owning smart pointer. So
Daniel Verkamp3527a272023-10-03 22:42:58423that's kind of like WeakPtr or SafeRef. These are also non-owning. And it's actually
Nigel Tao187a4792023-09-28 22:30:44424very similar in inspiration to what WeakPtr is. So it has a side channel where
Daniel Verkamp3527a272023-10-03 22:42:58425it can see if the thing it's pointing to is alive or gone. So for WeakPtr, it
426talks to the WeakPtrFactory and says "am I deleted?" And for `raw_ptr`, what it
Nigel Tao187a4792023-09-28 22:30:44427does is it keeps a reference count, kind of like `scoped_refptr`, but it's a
428weak reference count. It's not owning. And it keeps this reference count in the
Daniel Verkamp3527a272023-10-03 22:42:58429memory allocator. So Chrome has its own memory allocator for `new` and `delete`
Nigel Tao187a4792023-09-28 22:30:44430called PartitionAlloc. And that lets us do some interesting stuff. And this is
431one of them. And so what happens is as long as there is `raw_ptr` around, this
432reference count is non-zero. So even if you go and you delete the object, the
433allocator knows there is some pointer to it. It's still out there. And so it
434doesn't free it. It holds it. And it poisons the memory, so that just means
435it's going to write some bit pattern over it, so it's not really useful
436anymore. It's basically re-initialized the memory. And so later, if you go and
437use this `raw_ptr`, you get access to just dead memory. It's there, but it's
438not useful anymore. You're not going to be able to create security bugs in the
439same way. Because when we first started talking about a Use-After-Free - you
440have your goat, you free it, a cow is there, and now your pointer is pointing
441at the wrong thing - you can't do that because as long as there's this
442`raw_ptr` to your goat, the goat can be gone, but nothing else is going to come
443back here. It's still taken by that poisoned memory until all the `raw_ptr`s
444are gone. So that's their job, to protect us from a Use-After-Free being
445exploitable. It doesn't necessarily crash when you use it incorrectly, you just
446get to use this bad memory inside of it. If you try to use it as a pointer,
447then you're using a bad pointer, you're going to probably crash. But it's a
448little bit different than a WeakPtr, which is going to deterministically crash
449as soon as you try to use it when it's gone. It's really just a protection or a
450mitigation against security exploits through Use-After-Free. And then we
451recently just added `raw_ref`, which is really the same as `raw_ptr`, except
Daniel Verkamp3527a272023-10-03 22:42:58452addressing nullability. So smart pointers in C++ have historically all allowed
Nigel Tao187a4792023-09-28 22:30:44453a null state. That's representative of what native pointers did in C and C++.
454And so this is kind of just bringing this along in this obvious, historical
455way. But if you look at other languages that have been able to break with
456history and make their own choices kind of fresh, we see that they make choices
457like not having null pointers, not having null smart pointers. And that
458increases the readability and the understanding of your code greatly. So just
459like for WeakPtr, how we said, we just check if it's there or not. And if it's
460not, we return, and so on. It's every time you have a WeakPtr, if you were
461thinking of a timeline, every time you touch a WeakPtr, your timeline splits.
462And so you get this exponential timeline of possible states that your
Daniel Verkamp3527a272023-10-03 22:42:58463software's in. That's really intense. Whereas every time you can not do that,
Nigel Tao187a4792023-09-28 22:30:44464say this can't be null, so instead of WeakPtr, you're using SafeRef. This can't
Daniel Verkamp3527a272023-10-03 22:42:58465be not here or null, actually - WeakPtr can just be straight up null - this is
Nigel Tao187a4792023-09-28 22:30:44466always present. Then you don't have a split in your timeline, and that makes it
467a lot easier to understand what your software is doing. And so for `raw_ptr`,
468it followed this historical precedent. It lets you have a null value inside of
469it. And `raw_ref` is our kind of modern answer to this new take on nullability.
470And so `raw_ref` is a reference wrapper, meaning it holds a reference inside of
471it, conceptually, meaning it just can't be null. That is just basically - it's
472a pointer, but it can't be null.
473
47433:24 SHARON: So these do sound the most straightforward to use. So basically,
Daniel Verkamp3527a272023-10-03 22:42:58475if you're not sure - for your class members at least - any time you would use a
Nigel Tao187a4792023-09-28 22:30:44476native pointer or an ampersand, basically you should always just put those in
477either a `raw_ptr` or a `raw_ref`, right?
478
47933:45 DANA: Yeah, that's what our style guide recommends, with one nuance. So
480because `raw_ptr` and `raw_ref` interact with the memory allocator, they have
Daniel Verkamp3527a272023-10-03 22:42:58481the ability to be like, turned on or off dynamically at runtime. And there's a
Nigel Tao187a4792023-09-28 22:30:44482performance hit on keeping this reference count around. And so at the moment,
483they are not turned on in the renderer process because it's a really
Daniel Verkamp3527a272023-10-03 22:42:58484performance-critical place. And the impact of security bugs there is a little
485less than in the browser process, where you just immediately get access to the
Nigel Tao187a4792023-09-28 22:30:44486whole system. And so we're working on turning it on there. But if you're
487writing code that's only in the renderer process, then there's no point to use
488it. And we don't recommend that you use it. But the default rule is yes. Don't
489use a native pointer, don't use a native reference. As a field to an object,
Daniel Verkamp3527a272023-10-03 22:42:58490use a `raw_ptr`, use a `raw_ref`. Prefer `raw_ref` - prefer something with less states, always,
491because you get less branches in your timeline. And then you can make it
492`const` if you don't want it to be able to rebound to a new object, if you
493don't want the pointer to change. Or you can make it mutable if you wanted to
494be able to.
Nigel Tao187a4792023-09-28 22:30:44495
49634:58 SHARON: So you did mention that these types are ref counted, but earlier
Daniel Verkamp3527a272023-10-03 22:42:58497you said that you should avoid ref counting things. So -
Nigel Tao187a4792023-09-28 22:30:44498
49935:04 DANA: Yes.
500
50135:11 SHARON: So what's the balance there? Is it because with a
502`scoped_refptr`, you're a bit more involved in the ref counting, or is it just,
503this is we've done it for you, you can use it. This is OK.
504
50535:19 DANA: No, this is a really good question. Thank you for asking that. So
506there's two kinds of ref counts going on here. I tried to kind of allude to it,
507but it's great to make it clear. So `scoped_refptr` is a strong ref count,
508meaning the ref count owns the object. So the destructor runs, the object is
509gone and deleted when that ref count goes to 0. `raw_ref` and `raw_ptr` are a
Daniel Verkamp3527a272023-10-03 22:42:58510weak ref count. They could be pointing to something owned in a
Nigel Tao187a4792023-09-28 22:30:44511`scoped_refptr` even. So they can exist at the same time. You can have both
512kind of ref counts going at the same time. A weak ref count, in this case, is
513holding the memory alive so that it doesn't get re-used. But it's not keeping
514the object in that memory alive. And so from a programming state point-of-view,
515the weak refs don't matter. They're helping protect you from security bugs.
Daniel Verkamp3527a272023-10-03 22:42:58516When things go wrong, when a bug happens, they're helping to make it less
517impactful. But they don't change your program in a visible way. Whereas, strong
518references do. That destrutor's timing is based on when the ref count goes to 0
519for a strong reference. So that's the difference between these two.
Nigel Tao187a4792023-09-28 22:30:44520
52136:46 SHARON: So when you say don't use ref counting, you mean don't use strong
522ref counting.
523
52436:46 DANA: I do, yes.
525
52636:51 SHARON: And if you want to learn more about the raw pointer, `raw_ptr`,
Daniel Verkamp3527a272023-10-03 22:42:58527`raw_ref`, that's all part of the [MiraclePtr] project, and there's a talk about
Nigel Tao187a4792023-09-28 22:30:44528that from BlinkOn. I'll link that below also. So in terms of other base types,
529there's a new one that's called `base::expected`. I haven't even really seen
530this around. So can you tell us a bit more about how we use that, and what
531that's for?
532
Daniel Verkamp3527a272023-10-03 22:42:5853337:09 DANA: `base::expected` is a backport from C++23, I want to say. So the
Nigel Tao187a4792023-09-28 22:30:44534proposal for `base::expected` actually cites a Rust type as inspiration, which
535is called `std::result` in Rust. And it's a lot like `optional`, so it's used
536for return values. And it's more or less kind of a replacement for exceptions.
537So Chrome doesn't compile with exceptions enabled even, so we've never relied
538on exceptions to report errors. But we have to do complicated things, like with
539`optional` to return a bool or an enum. And then maybe some value. And so this
540kind of compresses all that down into a single type, but it's got more state
541than just an option. So `expected` gives you two choices. It either returns
542your value, like `optional` can, or it returns an error. And so that's the
543difference between `optional` and `expected`. You can give a full error type.
544And so this is really useful when you want to give more context on what went
545wrong, or why you're not returning the value. So it makes a lot of sense in
Daniel Verkamp3527a272023-10-03 22:42:58546stuff like file IO. So you're opening a file, and it can fail for various
Nigel Tao187a4792023-09-28 22:30:44547reasons, like I don't have permission, it doesn't exist, whatever. And so in
548that case, the way you would express that in a modern way would be to return
549`base::expected` of your file handle or file class. And as an error, some
550enumerator, perhaps, or even an object that has additional state beyond just I
551couldn't open the file. But maybe a string about why you couldn't open the file
552or something like this. And so it gives you a way to return a structured error
553result.
554
55539:05 SHARON: That's found useful in lots of cases. So all of these types are
556making up for basically what is lacking in C++, which is memory safety. C++, it
557does a lot. It's been around for a long time. Most of Chrome is written in it.
558But there are all these memory issues. And a lot of our security bugs are a
559result of this. So you are working on bringing Rust to Chromium. Why is that a
560good next step? Why does that solve these problems we're currently facing?
561
56239:33 DANA: So Rust has some very cool properties to it. Its first property
563that is really important to this conversation is the way that it handles
564pointers, which in Rust would be treated pretty much exclusively as references.
565And what Rust does is it requires you to tell the compiler the relationships
566between the lifetimes of your references. And the outcome of this additional
567knowledge to the compiler is memory safety. And so what does that mean? It
568means that you can't write a Use-After-Free bug in Rust unless you're going
569into the unsafe part of the language, which is where scariness exists. But you
570don't need to go there to write a normal program. So we'll ignore it. And so
571what that means is you can't write the bug. And so that doesn't just mean I
572also like to believe I can write C++ without a bug. That's not true. But I
573would love to believe that. But it means that later, when I come back and
574refactor my code, or someone comes who's never seen this before and fixes some
575random bug somewhere related to it, they can't introduce a Use-After-Free
576either. Because if they do, the compiler is like, hey - it's going to outlive
577it. You can't use it. Sorry. And so there's this whole class of bugs that you
578never have to debug, you never ship, they never affect users. And so this is a
579really nice promise, really appealing for a piece of software like Chrome,
580where our basic purpose is to handle arbitrary and adversarial data. You want
581to be able to go on some web page, maybe it's hostile, maybe not. You just get
582a link. You want to be able to click that link and trust that even if it's
583really hostile and wanting to destroy you, it can't. Chrome is that safety net
584for you. And so Rust is that kind of safety net for our code, to say no matter
585how you change it over time, it's got your back. You can't introduce this kind
586of bug.
587
Daniel Verkamp3527a272023-10-03 22:42:5858842:03 SHARON: So this Rust project sounds really cool. If people want to learn
589more or get involved - if you're into the whole languages, memory safety kind
590of thing - where can people go to learn more?
Nigel Tao187a4792023-09-28 22:30:44591
59242:09 DANA: So if you're interested in helping out with our Rust experiment,
593then you can look for us in the Rust channel on Slack. If you're interested in
594C++ language stuff, you can find us in the CXX channel on Slack, as well. As
Daniel Verkamp3527a272023-10-03 22:42:58595well as the [[email protected]] mailing list. And there is, of course, the
596[[email protected]] mailing list if you want to use email to reach us as
Nigel Tao187a4792023-09-28 22:30:44597well.
598
59942:44 SHARON: Thank you very much, Dana. There will be notes from all of this
600also linked in the description box. And thank you very much for this first
601episode.
602
60342:52 DANA: Thanks, Sharon This was fun.
Daniel Verkamp3527a272023-10-03 22:42:58604
605[Life of a Vulnerability]: https://www.youtube.com/watch?v=HAJAEQrPUN0
606[MiraclePtr]: (https://www.youtube.com/watch?v=WhI1NWbGvpE)
607[[email protected]]: https://groups.google.com/a/chromium.org/g/cxx
608[[email protected]]: https://groups.google.com/a/chromium.org/g/rust-dev