312-scaling.vtt

WEBVTT

00:00:00.000 --> 00:00:05.000
- Everyone out there in the live stream, welcome, welcome.


00:00:05.000 --> 00:00:07.000
Thank you for joining us.


00:00:07.000 --> 00:00:08.420
It's really great to have you here.


00:00:08.420 --> 00:00:11.120
So if you've got thoughts, comments you'd like to add


00:00:11.120 --> 00:00:13.600
to the show, please put them into the YouTube live chat


00:00:13.600 --> 00:00:18.480
and we'll see what we can do to make that part of the show.


00:00:18.480 --> 00:00:20.800
All right, Julian, you ready to kick this off?


00:00:20.800 --> 00:00:21.900
- Yeah, sure.


00:00:21.900 --> 00:00:26.240
- Julian, welcome to Talk Python to Me.


00:00:26.240 --> 00:00:28.640
- Thank you.


00:00:28.640 --> 00:00:30.640
- Yeah, it's great to have you here.


00:00:30.640 --> 00:00:34.560
And I think we've got a bunch of fun stuff to talk about.


00:00:34.560 --> 00:00:37.220
It's really interesting to think about


00:00:37.220 --> 00:00:39.880
how we go about building software at scale.


00:00:39.880 --> 00:00:41.260
And one of the things that just,


00:00:41.260 --> 00:00:43.140
I don't know how you feel about it, reading your book,


00:00:43.140 --> 00:00:46.340
I feel like you must have some opinions on this.


00:00:46.340 --> 00:00:47.660
But when I go to a website


00:00:47.660 --> 00:00:50.420
that is clearly not a small little company,


00:00:50.420 --> 00:00:53.620
it's obviously a large company with money to put behind,


00:00:53.620 --> 00:00:55.720
you know, professional developers and stuff.


00:00:55.720 --> 00:00:58.420
And you click on it and it takes four seconds


00:00:58.420 --> 00:00:59.900
for every page load.


00:00:59.900 --> 00:01:01.620
It's just like, how is it possible


00:01:01.620 --> 00:01:04.300
that you're building this software with so much,


00:01:04.300 --> 00:01:06.540
oh, this is the face of your business.


00:01:06.540 --> 00:01:08.140
And sometimes they decide to fix it


00:01:08.140 --> 00:01:09.140
with front-end frameworks.


00:01:09.140 --> 00:01:11.260
So then you get like a quick splash


00:01:11.260 --> 00:01:13.580
of like a box with a little UI,


00:01:13.580 --> 00:01:15.420
and then it's just loading for four seconds,


00:01:15.420 --> 00:01:17.300
which to me feels no better.


00:01:17.300 --> 00:01:20.460
So I don't know, I feel like building scalable software


00:01:20.460 --> 00:01:22.220
is really important, and still people are getting it


00:01:22.220 --> 00:01:23.500
quite wrong quite often.


00:01:23.500 --> 00:01:27.700
- Yeah, I mean, I think it's also,


00:01:27.700 --> 00:01:30.880
Because there's a lot of things what you want to do when you do that,


00:01:30.880 --> 00:01:34.400
which is like, well, I mean, write proper code for sure,


00:01:34.400 --> 00:01:37.280
but also you want to be able to measure everything,


00:01:37.280 --> 00:01:40.800
like to understand where the bottleneck might be.


00:01:40.800 --> 00:01:45.340
And that's not the easy part, like writing code and fixing bugs and stuff.


00:01:45.340 --> 00:01:49.440
We all know how to do that. But then if we are asking you to optimize,


00:01:49.440 --> 00:01:53.200
that's one of the things that I usually use as an example


00:01:53.200 --> 00:01:56.940
when I talk about profiling is like, well, if I were to ask you to more like,


00:01:56.940 --> 00:01:59.520
I want you to tell me which part of your code


00:01:59.520 --> 00:02:02.260
is using 20% of the CPU.


00:02:02.260 --> 00:02:03.100
You really don't know.


00:02:03.100 --> 00:02:04.460
You can guess.


00:02:04.460 --> 00:02:07.320
You can probably do a good guess most of the time,


00:02:07.320 --> 00:02:08.900
but for real, you don't know.


00:02:08.900 --> 00:02:11.660
You have no clue until you actually look at the data,


00:02:11.660 --> 00:02:14.860
use a profiler or any tool for that being


00:02:14.860 --> 00:02:16.660
that will give you this information.


00:02:16.660 --> 00:02:21.180
- Yeah, we're really bad in using our intuition


00:02:21.180 --> 00:02:22.060
for those things.


00:02:22.060 --> 00:02:24.380
I remember the most extreme example I ever had of this


00:02:24.380 --> 00:02:25.540
was I was working on this project


00:02:25.540 --> 00:02:29.320
that was doing huge amounts of math, wavelet decomposition,


00:02:29.320 --> 00:02:32.900
kind of like Fourier analysis, but I think kind of worse.


00:02:32.900 --> 00:02:34.920
And I thought, okay, this is too slow.


00:02:34.920 --> 00:02:37.860
It must be in all this complicated math area.


00:02:37.860 --> 00:02:39.500
And I don't understand the math very well,


00:02:39.500 --> 00:02:40.480
and I don't want to change it,


00:02:40.480 --> 00:02:42.180
but this is gotta be here, right?


00:02:42.180 --> 00:02:43.120
It's slow.


00:02:43.120 --> 00:02:44.780
And I put it into the profiler.


00:02:44.780 --> 00:02:47.060
And the thing that turned out was


00:02:47.060 --> 00:02:48.740
we were spending 80% of our time


00:02:48.740 --> 00:02:53.180
just doing like finding the index of an element in a list.


00:02:53.180 --> 00:02:54.300
- Yeah, which is--


00:02:54.300 --> 00:02:55.460
- It's insane.


00:02:55.460 --> 00:02:59.140
- Yeah, my favorite, I think, programming quote


00:02:59.140 --> 00:03:01.060
is from Donald Knuth, which is,


00:03:01.060 --> 00:03:04.220
"Early optimization is the root of all evil."


00:03:04.220 --> 00:03:07.300
Like, it's widely known, but I mean,


00:03:07.300 --> 00:03:11.560
I think I will quote it every week or so now.


00:03:11.560 --> 00:03:13.500
- Yeah, it's fantastic, it's fantastic.


00:03:13.500 --> 00:03:15.780
Yeah, in my case, we switched it to a dictionary


00:03:15.780 --> 00:03:17.820
and it went five times faster, and that was it.


00:03:17.820 --> 00:03:19.740
Like, it was incredibly easy to fix,


00:03:19.740 --> 00:03:22.280
but understanding that that was where the problem was,


00:03:22.280 --> 00:03:23.940
I would have never guessed.


00:03:23.940 --> 00:03:25.900
So yeah, it's hard to understand


00:03:25.900 --> 00:03:28.540
and we're gonna talk about finding these challenges


00:03:28.540 --> 00:03:30.620
and also some of the design patterns.


00:03:30.620 --> 00:03:32.380
You've written a really cool book


00:03:32.380 --> 00:03:34.740
called "The Hacker's Guide to Scaling Python"


00:03:34.740 --> 00:03:36.580
and we're gonna dive into some of the ideas you cover there.


00:03:36.580 --> 00:03:39.460
Also talk about some of your work at Datadog


00:03:39.460 --> 00:03:41.620
that where you're doing some of the profiling stuff,


00:03:41.620 --> 00:03:43.580
not necessarily for you internally,


00:03:43.580 --> 00:03:45.280
although I'm sure there is some,


00:03:45.280 --> 00:03:47.540
but it also could be for so many people


00:03:47.540 --> 00:03:50.300
like you guys basically have profiling as a service


00:03:50.300 --> 00:03:52.660
and runtime as a service,


00:03:52.660 --> 00:03:54.500
runtime analysis of the service, which is great,


00:03:54.500 --> 00:03:55.340
and we'll get into that.


00:03:55.340 --> 00:03:57.140
But before we do, let's start with your story.


00:03:57.140 --> 00:03:59.140
How'd you get into programming in Python?


00:03:59.140 --> 00:04:00.980
- That was a good question.


00:04:00.980 --> 00:04:04.740
So actually, I think I started like 15 years ago or so.


00:04:04.740 --> 00:04:07.340
I actually started to learn Perl,


00:04:07.340 --> 00:04:08.860
the first programming language,


00:04:08.860 --> 00:04:10.940
like, you know, kind of scripting language,


00:04:10.940 --> 00:04:13.420
like we used to call them at least a few years ago.


00:04:13.420 --> 00:04:16.700
And yeah, I mean, I like Perl,


00:04:16.700 --> 00:04:20.060
but I wanted to learn like object-oriented programming,


00:04:20.060 --> 00:04:23.420
and I never understood Perl object-oriented programming.


00:04:23.420 --> 00:04:26.540
Their model was so weird for me.


00:04:26.540 --> 00:04:28.860
Maybe because I was young and I don't know.


00:04:28.860 --> 00:04:32.700
But somebody talked to me about Python.


00:04:32.700 --> 00:04:38.460
I bought the book, the RE book about Python.


00:04:38.460 --> 00:04:41.900
And I kept it around for a year or so because I had no project at all,


00:04:41.900 --> 00:04:45.340
no idea. Most of my job back then was to be a sysadmin,


00:04:45.340 --> 00:04:47.820
so not really anything to do with Python.


00:04:47.820 --> 00:04:51.500
And some day I was working on DPM,


00:04:51.500 --> 00:04:53.620
like the line distribution,


00:04:53.620 --> 00:04:56.540
and I was like, oh, I need to do something like a new project


00:04:56.540 --> 00:04:58.540
and I'm going to do that with Python.


00:04:58.540 --> 00:05:00.700
And I started to learn Python this way


00:05:00.700 --> 00:05:03.900
with my project on one side, the book on the other side.


00:05:03.900 --> 00:05:05.940
I was like, that's amazing, I love it.


00:05:05.940 --> 00:05:08.380
And I never stopped doing Python after that.


00:05:08.380 --> 00:05:09.660
- That's fantastic.


00:05:09.660 --> 00:05:12.020
It feels like it very much was a


00:05:12.020 --> 00:05:14.900
automate the boring stuff type of introduction.


00:05:14.900 --> 00:05:16.100
Like there's these little problems


00:05:16.100 --> 00:05:20.100
and Bash is too small or too hard to make it solve those problems.


00:05:20.100 --> 00:05:24.100
What else could I use? And Python was a good fit for that.


00:05:24.100 --> 00:05:28.100
- Yeah, it's a great way. I have a lot of people coming to me over the years


00:05:28.100 --> 00:05:32.100
and being like, "Julien, I want you to contribute to a project.


00:05:32.100 --> 00:05:36.100
I want to start something in Python. What should I do?" I don't know. What's your problem you want to solve?


00:05:36.100 --> 00:05:40.100
If you want to find a boring thing you want to automate or anything that's the best


00:05:40.100 --> 00:05:44.100
idea you can have to... If it's an open source project that exists already,


00:05:44.100 --> 00:05:45.940
I mean, good for you, it's even better,


00:05:45.940 --> 00:05:49.220
but I mean, just write a script or whatever you want to,


00:05:49.220 --> 00:05:51.140
to start hacking and learning.


00:05:51.140 --> 00:05:54.700
That's the best ways to scratch your own itch.


00:05:54.700 --> 00:05:56.340
- Yeah, absolutely.


00:05:56.340 --> 00:05:57.460
It's so easy to think of,


00:05:57.460 --> 00:05:58.800
well, I want to build this great big thing,


00:05:58.800 --> 00:06:01.260
but we all have these little problems that need solving


00:06:01.260 --> 00:06:04.180
and it's good to start small and practice small


00:06:04.180 --> 00:06:07.140
and build up and yeah, I find it really valuable.


00:06:07.140 --> 00:06:09.040
People often ask me like, oh, I want to get started.


00:06:09.040 --> 00:06:09.880
What should I do?


00:06:09.880 --> 00:06:11.140
Should I build a website like this?


00:06:11.140 --> 00:06:13.420
Maybe a machine learning thing like that.


00:06:13.420 --> 00:06:14.260
I'm like, whoa, whoa, whoa.


00:06:14.260 --> 00:06:16.460
Like that's, yes, you definitely want to get there.


00:06:16.460 --> 00:06:18.340
But you know, if you're really, really just starting,


00:06:18.340 --> 00:06:20.100
like, you know, don't kill yourself


00:06:20.100 --> 00:06:21.540
by trying to take on too much at once.


00:06:21.540 --> 00:06:24.260
So it sounds like it worked well for you.


00:06:24.260 --> 00:06:25.100
How about now?


00:06:25.100 --> 00:06:25.920
What are you doing day to day?


00:06:25.920 --> 00:06:26.860
I hinted at Datadog.


00:06:26.860 --> 00:06:30.780
- Yeah, so I've been doing Python for,


00:06:30.780 --> 00:06:32.340
I mean, the next 10 years,


00:06:32.340 --> 00:06:35.500
after I learned Python, I've been working on OpenStack,


00:06:35.500 --> 00:06:37.660
which is a huge Python project,


00:06:37.660 --> 00:06:41.220
implementing a open cloud system


00:06:41.220 --> 00:06:44.220
where you can host your own AWS, basically.


00:06:44.220 --> 00:06:46.220
And so everything is in Python there.


00:06:46.220 --> 00:06:52.220
So I work on one of the largest Python projects,


00:06:52.220 --> 00:06:54.220
which is up on the stack for a few years.


00:06:54.220 --> 00:06:57.220
And then I decided to go for a change.


00:06:57.220 --> 00:07:01.220
And then I was looking into building a profiling team,


00:07:01.220 --> 00:07:04.220
building a profiler, a continuous profiler,


00:07:04.220 --> 00:07:08.220
which means you would not profile your script on your laptop,


00:07:08.220 --> 00:07:11.660
to profile your application running on your production system for real.


00:07:11.660 --> 00:07:17.180
And I was like, "That's not something I think anyone did before in Python, at least.


00:07:17.180 --> 00:07:20.540
So I want to do that." And that's what I started to do two years ago.


00:07:20.540 --> 00:07:22.860
That's really interesting because normally,


00:07:22.860 --> 00:07:27.580
you have this quantum mechanics problem with profilers and debuggers,


00:07:27.580 --> 00:07:32.180
especially profilers like the line-by-line ones so much


00:07:32.180 --> 00:07:34.460
where it runs at one speed normally,


00:07:34.460 --> 00:07:37.060
and then you hit it with CProfile or something,


00:07:37.060 --> 00:07:39.560
and it's five times slower or whatever it turns out to be.


00:07:39.560 --> 00:07:43.680
And you're like, whoa, this is a lot slower.


00:07:43.680 --> 00:07:48.680
Hopefully it gives you just a factor of slowness over it.


00:07:48.680 --> 00:07:53.040
Like if it says it spends 20% here and 40% there,


00:07:53.040 --> 00:07:55.720
hopefully it's still true at normal speed,


00:07:55.720 --> 00:07:58.080
but sometimes it really depends, right?


00:07:58.080 --> 00:08:00.060
Like if you're calling a function


00:08:00.060 --> 00:08:02.240
that goes out of your system and that's 20%,


00:08:02.240 --> 00:08:04.880
and then you're doing a really tight loop with lots of code,


00:08:04.880 --> 00:08:07.440
the profiler will introduce more overhead


00:08:07.440 --> 00:08:10.840
in your tight loop part than it will in the external system


00:08:10.840 --> 00:08:13.120
where it adds basically zero overhead.


00:08:13.120 --> 00:08:17.640
And so that's a big challenge of understanding profiling


00:08:17.640 --> 00:08:21.240
results in general, and it's a really big reason


00:08:21.240 --> 00:08:24.300
to not just run the profiler constantly in production.


00:08:24.300 --> 00:08:26.440
- Yeah, exactly.


00:08:26.440 --> 00:08:30.800
It's, I mean, and people do that now.


00:08:30.800 --> 00:08:33.120
I mean, if you have the right profiler,


00:08:33.120 --> 00:08:36.400
The way CProfile works, and we can dig a bit into that,


00:08:36.400 --> 00:08:39.200
but CProfile, the way it works, it's going to intercept everything.


00:08:39.200 --> 00:08:41.200
It's what we call a data mystic profiler,


00:08:41.200 --> 00:08:44.000
where if you run the same program twice,


00:08:44.000 --> 00:08:46.400
you will get the same CProfile output for sure.


00:08:46.400 --> 00:08:49.600
It's intercepting all the function calls that you have.


00:08:49.600 --> 00:08:51.680
So if you have a ton of function calls,


00:08:51.680 --> 00:08:53.840
it makes things, like you were saying,


00:08:53.840 --> 00:08:56.000
five times slower for sure at least.


00:08:56.000 --> 00:08:58.880
And it'll inject a little bit of bytecode


00:08:58.880 --> 00:09:00.640
at the beginning and end of every function,


00:09:00.640 --> 00:09:02.880
all sorts of stuff, and it actually changes what happens.


00:09:02.880 --> 00:09:07.880
Yeah, exactly. So it can change the timing.


00:09:07.880 --> 00:09:10.880
So it's a good solution to have a


00:09:10.880 --> 00:09:14.880
blackboard estimate of what's going on


00:09:14.880 --> 00:09:16.880
and it gives you pretty good results.


00:09:16.880 --> 00:09:17.880
Usually it's a good tool. I used it a lot of times over the years.


00:09:17.880 --> 00:09:22.880
It always gave me good information.


00:09:22.880 --> 00:09:25.880
The problem with C++ is that you can't use it in production


00:09:25.880 --> 00:09:27.880
because it's too slow.


00:09:27.880 --> 00:09:28.880
It's also not providing information.


00:09:28.880 --> 00:09:31.880
It gives you the world time that you use,


00:09:28.880 --> 00:09:31.880
but not necessarily the CPU time of each of your thread,


00:09:31.880 --> 00:09:33.880
et cetera, that you're going to use.


00:09:33.880 --> 00:09:35.880
So the information is not really fine-grained.


00:09:35.880 --> 00:09:37.880
It's a rough--


00:09:37.880 --> 00:09:39.880
- Yeah, it's probably not streaming either, right?


00:09:39.880 --> 00:09:41.880
It runs and then gives you the answer,


00:09:41.880 --> 00:09:45.880
not some sort of real-time stream of what's happening.


00:09:45.880 --> 00:09:48.880
- So for one of the case, like we were mentioning previously,


00:09:48.880 --> 00:09:51.880
where you know this part of the code,


00:09:51.880 --> 00:09:53.880
like this scenario that you can recreate


00:09:53.880 --> 00:09:55.880
in a one-minute script or something,


00:09:55.880 --> 00:09:57.880
it's slow and it should take only 30 seconds,


00:09:57.880 --> 00:10:01.560
seconds, you can run CProfiler on it, on one minute on your laptop and be okay,


00:10:01.560 --> 00:10:04.680
I'm going to show you this piece of code. Then if you want to see what's


00:10:04.680 --> 00:10:07.400
happening on prediction with a real workload for real,


00:10:07.400 --> 00:10:11.000
and like you were saying, streaming the data to see


00:10:11.000 --> 00:10:14.600
in real time what's going on, well CProfiler doesn't fit.


00:10:14.600 --> 00:10:18.520
And also, any data mystic profiler which tries to catch


00:10:18.520 --> 00:10:22.680
everything your program does will not work with good performance. So you have


00:10:22.680 --> 00:10:27.560
to do another approach, which is what most profiling profilers


00:10:27.560 --> 00:10:30.800
for continuous profiling, which is statistical profiling,


00:10:30.800 --> 00:10:32.560
where you actually sample your program


00:10:32.560 --> 00:10:35.680
and you try to look what it does most of the time


00:10:35.680 --> 00:10:36.520
and the most of them.


00:10:36.520 --> 00:10:38.440
So it's not a true representation.


00:10:38.440 --> 00:10:40.520
It's not like the reality 100%.


00:10:40.520 --> 00:10:41.960
It's a good statistical approach


00:10:41.960 --> 00:10:44.040
of what your program is doing most of the time.


00:10:44.040 --> 00:10:47.780
- I see, is that more of the sampling style of profilers


00:10:47.780 --> 00:10:51.860
where it's like every 200 milliseconds,


00:10:51.860 --> 00:10:53.000
like what are you doing now?


00:10:53.000 --> 00:10:54.120
What are you doing now?


00:10:54.120 --> 00:10:54.960
What are you doing now?


00:10:54.960 --> 00:10:56.800
It's like a really annoying young child,


00:10:56.800 --> 00:10:58.800
"What are you doing now? What are you doing now?"


00:10:58.800 --> 00:11:00.800
And it's going to miss some things.


00:11:00.800 --> 00:11:02.800
If there's a function you call and it's really quick,


00:11:02.800 --> 00:11:04.800
it's like, "Well, you never called that function as far as the


00:11:04.800 --> 00:11:07.360
profiler is concerned because it just didn't line up."


00:11:07.360 --> 00:11:10.000
But if you do it enough over time,


00:11:10.000 --> 00:11:12.000
you'll get to see a good picture of...


00:11:12.000 --> 00:11:14.000
You don't care about the stuff that happens really fast,


00:11:14.000 --> 00:11:16.000
but you care about the stuff that happens really slow,


00:11:16.000 --> 00:11:18.000
and those are going to show up pretty largely


00:11:18.000 --> 00:11:20.000
in these sorts of... - Exactly.


00:11:20.000 --> 00:11:22.000
So if you use cProfile, you will see this very small


00:11:22.000 --> 00:11:24.000
function call because it catches everything.


00:11:24.000 --> 00:11:27.720
But in reality, for the purpose of optimizing your program,


00:11:27.720 --> 00:11:30.960
you actually don't care. You don't care if you don't see them statistically,


00:11:30.960 --> 00:11:34.240
because they're not important. So that's not what you want to optimize.


00:11:34.240 --> 00:11:36.800
In the end, that's not where your problem lies, probably.


00:11:36.800 --> 00:11:41.200
It's in the outliers, the ones that you see often in your profile,


00:11:41.200 --> 00:11:44.640
the ones you see 80% of the time when the profiler asks your program,


00:11:44.640 --> 00:11:47.200
"What are you doing?" It's always the same function being called,


00:11:47.200 --> 00:11:49.200
"What's the one you want to look at?"


00:11:51.200 --> 00:11:55.240
So I think that fits quite well with the production style.


00:11:55.240 --> 00:11:56.320
I know I was gonna ask you about your book,


00:11:56.320 --> 00:11:59.440
but we're sort of down in this profiling story.


00:11:59.440 --> 00:12:00.280
- That's fine.


00:12:00.280 --> 00:12:05.120
- And you know, I've used Datadog's tools


00:12:05.120 --> 00:12:08.400
for error handling and like exception,


00:12:08.400 --> 00:12:10.040
you know, let me know when there's an error type thing.


00:12:10.040 --> 00:12:11.640
So I have that set up on like Talk Python,


00:12:11.640 --> 00:12:14.680
the podcast site and the Talk Python Training courses site.


00:12:14.680 --> 00:12:15.720
And of course, when you turn it on,


00:12:15.720 --> 00:12:18.920
you get all these errors that are like happening


00:12:18.920 --> 00:12:20.720
in the site, but nobody's complained to you


00:12:20.720 --> 00:12:23.120
that you didn't realize, there's some edge case or whatever.


00:12:23.120 --> 00:12:24.280
And it's really fantastic.


00:12:24.280 --> 00:12:25.720
But something I've never looked at


00:12:25.720 --> 00:12:29.720
is the real-time profiling stuff that you're


00:12:29.720 --> 00:12:30.840
working on these days.


00:12:30.840 --> 00:12:32.560
So maybe just--


00:12:32.560 --> 00:12:34.000
I have no idea what this is like.


00:12:34.000 --> 00:12:35.520
I can imagine maybe what it's like.


00:12:35.520 --> 00:12:39.960
But give me a sense of what kind of stuff do I get out of it.


00:12:39.960 --> 00:12:40.800
>>Nicolas: Sure.


00:12:40.800 --> 00:12:41.680
Yeah.


00:12:41.680 --> 00:12:43.200
So what you'll get, the first thing you'll get,


00:12:43.200 --> 00:12:44.240
it's to profile.


00:12:44.240 --> 00:12:46.320
So you'll get frame charts, essentially,


00:12:46.320 --> 00:12:49.840
which are these kind of charts where you--


00:12:49.840 --> 00:12:54.080
I mean, they look like flames, usually because they are orange and red,


00:12:54.080 --> 00:13:00.160
and going up and down, and the height being the depth


00:13:00.160 --> 00:13:04.720
of your stack trace, and the width being the percent of time


00:13:04.720 --> 00:13:08.640
of resources that you use. So usually, it's time you're going to meter.


00:13:08.640 --> 00:13:10.720
For example, we meter a wall time in CPU Jam.


00:13:10.720 --> 00:13:13.920
So what you'll see is if your function is using a lot of wall time,


00:13:13.920 --> 00:13:17.760
is it waiting for something? Is it waiting for a socket to be read,


00:13:17.760 --> 00:13:19.760
for a lock to be acquired.


00:13:19.760 --> 00:13:21.760
But one of the other profiles we gather is


00:13:21.760 --> 00:13:23.760
how much CPU it's actually using.


00:13:23.760 --> 00:13:26.380
If you want to know if your program is CPU-bound,


00:13:26.380 --> 00:13:28.880
you will see which function is actually using the most CPU


00:13:28.880 --> 00:13:30.880
in your program.


00:13:30.880 --> 00:13:33.500
I could go to my hosting provider, and I could check the box


00:13:33.500 --> 00:13:35.500
and say, "No, I don't want to pay $20 a month.


00:13:35.500 --> 00:13:38.120
I want to pay $50 a month to make this go


00:13:38.120 --> 00:13:40.120
two and a half times faster."


00:13:40.120 --> 00:13:42.500
If I'm CPU-bound, that might actually work.


00:13:42.500 --> 00:13:45.500
But if I'm not, it probably has no effect.


00:13:45.500 --> 00:13:47.500
- Exactly. - For small effects, right?


00:13:47.500 --> 00:13:50.140
So knowing that answer, knowing that would be really helpful.


00:13:50.140 --> 00:13:52.140
Can I scale this vertically?


00:13:52.140 --> 00:13:54.780
Or do I have to change something else?


00:13:54.780 --> 00:13:55.780
Yeah?


00:13:55.780 --> 00:13:58.980
Yeah, it's a whole story around profiling where, I mean,


00:13:58.980 --> 00:14:01.140
most of our users, when they come to us,


00:14:01.140 --> 00:14:03.140
like, we save thousands of dollars


00:14:03.140 --> 00:14:05.780
because we actually understood that we got the button,


00:14:05.780 --> 00:14:07.980
like, here or there, and we were able to downsize


00:14:07.980 --> 00:14:10.300
our deployment because we optimized this function,


00:14:10.300 --> 00:14:14.300
and we understood that this was blocked by this I/O, whatever.


00:14:14.300 --> 00:14:16.800
and when you understand all of that with profiling,


00:14:16.800 --> 00:14:18.400
whatever the language is, by the way,


00:14:18.400 --> 00:14:20.400
being Python or Java or anything,


00:14:20.400 --> 00:14:22.400
you actually save a lot.


00:14:22.400 --> 00:14:25.900
We have a terrific story about customers or internal users


00:14:25.900 --> 00:14:28.400
saving thousands of dollars just because


00:14:28.400 --> 00:14:31.400
they were able to understand what was going on in their program.


00:14:31.400 --> 00:14:34.700
And scaling up was not the solution,


00:14:34.700 --> 00:14:36.700
optimizing the right function was the solution.


00:14:36.700 --> 00:14:40.200
So you'll get CPU wall jam, and we also do memory profiling.


00:14:40.200 --> 00:14:43.500
So you'll see all the memory allocations that are done by Python,


00:14:43.500 --> 00:14:46.240
which are tied to the CPU usage


00:14:46.240 --> 00:14:49.240
because the more objects you're going to allocate...


00:14:49.240 --> 00:14:52.240
When I mean allocate, even if they don't stay around,


00:14:52.240 --> 00:14:55.240
instead you want to create a new string, a new object,


00:14:55.240 --> 00:14:59.740
even for a few seconds, milliseconds, it costs you memory.


00:14:59.740 --> 00:15:02.740
You have to call malloc, and you have to allocate memory,


00:15:02.740 --> 00:15:05.240
which takes time, so you will see that.


00:15:05.240 --> 00:15:09.240
If you use a lot of objects that are not reused, for example,


00:15:09.240 --> 00:15:11.740
you might want to see that.


00:15:11.740 --> 00:15:14.420
The latest one we shipped a few weeks ago,


00:15:14.420 --> 00:15:17.740
the eProfiler where you actually see a sample of your heap,


00:15:17.740 --> 00:15:20.300
like the memory you use in real time


00:15:20.300 --> 00:15:23.220
and what has been allocated on the heap.


00:15:23.220 --> 00:15:24.580
- Can you tell me what,


00:15:24.580 --> 00:15:28.980
how many of each type of object I have?


00:15:28.980 --> 00:15:32.220
Like you've got 20 megs in list,


00:15:32.220 --> 00:15:35.180
you've got 10 megs in strings, okay.


00:15:35.180 --> 00:15:36.540
- No, I mean, in theory, yes.


00:15:36.540 --> 00:15:38.940
In practice, no, and I'm actually fighting upstream


00:15:38.940 --> 00:15:41.060
with the CPython products to be able to do that.


00:15:41.060 --> 00:15:43.380
It's a limitation of CPython right now, technically.


00:15:43.380 --> 00:15:45.540
I can't really do that, but I'm able to give you


00:15:45.540 --> 00:15:47.700
the line number of the file, the function name,


00:15:47.700 --> 00:15:50.100
and the thread that has allocated the memory.


00:15:50.100 --> 00:15:52.500
And I wish I could know the class name,


00:15:52.500 --> 00:15:54.900
that would be amazing, like, for Java as that,


00:15:54.900 --> 00:15:58.380
and I want to have that in Python, so that will be my battle for next year.


00:15:58.380 --> 00:16:01.700
But if you have a memory leak, for example,


00:16:01.700 --> 00:16:05.940
which is quite common, where you keep adding more objects


00:16:05.940 --> 00:16:09.140
on top of each other, at some point, your memory grows forever,


00:16:09.140 --> 00:16:11.460
and you don't know where that comes from.


00:16:11.460 --> 00:16:13.380
With such a tool, a heap profiler,


00:16:13.380 --> 00:16:16.060
you're able to see which stack trace


00:16:16.060 --> 00:16:18.500
is going to add more and more and more memory forever


00:16:18.500 --> 00:16:19.980
and you'll be able to...


00:16:19.980 --> 00:16:22.780
No, it won't give you the solution to your problem.


00:16:22.780 --> 00:16:25.900
It will give you where to look at, which usually is--


00:16:25.900 --> 00:16:27.140
- It's still pretty good, yeah.


00:16:27.140 --> 00:16:29.540
- Yeah, it's like 90% of the problem, so yeah.


00:16:29.540 --> 00:16:32.740
- So can you talk a little bit about the internals


00:16:32.740 --> 00:16:34.860
of like how this works?


00:16:34.860 --> 00:16:37.740
I'm guessing it's not using cProfile.


00:16:37.740 --> 00:16:38.580
- No.


00:16:38.580 --> 00:16:43.580
Is it using other open source things to sort of put the service together


00:16:43.580 --> 00:16:46.580
or is it all custom stuff you did?


00:16:46.580 --> 00:16:49.580
- No, it's custom stuff. Everything is open source, so we want you to look at it.


00:16:49.580 --> 00:16:52.580
It's on our DigiTrace repository on GitHub.


00:16:52.580 --> 00:16:57.580
So the way it works for the CPU and wartime profiler, it's pretty easy.


00:16:57.580 --> 00:17:00.580
A lot of people know about that. You can actually ask CPython


00:17:00.580 --> 00:17:03.580
to give you the list of running threads.


00:17:03.580 --> 00:17:07.580
So if you do that 100 times per second, you get the list of running threads


00:17:07.580 --> 00:17:09.580
and you can get the stack trace running,


00:17:09.580 --> 00:17:11.580
like the function name, number of that running.


00:17:11.580 --> 00:17:13.580
So if you do that a lot of time,


00:17:13.580 --> 00:17:15.580
you get a pretty good picture of what your program


00:17:15.580 --> 00:17:17.580
and threads are doing most of the time.


00:17:17.580 --> 00:17:20.580
So, in summary, that's how it works.


00:17:20.580 --> 00:17:22.580
It's pretty easy, then there's a few tricks


00:17:22.580 --> 00:17:25.580
to get the CPU time, etc., using the Pthread API,


00:17:25.580 --> 00:17:27.580
but that's most of it.


00:17:27.580 --> 00:17:29.580
And for memory, there are actually a good thing


00:17:29.580 --> 00:17:31.580
that has been done by a friend,


00:17:31.580 --> 00:17:35.580
Victor Stiner, which is one of the--


00:17:35.580 --> 00:17:38.920
He's done a great amount of performance improvement,


00:17:38.920 --> 00:17:40.920
like really important stuff.


00:17:40.920 --> 00:17:43.420
And one of the things he did in Python 3.4,


00:17:43.420 --> 00:17:46.920
so it was a long time ago, is to add this module trace malloc,


00:17:46.920 --> 00:17:48.920
which we don't use.


00:17:48.920 --> 00:17:51.620
I actually built on top of it at some point,


00:17:51.620 --> 00:17:53.620
but we don't use it anymore.


00:17:53.620 --> 00:17:56.120
We wrote a lightweight version of it,


00:17:56.120 --> 00:17:59.720
but it opened the memory API of CPython,


00:17:59.720 --> 00:18:04.720
where we can actually plug your own memory allocator to CPython.


00:18:04.720 --> 00:18:06.520
And that's what we do with our profiler.


00:18:06.520 --> 00:18:08.800
We replace a memory allocator bar,


00:18:08.800 --> 00:18:11.640
a tiny wrapper that catches every location


00:18:11.640 --> 00:18:14.160
and then do profiling on top of it.


00:18:14.160 --> 00:18:15.000
- Right, exactly.


00:18:15.000 --> 00:18:17.520
So when it says allocate this,


00:18:17.520 --> 00:18:20.640
you say record this was allocated and then allocate it.


00:18:20.640 --> 00:18:21.880
Right? Something like that.


00:18:21.880 --> 00:18:23.620
Is this the thing you were talking about on the screen?


00:18:23.620 --> 00:18:24.920
This dd_trace_py?


00:18:24.920 --> 00:18:25.880
- Yeah, exactly.


00:18:25.880 --> 00:18:28.240
You have a dd_trace directory


00:18:28.240 --> 00:18:30.280
inside of the profiling directory


00:18:30.280 --> 00:18:31.640
and all the code is there.


00:18:31.640 --> 00:18:34.120
So you can take a look at how it works internally.


00:18:34.120 --> 00:18:41.560
Yeah, I mean, the way we build it is to be able to be easy to ship, to deploy,


00:18:41.560 --> 00:18:45.060
you don't require any extra permission.


00:18:45.060 --> 00:18:48.960
There are a lot of different ways of doing profiling using,


00:18:48.960 --> 00:18:53.920
for example, Linux capabilities, Perf, a lot of things that are external


00:18:53.920 --> 00:18:56.920
and not necessarily portable outside Linux.


00:18:56.920 --> 00:19:01.680
But the problem is that most of the time they require extra permission,


00:19:01.680 --> 00:19:04.400
like being root or anything, like using the PTRS API


00:19:04.400 --> 00:19:08.160
requires you extra permission, which is great.


00:19:08.160 --> 00:19:10.160
I mean, that's a great solution,


00:19:10.160 --> 00:19:13.720
maybe better technically for some point


00:19:13.720 --> 00:19:16.120
compared to what we do there,


00:19:16.120 --> 00:19:18.160
but they are very complicated to deploy.


00:19:18.160 --> 00:19:21.800
So that was another thing that drive us,


00:19:21.800 --> 00:19:23.600
I think, for writing this.


00:19:23.600 --> 00:19:25.840
- Right, so a simple pip install,


00:19:25.840 --> 00:19:29.640
plug in a line or two and off you go, right?


00:19:29.640 --> 00:19:33.760
I mean, the setup is pretty simple.


00:19:33.760 --> 00:19:41.040
And so for exporting the data, we use a Pprof format from Google, which is pretty standard.


00:19:41.040 --> 00:19:43.120
So you can actually use this profiler if you want.


00:19:43.120 --> 00:19:46.120
Even if you're not a Datapack customer and you want to give it a try, you can actually


00:19:46.120 --> 00:19:52.200
export the data to Pprof file and see what the data.


00:19:52.200 --> 00:19:56.920
You won't have the whole analytics that we provide and the fancy flame shaft with all


00:19:56.920 --> 00:19:59.800
their rainbow colors, but you can use a P-Prof Go tool,


00:19:59.800 --> 00:20:01.680
which is pretty okay.


00:20:01.680 --> 00:20:02.520
- Oh, interesting.


00:20:02.520 --> 00:20:04.200
So you can get basically the raw data out of it


00:20:04.200 --> 00:20:05.400
just by using it directly.


00:20:05.400 --> 00:20:09.760
It's just you guys provide the nice gathering.


00:20:09.760 --> 00:20:11.160
- Yeah, exactly.


00:20:11.160 --> 00:20:13.200
We provide the streamlining to our backend


00:20:13.200 --> 00:20:15.120
and we provide turn-off analysis.


00:20:15.120 --> 00:20:17.160
But if you are curious and want to take a look


00:20:17.160 --> 00:20:18.680
at how it works and what you can provide,


00:20:18.680 --> 00:20:20.680
I mean, it's a good way to do it too.


00:20:20.680 --> 00:20:21.520
- All right, awesome.


00:20:21.520 --> 00:20:23.640
I do want to dig into your scaling book,


00:20:23.640 --> 00:20:26.680
which is what we're going to spend a lot of time on.


00:20:26.680 --> 00:20:30.880
One final question though, can I diff profiles


00:20:30.880 --> 00:20:32.360
like from one version to another?


00:20:32.360 --> 00:20:35.080
Because one of the things that drives me crazy is


00:20:35.080 --> 00:20:37.680
I've done a bunch of recording, I got my numbers


00:20:37.680 --> 00:20:39.560
and then I'm gonna make a change.


00:20:39.560 --> 00:20:40.880
Is it better or is it worse?


00:20:40.880 --> 00:20:41.700
What has gotten better?


00:20:41.700 --> 00:20:42.540
What has gotten worse?


00:20:42.540 --> 00:20:45.400
Like, is there a way to say compare?


00:20:45.400 --> 00:20:48.400
- Yeah, that's something we are building a data log


00:20:48.400 --> 00:20:51.800
on our backend side to be able to track your releases


00:20:51.800 --> 00:20:53.920
and tell you where this is going faster,


00:20:53.920 --> 00:20:55.680
this is going slower and which functions


00:20:55.680 --> 00:20:59.680
or method are being the culprit of your slowness or whatever.


00:20:59.680 --> 00:21:02.480
So yeah, that's definitely something I want to do.


00:21:02.480 --> 00:21:06.480
- Yeah, that'd be so neat because you get into it,


00:21:06.480 --> 00:21:09.040
you do maybe take a whole week or two and you sit down


00:21:09.040 --> 00:21:11.520
and you make your code fast and you get it all polished.


00:21:11.520 --> 00:21:13.800
And then over time, it kind of degrades, right?


00:21:13.800 --> 00:21:15.360
As people add new stuff to it


00:21:15.360 --> 00:21:17.560
and they don't really necessarily do so


00:21:17.560 --> 00:21:19.480
thinking about performance.


00:21:19.480 --> 00:21:21.920
So it'd be cool to like, okay, here's how it's degraded


00:21:21.920 --> 00:21:23.320
and we can just focus our energy


00:21:23.320 --> 00:21:25.360
in making this part better again.


00:21:25.360 --> 00:21:26.560
I think that'd be great.


00:21:26.560 --> 00:21:28.440
- Yeah.


00:21:28.440 --> 00:21:32.560
- All right, well, tell us about your book.


00:21:32.560 --> 00:21:33.920
I find it fascinating.


00:21:33.920 --> 00:21:36.240
I kind of gave it a bit of an introduction,


00:21:36.240 --> 00:21:37.640
the idea of scaling it,


00:21:37.640 --> 00:21:41.920
but official title is "The Hacker's Guide to Scaling Python,"


00:21:41.920 --> 00:21:45.760
and the subtitle is "Build Apps that Scale to Billions."


00:21:45.760 --> 00:21:47.200
Billions of users, billions of requests,


00:21:47.200 --> 00:21:48.840
billions of whatever, I guess,


00:21:48.840 --> 00:21:51.800
but most apps don't do this,


00:21:51.800 --> 00:21:53.160
so I think they would be,


00:21:53.160 --> 00:21:55.720
A lot of people will be interested in hearing about it.


00:21:55.720 --> 00:21:57.240
- Right, I mean, most apps don't do this,


00:21:57.240 --> 00:21:59.520
but so many apps don't really need to do that.


00:21:59.520 --> 00:22:00.600
So that's not a problem.


00:22:00.600 --> 00:22:03.600
I wrote that book, I think four years ago now,


00:22:03.600 --> 00:22:07.160
because I was working, like I said, on OpenStack,


00:22:07.160 --> 00:22:10.240
where I would actually try to scale the things to billions,


00:22:10.240 --> 00:22:11.960
where it would be running,


00:22:11.960 --> 00:22:15.000
like it would be hops running on thousands of nodes.


00:22:15.000 --> 00:22:19.240
- Right, and maybe any individual app


00:22:19.240 --> 00:22:21.480
is not scaling to that level,


00:22:21.480 --> 00:22:25.520
but you guys basically being the platform as a service,


00:22:25.520 --> 00:22:28.840
in aggregate have a huge amount of load put on it, right?


00:22:28.840 --> 00:22:29.680
- Exactly.


00:22:29.680 --> 00:22:30.520
- Okay.


00:22:30.520 --> 00:22:32.640
- And which at that point,


00:22:32.640 --> 00:22:34.080
like when I started to write the book,


00:22:34.080 --> 00:22:38.160
a lot of people were flying outside Python,


00:22:38.160 --> 00:22:39.240
outside the Python realm,


00:22:39.240 --> 00:22:41.480
because while Python is slow,


00:22:41.480 --> 00:22:43.600
you can't do anything meaningful with Python, right?


00:22:43.600 --> 00:22:45.360
So slow, you have to switch to go.


00:22:45.360 --> 00:22:46.200
And that was the other thing.


00:22:46.200 --> 00:22:47.840
- Yeah, that's the first thing you'd have to do,


00:22:47.840 --> 00:22:49.680
is if I understand it.


00:22:49.680 --> 00:22:54.680
Python is slow, so you have to switch to Go, right?


00:22:54.680 --> 00:22:56.680
I hear it all the time.


00:22:56.680 --> 00:22:58.680
- So I was confronted about that.


00:22:58.680 --> 00:23:00.200
In the OpenStack project, somebody rewrote


00:23:00.200 --> 00:23:02.520
one of the software of OpenStack in Go


00:23:02.520 --> 00:23:04.360
because it was faster and was like,


00:23:04.360 --> 00:23:06.560
no, the Python architecture you use is slow.


00:23:06.560 --> 00:23:10.880
That's why the program is slow.


00:23:10.880 --> 00:23:12.320
It's not being ready to Python


00:23:12.320 --> 00:23:14.720
and there's no need to switch to Go.


00:23:14.720 --> 00:23:17.640
It's not related to the language.


00:23:17.640 --> 00:23:19.400
It's that the architecture was different.


00:23:15.760 --> 00:23:19.260
So that's what kind of motivated me at the beginning


00:23:19.260 --> 00:23:21.460
to write that book, to be able to share


00:23:21.460 --> 00:23:24.260
everything I learned for the years before


00:23:24.260 --> 00:23:26.260
building up on Stack, I mean part of it,


00:23:26.260 --> 00:23:29.660
and learning on what works and doesn't work


00:23:29.660 --> 00:23:33.260
at scale in Python, and to stop people switching


00:23:33.260 --> 00:23:35.260
to Go for bad reasons. There are good reasons


00:23:35.260 --> 00:23:37.260
to switch for Go for sure, but sometimes no.


00:23:37.260 --> 00:23:40.760
- Exactly, not just because it's estuary.


00:23:40.760 --> 00:23:43.460
Another example of this is people switching


00:23:43.460 --> 00:23:47.040
to Node.js because it can handle more connections.


00:23:47.040 --> 00:23:49.600
And the reason it can handle more connections


00:23:49.600 --> 00:23:52.240
is because it's written in an asynchronous way,


00:23:52.240 --> 00:23:53.320
a non-blocking way.


00:23:53.320 --> 00:23:55.360
And so if you write blocking Python,


00:23:55.360 --> 00:23:56.640
it doesn't matter if you write blocking C,


00:23:56.640 --> 00:23:58.200
it's not going to take it as well


00:23:58.200 --> 00:24:01.040
as if you write non-blocking Python.


00:24:01.040 --> 00:24:05.760
And so things like AsyncIO and those types,


00:24:05.760 --> 00:24:08.440
ASCII servers and whatnot can automatically


00:24:08.440 --> 00:24:11.800
sort of put you back in the game compared to those systems.


00:24:12.720 --> 00:24:14.860
The magic of, magic in quotes,


00:24:14.860 --> 00:24:17.400
the magic of Node was they made you do that from the start.


00:24:17.400 --> 00:24:19.080
They're like, oh, you can't call synchronous functions.


00:24:19.080 --> 00:24:22.400
So the only way you do this is you write crazy callbacks


00:24:22.400 --> 00:24:26.280
until better ways, like with promises and futures


00:24:26.280 --> 00:24:28.120
and then async and await get into the language.


00:24:28.120 --> 00:24:31.720
But they forced people to go down this pattern


00:24:31.720 --> 00:24:32.800
that allowed for scale.


00:24:32.800 --> 00:24:35.200
And then you can say, oh, look, our apps scale really well.


00:24:35.200 --> 00:24:37.280
And it's just that I think a lot of times


00:24:37.280 --> 00:24:38.880
people start with the easy way,


00:24:38.880 --> 00:24:40.520
which makes a lot of sense in Python,


00:24:40.520 --> 00:24:42.480
but that's not necessarily the scalable way.


00:24:42.480 --> 00:24:46.320
So start one way, but as you identify these problems,


00:24:46.320 --> 00:24:48.160
you know, maybe bring in some of the ideas of your book.


00:24:48.160 --> 00:24:49.000
Right?


00:24:49.000 --> 00:24:50.740
- Yeah, totally.


00:24:50.740 --> 00:24:52.880
I mean, that's why like, one of the first thing


00:24:52.880 --> 00:24:54.600
I like to say about that is like,


00:24:54.600 --> 00:24:56.680
Python is not fast or slow.


00:24:56.680 --> 00:24:59.320
First, it's like, you would say like English,


00:24:59.320 --> 00:25:01.200
English is slow or English is fast.


00:25:01.200 --> 00:25:02.040
Doesn't make any sense.


00:25:02.040 --> 00:25:04.760
You have people, I mean, speaking English very fast


00:25:04.760 --> 00:25:05.880
over notes.


00:25:05.880 --> 00:25:07.000
It's like Python the long way.


00:25:07.000 --> 00:25:10.200
When CPython is slow, okay, it's not the best VM


00:25:10.200 --> 00:25:14.280
actually, I think it's far from being the best VM out there,


00:25:14.280 --> 00:25:17.480
I mean, by VM, I mean, virtual machine of the language.


00:25:17.480 --> 00:25:21.800
Like, if you look at the state of the art of, I don't know, V8 for JavaScript or


00:25:21.800 --> 00:25:27.480
Or Grunt or whatever for Java or the JVM itself is pretty great nowadays.


00:25:27.480 --> 00:25:32.120
And I mean, if you look at all of that, Python, I mean, CPython is already


00:25:32.120 --> 00:25:39.160
looking bad, I think. But then it has over upside, which gives you a good,


00:25:39.160 --> 00:25:44.680
I mean, good things when you use Python and a good reason to keep using Python and see Python in the end.


00:25:44.680 --> 00:25:45.720
So


00:25:45.720 --> 00:25:50.040
I think it's a trade-off and people are not always putting the right


00:25:50.040 --> 00:25:53.240
weight at the right place for doing that trade-off.


00:25:53.240 --> 00:25:57.960
Yeah, I agree. One trade-off might be, oh, you could write it in,


00:25:57.960 --> 00:26:01.240
let's say Rust or something, for example, make it go faster.


00:26:01.240 --> 00:26:07.640
But then you're giving up the ability for people to come with just a very partial understanding of Python itself


00:26:07.880 --> 00:26:09.420
and still being really productive, right?


00:26:09.420 --> 00:26:11.420
Like people don't come to Rust and Java


00:26:11.420 --> 00:26:13.120
with very partial understandings of it


00:26:13.120 --> 00:26:14.720
and be super productive.


00:26:14.720 --> 00:26:16.260
They just don't, right?


00:26:16.260 --> 00:26:19.860
You got to take in a big bite of that whole,


00:26:19.860 --> 00:26:21.460
all the computer science ideas there,


00:26:21.460 --> 00:26:25.060
whereas like Python can start so simple and clean.


00:26:25.060 --> 00:26:26.860
And I think that's part of the magic.


00:26:26.860 --> 00:26:29.980
But I guess some of the patterns of that simple world


00:26:29.980 --> 00:26:31.700
don't always make sense, right?


00:26:31.700 --> 00:26:35.180
I do like that you pointed out that not everyone


00:26:36.180 --> 00:26:38.420
needs highly scalable apps.


00:26:38.420 --> 00:26:42.420
Right? Because it's really cool to hear,


00:26:42.420 --> 00:26:45.380
"Oh, they're doing this thing with Instagram."


00:26:45.380 --> 00:26:47.700
Right? Like, "Instagram turned off the garbage collector."


00:26:47.700 --> 00:26:50.820
And now they're getting better memory reuse


00:26:50.820 --> 00:26:52.180
across their web workers.


00:26:52.180 --> 00:26:53.380
And so maybe we should do that too.


00:26:53.380 --> 00:26:54.740
It's like, "Well, hold on now.


00:26:54.740 --> 00:26:57.540
How much of this infrastructure


00:26:57.540 --> 00:27:00.580
can you afford just 20 more dollars


00:27:00.580 --> 00:27:02.180
and not have to deal with this ever?"


00:27:02.180 --> 00:27:04.180
Right? I mean, they run their own version of CPython.


00:27:04.180 --> 00:27:06.580
that's a fork where they turn off the garbage collector.


00:27:06.580 --> 00:27:10.380
Like, do you really need to go that far?


00:27:10.380 --> 00:27:15.020
So I kind of put that out there just kind of a heads up


00:27:15.020 --> 00:27:16.900
for people before they dive in.


00:27:16.900 --> 00:27:18.780
Because kind of like design patterns,


00:27:18.780 --> 00:27:20.580
like I feel like when you learn some of this stuff,


00:27:20.580 --> 00:27:22.660
you're like, oh, let's just put all of this into place.


00:27:22.660 --> 00:27:25.660
And then you can end up with a more complicated system that


00:27:25.660 --> 00:27:28.020
didn't really need all those put in together at once.


00:27:28.020 --> 00:27:30.860
And maybe like there's no app that actually incorporates


00:27:30.860 --> 00:27:33.020
every single idea that you've mentioned here.


00:27:33.020 --> 00:27:36.020
They're all good ideas in their context, but not necessarily...


00:27:36.020 --> 00:27:39.280
You wouldn't order everything on a menu and put it all on one plate


00:27:39.280 --> 00:27:41.280
and then try to eat it.


00:27:41.280 --> 00:27:43.780
No. Especially because if you start...


00:27:43.780 --> 00:27:46.280
For example, the first thing people do usually is,


00:27:46.280 --> 00:27:49.280
you write a program. It's not fast enough.


00:27:49.280 --> 00:27:51.280
Let's not say it's slow. It's not fast enough for you.


00:27:51.280 --> 00:27:53.280
You're like, "Okay, I want to make it faster."


00:27:53.280 --> 00:27:57.020
If you can parallelize things, you're like,


00:27:57.020 --> 00:27:59.020
"Okay, I could run this and this in parallel."


00:27:59.020 --> 00:28:01.020
You go, "Okay, I'm going to use threads.


00:28:01.020 --> 00:28:04.020
All right, that's easy. There's a threading API,


00:28:04.020 --> 00:28:08.020
there are the concurrence, future API in Python.


00:28:08.020 --> 00:28:10.020
It's pretty easy to do,


00:28:10.020 --> 00:28:13.220
but it adds so much complexity to your program,


00:28:13.220 --> 00:28:15.220
you have to be sure it's really worth it.


00:28:15.220 --> 00:28:18.520
Because you know you're entering the world of concurrency,


00:28:18.520 --> 00:28:21.520
and when you're entering, you have to use locks.


00:28:21.520 --> 00:28:25.020
You have to be sure about your program is not having side effects


00:28:25.020 --> 00:28:29.020
between threads at the bad time or anything.


00:28:29.020 --> 00:28:32.180
And it adds so much complexity, it's actually very hard


00:28:32.180 --> 00:28:34.900
to make this kind of program right and to make sure it works.


00:28:34.900 --> 00:28:37.900
And there are so many new edge cases you're adding


00:28:37.900 --> 00:28:41.260
by adding concurrency, being threads or anything else.


00:28:41.260 --> 00:28:43.340
But you have to be sure it's worth it.


00:28:43.340 --> 00:28:45.740
And for a lot of people out there, it's really not worth it.


00:28:45.740 --> 00:28:48.660
You could have a pretty simple application with just one process


00:28:48.660 --> 00:28:52.620
or a couple of processes behind a Unicorn or a USB worker


00:28:52.620 --> 00:28:54.620
and be fine forever.


00:28:54.620 --> 00:28:57.100
But it doesn't make any sense to try to optimize.


00:28:57.100 --> 00:28:59.100
And like I was saying, early optimization,


00:28:59.100 --> 00:29:04.100
don't do it unless you are sure and you actually know why it's slow,


00:29:04.100 --> 00:29:09.100
you know where to optimize, which might be a good user for profiler or not,


00:29:09.100 --> 00:29:11.100
depending on what you're trying to optimize.


00:29:11.100 --> 00:29:14.100
But make sure that you understand the trade-offs you are doing.


00:29:14.100 --> 00:29:18.100
I saw so many people rushing into threads or anything like that


00:29:18.100 --> 00:29:22.100
and writing code that is invalid and then could crash in production


00:29:22.100 --> 00:29:25.100
because of race condition, etc., and they never thought about it.


00:29:25.100 --> 00:29:28.000
and it takes months, years to get things right again


00:29:28.000 --> 00:29:31.100
because it's very complex and writing


00:29:31.100 --> 00:29:34.600
like multi-faceted code is not something humans do very well.


00:29:34.600 --> 00:29:38.600
So if you can't afford to not do it, don't do it.


00:29:38.600 --> 00:29:42.800
Well, I think, you know, going back to the earlier discussion with profiling and stuff,


00:29:42.800 --> 00:29:45.600
either in production or just with C profiling,


00:29:45.600 --> 00:29:47.400
measure first, right?


00:29:47.400 --> 00:29:52.600
Because then you know so much better what can be applied to solve that problem


00:29:52.600 --> 00:29:55.960
Because if the slow thing is you're waiting on the database,


00:29:55.960 --> 00:30:00.280
well, you sure don't need more threads to worry about that, right?


00:30:00.280 --> 00:30:02.440
You might consider caching.


00:30:02.440 --> 00:30:03.800
That could be an option.


00:30:03.800 --> 00:30:06.120
You might consider indexes to make the database faster.


00:30:06.120 --> 00:30:10.320
But you could really easily introduce lots of complexity


00:30:10.320 --> 00:30:13.320
into the system by applying the wrong fix


00:30:13.320 --> 00:30:14.920
and not really get a whole lot better.


00:30:14.920 --> 00:30:16.920
- Yeah. - So, yeah.


00:30:16.920 --> 00:30:18.920
All right.


00:30:18.920 --> 00:30:20.240
Let's talk about scaling.


00:30:20.240 --> 00:30:24.920
I think scaling is just the definition of scaling


00:30:24.920 --> 00:30:28.120
is really interesting because a lot of people see here


00:30:28.120 --> 00:30:30.520
that like, I want an app that scales or like,


00:30:30.520 --> 00:30:31.760
man, YouTube is so awesome.


00:30:31.760 --> 00:30:34.760
It scales to a million requests a second or whatever.


00:30:34.760 --> 00:30:36.400
I want that.


00:30:36.400 --> 00:30:40.800
And then they have one, they have their app running


00:30:40.800 --> 00:30:43.960
and they click the link and it takes three seconds


00:30:43.960 --> 00:30:47.360
or they run the function and it takes three seconds.


00:30:47.360 --> 00:30:52.640
Well, if that app scaled well, it would mean it could run in three seconds


00:30:52.640 --> 00:30:55.960
for a hundred people as well as it could run in three seconds for one person.


00:30:55.960 --> 00:30:57.760
That doesn't necessarily mean faster.


00:30:57.760 --> 00:31:00.560
So there's this distinction I think people need to make


00:31:00.560 --> 00:31:04.120
between high-performance fast code, quickly responding,


00:31:04.120 --> 00:31:09.000
and then scaling like it doesn't degrade as it takes on more users.


00:31:09.000 --> 00:31:11.560
Maybe you want to riff on that a little bit?


00:31:11.560 --> 00:31:14.680
Yeah, there are two dimensions, basically, which is like we were saying.


00:31:14.680 --> 00:31:19.680
One is more users, which is more in parallel, let's say.


00:31:19.680 --> 00:31:23.440
And one is faster, like having the page being loaded faster.


00:31:23.440 --> 00:31:27.320
So there are two different things.


00:31:27.320 --> 00:31:29.280
If you want to really optimize one particular use case,


00:31:29.280 --> 00:31:31.960
like page being loaded or whatever,


00:31:31.960 --> 00:31:34.520
it's really a good practice.


00:31:34.520 --> 00:31:35.960
I mean, you can't really scale that request,


00:31:35.960 --> 00:31:39.320
a web request on multiple nodes,


00:31:39.320 --> 00:31:41.760
unless it's very complicated,


00:31:41.760 --> 00:31:42.920
but to load a page or REST API or anything like that.


00:31:41.600 --> 00:31:45.300
you really want to profile that part of the code to be sure.


00:31:45.300 --> 00:31:46.880
You don't have that--


00:31:46.880 --> 00:31:49.600
That's a case where profiling locally with CProfile


00:31:49.600 --> 00:31:51.600
might actually be really good.


00:31:51.600 --> 00:31:53.600
One request is actually quite slow.


00:31:53.600 --> 00:31:57.600
You could learn a lot about that, running that in a profiler,


00:31:57.600 --> 00:32:00.240
and adding the horizontal scalability stuff


00:32:00.240 --> 00:32:03.440
might actually make it a tiny bit slower for each individual request,


00:32:03.440 --> 00:32:06.000
but allow many more of them to happen.


00:32:06.000 --> 00:32:08.600
So you got to figure out what part you're working on.


00:32:09.200 --> 00:32:11.800
Yeah, keeping in mind, if you see profile on your laptop,


00:32:11.800 --> 00:32:14.600
it's going to be different than if you see profile on AWS


00:32:14.600 --> 00:32:17.500
or anything you run, because your database is going to be different,


00:32:17.500 --> 00:32:20.600
the latency is going to be different, so it's hard to reproduce


00:32:20.600 --> 00:32:23.100
the same condition on your developer laptop


00:32:23.100 --> 00:32:25.600
that you have in production, you don't have the same system.


00:32:25.600 --> 00:32:30.600
So it's really a good way, and you can do 80% of the job,


00:32:30.600 --> 00:32:34.500
but in some cases, it's great to have continuous profiling


00:32:34.500 --> 00:32:36.500
on your production system.


00:32:36.500 --> 00:32:39.700
And that gives you a good way to optimize your code


00:32:39.700 --> 00:32:42.960
and to make sure that this dimension of being faster is covered.


00:32:42.960 --> 00:32:46.700
Then the dimension of, well, let's scale to thousands of users


00:32:46.700 --> 00:32:50.280
and still have the three-second load for everyone,


00:32:50.280 --> 00:32:53.380
then that's another problem. That's where you actually don't need


00:32:53.380 --> 00:32:56.620
a profiler, but you need a good architecture of your program


00:32:56.620 --> 00:33:01.320
and your code, and be able to spawn new process,


00:33:01.320 --> 00:33:04.620
new threads, new nodes, new anything that can process things


00:33:04.620 --> 00:33:09.620
in parallel for you and decouple like split your program


00:33:09.620 --> 00:33:11.820
in different parts and having a good architecture.


00:33:11.820 --> 00:33:14.140
And there you can do that with Python


00:33:14.140 --> 00:33:16.980
with any programming language, I mean, honestly,


00:33:16.980 --> 00:33:19.420
but you can do it also that with Python version


00:33:19.420 --> 00:33:20.620
to switch to any other language


00:33:20.620 --> 00:33:22.220
if you know what you're doing.


00:33:22.220 --> 00:33:24.860
- Right, it makes such an important difference there.


00:33:24.860 --> 00:33:26.260
All right, so let's go, I thought it'd be fun


00:33:26.260 --> 00:33:28.420
to go through a couple of the chapters of your book


00:33:28.420 --> 00:33:32.340
and just talk about some of the big ideas there.


00:33:32.340 --> 00:33:34.780
And the first, you kind of build your way up


00:33:34.780 --> 00:33:36.460
to larger systems, right?


00:33:36.460 --> 00:33:38.060
Like you started out,


00:33:38.060 --> 00:33:39.220
we already talked about what is scaling,


00:33:39.220 --> 00:33:42.100
but the next one that you really focused on is,


00:33:42.100 --> 00:33:44.460
how do I scale to take full advantage


00:33:44.460 --> 00:33:46.060
of my current computer?


00:33:46.060 --> 00:33:49.180
The one I'm recording on here is my Mac mini M1.


00:33:49.180 --> 00:33:50.820
It has four cores.


00:33:50.820 --> 00:33:53.420
Over there, I have my sim racing setup.


00:33:53.420 --> 00:33:55.380
It has 16 cores.


00:33:55.380 --> 00:33:56.940
Let's suppose I'm running on that one.


00:33:56.940 --> 00:33:58.620
If I run my Python code over there


00:33:58.620 --> 00:33:59.660
and I create a bunch of threads


00:33:59.660 --> 00:34:01.020
and it says a bunch of Python things,


00:34:01.020 --> 00:34:05.220
there's a good chance it's using 1/16 of that CPU, right?


00:34:05.220 --> 00:34:06.780
- Yeah, yeah, exactly.


00:34:06.780 --> 00:34:09.460
It's, I mean, people who start with Python,


00:34:09.460 --> 00:34:13.220
usually it's that issue, I mean, pretty soon,


00:34:13.220 --> 00:34:15.120
where you want to run multiple,


00:34:15.120 --> 00:34:18.260
I mean, you want to run multiple threads in parallel,


00:34:18.260 --> 00:34:20.260
for example, to make sure your code is faster,


00:34:20.260 --> 00:34:23.100
and then, which is a proper way, I mean,


00:34:23.100 --> 00:34:24.820
outside Python, it's a proper way to scale,


00:34:24.820 --> 00:34:26.140
like running your threads,


00:34:26.140 --> 00:34:30.180
allow you to run another execution thread


00:34:30.180 --> 00:34:33.820
of your program in another CPU.


00:34:33.820 --> 00:34:36.020
And Threads were not used that much 20 years ago


00:34:36.020 --> 00:34:37.780
because all computers, every computer


00:34:37.780 --> 00:34:38.980
had only one core, right?


00:34:38.980 --> 00:34:41.620
I mean, your personal computer, it was a Pentium


00:34:41.620 --> 00:34:44.060
with only one core and nobody cared about the Threads.


00:34:44.060 --> 00:34:47.300
Now that everybody has 16 cores in their pockets,


00:34:47.300 --> 00:34:49.980
it's like, "Whoa, we should do something about Threads."


00:34:49.980 --> 00:34:54.980
So, yeah, that's where you started 10 years ago,


00:34:54.980 --> 00:34:56.820
seeing more and more people being interested


00:34:56.820 --> 00:34:58.980
in using Threads in Python because,


00:34:58.980 --> 00:35:02.320
Well, I'm doing this computation, and I could do it twice to go faster,


00:35:02.320 --> 00:35:04.920
so I'm spawning on new threads, and we got...


00:35:04.920 --> 00:35:07.920
Except that if you do that in Python, it doesn't work very well,


00:35:07.920 --> 00:35:12.160
because there's this global interpreter lock, the GIL,


00:35:12.160 --> 00:35:16.720
which actually makes sure that your Python code works nice


00:35:16.720 --> 00:35:20.600
on multiple threads, but every thread running Python code,


00:35:20.600 --> 00:35:23.300
executing bytecode, they have to acquire this lock,


00:35:23.300 --> 00:35:28.240
and L2 it forever until they're finished or until they get interrupted,


00:35:28.240 --> 00:35:32.640
which means you can only have one thread in Python running at a time


00:35:32.640 --> 00:35:35.640
on the Python VM and the other threads have to wait.


00:35:35.640 --> 00:35:38.280
Or do other stuff which are not Python-related,


00:35:38.280 --> 00:35:41.440
which is what a lot of C extensions like NumPy


00:35:41.440 --> 00:35:44.400
or other C extensions you may be using are doing.


00:35:44.400 --> 00:35:47.640
They're releasing the GIL and doing things which are not Python,


00:35:47.640 --> 00:35:49.640
but still doing things that are useful for you.


00:35:49.640 --> 00:35:52.080
But in that case, they're able to release the GIL


00:35:52.080 --> 00:35:54.720
and let the rest of the Python program run.


00:35:54.720 --> 00:35:57.240
But if your program is 100% Python,


00:35:57.240 --> 00:35:59.320
don't use any kind of C extension,


00:35:59.320 --> 00:36:01.320
not that you've got anything, then all


00:36:01.320 --> 00:36:03.080
your threads have this giant bottleneck,


00:36:03.080 --> 00:36:04.520
which is a gil,


00:36:04.520 --> 00:36:06.920
which blocks every thread. So if you run...


00:36:06.920 --> 00:36:09.880
I think my record is like


00:36:09.880 --> 00:36:13.400
1.6 cores with a ton of threads


00:36:13.400 --> 00:36:15.720
on the Python program. You can't really


00:36:15.720 --> 00:36:16.440
use...


00:36:16.440 --> 00:36:18.760
I never managed to use two cores with a


00:36:18.760 --> 00:36:19.720
Python,


00:36:19.720 --> 00:36:21.800
I mean a single Python program and a lot


00:36:21.800 --> 00:36:24.280
of threads. It's very hard to get to that


00:36:24.280 --> 00:36:26.520
two cores being used. So when you have a


00:36:26.520 --> 00:36:31.320
3264 cores machine that you rent or that you use,


00:36:31.320 --> 00:36:34.360
it's a pretty waste of resources.


00:36:34.360 --> 00:36:36.920
Yeah. So this is super interesting.


00:36:36.920 --> 00:36:41.480
And people often see this as like Python doing a threading restriction.


00:36:41.480 --> 00:36:44.280
And what's really interesting is the GIL


00:36:44.280 --> 00:36:50.200
is really about protecting the memory allocation and cleanup, right?


00:36:50.200 --> 00:36:53.000
It's incrementing and decrementing that ref count.


00:36:53.000 --> 00:36:56.440
So you don't have to take a lock every time you touch an object


00:36:56.440 --> 00:36:58.840
or assign a variable, which would make it really slow.


00:36:58.840 --> 00:37:01.240
It'd be more scalable, but it would be slower,


00:37:01.240 --> 00:37:03.560
even in the single use case.


00:37:03.560 --> 00:37:05.560
Yeah, there are many experiences to do that.


00:37:05.560 --> 00:37:08.200
This is actually what you have in Java.


00:37:08.200 --> 00:37:10.200
There are this kind of monitor, I think they call it,


00:37:10.200 --> 00:37:11.880
where you have a lock per object,


00:37:11.880 --> 00:37:15.400
and it works well for them, but don't ask me the details.


00:37:15.400 --> 00:37:18.600
But for Python, there are a few experiments to do that,


00:37:18.600 --> 00:37:22.200
and yeah, it makes the thing very, very slower, unfortunately.


00:37:22.200 --> 00:37:25.800
so it's not a good option to do to go that road.


00:37:25.800 --> 00:37:28.760
And I mean, there have been, if you look at the history of the Guild,


00:37:28.760 --> 00:37:33.240
there has been a project called the Gilectomy a few years ago to remove the Guild.


00:37:33.240 --> 00:37:37.320
I mean, there have been plenty of experiments to try to get rid of that.


00:37:37.320 --> 00:37:40.760
And the other problem is that if we ever do that at some point,


00:37:40.760 --> 00:37:43.960
it will break the language and a lot of assumptions around the language.


00:37:43.960 --> 00:37:47.400
Because like in Python, when you add an item to a list,


00:37:47.400 --> 00:37:51.000
it is thread safe by definition, because of the Guild for sure.


00:37:51.000 --> 00:37:55.000
But if we start by saying, "Well, each time you want to


00:37:55.000 --> 00:37:59.000
add an item to a list, you need to use a lock to do that."


00:37:59.000 --> 00:38:02.180
Either we do it implicitly, but it's very slow,


00:38:02.180 --> 00:38:04.680
or you do it explicitly as a programmer,


00:38:04.680 --> 00:38:06.680
then it's going to be very tedious, for sure,


00:38:06.680 --> 00:38:10.840
and it's not going to be compatible with the Python we know right now,


00:38:10.840 --> 00:38:13.840
which is not a good option. So we are stuck.


00:38:13.840 --> 00:38:18.700
Well, have you been tracking the work


00:38:18.700 --> 00:38:22.540
on PEP 554 and multiple sub-interpreters


00:38:22.540 --> 00:38:24.460
with what Eric Snow has been doing.


00:38:24.460 --> 00:38:25.580
Yeah, a little bit.


00:38:25.580 --> 00:38:29.500
Which, yeah, that offers some really interesting opportunities there.


00:38:29.500 --> 00:38:32.460
Yeah, I think it's a different approach,


00:38:32.460 --> 00:38:36.300
but it's a mix, like it's a trade-off between


00:38:36.300 --> 00:38:38.060
multi-threading and multi-processing.


00:38:38.060 --> 00:38:41.100
Yeah, it's like a blend, like a half and half of them, yeah.


00:38:41.100 --> 00:38:44.940
But I think it's the most promising thing we have right now,


00:38:44.940 --> 00:38:47.500
because I don't think the GIL is going to go away anytime soon,


00:38:47.500 --> 00:38:53.660
unless somebody really take a giant project and do that.


00:38:53.660 --> 00:38:57.180
But there's nobody, unfortunately, outside,


00:38:57.180 --> 00:38:58.860
I mean, inside the Python community,


00:38:58.860 --> 00:39:02.140
there's no company trying to sponsor any kind of effort like that.


00:39:02.140 --> 00:39:05.980
A lot of the Python upstream stuff, from what I see,


00:39:05.980 --> 00:39:09.260
are run by people willing to do that on their free time.


00:39:09.260 --> 00:39:12.140
And some are sponsored, for sure, or hired by a company,


00:39:12.140 --> 00:39:14.140
but a lot of them are not.


00:39:14.140 --> 00:39:19.500
and there are nobody like a giant big tech company trying to push something like that forward.


00:39:19.500 --> 00:39:22.940
So it's probably what's also not helping Python.


00:39:22.940 --> 00:39:25.820
So it's the multi-interpreter thing is probably the next best thing we'll get.


00:39:25.820 --> 00:39:30.460
I think it is as well because I just don't see the gig going away


00:39:30.460 --> 00:39:32.700
unless we say we're going to give up reference counting.


00:39:32.700 --> 00:39:37.500
And if you give up reference counting and then you add like a JIT and you get like,


00:39:37.500 --> 00:39:39.260
I mean, that's a really different change.


00:39:39.260 --> 00:39:43.900
And if you think of all the trouble just changing strings from Python 2 to Python 3,


00:39:43.900 --> 00:39:44.740
- Yeah.


00:39:44.740 --> 00:39:46.060
- Like this is the next level, right?


00:39:46.060 --> 00:39:47.380
It's crazy.


00:39:47.380 --> 00:39:48.580
- Okay, we're not finished yet.


00:39:48.580 --> 00:39:51.020
I still have to maintain a lot of Python 2 code,


00:39:51.020 --> 00:39:51.860
to be honest.


00:39:51.860 --> 00:39:55.180
So I'm not ready to do Python 4 yet.


00:39:55.180 --> 00:39:56.580
- I don't think we're ready for it either.


00:39:56.580 --> 00:39:58.380
So I think subinterpreters is interesting


00:39:58.380 --> 00:40:01.980
and subinterpreters are interesting because


00:40:01.980 --> 00:40:08.340
they take the ways in which you sort of do multiprocessing,


00:40:08.340 --> 00:40:10.420
which does work for scaling out, right?


00:40:10.420 --> 00:40:12.020
You kind of do message passing


00:40:12.020 --> 00:40:17.300
do message passing and each process owns its own data structures and stuff like that.


00:40:17.300 --> 00:40:20.340
And it just says, well, you don't have to create processes to make that happen.


00:40:20.340 --> 00:40:22.340
So it's faster, basically.


00:40:22.340 --> 00:40:27.700
Yeah, and I think one of the problems with multiprocessing is also serializing the data


00:40:27.700 --> 00:40:33.700
between the different process, which is always, I think Stack Overflow is filled with that.


00:40:33.700 --> 00:40:39.180
People are complaining about unable to pick all the data between multiple processes, which


00:40:39.180 --> 00:40:40.180
is very tedious.


00:40:40.180 --> 00:40:44.540
I hope that having the subatom thing will solve part of that,


00:40:44.540 --> 00:40:45.900
not having to serialize everything.


00:40:45.900 --> 00:40:47.900
Also in terms of performance, in terms of usage for sure,


00:40:47.900 --> 00:40:49.460
but also in terms of performance,


00:40:49.460 --> 00:40:51.940
you don't have to serialize and serialize everything


00:40:51.940 --> 00:40:55.020
every time you want to pass something to a subprocess


00:40:55.020 --> 00:40:58.020
with a very huge serial.


00:40:58.020 --> 00:41:01.900
- So in danger of not making it all the way


00:41:01.900 --> 00:41:04.500
through the other topics, let me ask you,


00:41:04.500 --> 00:41:07.060
just a couple of other real quick questions or comments


00:41:07.060 --> 00:41:09.140
I'll let you call out a couple of things.


00:41:09.140 --> 00:41:12.140
One, like this CPU scaling is a problem,


00:41:12.140 --> 00:41:14.140
except for when it's not.


00:41:14.140 --> 00:41:16.140
Like sometimes it's not a problem at all.


00:41:16.140 --> 00:41:18.140
And the example I'm thinking of, if I'm writing an API,


00:41:18.140 --> 00:41:20.140
if I'm writing a website, we need those things.


00:41:20.140 --> 00:41:23.140
The way that you host those is you go to a server,


00:41:23.140 --> 00:41:26.140
or you use a platform as a service, which does this for you,


00:41:26.140 --> 00:41:29.140
and you run it in something like microWSGI


00:41:29.140 --> 00:41:31.140
or gUnicorn or something.


00:41:31.140 --> 00:41:33.140
And what those do immediately is they say,


00:41:33.140 --> 00:41:35.140
"Well, we're not really going to run it in the main process.


00:41:35.140 --> 00:41:37.140
The main process is going to look at the other ones,


00:41:37.140 --> 00:41:41.140
I want to create four or eight or 10 copies of your app running


00:41:41.140 --> 00:41:45.140
all at the same time and it will send the request off to these different processes.


00:41:45.140 --> 00:41:49.140
And then all of a sudden, hey, if you have less than 10 cores,


00:41:49.140 --> 00:41:51.140
you're scaling again.


00:41:51.140 --> 00:41:56.140
- Yeah, exactly. So that's why threads are great for things like I/O,


00:41:56.140 --> 00:41:59.140
but if you don't really want to scale for CPU and cores,


00:41:59.140 --> 00:42:01.140
threads are not the right solution.


00:42:01.140 --> 00:42:05.140
And it's way better to use multiple processes.


00:42:05.140 --> 00:42:09.140
Either way, Unicorn, USB, very good solution for web apps.


00:42:09.140 --> 00:42:14.000
Or there are alternatives to that.


00:42:14.000 --> 00:42:19.780
Or a framework like Celery for doing tasks, for example,


00:42:19.780 --> 00:42:22.920
which is out of the box to spawn multiple processes


00:42:22.920 --> 00:42:25.920
to handle all of your tasks on multiple CPUs.


00:42:25.920 --> 00:42:30.140
And usually, if you don't choose any kind of


00:42:30.140 --> 00:42:35.140
E-Sync, IO-like framework, or Donado, or anything like that,


00:42:35.140 --> 00:42:38.780
where you only have one process running one task at a time,


00:42:38.780 --> 00:42:42.220
you can spawn multiple processes, even more processes than you have cores.


00:42:42.220 --> 00:42:45.920
If you have 16 cores, you can start, I don't know, 100 processes


00:42:45.920 --> 00:42:48.220
if you have enough memory, for sure.


00:42:48.220 --> 00:42:51.520
But memory is really not a problem, unless you...


00:42:51.520 --> 00:42:54.420
It depends on what you do, for sure. But for a REST API,


00:42:54.420 --> 00:42:57.220
it's really not a big problem. You're not using gigabytes of memory per process


00:42:57.220 --> 00:43:00.820
and better regress. So it's fine. It's winning a lot of Unicorn walkers.


00:43:00.820 --> 00:43:05.940
Yeah, it depends on that for sure. So two things that I ran across that were


00:43:05.940 --> 00:43:11.060
interested in this chapter that you covered were Futurist and Kotlin.


00:43:11.060 --> 00:43:13.140
I'm not sure how you say that second one. Yeah.


00:43:13.140 --> 00:43:16.740
Can you tell people about these two little helper library packages?


00:43:16.740 --> 00:43:21.220
Yeah, Futurist is actually, it's a tiny wrapper around concurrency on the futures


00:43:21.220 --> 00:43:26.740
that you might know in Python. The thing, it does a few things that are not there,


00:43:26.740 --> 00:43:31.740
like the ability to have statistics about your pool of threads


00:43:31.740 --> 00:43:35.080
or your pool of anything you use process,


00:43:35.080 --> 00:43:38.080
which gives you a pretty good idea.


00:43:38.080 --> 00:43:40.220
A lot of applications out there, they're like,


00:43:40.220 --> 00:43:43.220
"Okay, I can scale on 32 threads, 64,"


00:43:43.220 --> 00:43:45.420
and you have a setting usually to scale that,


00:43:45.420 --> 00:43:48.780
and you don't really know as a user or even as a developer


00:43:48.780 --> 00:43:50.880
how many threads you're supposed to start and to run


00:43:50.880 --> 00:43:54.040
to handle your workload.


00:43:54.040 --> 00:43:57.640
you're just typing a number randomly and see if it works or not.


00:43:57.640 --> 00:44:00.360
And having statistics around that is pretty useful.


00:44:00.360 --> 00:44:02.360
There are some features, if I remember correctly,


00:44:02.360 --> 00:44:05.480
where you can actually control the backlog.


00:44:05.480 --> 00:44:08.120
Usually, you have a pool of threads or processes


00:44:08.120 --> 00:44:11.800
or a pool of anything trying to handle your task.


00:44:11.800 --> 00:44:14.440
But the more you add, it can grow forever,


00:44:14.440 --> 00:44:18.520
so I think the ability to control your backlog


00:44:18.520 --> 00:44:21.000
and say, "Okay, I have enough tasks in the queue,


00:44:21.000 --> 00:44:23.000
now you have to do something."


00:44:23.000 --> 00:44:26.000
I'm not going to take any more tags.


00:44:26.000 --> 00:44:29.000
So that's the pattern you see a lot in Q system.


00:44:29.000 --> 00:44:31.000
Usually people when they design a Q system,


00:44:31.000 --> 00:44:33.000
they design the Q system with like, there's a Q,


00:44:33.000 --> 00:44:36.000
I'm going to take things out of it and process them.


00:44:36.000 --> 00:44:38.000
And they don't think about controlling the size of the Q,


00:44:38.000 --> 00:44:42.000
so the Q can grow forever, which in theory is great,


00:44:42.000 --> 00:44:44.000
but in practice you don't have infinite resources


00:44:44.000 --> 00:44:46.000
to store the Q and then to process it,


00:44:46.000 --> 00:44:49.000
so you won't be able to reject works.


00:44:49.000 --> 00:44:52.000
Exactly, and one of the big complaints or criticisms,


00:44:52.000 --> 00:44:55.000
I guess I should say is in these async systems,


00:44:55.000 --> 00:44:57.000
they don't provide back pressure.


00:44:57.000 --> 00:45:00.000
A bunch of work comes into the front and it piles into asyncio,


00:45:00.000 --> 00:45:03.000
which then piles just massively on top of the database,


00:45:03.000 --> 00:45:05.000
and then the database dies.


00:45:05.000 --> 00:45:09.000
And there's no place further back before the dying of the database


00:45:09.000 --> 00:45:12.000
where it kind of slows down.


00:45:12.000 --> 00:45:15.000
And so this is something that would allow you to do that


00:45:15.000 --> 00:45:17.000
for threading and multiprocessing, right?


00:45:17.000 --> 00:45:19.000
Yeah, exactly.


00:45:19.000 --> 00:45:22.440
And which is one of the other chapters of the book,


00:45:22.440 --> 00:45:25.600
which, well, that title is Design for Failure.


00:45:25.600 --> 00:45:27.640
And you could write an entire book on that,


00:45:27.640 --> 00:45:31.040
which is when you write your application,


00:45:31.040 --> 00:45:34.160
usually you write something in a very optimistic way


00:45:34.160 --> 00:45:35.720
because you are in a good mood and you're like,


00:45:35.720 --> 00:45:37.200
everything's going to work, right?


00:45:37.200 --> 00:45:38.040
And the reality is--


00:45:38.040 --> 00:45:41.240
- Well, you test with small data and few clients, right?


00:45:41.240 --> 00:45:42.080
- Exactly.


00:45:42.080 --> 00:45:43.960
And the more you scale,


00:45:43.960 --> 00:45:45.360
like the more you add threads,


00:45:45.360 --> 00:45:46.320
the more you add processes,


00:45:46.320 --> 00:45:47.400
the more you add nodes,


00:45:47.400 --> 00:45:50.280
or your network, you're going to use Kubernetes


00:45:50.280 --> 00:45:53.200
to spawn hundreds of nodes and versions of your application.


00:45:53.200 --> 00:45:54.800
And the more likely it is to fail,


00:45:54.800 --> 00:45:56.840
somebody's going to unplug a cable somewhere,


00:45:56.840 --> 00:45:58.640
anything can happen for sure.


00:45:58.640 --> 00:46:01.240
And you're not designing for that.


00:46:01.240 --> 00:46:03.680
Usually you're designing in a very optimistic way


00:46:03.680 --> 00:46:05.360
because most of the time it works.


00:46:05.360 --> 00:46:08.160
And when it doesn't, if you really want to go at scale,


00:46:08.160 --> 00:46:10.160
I mean, if you really want to go far


00:46:10.160 --> 00:46:13.880
and you want to work even in extreme conditions


00:46:13.880 --> 00:46:18.380
when the weather is really rainy, it's a lot of work.


00:46:18.380 --> 00:46:20.460
So that's why I was saying at the beginning,


00:46:20.460 --> 00:46:22.180
like it's a trade-off to add even threads


00:46:22.180 --> 00:46:24.400
'cause then you have to, and what happens


00:46:24.400 --> 00:46:26.420
when I can't start a new thread anymore


00:46:26.420 --> 00:46:28.140
because I don't know my system is out of,


00:46:28.140 --> 00:46:29.460
which is pretty rare nowadays.


00:46:29.460 --> 00:46:31.260
You have a lot of space for threads usually,


00:46:31.260 --> 00:46:33.340
but if you are on a, I mean,


00:46:33.340 --> 00:46:35.700
very limited resource system or whatever,


00:46:35.700 --> 00:46:36.900
like what do you want out of that?


00:46:36.900 --> 00:46:38.380
- Threads pre-allocate a lot of memory,


00:46:38.380 --> 00:46:39.820
like stack space and stuff.


00:46:39.820 --> 00:46:41.260
- Yeah.


00:46:41.260 --> 00:46:43.420
- Have you heard of Locus at locus.io?


00:46:43.420 --> 00:46:44.540
Have you seen this thing?


00:46:44.540 --> 00:46:46.260
- No, I don't think so.


00:46:46.260 --> 00:46:47.620
- Yeah, so speaking of back pressure


00:46:47.620 --> 00:46:48.740
and just knowing what your system can take,


00:46:48.740 --> 00:46:50.800
this thing is super, super cool.


00:46:50.800 --> 00:46:54.260
It's a Python load testing thing


00:46:54.260 --> 00:46:56.200
that allows you to even do distributed load.


00:46:56.200 --> 00:46:57.560
But what you do that's really interesting


00:46:57.560 --> 00:47:00.780
is you create a class in Python


00:47:00.780 --> 00:47:03.500
and you say you give it tasks


00:47:03.500 --> 00:47:05.020
and then those tasks have URLs.


00:47:05.020 --> 00:47:06.220
And then you can say, well,


00:47:06.220 --> 00:47:07.780
this one I want 70% of the time,


00:47:07.780 --> 00:47:08.780
the user's gonna go there


00:47:08.780 --> 00:47:09.940
and then they're gonna go here.


00:47:09.940 --> 00:47:11.980
And then I want you to kind of delay


00:47:11.980 --> 00:47:13.140
to like, they click around,


00:47:13.140 --> 00:47:15.860
maybe every 10 seconds or so, so randomly around that time,


00:47:15.860 --> 00:47:16.860
have them click around.


00:47:16.860 --> 00:47:19.700
And it's just a really good library for a tool for people


00:47:19.700 --> 00:47:23.660
who are like, "Well, I didn't test it with enough users


00:47:23.660 --> 00:47:24.660
because I'm just me."


00:47:24.660 --> 00:47:25.660
But what does it actually...


00:47:25.660 --> 00:47:28.460
Like something like this would be a really good option, I think.


00:47:28.460 --> 00:47:29.460
- Yeah, that's a good...


00:47:29.460 --> 00:47:32.860
Even to gather data for profiling after,


00:47:32.860 --> 00:47:33.860
that's a pretty good...


00:47:33.860 --> 00:47:34.860
If you're about to reproduce something...


00:47:34.860 --> 00:47:35.860
- Oh, yeah.


00:47:35.860 --> 00:47:36.860
- Yeah.


00:47:36.860 --> 00:47:37.860
- That's interesting.


00:47:37.860 --> 00:47:38.860
- Yeah.


00:47:38.860 --> 00:47:41.140
- Because you want to profile a realistic scenario.


00:47:41.140 --> 00:47:44.140
So instead of hitting one person, you hit it with like 100


00:47:44.140 --> 00:47:46.140
and then get the profiling results of that.


00:47:46.140 --> 00:47:48.140
Okay, that's a good idea.


00:47:48.140 --> 00:47:51.140
Yeah, that's really the good thing with continuous profiling


00:47:51.140 --> 00:47:53.140
is that you're able to see for real what happens.


00:47:53.140 --> 00:47:57.140
But if you know how to reproduce it, that's also a valuable option.


00:47:57.140 --> 00:47:59.140
Yeah, okay, very interesting.


00:47:59.140 --> 00:48:01.140
All right, so CPU scaling is interesting.


00:48:01.140 --> 00:48:06.140
And then Python around 3.4.3.5 came out with this really interesting idea


00:48:06.140 --> 00:48:09.140
of async.io and especially async.await


00:48:09.140 --> 00:48:11.140
that makes AsyncIO so much better to work with.


00:48:11.140 --> 00:48:15.460
And what's interesting is that has nothing to do with threading.


00:48:15.460 --> 00:48:17.460
That doesn't suffer any of the problems of,


00:48:17.460 --> 00:48:20.980
almost, the problems of the GIL and so on,


00:48:20.980 --> 00:48:22.980
because it's all about waiting on things.


00:48:22.980 --> 00:48:24.980
And when you're waiting, usually you release the GIL.


00:48:24.980 --> 00:48:30.500
Yeah. So threads are a good solution for I/O


00:48:30.500 --> 00:48:32.500
when you can't actually use something like AsyncIO,


00:48:32.500 --> 00:48:36.180
because let's be honest, if you're using an old library


00:48:36.180 --> 00:48:38.180
which was designed five years ago,


00:48:38.180 --> 00:48:40.660
it's not designed to be a single also.


00:48:40.660 --> 00:48:43.060
- Well, you're using an old ORM


00:48:43.060 --> 00:48:46.300
and the old ORM doesn't have an async version of it.


00:48:46.300 --> 00:48:48.780
And it's either rewrite it in a new one


00:48:48.780 --> 00:48:50.220
or use the old non-async, right?


00:48:50.220 --> 00:48:51.060
Something like that.


00:48:51.060 --> 00:48:53.260
Then maybe threads, I don't know.


00:48:53.260 --> 00:48:57.180
- Yeah, usually, I mean, ORM, it's a good bad example


00:48:57.180 --> 00:48:58.740
in the sense that it's a good example technically,


00:48:58.740 --> 00:49:02.580
but usually the problem is people writing bad queries.


00:49:02.580 --> 00:49:03.820
- Yeah, I know.


00:49:03.820 --> 00:49:05.020
Better queries than an index.


00:49:05.020 --> 00:49:06.140
I'll probably solve that most of the time.


00:49:06.140 --> 00:49:06.980
- Exactly.


00:49:06.980 --> 00:49:11.980
But in theory, you're right.


00:49:11.980 --> 00:49:13.380
Technically, it's a good example.


00:49:13.380 --> 00:49:15.340
And yeah, it looks like it's in Kayo.


00:49:15.340 --> 00:49:17.420
It's magic because like you were saying,


00:49:17.420 --> 00:49:19.020
it's like the node thing that brought that back to life


00:49:19.020 --> 00:49:22.020
where it has been used and there for the last 40 years.


00:49:22.020 --> 00:49:25.420
I don't know.


00:49:25.420 --> 00:49:27.340
And it's like suddenly everybody's like,


00:49:27.340 --> 00:49:29.180
"Wow, that's amazing."


00:49:29.180 --> 00:49:30.140
And that's how you would write anyway,


00:49:30.140 --> 00:49:31.860
any web servers for the last 30 years.


00:49:31.860 --> 00:49:34.540
But it's great.


00:49:34.540 --> 00:49:35.580
Now it's built in in Python,


00:49:35.580 --> 00:49:36.660
so it's pretty easy to use and it's very, very well.


00:49:35.100 --> 00:49:38.380
I mean, I think it progressed a lot over the years.


00:49:38.380 --> 00:49:40.660
Like a couple of years ago,


00:49:40.660 --> 00:49:43.420
where everybody was using Flask or Django,


00:49:43.420 --> 00:49:44.380
which is still true,


00:49:44.380 --> 00:49:47.900
but there's a lot, a lot of better alternative


00:49:47.900 --> 00:49:50.620
in the sense that like Starlette, et cetera,


00:49:50.620 --> 00:49:55.620
but the first API that you can use to build an API website.


00:49:55.620 --> 00:49:59.180
- Yeah, this whole Async.io world has really flourished


00:49:59.180 --> 00:50:00.780
since you wrote the book, hasn't it?


00:50:00.780 --> 00:50:01.620
- Yeah.


00:50:01.620 --> 00:50:02.460
- Yeah.


00:50:02.460 --> 00:50:03.700
- Yeah, when I wrote the book,


00:50:03.700 --> 00:50:07.700
I actually learned async.io writing the book and there was nothing.


00:50:07.700 --> 00:50:11.700
The main problem is that I want to use Redis


00:50:11.700 --> 00:50:15.700
or like you were saying, a database and there's nothing.


00:50:15.700 --> 00:50:19.700
Or it's very low-level stuff and it's not going to be...


00:50:19.700 --> 00:50:23.700
Yeah, I can use it but it's going to take me hours of work to get anything


00:50:23.700 --> 00:50:27.700
close to what I would do with a synchronous version.


00:50:27.700 --> 00:50:31.700
Nowadays, it's totally better. I actually do a lot of work with async.io myself.


00:50:31.700 --> 00:50:35.700
I mean, everything's there for every


00:50:35.700 --> 00:50:39.700
thing doing socket.io, file.io, everything is available. There are


00:50:39.700 --> 00:50:43.460
sometimes multiple versions of the same library because


00:50:43.460 --> 00:50:47.060
people don't agree on how to do Redis or whatever, which


00:50:47.060 --> 00:50:51.940
gives you choice. So that's great and it's a very, very good way to


00:50:51.940 --> 00:50:54.900
not use threads. So you still have the concurrency problem where you can have


00:50:54.900 --> 00:50:59.460
multiple tasks from a single running, not at the same time in our space-time


00:50:59.460 --> 00:51:03.460
dimension, but being static means pause, being resumed later,


00:51:03.460 --> 00:51:06.980
so you have to still take care and you actually can use lock too


00:51:06.980 --> 00:51:12.100
in SDKO, but it's still a little less of a problem and issue with threads,


00:51:12.100 --> 00:51:15.300
and you're not going to use more than one CPU, for sure,


00:51:15.300 --> 00:51:18.340
that's not designed to do that, but you will be able,


00:51:18.340 --> 00:51:21.460
maybe more easily, because you will have less overhead


00:51:21.460 --> 00:51:24.500
than with threads to be able to use 100% of your CPU,


00:51:24.500 --> 00:51:28.260
like we were able to max out your CPU resource,


00:51:28.260 --> 00:51:30.660
And then when you do that with one Python process,


00:51:30.660 --> 00:51:35.660
well, just start a new one using Celery, Unicorn, whatever,


00:51:35.660 --> 00:51:39.460
or Quotidien, you were mentioning, which is a good tool to do that,


00:51:39.460 --> 00:51:43.060
which is actually able to spawn multiple processes


00:51:43.060 --> 00:51:45.360
and manage them for you because usually that's the problem


00:51:45.360 --> 00:51:48.960
when you have a demand, you want to do a lot of work.


00:51:48.960 --> 00:51:51.760
Like the Celery model for Q is a pretty good example


00:51:51.760 --> 00:51:55.860
when you have multiple workers and each worker is on thread


00:51:55.860 --> 00:51:59.860
doing things in the background if you're not using a framework such as Celery.


00:51:59.860 --> 00:52:03.860
And ContinuDone is a good small library to do that where you can


00:52:03.860 --> 00:52:07.860
implement a class and have each class being


00:52:07.860 --> 00:52:11.860
spun as a process, basically, and be managed by a master process


00:52:11.860 --> 00:52:15.860
like you would do with UWSKI or Unicorn, and managing the child,


00:52:15.860 --> 00:52:19.860
restarting them if they die, etc. So that's a lot of work to do. You can


00:52:19.860 --> 00:52:23.860
totally do that yourself, but ContinuDone does that for you out of the box.


00:52:23.860 --> 00:52:28.100
- Yeah, that's a cool way to create those sub processes


00:52:28.100 --> 00:52:31.980
and stuff, but yeah, I think async.io is a lot of promise.


00:52:31.980 --> 00:52:34.300
It's coming along, it's really been growing.


00:52:34.300 --> 00:52:37.960
SQLAlchemy just released their 1.4 version,


00:52:37.960 --> 00:52:40.740
which actually, so SQLAlchemy now just in like


00:52:40.740 --> 00:52:45.420
a couple of weeks ago now supports a wait session query


00:52:45.420 --> 00:52:47.380
of thing, not exactly that syntax,


00:52:47.380 --> 00:52:50.540
but almost, they've slightly adjusted it, but pretty cool.


00:52:50.540 --> 00:52:52.260
All right, and then one of the things that you talked about


00:52:52.260 --> 00:52:56.020
of scaling that I agree on that is super important


00:52:56.020 --> 00:52:57.660
is statelessness.


00:52:57.660 --> 00:53:02.020
So if you wanna, and I suspect going from one to two


00:53:02.020 --> 00:53:06.300
is harder than going from two to 10


00:53:06.300 --> 00:53:07.940
in terms of scaling, right?


00:53:07.940 --> 00:53:10.520
Soon as you're like, okay, this is gonna be in two places.


00:53:10.520 --> 00:53:13.280
That means it has to be stateless and it's communication.


00:53:13.280 --> 00:53:14.120
All right, all these things.


00:53:14.120 --> 00:53:15.660
If you're just putting stuff in memory


00:53:15.660 --> 00:53:17.500
and sharing the pointers and kind of storing


00:53:17.500 --> 00:53:20.140
a persistent memory session for somebody,


00:53:20.140 --> 00:53:22.020
well then putting that in two places


00:53:22.020 --> 00:53:24.020
is really fraught.


00:53:24.020 --> 00:53:26.020
Yeah, it's really the thing where


00:53:26.020 --> 00:53:28.020
I like to say that if you start


00:53:28.020 --> 00:53:30.020
using multiple processes, you're actually


00:53:30.020 --> 00:53:32.020
pretty ready to handle multiple nodes


00:53:32.020 --> 00:53:34.020
like over a network because


00:53:34.020 --> 00:53:36.020
using multiple threads, you are always


00:53:36.020 --> 00:53:38.020
in the same program, so it's tempting to


00:53:38.020 --> 00:53:40.020
share the state of everything between


00:53:40.020 --> 00:53:42.020
your different threads, and then you have


00:53:42.020 --> 00:53:44.020
concurrency issues and you need to lock, etc.


00:53:44.020 --> 00:53:46.020
But a lot of people go that road


00:53:46.020 --> 00:53:48.020
being, I don't know, maybe


00:53:48.020 --> 00:53:50.020
a bit naive and saying, "Oh, that's easy.


00:53:50.020 --> 00:53:54.580
have to leave my program, but if you


00:53:54.580 --> 00:53:56.340
are ready to go to the step where you


00:53:56.340 --> 00:53:57.540
actually okay to split your work


00:53:57.540 --> 00:53:59.620
into multiple processes,


00:53:59.620 --> 00:54:00.900
which might have to communicate between


00:54:00.900 --> 00:54:02.980
themselves for sure,


00:54:02.980 --> 00:54:04.420
and they can start by communicating over


00:54:04.420 --> 00:54:06.900
the same host,


00:54:06.900 --> 00:54:08.740
but then you just add networks in


00:54:08.740 --> 00:54:10.260
between the process and you can scale to


00:54:10.260 --> 00:54:12.820
multiple


00:54:12.820 --> 00:54:13.380
nodes and over whatever number of


00:54:13.380 --> 00:54:15.060
nodes you want.


00:54:15.060 --> 00:54:16.820
Then the problem is to handle


00:54:16.820 --> 00:54:18.340
connectivity issue that you don't have


00:54:16.820 --> 00:54:18.820
if you run on a single host, between processes,


00:54:18.820 --> 00:54:22.660
you don't have somebody unplugging the invisible cable.


00:54:22.660 --> 00:54:26.580
But if you're ready to handle all of that network failure,


00:54:26.580 --> 00:54:28.420
which will happen for sure,


00:54:28.420 --> 00:54:29.700
between your different processes,


00:54:29.700 --> 00:54:32.660
then you can scale pretty easily on different nodes.


00:54:32.660 --> 00:54:35.060
But as you were saying, it's like,


00:54:35.060 --> 00:54:37.700
you have to switch your padding when you write your program,


00:54:37.700 --> 00:54:40.020
which is being as stateless as possible.


00:54:40.020 --> 00:54:42.340
Which is why I wrote a non-territorial chapter


00:54:42.340 --> 00:54:43.380
on functional programming,


00:54:43.380 --> 00:54:45.300
because I love functional programming.


00:54:45.300 --> 00:54:50.820
I love lisp and and I would do lisp if it was more popular, but I have to do python


00:54:50.820 --> 00:54:53.140
So i'll do python. It's great. It's a great lisp


00:54:53.140 --> 00:54:59.140
But and and then for sure program gives you a pretty good way of writing code and give you a good mindset


00:54:59.140 --> 00:55:01.140
I was I would say to write code but


00:55:01.140 --> 00:55:06.900
Avoid to do side effects and then that makes your program stateless most of the time which make it very easy to scale


00:55:06.900 --> 00:55:11.380
right the more statelessness you can have the easier it is going to scale and


00:55:12.180 --> 00:55:16.180
you get it down to a point, well, maybe the state is now stored in a Redis server


00:55:16.180 --> 00:55:20.180
that's shared between them or some, or even in a database, like a


00:55:20.180 --> 00:55:24.180
a real common example is just put it in a database, right? So like on


00:55:24.180 --> 00:55:28.180
the training site that I have, people come in, the only piece of state that is shared is


00:55:28.180 --> 00:55:32.180
who is logged in. And when it comes back, it goes back


00:55:32.180 --> 00:55:36.180
to the database, it's okay, well, who is this person actually?


00:55:36.180 --> 00:55:40.180
Do they have a course? Can they access this course? Like all those things are asked every time. And my first


00:55:40.180 --> 00:55:42.940
So my first impression of like writing code like that was like,


00:55:42.940 --> 00:55:45.300
well, if I have to query the database for every request


00:55:45.300 --> 00:55:47.940
to get back all the information about, you know,


00:55:47.940 --> 00:55:51.220
whatever it is I care about for I'm tracking on this request,


00:55:51.220 --> 00:55:54.940
it's going to be so slow, except for it's not really.


00:55:54.940 --> 00:55:56.060
It works really well, actually.


00:55:56.060 --> 00:55:58.540
And it definitely lets you scale better.


00:55:58.540 --> 00:55:59.980
- Yeah.


00:55:59.980 --> 00:56:01.260
- Yeah, yeah, it's pretty interesting.


00:56:01.260 --> 00:56:03.700
Okay, so stateless programming,


00:56:03.700 --> 00:56:05.740
which means like functional programming.


00:56:05.740 --> 00:56:07.580
You want to just like call out that example


00:56:07.580 --> 00:56:11.580
remove last item that you have on the screen here,


00:56:11.580 --> 00:56:14.580
the first page of that book.


00:56:14.580 --> 00:56:17.580
I think it will give people a sense of what you're talking about.


00:56:17.580 --> 00:56:20.580
Yeah, exactly. I was trying to explain in that chapter


00:56:20.580 --> 00:56:23.580
what's a pure and non-pure function, where you actually


00:56:23.580 --> 00:56:26.580
have one function doing the side effect.


00:56:26.580 --> 00:56:29.580
I mean, when you pass argument...


00:56:29.580 --> 00:56:31.580
Functional programming, if you've never heard of it, is pretty simple.


00:56:31.580 --> 00:56:33.580
Imagine that all your functions are black boxes,


00:56:33.580 --> 00:56:35.580
and that you are going to put something in it,


00:56:35.580 --> 00:56:38.580
and when you get something out, you can't reuse the thing that you put inside.


00:56:38.580 --> 00:56:41.580
You're going to use only what's being with put.


00:56:41.580 --> 00:56:44.580
So when you don't do a pure function and function programming,


00:56:44.580 --> 00:56:46.580
you're going to pass a list, for example,


00:56:46.580 --> 00:56:49.580
and you're going to modify it and not returning anything


00:56:49.580 --> 00:56:52.580
because you actually modified the list that you passed as an argument,


00:56:52.580 --> 00:56:54.580
which is not functional at all.


00:56:54.580 --> 00:56:57.580
Maybe like list.sort would be an example, right?


00:56:57.580 --> 00:57:00.580
The thing you're calling sort on is changing the list itself.


00:57:00.580 --> 00:57:01.580
Yeah.


00:57:01.580 --> 00:57:07.820
Yeah, which is, yeah, and that's a trade-off because list.sort is really faster than sorted,


00:57:07.820 --> 00:57:10.780
putting sorted on the list, but it's not functional.


00:57:10.780 --> 00:57:14.380
But if you call sorted or if you return the list minus the last item


00:57:14.380 --> 00:57:18.060
for a function that removes the last item, then it's functional.


00:57:18.060 --> 00:57:20.780
You're not returning the same list, you're writing a new list


00:57:20.780 --> 00:57:24.700
with a different output, like the last item being removed,


00:57:24.700 --> 00:57:29.980
but it is stateless, like you can lose the first, what you put as an input,


00:57:29.980 --> 00:57:32.620
you don't care anymore, you have something that is outside.


00:57:32.620 --> 00:57:34.620
And if you design all your program like that,


00:57:34.620 --> 00:57:39.140
it's pretty easy to imagine having a large input of data,


00:57:39.140 --> 00:57:42.140
putting that into a queue, having a worker taking that,


00:57:42.140 --> 00:57:44.980
doing whatever we need to do, and then putting something


00:57:44.980 --> 00:57:48.340
and putting that into a queue, database, whatever you might want to do.


00:57:48.340 --> 00:57:52.060
And that's the basis of anything that scales,


00:57:52.060 --> 00:57:54.540
is to be able to do that, to be able to scale


00:57:54.540 --> 00:57:57.540
and do asynchronous tasks in the background.


00:57:58.540 --> 00:58:04.380
Cool. Yeah, I think list.sort versus sorted of list is like the perfect comparison there.


00:58:04.380 --> 00:58:04.880
Yeah.


00:58:04.880 --> 00:58:13.020
All right. You touched on queues, and I think queues have this possibility to just allow


00:58:13.020 --> 00:58:20.700
incredible scale, right? Instead of every request trying to answer the question or do the task


00:58:20.700 --> 00:58:26.700
it's meant to do entirely, all it has to do is start a task and say, "Hey, that started. Off we


00:58:26.700 --> 00:58:31.700
and put it into something like RabbitMQ or Celery,


00:58:31.700 --> 00:58:33.800
RedisQ, something like that.


00:58:33.800 --> 00:58:35.960
And some other thing that's gonna pull it out of there


00:58:35.960 --> 00:58:38.440
and get to it when it gets done, right?


00:58:38.440 --> 00:58:39.280
- Yeah, exactly.


00:58:39.280 --> 00:58:41.680
It's really depends on what you do


00:58:41.680 --> 00:58:43.280
and what you're trying to solve


00:58:43.280 --> 00:58:45.880
with your application library, whatever.


00:58:45.880 --> 00:58:50.880
But as a general thing, it's a pretty good way


00:58:50.880 --> 00:58:53.880
and architecture of a program to have that.


00:58:53.880 --> 00:58:56.120
Like if you, for example, a REST API,


00:58:56.120 --> 00:59:00.120
which is what people do most of the time now.


00:59:00.120 --> 00:59:03.680
I mean, you can definitely process a request right away.


00:59:03.680 --> 00:59:05.680
And if you know that it's going to take,


00:59:05.680 --> 00:59:08.200
I don't know, less than one second, okay, it's fine.


00:59:08.200 --> 00:59:09.440
You can do that right away.


00:59:09.440 --> 00:59:11.800
But if you know it's going to take 10 seconds, 20 seconds,


00:59:11.800 --> 00:59:15.160
it's very, very impractical for a client


00:59:15.160 --> 00:59:17.200
to keep a connection open for 30 seconds


00:59:17.200 --> 00:59:19.000
for good and bad reasons.


00:59:19.000 --> 00:59:22.640
- But almost anyone's gonna think it's broken.


00:59:22.640 --> 00:59:25.440
Even if it technically would have worked.


00:59:25.440 --> 00:59:27.880
it's been 10 seconds and there's something wrong.


00:59:27.880 --> 00:59:28.880
This is not okay, right?


00:59:28.880 --> 00:59:32.160
Like it's just not the right response.


00:59:32.160 --> 00:59:34.480
- Yeah, and I mean, connection can be cut.


00:59:34.480 --> 00:59:37.480
So if you need 20 seconds to do anything


00:59:37.480 --> 00:59:40.760
and then it is being cut at 18 seconds,


00:59:40.760 --> 00:59:43.360
then you lost your time and the client has to retry.


00:59:43.360 --> 00:59:45.360
So it has to repost the same payload


00:59:45.360 --> 00:59:47.760
and then you have to reprocess it for 20 seconds.


00:59:47.760 --> 00:59:49.320
So you are actually losing time.


00:59:49.320 --> 00:59:52.320
So it is way better to take the input,


00:59:52.320 --> 00:59:58.560
store it in a queue, reply with 200, "Okay, I got the payload, I'm going to take care of it, and then I will


00:59:58.560 --> 01:00:04.560
notify you with the webhook, I'm going to give you the result at this address," whatever mechanism you can use


01:00:04.560 --> 01:00:10.960
to do asynchronous. But I mean, building this kind of system asynchronously, when you have the worker


01:00:10.960 --> 01:00:16.000
taking a message from the queue, processing them, putting the result somewhere else, it's


01:00:16.000 --> 01:00:20.160
it's already a really good way to scale your application.


01:00:20.160 --> 01:00:22.160
And you can start without...


01:00:22.160 --> 01:00:25.440
You can start with Python itself. There's a queue in Python.


01:00:25.440 --> 01:00:27.440
There's multiprocess.


01:00:27.440 --> 01:00:29.440
And you don't have to deploy it into whatever.


01:00:29.440 --> 01:00:31.440
You can actually start if you know your program.


01:00:31.440 --> 01:00:33.440
You can even just have a background thread and a list.


01:00:33.440 --> 01:00:37.680
Yeah, exactly. You can start with something very simple for this pattern.


01:00:37.680 --> 01:00:40.000
You don't have to use a huge framework or whatever.


01:00:40.000 --> 01:00:42.800
If you know the pattern and you know it applies to what you're doing,


01:00:42.800 --> 01:00:44.800
you actually can use it. And you know, for example,


01:00:44.800 --> 01:00:49.800
that you will never need more than one host,


01:00:49.800 --> 01:00:52.800
one node, one computer will be enough forever for your program,


01:00:52.800 --> 01:00:55.800
while you don't need to deploy your network-based queue system


01:00:55.800 --> 01:00:58.800
like Redis, Rabbit, or whatever. You can use Python itself,


01:00:58.800 --> 01:01:04.800
use a multiprocess queue, and that will solve all your problems perfectly.


01:01:04.800 --> 01:01:07.800
Yeah, that's a great example.


01:01:07.800 --> 01:01:11.800
And the multiprocessing has an actual queue data structure


01:01:11.800 --> 01:01:15.220
that it properly shares across with notifications


01:01:15.220 --> 01:01:20.780
and everything across these multi-process processes,


01:01:20.780 --> 01:01:22.660
where the multi-process thing can just


01:01:22.660 --> 01:01:25.420
say, I'm going to block until I get something from the queue.


01:01:25.420 --> 01:01:27.880
And then as soon as you put it in, it picks it up and goes.


01:01:27.880 --> 01:01:30.180
But otherwise, it just chills in the background.


01:01:30.180 --> 01:01:31.380
>>Yeah.


01:01:31.380 --> 01:01:33.820
>>Yeah, very nice.


01:01:33.820 --> 01:01:35.700
All right, moving on.


01:01:35.700 --> 01:01:38.500
Designing for failure, that's a good one.


01:01:38.500 --> 01:01:40.540
You know, the thing that comes to mind is,


01:01:40.540 --> 01:01:42.700
at the extreme end of this.


01:01:42.700 --> 01:01:43.980
When I talked about scalability,


01:01:43.980 --> 01:01:46.700
I maybe said YouTube in a million requests a second.


01:01:46.700 --> 01:01:49.740
This one is chaos monkey in Netflix.


01:01:49.740 --> 01:01:50.580
- Yeah, exactly.


01:01:50.580 --> 01:01:52.100
Yeah, you have to design for that.


01:01:52.100 --> 01:01:54.140
So like I was saying, like a lot of people


01:01:54.140 --> 01:01:57.900
that try to write their code with a very optimistic mindset,


01:01:57.900 --> 01:01:59.180
like everything is going to be fine


01:01:59.180 --> 01:02:01.300
and I don't really care about error or the exceptions


01:02:01.300 --> 01:02:06.300
where you actually want to write proper exceptions,


01:02:06.300 --> 01:02:08.720
like proper classes of exceptions


01:02:08.720 --> 01:02:10.980
and proper handling of this exception in your program


01:02:10.980 --> 01:02:13.060
and making sure that when you, for example,


01:02:13.060 --> 01:02:14.380
when you use, I don't know, Redis


01:02:14.380 --> 01:02:17.540
and use a Redis library, you want to be sure,


01:02:17.540 --> 01:02:18.380
to be aware, and that's something


01:02:18.380 --> 01:02:20.580
that's not very obvious, honestly,


01:02:20.580 --> 01:02:23.340
'cause it's honestly not really well documented.


01:02:23.340 --> 01:02:27.540
Like, you can read the API of a Redis library


01:02:27.540 --> 01:02:30.740
and see, okay, it takes that type as an argument


01:02:30.740 --> 01:02:32.820
and you're going to return that type,


01:02:32.820 --> 01:02:34.980
but you don't know which exception is going to be raised.


01:02:34.980 --> 01:02:39.780
So sometimes you have to see it with your own eyes


01:02:39.780 --> 01:02:41.260
in production, like, oh, it's broken.


01:02:41.260 --> 01:02:42.660
It's going to raise connection error.


01:02:42.660 --> 01:02:44.380
Okay, now I know I need to fix it.


01:02:44.380 --> 01:02:47.960
- Well, the tricky part of that is not necessarily


01:02:47.960 --> 01:02:51.560
seeing the exception and knowing it, but now what?


01:02:51.560 --> 01:02:54.820
Like when I get the connection error,


01:02:54.820 --> 01:02:56.140
what means I can't talk to the database,


01:02:56.140 --> 01:02:58.620
it's overloaded or it's rebooting 'cause it's patching,


01:02:58.620 --> 01:03:00.500
but then what happens, right?


01:03:00.500 --> 01:03:03.060
Like, how do I not just go, well, there was an error.


01:03:03.060 --> 01:03:04.540
Tell the Datadog people there was an error.


01:03:04.540 --> 01:03:06.540
so we know, and then crash for the user.


01:03:06.540 --> 01:03:08.540
What do you do beyond that?


01:03:08.540 --> 01:03:10.540
- Yeah, and the answer is not obvious.


01:03:10.540 --> 01:03:12.540
It truly depends on what you're doing.


01:03:12.540 --> 01:03:14.540
If you are in a REST API,


01:03:14.540 --> 01:03:16.540
and your database connection is broken,


01:03:16.540 --> 01:03:18.540
you can't connect to the database.


01:03:18.540 --> 01:03:20.540
Are you going to retry?


01:03:20.540 --> 01:03:22.540
How many times?


01:03:22.540 --> 01:03:24.540
How many seconds are you going to retry?


01:03:24.540 --> 01:03:26.540
Because the other guy is waiting


01:03:26.540 --> 01:03:28.540
on the other side of the line.


01:03:28.540 --> 01:03:30.540
So you can't do that for 10 seconds.


01:03:30.540 --> 01:03:32.540
It's too long.


01:03:32.540 --> 01:03:34.220
So you have to do that a few times.


01:03:34.220 --> 01:03:35.220
Then what do you do?


01:03:35.220 --> 01:03:37.860
Do you return a 500 error and crash?


01:03:37.860 --> 01:03:41.340
Or do you return something that is like a retry later?


01:03:41.340 --> 01:03:42.340
I mean, there's a lot of...


01:03:42.340 --> 01:03:43.620
And you have to think about all of that.


01:03:43.620 --> 01:03:48.580
Like when to say to the client to retry, if they can retry or just, yeah, crash.


01:03:48.580 --> 01:03:51.420
That's also an option sometimes.


01:03:51.420 --> 01:03:53.780
And there are so many patterns.


01:03:53.780 --> 01:03:58.020
Most of the time, network errors, but it might be disk full or whatever.


01:03:58.020 --> 01:03:59.020
And you have to...


01:03:59.020 --> 01:04:01.780
So you can't think about everything at the beginning for sure.


01:04:01.780 --> 01:04:03.700
you have to have a good report system.


01:04:03.700 --> 01:04:04.540
- Are you a fan?


01:04:04.540 --> 01:04:06.900
I totally agree about the reporting system.


01:04:06.900 --> 01:04:09.140
It's hugely valuable and notifications as well.


01:04:09.140 --> 01:04:12.140
'Cause if you don't look at the reporting system,


01:04:12.140 --> 01:04:14.940
the log is full of errors and nobody would look for a week.


01:04:14.940 --> 01:04:18.580
But are you a fan of the retry decorators?


01:04:18.580 --> 01:04:19.900
You know, I'm talking about some of those things


01:04:19.900 --> 01:04:23.380
you can say, here's like an expensional back off.


01:04:23.380 --> 01:04:26.380
I like you to try five times and like first


01:04:26.380 --> 01:04:28.140
after like a second and then five seconds,


01:04:28.140 --> 01:04:29.900
then 10 seconds, what do you think?


01:04:29.900 --> 01:04:33.260
I'm the author of Tenacity, which is one of the most widely used.


01:04:33.260 --> 01:04:36.300
Yeah, that's one. That answers that question.


01:04:36.300 --> 01:04:38.300
- You're a fan. - Exactly.


01:04:38.300 --> 01:04:41.580
I am a fan, and it's all 80% of the problem.


01:04:41.580 --> 01:04:45.260
I mean, it's up to you to know how to retry,


01:04:45.260 --> 01:04:48.460
but it's a very, very good pattern to use,


01:04:48.460 --> 01:04:51.740
and Tenacity provides that as a decorator,


01:04:51.740 --> 01:04:55.980
which is not the best strategy if you want to have different strategies,


01:04:55.980 --> 01:04:58.140
like this function should be retried this number of times,


01:04:58.140 --> 01:05:00.740
depending on who the caller is.


01:05:00.740 --> 01:05:03.380
But most of the time, it's good enough.


01:05:03.380 --> 01:05:04.900
Actually, it's good enough.


01:05:04.900 --> 01:05:07.580
Most of the time, it's better to use that


01:05:07.580 --> 01:05:10.980
in a naive way where you just retry five times


01:05:10.980 --> 01:05:13.820
for five seconds or whatever, but not doing anything.


01:05:13.820 --> 01:05:15.620
Because if you know...


01:05:15.620 --> 01:05:17.100
It's also not a silver bullet.


01:05:17.100 --> 01:05:20.140
I see sometimes people using it to like,


01:05:20.140 --> 01:05:22.420
"Well, if anything wrong happens, I'm just going to retry."


01:05:22.420 --> 01:05:23.460
Which is not...


01:05:23.460 --> 01:05:25.420
Like, please use proper exception types.


01:05:25.420 --> 01:05:27.860
Like, catch the right thing and retry for the right reason,


01:05:27.860 --> 01:05:28.980
Not for everything.


01:05:28.980 --> 01:05:31.660
- Like maybe retry with connection timeout, but...


01:05:31.660 --> 01:05:33.900
- Yes, but like if-


01:05:33.900 --> 01:05:35.100
- Some other thing that crashes,


01:05:35.100 --> 01:05:36.500
like authorization failure.


01:05:36.500 --> 01:05:38.780
Like that's never gonna get better.


01:05:38.780 --> 01:05:39.620
- Exactly.


01:05:39.620 --> 01:05:41.380
But sometimes you see people writing,


01:05:41.380 --> 01:05:43.900
you're like, "I'm going to retry exception.


01:05:43.900 --> 01:05:45.540
Whatever is raised, I'm going to retry."


01:05:45.540 --> 01:05:46.940
Which is really not a good idea,


01:05:46.940 --> 01:05:50.020
'cause yeah, it's going to be fine for network errors,


01:05:50.020 --> 01:05:51.020
but like you were saying,


01:05:51.020 --> 01:05:52.900
if it's an authentication, you don't want to retry.


01:05:52.900 --> 01:05:54.380
So, I mean, be careful in that,


01:05:54.380 --> 01:05:58.780
But then if you know that you've got an I/O error most of the time


01:05:58.780 --> 01:06:02.580
because the network is done one day,


01:06:02.580 --> 01:06:07.580
it's fine to do that, and it's a really good way to design easily


01:06:07.580 --> 01:06:09.180
for this kind of thing.


01:06:09.180 --> 01:06:14.980
It doesn't solve everything, and if you have a large job, for example,


01:06:14.980 --> 01:06:18.280
that you know is going to take 10 minutes to compute, etc.,


01:06:18.280 --> 01:06:23.580
I mean, having this kind of free trial is going to save you


01:06:23.580 --> 01:06:25.580
because in a framework like Celery, for example,


01:06:25.580 --> 01:06:27.580
if your job fails after five minutes,


01:06:27.580 --> 01:06:30.420
for whatever reason, it's just going to put it back in the queue


01:06:30.420 --> 01:06:33.820
and retry later, but you lost the five first minutes that you used.


01:06:33.820 --> 01:06:36.520
And you can end up in these poison message scenarios


01:06:36.520 --> 01:06:39.220
where it tries, it fails, it goes back, it tries, it fails, it goes back,


01:06:39.220 --> 01:06:42.060
it tries, and then, yeah, then it's not so great.


01:06:42.060 --> 01:06:45.760
All right, just a little bit of time for some more.


01:06:45.760 --> 01:06:47.760
Deployment.


01:06:47.760 --> 01:06:50.960
You talk about, in your book, you talk about deploying on a platform


01:06:50.960 --> 01:06:52.960
as a server, as a PaaS, like Heroku.


01:06:52.960 --> 01:06:57.960
There's always VMs. These days we have Docker and Kubernetes.


01:06:57.960 --> 01:07:02.960
I mean, honestly, it's not simple to know what to do as somebody who is a newcomer, I think.


01:07:02.960 --> 01:07:07.960
Yeah, and I think it's a lot since I wrote the book, but nowadays, I still, I mean,


01:07:07.960 --> 01:07:10.960
Heroku is still there and pretty widely used because it's a good solution.


01:07:10.960 --> 01:07:16.960
The thing is that deploying a Python application, like for myself, I'm a pretty good Python programmer,


01:07:16.960 --> 01:07:21.960
but then I was out of Python, like infrastructure and Kubernetes, I barely know anything about it.


01:07:21.960 --> 01:07:25.560
It's a full-time job and it's not my job.


01:07:25.560 --> 01:07:28.480
It's another side of another job.


01:07:28.480 --> 01:07:31.600
So I could learn for sure, and I could be an expert in Kubernetes


01:07:31.600 --> 01:07:33.280
and deployment of anything.


01:07:33.280 --> 01:07:36.720
But I mean, it's fine to do that if you want to do that.


01:07:36.720 --> 01:07:39.400
But as a Python developer, I don't really want to do it.


01:07:39.400 --> 01:07:41.720
I'm happy to use any kind of platform,


01:07:41.720 --> 01:07:44.960
the service like Heroku, where I can actually,


01:07:44.960 --> 01:07:49.120
like using the Kubernetes container approach of deploying


01:07:49.120 --> 01:07:52.600
and spending a lot of this to scale is not my responsibility,


01:07:52.600 --> 01:07:55.840
but I can outsource it to somebody that knows how to do that.


01:07:55.840 --> 01:07:59.440
So there's plenty of options. I think I wrote about Heroku, OpenShift,


01:07:59.440 --> 01:08:04.840
it does that to Amazon or Microsoft or Google or other solutions.


01:08:04.840 --> 01:08:06.560
Yeah, they might have something, yeah.


01:08:06.560 --> 01:08:09.640
Yeah, and I mean, there's no reason if you really know


01:08:09.640 --> 01:08:13.360
that your application is going to scale, you don't want to spend a lot of time


01:08:13.360 --> 01:08:18.320
on infrastructure and learning, I mean, Kubernetes, Docker, whatever.


01:08:18.320 --> 01:08:23.180
I mean, you can spin easily on a picture on top of,


01:08:23.180 --> 01:08:25.920
of Heroku and then click a button to have two nodes,


01:08:25.920 --> 01:08:27.800
three nodes, four nodes, 10 nodes,


01:08:27.800 --> 01:08:29.560
and then wallet, and it's expensive,


01:08:29.560 --> 01:08:30.800
but that's another issue.


01:08:30.800 --> 01:08:35.000
- Yeah, the platform as a service often,


01:08:35.000 --> 01:08:42.120
they exchange complete ease of use with maybe two things,


01:08:42.120 --> 01:08:45.560
one with cost, and then the other is with flexibility,


01:08:45.560 --> 01:08:46.400
right?


01:08:46.400 --> 01:08:48.120
Like you kind of got to fit their way.


01:08:48.120 --> 01:08:51.000
"Well, the way you use the database is our managed database service."


01:08:51.000 --> 01:08:54.280
And if you don't like that, well, then, I don't know,


01:08:54.280 --> 01:08:56.040
you got to just use our managed service.


01:08:56.040 --> 01:08:56.520
You know what I mean?


01:08:56.520 --> 01:08:58.840
Things like that are kind of somewhat fixed.


01:08:58.840 --> 01:09:01.560
But I think it's really good for a lot of people.


01:09:01.560 --> 01:09:02.120
Yeah, exactly.


01:09:02.120 --> 01:09:04.120
I mean, it covers 90% of the market, right?


01:09:04.120 --> 01:09:07.160
I mean, most people that are going to start with,


01:09:07.160 --> 01:09:09.880
even if it's not a bad project,


01:09:09.880 --> 01:09:11.880
but like you're starting your company,


01:09:11.880 --> 01:09:13.160
you're doing a small project,


01:09:13.160 --> 01:09:16.600
and you know, maybe it's one day you will be the next Google you have to scale.


01:09:16.600 --> 01:09:18.600
But at that time, you'll solve the problem.


01:09:18.600 --> 01:09:20.600
You'll get plenty of money to solve it.


01:09:20.600 --> 01:09:22.600
But until then, you don't have a lot of time,


01:09:22.600 --> 01:09:24.600
you don't have a lot of money. It's actually pretty cheap


01:09:24.600 --> 01:09:26.600
compared to the time you would spend learning


01:09:26.600 --> 01:09:28.600
the ropes of Kubernetes.


01:09:28.600 --> 01:09:30.600
I mean, a secure deployment


01:09:30.600 --> 01:09:32.600
at scale of Kubernetes, I'm sure


01:09:32.600 --> 01:09:34.600
it's pretty more complicated than writing


01:09:34.600 --> 01:09:36.600
a simple Flask application.


01:09:36.600 --> 01:09:38.600
So it's a trade-off.


01:09:38.600 --> 01:09:40.600
I think it's a pretty good trade-off if you really want to


01:09:40.600 --> 01:09:42.600
start saying, "Okay,


01:09:42.600 --> 01:09:44.600
I think at some point I will need to scale.


01:09:44.600 --> 01:09:47.600
I can't run on my laptop anymore, I need to run that somewhere.


01:09:47.600 --> 01:09:50.200
While using a platform like that is a pretty good trade-off.


01:09:50.200 --> 01:09:53.840
Yeah, and I think it's so easy to dream big and think,


01:09:53.840 --> 01:09:56.560
"Oh, I'm going to have to scale. If I'm going to deploy this,


01:09:56.560 --> 01:09:59.920
what is it going to be like if I get the first 100,000 users?"


01:09:59.920 --> 01:10:03.160
You should be so lucky that you have that problem.


01:10:03.160 --> 01:10:06.680
So often, things get built and they just stagnate,


01:10:06.680 --> 01:10:09.040
or they don't go anywhere, or the reason they stagnate


01:10:09.040 --> 01:10:11.880
is you're not adding features fast enough because you spent so much time


01:10:11.880 --> 01:10:14.680
and building complicated architectures


01:10:14.680 --> 01:10:17.040
for a case in the future,


01:10:17.040 --> 01:10:19.420
when reality is on the past,


01:10:19.420 --> 01:10:22.560
you could just pay 10 times as much for two weeks


01:10:22.560 --> 01:10:24.160
and then completely move to something else


01:10:24.160 --> 01:10:27.460
and you could buy yourself that time for $500, right?


01:10:27.460 --> 01:10:30.240
But you could spend months building something, right?


01:10:30.240 --> 01:10:33.600
Like that's going to support some insane future


01:10:33.600 --> 01:10:34.480
that doesn't exist.


01:10:34.480 --> 01:10:36.300
And so, you know, a lot of people,


01:10:36.300 --> 01:10:38.360
they'd be better off to just move forward


01:10:38.360 --> 01:10:41.840
and then evolve and realize it's not forever.


01:10:41.840 --> 01:10:45.880
it's a path towards where you're going to be.


01:10:45.880 --> 01:10:47.840
- Yeah, and then learn marketing


01:10:47.840 --> 01:10:51.320
to have people coming to your project.


01:10:51.320 --> 01:10:52.920
- That is the problem, yes.


01:10:52.920 --> 01:10:56.520
- That might be my next book about doing marketing


01:10:56.520 --> 01:10:59.840
to getting people on your project to be able to scale them.


01:10:59.840 --> 01:11:02.680
- Yeah, I'll definitely read that book if you write it.


01:11:02.680 --> 01:11:04.520
All right, we got some more topics to cover,


01:11:04.520 --> 01:11:05.920
but we're down to just one


01:11:05.920 --> 01:11:07.960
that I think we have a little time to touch on


01:11:07.960 --> 01:11:09.600
because it's like magic sauce.


01:11:09.600 --> 01:11:12.960
For database, the magic sauce is indexes.


01:11:12.960 --> 01:11:16.320
For many other things, the magic sauce is caching, right?


01:11:16.320 --> 01:11:17.360
If this thing is really slow,


01:11:17.360 --> 01:11:19.320
I'll give you an example from Talk Python.


01:11:19.320 --> 01:11:21.920
We have, or maybe even better from Python Bytes,


01:11:21.920 --> 01:11:23.560
that's a more interesting example,


01:11:23.560 --> 01:11:28.560
is that RSS feed for that thing is made up out of,


01:11:28.560 --> 01:11:30.280
I think we had to limit it 'cause we got too many,


01:11:30.280 --> 01:11:32.240
but for a while it was made up of 200 episodes.


01:11:32.240 --> 01:11:35.080
Each episode is like five pages of markdown.


01:11:35.080 --> 01:11:39.560
In order to render that RSS feed on demand,


01:11:39.560 --> 01:11:41.660
I've got to render, I've got to go to the database,


01:11:41.660 --> 01:11:46.000
query 200 things, and then mark downify them 200 times,


01:11:46.000 --> 01:11:49.600
and then put that into an XML document and return that.


01:11:49.600 --> 01:11:53.200
And that is not super fast, but you know what?


01:11:53.200 --> 01:11:54.800
If I take that result I would have returned


01:11:54.800 --> 01:11:56.720
and save it in the database


01:11:56.720 --> 01:12:00.080
and just generate it once a minute, then it's fine.


01:12:00.080 --> 01:12:00.920
Right?


01:12:00.920 --> 01:12:02.340
Those, it's like magic.


01:12:02.340 --> 01:12:06.060
It goes from one second to one millisecond,


01:12:07.300 --> 01:12:09.300
and you just get so much scale from it.


01:12:09.300 --> 01:12:13.020
- Yeah, that's exactly what you're saying.


01:12:13.020 --> 01:12:16.380
Like it's a pretty good pattern when you have to optimize.


01:12:16.380 --> 01:12:20.300
So that would be more for the performance dimension


01:12:20.300 --> 01:12:22.660
when you want your code to be faster,


01:12:22.660 --> 01:12:25.020
not necessarily to scale in number of user.


01:12:25.020 --> 01:12:26.460
Even if at some time connected,


01:12:26.460 --> 01:12:29.540
like if you have 200 people requesting at the same time,


01:12:29.540 --> 01:12:31.980
your RSS feed and you have to do that


01:12:31.980 --> 01:12:34.740
at the same time, 200 times, that's pretty useless.


01:12:34.740 --> 01:12:36.840
So I mean, caching is pretty good pattern.


01:12:36.840 --> 01:12:39.560
There's nothing actually very specific to Python there.


01:12:39.560 --> 01:12:42.600
I mean, even in this chapter of the book, it's actually pretty like,


01:12:42.600 --> 01:12:45.920
I'm talking about how to use memcache or already,


01:12:45.920 --> 01:12:47.720
so whatever you want to cache,


01:12:47.720 --> 01:12:51.080
and there are plenty of good solution to cache over the network.


01:12:51.080 --> 01:12:53.600
You can start by caching locally in your own process.


01:12:53.600 --> 01:12:55.040
Like, I remember I see.


01:12:55.040 --> 01:12:58.040
Like a Python dictionary is a really good cache, right?


01:12:58.040 --> 01:12:59.440
- For certain things. - Exactly.


01:12:59.440 --> 01:13:03.240
And there are in Python 3 something that is the,


01:13:03.240 --> 01:13:05.440
- there were the LRU cache. - Yeah.


01:13:05.440 --> 01:13:07.440
- There are a lot of cache decorators. - Yeah.


01:13:07.440 --> 01:13:10.320
Cache tools, the cache tools library in Python,


01:13:10.320 --> 01:13:13.320
there's a lot of different algorithms if you want to cache locally


01:13:13.320 --> 01:13:15.820
in your own Python program.


01:13:15.820 --> 01:13:19.280
If you know you're going to call this method a hundred of times,


01:13:19.280 --> 01:13:21.520
and the result is going to be the same,


01:13:21.520 --> 01:13:25.220
just cache it if it's expensive to compute.


01:13:25.220 --> 01:13:27.220
I mean, expensive to compute might be...


01:13:27.220 --> 01:13:31.360
Well, expensive in terms of CPU, but it also might be expensive for database,


01:13:31.360 --> 01:13:36.360
or sometimes the expansiveness is going to be the network,


01:13:36.360 --> 01:13:39.360
you're going to request some data over the network,


01:13:39.360 --> 01:13:42.360
and it's very far or it's a very slow system,


01:13:42.360 --> 01:13:45.360
or very unreliable system.


01:13:45.360 --> 01:13:48.360
So using caching system is a pretty good solution to avoid this,


01:13:48.360 --> 01:13:51.360
which is also linked to the design for failure we talked about before.


01:13:51.360 --> 01:13:54.360
Right, if you're consuming third-party services,


01:13:54.360 --> 01:13:57.360
you can't necessarily depend on their uptime, on their reliability,


01:13:57.360 --> 01:14:00.360
on their response time, all those types of things.


01:14:00.360 --> 01:14:03.860
So when you go to our courses,


01:14:03.860 --> 01:14:06.440
we've got 12 video servers throughout the world,


01:14:06.440 --> 01:14:07.700
and we want to serve you the video


01:14:07.700 --> 01:14:09.200
from the one closest to you.


01:14:09.200 --> 01:14:11.200
So we have a service that we call


01:14:11.200 --> 01:14:13.660
that takes your IP address, figures out where you are,


01:14:13.660 --> 01:14:16.860
and then chooses a video server for you


01:14:16.860 --> 01:14:18.360
so you get the best time.


01:14:18.360 --> 01:14:20.440
That costs a little tiny bit of money each time,


01:14:20.440 --> 01:14:24.360
but with enough requests, it would be, you know, hundreds,


01:14:24.360 --> 01:14:25.860
maybe even, I don't know,


01:14:25.860 --> 01:14:29.860
definitely into the hundreds per month of video.


01:14:29.860 --> 01:14:33.460
of where is this person API charges.


01:14:33.460 --> 01:14:34.900
And so we just cache that.


01:14:34.900 --> 01:14:37.100
Like if this IP address is from this city


01:14:37.100 --> 01:14:39.900
or this country, we just put that in our database.


01:14:39.900 --> 01:14:41.660
And first we check the database,


01:14:41.660 --> 01:14:42.980
do we know where this IP address is?


01:14:42.980 --> 01:14:44.420
No, go to the service.


01:14:44.420 --> 01:14:45.900
Otherwise, just get it from the database.


01:14:45.900 --> 01:14:48.380
It's both faster and it literally doesn't cost as much.


01:14:48.380 --> 01:14:51.420
It's less expensive in the most direct meaning of that.


01:14:51.420 --> 01:14:52.220
Yeah.


01:14:52.220 --> 01:14:54.940
And then you were hitting on the first


01:14:54.940 --> 01:14:56.940
and biggest issue in computer science,


01:14:56.940 --> 01:14:58.580
which is cache invalidation,


01:14:58.580 --> 01:15:02.900
which is, well, in your case, the IP might not change of country.


01:15:02.900 --> 01:15:05.540
It can change, not very often, but it can change.


01:15:05.540 --> 01:15:09.700
Yeah, so for our, just for our example, what I did is it's in MongoDB, so I set up a


01:15:09.700 --> 01:15:14.500
an index that will remove it from the database after six months.


01:15:14.500 --> 01:15:18.820
Yeah, which is fine, but arbitrary, right?


01:15:18.820 --> 01:15:20.820
Yes, it's totally, I could be wrong for a while.


01:15:20.820 --> 01:15:25.460
I mean, exactly, but the failure case is it's slow streaming with buffering, potentially.


01:15:25.460 --> 01:15:28.180
It's not complete failure, completely the wrong answer, right?


01:15:28.180 --> 01:15:30.180
So for us, it's acceptable.


01:15:30.180 --> 01:15:32.820
Exactly. So you made a trade-off, which is totally fine for your risk-based.


01:15:32.820 --> 01:15:35.940
And that's a lot of things that you do when you want to scale is trade-off.


01:15:35.940 --> 01:15:40.100
And sometimes you don't get things totally right, but it's fine.


01:15:40.100 --> 01:15:44.260
It's just not the best experience for your user in your case,


01:15:44.260 --> 01:15:46.020
but it's fine and you can live with it.


01:15:46.020 --> 01:15:49.460
And I think it's a change of mindset when you go from,


01:15:49.460 --> 01:15:54.500
"I'm writing a Python program which has to be perfect and works 100% of the time."


01:15:54.500 --> 01:15:56.900
And then when you want to scale, you have to do a lot of trade-off.


01:15:56.900 --> 01:16:00.220
or like, it will work fine for 80% of the people.


01:16:00.220 --> 01:16:03.500
And for some cases, 5% of the time,


01:16:03.500 --> 01:16:06.260
while that might be not optimal, but it's fine.


01:16:06.260 --> 01:16:08.900
And that's a lot of doing things at scale


01:16:08.900 --> 01:16:11.500
are changing this mindset to,


01:16:11.500 --> 01:16:13.820
well, it works always, it's always true.


01:16:13.820 --> 01:16:15.980
And sometimes it's not really true.


01:16:15.980 --> 01:16:18.860
I mean, if you had a way for you to be aware


01:16:18.860 --> 01:16:21.100
and notified that an IP address changed in this country,


01:16:21.100 --> 01:16:22.940
you could invalidate your cache


01:16:22.940 --> 01:16:25.620
and then make it totally reliable.


01:16:25.620 --> 01:16:27.820
I mean, for a few seconds, maybe it won't be up to date,


01:16:27.820 --> 01:16:29.820
but that would be close to perfection.


01:16:29.820 --> 01:16:33.020
But you don't have that system, so you have to do what you did,


01:16:33.020 --> 01:16:35.020
which is a good trade-off. It's pragmatic.


01:16:35.020 --> 01:16:37.820
You have to be very pragmatic when you do things at scale.


01:16:37.820 --> 01:16:40.820
Yeah, and also kind of designed for failure.


01:16:40.820 --> 01:16:43.320
What's the worst case if that goes wrong?


01:16:43.320 --> 01:16:46.320
It's like streaming halfway around the world or something.


01:16:46.320 --> 01:16:49.320
Whereas other things, like if the database goes down,


01:16:49.320 --> 01:16:51.820
you've got to deal with that entirely differently.


01:16:53.320 --> 01:16:54.320
Yeah.


01:16:54.320 --> 01:16:56.320
It's a hard one to fix though. I don't really know what to do there.


01:16:56.320 --> 01:17:00.320
Well, I mean, caching, like you could cache, for example,


01:17:00.320 --> 01:17:06.320
you could cache an older version and reply to the client,


01:17:06.320 --> 01:17:10.320
like it might be an older version, I'm sorry, like what the time system is there.


01:17:10.320 --> 01:17:13.320
I mean, depending on what you build, obviously, you have to know the use case


01:17:13.320 --> 01:17:15.320
and the things you're going to do, but caching could be a solution.


01:17:15.320 --> 01:17:18.320
That's a good idea because you might actually be able to say,


01:17:18.320 --> 01:17:22.320
if I'm going to go to Redis, and if it's not there, then I'm going to go to database.


01:17:22.320 --> 01:17:26.320
many of the requests that come in might never need to go to the database again.


01:17:26.320 --> 01:17:31.320
If you say, "Oh, whoops, the database is down, we're just going to start with out of the cache,"


01:17:31.320 --> 01:17:35.320
it could go for a while until there's some right operation.


01:17:35.320 --> 01:17:38.320
As long as it's read-only, it might not matter.


01:17:38.320 --> 01:17:41.320
Which is what services like Cloudflare does for the web, for example.


01:17:41.320 --> 01:17:44.320
Like with caching, it protects you, and if you're down,


01:17:44.320 --> 01:17:48.320
they're just going to show the page they had a few seconds ago until you're back up,


01:17:48.320 --> 01:17:50.320
and nobody will notice.


01:17:50.320 --> 01:17:52.320
You can apply that.


01:17:52.320 --> 01:17:55.320
And the thing you have to keep in mind when you do caching


01:17:55.320 --> 01:17:57.320
is to be able to invalidate your cache.


01:17:57.320 --> 01:18:00.320
If you're caching database and something changes in the database,


01:18:00.320 --> 01:18:04.320
you have to have this callback mechanism


01:18:04.320 --> 01:18:06.320
where your database can say to your cache,


01:18:06.320 --> 01:18:08.320
"By the way, that change, you need to update it."


01:18:08.320 --> 01:18:10.320
You need to be aware of that,


01:18:10.320 --> 01:18:13.320
otherwise you have to put arbitrary timestamp,


01:18:13.320 --> 01:18:16.320
like you said, for your six months.


01:18:16.320 --> 01:18:18.320
It's going to be six months and that's it,


01:18:18.320 --> 01:18:20.160
which is fine for such a use case,


01:18:20.160 --> 01:18:22.920
but for a lot of people who can like to RSS feed,


01:18:22.920 --> 01:18:24.400
but wouldn't work very well probably


01:18:24.400 --> 01:18:26.600
if you were doing six months.


01:18:26.600 --> 01:18:27.600
- Yeah, that would be bad.


01:18:27.600 --> 01:18:31.540
All of a sudden now there's 24 new episodes,


01:18:31.540 --> 01:18:33.280
all of a sudden or something, yeah.


01:18:33.280 --> 01:18:35.120
So this is where you write that cron job


01:18:35.120 --> 01:18:38.440
that just restarts Redis once an hour and you'll be fine.


01:18:38.440 --> 01:18:39.320
No, just kidding.


01:18:39.320 --> 01:18:42.080
You're right, this cache validation really, really


01:18:42.080 --> 01:18:44.800
is tricky because if you check the database,


01:18:44.800 --> 01:18:46.800
then you're not really caching anymore.


01:18:46.800 --> 01:18:48.800
You might as well not have the cache.


01:18:48.800 --> 01:18:52.800
So yeah, it's super tricky, but definitely a bit of the magic sauce.


01:18:52.800 --> 01:18:55.800
All right, I think that there's plenty more we could cover


01:18:55.800 --> 01:18:57.800
in the book and about scaling and architecture,


01:18:57.800 --> 01:18:59.800
and it would be really fun, but we're way over time,


01:18:59.800 --> 01:19:03.800
so we should probably just wrap it up with a couple of questions here


01:19:03.800 --> 01:19:05.800
that I always ask at the end of the show.


01:19:05.800 --> 01:19:07.800
So, Julien, if you're going to write some Python code,


01:19:07.800 --> 01:19:09.800
what editor do you use?


01:19:09.800 --> 01:19:13.800
Emacs. I've been an Emacs user for the last 10 years.


01:19:13.800 --> 01:19:17.960
I think I still have the commit access to Emacs itself.


01:19:17.960 --> 01:19:19.560
- Oh cool, you did say you love Lisp


01:19:19.560 --> 01:19:23.400
and you have your code powered by Lisp.


01:19:23.400 --> 01:19:25.280
- Exactly, yeah.


01:19:25.280 --> 01:19:28.720
I stopped but I wrote a lot of Lisp back 10 years ago.


01:19:28.720 --> 01:19:29.720
- Yeah, cool.


01:19:29.720 --> 01:19:33.400
The notable PyPI package, if you want you can shout out


01:19:33.400 --> 01:19:36.760
Tenacity which we covered or something else if you'd like.


01:19:36.760 --> 01:19:39.320
- Tenacity and DyeQuery which I love.


01:19:39.320 --> 01:19:41.840
DyeQuery is a tiny wrapper around the logging system


01:19:41.840 --> 01:19:45.840
The story is that I never remember how to configure the logging system in Python.


01:19:45.840 --> 01:19:49.840
I do import logging and I don't know how to configure it


01:19:49.840 --> 01:19:53.840
to work like I want. So Decory does that. It's pretty easy to use.


01:19:53.840 --> 01:19:57.840
It has a functional approach like Tenacity in its design.


01:19:57.840 --> 01:20:01.840
And it's basically two lines to use the logging system. You have something that works


01:20:01.840 --> 01:20:03.840
out of the box with callovers, etc.


01:20:03.840 --> 01:20:07.840
- Oh, fantastic. Yeah, I always forget how to set up the logging system and use something else


01:20:07.840 --> 01:20:14.800
as well. I don't need to remember this. Fantastic. All right. So thank you for being here. Thank


01:20:14.800 --> 01:20:20.600
you for talking about this and covering all these ideas. They're fascinating. And they


01:20:20.600 --> 01:20:24.680
always require some trade-offs, right? Like it's when should I use this thing or that


01:20:24.680 --> 01:20:29.800
thing? But if people want to get started, one, where do they find your book? Two, what


01:20:29.800 --> 01:20:34.880
advice in general do you have for them going down this path?


01:20:34.880 --> 01:20:38.360
I think, I mean, yeah, I mean, you can find my book,


01:20:38.360 --> 01:20:41.520
scalingpython.com if you want to take a look.


01:20:41.520 --> 01:20:44.160
It's a pretty good read, I think, to give you the right mindset


01:20:44.160 --> 01:20:48.600
to understand what the trade-offs you might need to do in your program.


01:20:48.600 --> 01:20:50.920
And I think that's what boils down to that,


01:20:50.920 --> 01:20:54.960
like, what are you ready to change and how to design your program?


01:20:54.960 --> 01:20:57.760
And what are you going to...


01:20:57.760 --> 01:20:59.080
What is going to be your real use case?


01:20:59.080 --> 01:21:00.760
Like, why do you want to scale?


01:21:00.760 --> 01:21:02.960
And are you going to scale for real?


01:21:02.960 --> 01:21:05.960
or are you just thinking that you will need to scale in the future?


01:21:05.960 --> 01:21:09.660
And do the right trade-off and don't overcomplicate things


01:21:09.660 --> 01:21:13.260
because you're just going to shoot yourself in the foot by doing that.


01:21:13.260 --> 01:21:17.140
Yeah, it's super tricky. I would just add on to that.


01:21:17.140 --> 01:21:19.840
If what you think you might need to scale in the future


01:21:19.840 --> 01:21:24.340
is a web app or web API, use Locust to actually measure it.


01:21:24.340 --> 01:21:26.340
And if what it is something more local,


01:21:26.340 --> 01:21:29.540
like a data science type of thing or something computationally locally,


01:21:29.540 --> 01:21:33.560
run C profile with it and just measure, right?


01:21:33.560 --> 01:21:35.360
However you go about your measuring.


01:21:35.360 --> 01:21:38.000
- Yep. - Yeah, fantastic.


01:21:38.000 --> 01:21:39.440
All right, thank you, Julian.


01:21:39.440 --> 01:21:40.640
It's been great to chat with you


01:21:40.640 --> 01:21:42.440
and sharing these ideas.


01:21:42.440 --> 01:21:43.840
Great book.


01:21:43.840 --> 01:21:45.080
- Thank you, Michael.


01:21:45.080 --> 01:21:46.720
- Yep.


01:21:46.720 --> 01:21:48.560
Bye, and bye everyone out there in the live stream.


01:21:48.560 --> 01:21:49.660
Thanks for being here.


01:21:49.660 --> 01:21:59.660
[BLANK_AUDIO]