This page is designed to help Chromium developers debug memory issues.
When in doubt, reach out to [email protected].
Let‘s say that there’s a CL or feature that reproducibly increases memory usage when it's landed/enabled, given a particular set of repro steps.
Repeat the above steps, but this time also take a heap dump. Confirm that the regression is also visible in the heap dump, and then compare the two heap dumps to find the difference. You can also use diff_heap_profiler.py to perform the diff.
Hopefully the MemoryDumpProvider has sufficient information to help diagnose the leak. Depending on the whether the leaked object is allocated via malloc or new
For a small set of Chrome users in the wild, Chrome will record and upload anonymized heap dumps. This has the benefit of wider coverage for real code paths, at the expense of reproducibility.
These heap dumps can take some time to grok, but frequently yield valuable insight. At the time of this writing, heap dumps from the wild have resulted in real, high impact bugs being found in Chrome code ~90% of the time.
For an example investigation of a real heap dump, see this link.
Navigate to chrome://flags and search for memlog. There are several options that can be used to configure heap dumps. All of these options are also available as command line flags, for automated test runs [e.g. telemetry].
#memlog controls which processes are profiled. It's also possible to manually specify the process via the interface at chrome://memory-internals.#memlog-in-process makes the profiling service to be run within the Chrome browser process. Defaults to run the service as a separate dedicated process.#memlog-sampling-rate specifies the sampling interval in bytes. The lower the interval, the more precise is the profile. However it comes at the cost of performance. Default value is 100KB, that is enough to observe allocation sites that make allocations >500KB total, where total equals to a single allocation size times the number of such allocations at the same call site.#memlog-stack-mode describes the type of metadata recorded for each allocation. native stacks provide the most utility. The only time the other options should be considered is for Android official builds, most of which do not support native stacks.Once the flags have been set appropriately, restart Chrome and take a memory-infra trace. The results will have a heap dump.
In case you can reproduce the corruption locally, you are advised to run sanitizers (e.g. ASan) to locate and fix UB.
Otherwise, you can look into minidump (link Googlers-only) if available.
Memory allocation goes through multiple states, and its payload sometimes has a distinctive pattern. You may also see some variance on lower bits, introduced by e.g. an offset within struct.
free() may or may not be overwritten.free()d memory may be caught as “free-list corruption”.0xCDCDCDCDCDCDCDCD: when allocation gets returned to PartitionAlloc.PA_BUILDFLAG(EXPENSIVE_DCHECKS_ARE_ON) builds.free()d memory in quarantine for a while before returning it into a free-list to detect and mitigate UaF bugs.0xCDCDCDCDCDCDCDCD: PartitionAlloc's FreeFlags::kZap.0xEFEFEFEFEFEFEFEF: In BRP quarantine.0xEFED????????8000: In LUD quarantine.free() stack trace on crashpad.0xECEC????????8000: In E-LUD quarantine.In principle, once initialized you should only see values written by your code while your allocation is alive. However, in rare case, you may see values from Write-after-Free.
void YourFunc() { | void TheirFunc() {
| int* p1 = new int;
| delete p1;
// The allocator may |
// redistribute `p1` to `p2` |
int* p2 = new int; |
*p2 = 123; |
| // Write-after-Free
| *p1 = 456;
// 456 may show up |
printf("%d\n", *p2); |
} | }
...or values from Double-Free.
void YourFunc() { | void TheirFunc() {
| int* p1 = new int;
| delete p1;
// The allocator may |
// redistribute `p1` to `p2` |
int* p2 = new int; |
*p2 = 123; |
| // Double-Free
| delete p1;
|
| // The allocator may
| // redistribute `p2` to `p3`
| int* p3 = new int;
| *p3 = 456;
// 456 may show up |
printf("%d\n", *p2); |
} | }
0x0000000000000000: zero initialization.0x0000000000000000: PartitionAlloc's AllocFlags::kZeroFill.calloc().0xABABABABABABABAB: PartitionAlloc's newly allocated memory.PA_BUILDFLAG(EXPENSIVE_DCHECKS_ARE_ON) builds.You may see random values written by someone else if you keep using pointers to free()d region.
void YourFunc() { | void TheirFunc() {
int* p1 = new int; |
*p1 = 123; |
delete p1; |
| // The allocator may
| // redistribute `p1` to `p2`
| int* p2 = new int;
| *p2 = 456;
// Use-after-Free; |
// 456 may show up |
printf("%d\n", *p1); |
} | }