Vlad Tsyrklevich | 08bc0525 | 2018-12-04 06:58:54 | [diff] [blame] | 1 | # GWP-ASan |
| 2 | |
| 3 | GWP-ASan is a debug tool intended to detect heap memory errors in the wild. It |
Vlad Tsyrklevich | f9c9065 | 2018-12-28 21:15:03 | [diff] [blame] | 4 | samples allocations to a debug allocator, similar to ElectricFence or Page Heap, |
Vlad Tsyrklevich | 6e6402a | 2019-01-22 07:50:20 | [diff] [blame] | 5 | causing memory errors to crash and report additional debugging context about |
| 6 | the error. |
Vlad Tsyrklevich | 08bc0525 | 2018-12-04 06:58:54 | [diff] [blame] | 7 | |
Vlad Tsyrklevich | ee4629b | 2019-10-24 20:07:06 | [diff] [blame^] | 8 | It is also known by its recursive backronym, GWP-ASan Will Provide Allocation |
| 9 | Sanity. |
| 10 | |
Vlad Tsyrklevich | 08bc0525 | 2018-12-04 06:58:54 | [diff] [blame] | 11 | ## Allocator |
| 12 | |
| 13 | The GuardedPageAllocator returns allocations on pages buffered on both sides by |
| 14 | guard pages. The allocations are either left- or right-aligned to detect buffer |
| 15 | overflows and underflows. When an allocation is freed, the page is marked |
| 16 | inaccessible so use-after-frees cause an exception (until that page is reused |
| 17 | for another allocation.) |
| 18 | |
| 19 | The allocator saves stack traces on every allocation and deallocation to |
| 20 | preserve debug context if that allocation results in a memory error. |
| 21 | |
Vlad Tsyrklevich | dc1a9a5e8 | 2018-12-18 18:04:01 | [diff] [blame] | 22 | The allocator implements a quarantine mechanism by allocating virtual memory for |
Vlad Tsyrklevich | 4a2e4d20 | 2019-04-25 00:22:43 | [diff] [blame] | 23 | more allocations than the total number of physical pages it can return at any |
| 24 | given time. The difference forms a rudimentary quarantine. |
| 25 | |
| 26 | Because pages are re-used for allocations, it's possible that a long-lived |
| 27 | use-after-free will cause a crash long after the original allocation has been |
| 28 | replaced. In order to decrease the likelihood of incorrect stack traces being |
| 29 | reported, we allocate a lot of virtual memory but don't store metadata for every |
| 30 | allocation. That way though we may not be able to report the metadata for an old |
| 31 | allocation, we will not report incorrect stack traces. |
Vlad Tsyrklevich | dc1a9a5e8 | 2018-12-18 18:04:01 | [diff] [blame] | 32 | |
Vlad Tsyrklevich | 08bc0525 | 2018-12-04 06:58:54 | [diff] [blame] | 33 | ## Crash handler |
| 34 | |
| 35 | The allocator is designed so that memory errors with GWP-ASan allocations |
| 36 | intentionally trigger invalid access exceptions. A hook in the crashpad crash |
| 37 | handler process inspects crashes, determines if they are GWP-ASan exceptions, |
| 38 | and adds additional debug information to the crash minidump if so. |
| 39 | |
| 40 | The crash handler hook determines if the exception was related to GWP-ASan by |
| 41 | reading the allocator internals and seeing if the exception address was within |
| 42 | the bounds of the allocator region. If it is, the crash handler hook extracts |
| 43 | debug information about that allocation, such as thread IDs and stack traces |
| 44 | for allocation (and deallocation, if relevant) and writes it to the crash dump. |
| 45 | |
| 46 | The crash handler runs with elevated privileges so parsing information from a |
| 47 | lesser-privileged process is security sensitive. The GWP-ASan hook is specially |
| 48 | structured to minimize the amount of allocator logic it relies on and to |
| 49 | validate the allocator internals before reasoning about them. |
| 50 | |
| 51 | ## Status |
| 52 | |
Vlad Tsyrklevich | ee4629b | 2019-10-24 20:07:06 | [diff] [blame^] | 53 | GWP-ASan is implemented for malloc and PartitionAlloc. It is enabled by default |
| 54 | on Windows and macOS. The allocator parameters can be manually modified by using |
| 55 | an invocation like the following: |
Vlad Tsyrklevich | 08bc0525 | 2018-12-04 06:58:54 | [diff] [blame] | 56 | |
| 57 | ```shell |
| 58 | chrome --enable-features="GwpAsanMalloc<Study" \ |
| 59 | --force-fieldtrials=Study/Group1 \ |
Vlad Tsyrklevich | 4a2e4d20 | 2019-04-25 00:22:43 | [diff] [blame] | 60 | --force-fieldtrial-params=Study.Group1:MaxAllocations/128/MaxMetadata/255/TotalPages/4096/AllocationSamplingFrequency/1000/ProcessSamplingProbability/1.0 |
Vlad Tsyrklevich | 08bc0525 | 2018-12-04 06:58:54 | [diff] [blame] | 61 | ``` |
| 62 | |
Vlad Tsyrklevich | 04d8664 | 2019-05-21 00:22:50 | [diff] [blame] | 63 | GWP-ASan is tuned more aggressively in canary/dev, to increase the likelihood we |
| 64 | catch newly introduced bugs, and for specific processes depending on the |
| 65 | particular allocator. |
Vlad Tsyrklevich | 4a2e4d20 | 2019-04-25 00:22:43 | [diff] [blame] | 66 | |
Vlad Tsyrklevich | 6e6402a | 2019-01-22 07:50:20 | [diff] [blame] | 67 | A [hotlist of bugs discovered by by GWP-ASan](https://bugs.chromium.org/p/chromium/issues/list?can=1&q=Hotlist%3DGWP-ASan) |
Vlad Tsyrklevich | 4a2e4d20 | 2019-04-25 00:22:43 | [diff] [blame] | 68 | exists, though GWP-ASan crashes are filed without external visibility by |
| 69 | default. |
Vlad Tsyrklevich | 6e6402a | 2019-01-22 07:50:20 | [diff] [blame] | 70 | |
Vlad Tsyrklevich | 04d8664 | 2019-05-21 00:22:50 | [diff] [blame] | 71 | ## Limitations |
| 72 | |
| 73 | - GWP-ASan is configured with a small fixed-size amount of memory, so |
| 74 | long-lived allocations can quickly deplete the page pool and lead the |
| 75 | allocator to run out of memory. Depending on the sampling frequency and |
| 76 | distribution of allocation lifetimes this may lead to only allocations early |
| 77 | in the process lifetime being sampled. |
| 78 | - Allocations over a page in size are not sampled. |
| 79 | - The allocator skips zero-size allocations. Zero-size allocations on some |
| 80 | platforms return valid pointers and may be subject to lifetime and bounds |
| 81 | issues. |
| 82 | - GWP-ASan does not intercept allocations for Oilpan or the v8 GC. |
| 83 | - GWP-ASan does not hook PDFium's fork of PartitionAlloc. |
| 84 | - Right-aligned allocations to catch overflows are not perfectly right-aligned, |
| 85 | so small out-of-bounds accesses may be missed. |
Vlad Tsyrklevich | ee4629b | 2019-10-24 20:07:06 | [diff] [blame^] | 86 | - GWP-ASan does not sample some early allocations that occur before field trial |
| 87 | initialization. |
| 88 | - Depending on the platform, GWP-ASan may or may not hook malloc allocations |
| 89 | that occur in code not linked directly against Chrome. |
Vlad Tsyrklevich | 04d8664 | 2019-05-21 00:22:50 | [diff] [blame] | 90 | |
Vlad Tsyrklevich | 08bc0525 | 2018-12-04 06:58:54 | [diff] [blame] | 91 | ## Testing |
| 92 | |
| 93 | There is [not yet](https://crbug.com/910751) a way to intentionally trigger a |
| 94 | GWP-ASan exception. |
| 95 | |
| 96 | There is [not yet](https://crbug.com/910749) a way to inspect GWP-ASan data in |
Vlad Tsyrklevich | 4a2e4d20 | 2019-04-25 00:22:43 | [diff] [blame] | 97 | a minidump (crash report) without access to Google's crash service. |