If you are a regular reader of our Chrome release blog, you may have noticed that phrases like 'exploit for CVE-1234-567 exists in the wild' have been appearing more often recently. In this post we'll explore why there seems to be such an increase in exploits, and clarify some misconceptions in the process. We'll then share how Chrome is continuing to make it harder for attackers to achieve their goals.
How things work today
While the increase may initially seem concerning, it’s important to understand the reason behind this trend. If it's because there are many more exploits in the wild, it could point to a worrying trend. On the other hand, if we’re simply gaining more visibility into exploitation by attackers, it's actually a good thing! It’s good because it means we can respond by providing bug fixes to our users faster, and we can learn more about how real attackers operate.
So, which is it? It’s likely a little of both.
Our colleagues at Project Zero publicly track all known in-the-wild “zero day” bugs. Here’s what they’ve reported for browsers:
First, we don’t believe there was no exploitation of Chromium based browsers between 2015 and 2018. We recognize that we don’t have full view into active exploitation, and just because we didn’t detect any zero-days during those years, doesn’t mean exploitation didn’t happen. Available exploitation data suffers from sampling bias.
Teams like Google’s Threat Analysis Group are also becoming increasingly sophisticated in their efforts to protect users by discovering zero-days and in-the-wild attacks. A good example is a bug in our Portals feature that we fixed last fall. This bug was discovered by a team member in Switzerland and reported to Chrome through our bug tracker. While Chrome normally keeps each web page locked away in a box called the “renderer sandbox,” this bug allowed the code to break out, potentially allowing attackers to steal information. Working across multiple time zones and teams, it took the team three days to come up with a fix and roll it out, as detailed in our video on the process:
Why so many exploits?
There are a number of factors at play, from changes in vendor and attacker behavior, to changes in the software itself. Here are four in particular that we've been discussing and exploring as a team.
First, we believe we’re seeing more bugs thanks to vendor transparency. Historically, many browser makers didn’t announce that a bug was being exploited in the wild, even if they knew it was happening. Today, most major browser makers have increased transparency via publishing details in release communications, and that may account for more publicly tracked “in the wild” exploitation. These efforts have been spearheaded by both browser security teams and dedicated research groups, such as Project Zero.
Second, we believe we’re seeing more exploits due to evolved attacker focus. There are two reasons to suspect attackers might be choosing to attack Chrome more than they did in the past.
Third, some attacks that could previously be accomplished with a single bug now require multiple bugs. Before 2015, only a single in-the-wild bug was required to steal a user’s secrets from other websites, because multiple web pages lived together in a single renderer process. If an attacker could compromise the renderer process belonging to a malicious website that a user visited, they might have been able to access the credentials for some other more sensitive website.
With Chrome’s multiyear Site Isolation project largely complete, a single bug is almost never sufficient to do anything really bad. Attackers often need to chain at least two bugs: first, to compromise the renderer process, and second, to jump into the privileged Chrome browser process or directly into the device operating system. Sometimes multiple bugs are needed to achieve one or both of these steps.
So, to achieve the same result, an attacker generally now has to use more bugs than they previously did. For exactly the same level of attacker success, we’d see more in-the-wild bugs reported over time, as we add more layers of defense that the attacker needs to bypass.
Fourth, there’s simply the fact that software has bugs. Some fraction of those bugs are exploitable. Browsers increasingly mirror the complexity of operating systems — providing access to your peripherals, filesystem, 3D rendering, GPUs — and more complexity means more bugs.
Ultimately, we believe data is an important part of the story, but the absolute number of exploited bugs isn't a sufficient measure of security risk. Since some security bugs are inevitable, how a software vendor architects their software (so that the impact of any single bug is limited) and responds to critical security bugs is often much more important than the specifics of any single bug.
How Chrome is raising the bar
The Chrome team works hard to both detect and fix bugs before releases and get bug fixes out to users as quickly as possible. We’re proud of our record at fixing serious bugs quickly, and we are continually working to do better.
For example, one area of concern for us is the risk of n-day attacks: that is, exploitation of bugs we’ve already fixed, where the fixes are visible in our open-source code repositories. We have greatly reduced our “patch gap” from 35 days in Chrome 76 to an average of 18 days in subsequent milestones, and we expect this to reduce slightly further with Chrome’s faster release cycle.
Irrespective of how quickly bugs are fixed, any in-the-wild exploitation is bad. Chrome is working hard to make it expensive and difficult for attackers to achieve their goals.
Some examples of the projects ongoing:
We are well past the stage of having “easy wins” when it comes to raising the bar for security. All of these are long term projects with significant engineering challenges. But as we've shown with Site Isolation, Chrome isn't afraid of making long term investments in major security engineering projects. One of the major challenges is performance: all of these technologies (except memory safe languages) could risk slowing the browser. Expect a series of blog posts over the coming months as we explore performance vs. security trade-offs. These decisions are really hard: we do not want to make Chrome slower for billions of people, especially as this disproportionately hits users with slower devices – we strive to make Chrome secure for all our users, not just those with the high end systems.
How you can help
Above all: if Chrome is reminding you to update, please do!
If you’re an enterprise IT professional, keep your users up-to-date by keeping auto-update on, and familiarize yourself with the added enterprise policies and controls that you can apply to Chrome within your organization. We strongly advise not focusing on zero-days when making decisions about updates, but instead to assume any Chrome security bug is under exploitation as an n-day.
If you're a security researcher, you can report bugs you find to the Chrome Vulnerability Rewards Program — and thanks for helping us make Chrome safer for everyone!
The v5.16 release of the Linux kernel implements support for THREAD_INFO_IN_TASK when running on ARMv7 SMP systems. This protects the kernel's per-task bookkeeping (called thread_info), which lives on the far (and normally unused) end of the stack, against stack overflows which may occur in rare -yet sometimes exploitable- cases where the control flow of the program simply ends up accumulating more state than the stack can hold. (Note that a stack overflow is not the same as a stack buffer overflow, where the overflow happens in the opposite direction.)
By moving thread_info off the stack and into the kernel heap, and by using a special SMP CPU register to keep track of its location, we can mitigate the risk of stack overflows resulting in thread_info corruption. However, it does not prevent stack overflows themselves: these may still occur, and result in corruption of other data structures that happen to be adjacent to the task stack in memory.
For CPUs that lack this special SMP CPU register, we also proposed an implementation of THREAD_INFO_IN_TASK that is expected to land in v5.18. Instead of a special register, it uses a global variable to keep track of the location of thread_info.
Preventing stack overflows from corrupting unrelated memory contents is the goal of VMAP_STACK, which we are enabling for 32-bit ARM as well. When VMAP_STACK is enabled, kernel mode stacks are allocated from the kernel heap as before, but mapped into a different part of the kernel's address space, and surrounded by guard regions, which are guaranteed to be kept unpopulated. Given that accesses to such unpopulated regions will trigger an exception, the kernel's memory management layer can step in and terminate the program as soon as a stack overflow occurs, and prevent it from causing memory corruption.
Last year was another record setter for our Vulnerability Reward Programs (VRPs). Throughout 2021, we partnered with the security researcher community to identify and fix thousands of vulnerabilities – helping keep our users and the internet safe.
Thanks to these incredible researchers, Vulnerability Reward Programs across Google continued to grow, and we are excited to report that in 2021 we awarded a record breaking $8,700,000 in vulnerability rewards – with researchers donating over $300,000 of their rewards to a charity of their choice.
We also launched bughunters.google.com in 2021, a public researcher portal dedicated to keeping Google products and the internet safe and secure. This new platform brings all of our VRPs (Google, Android, Abuse, Chrome, and Google Play) closer together and provides a single intake form, making security bug submission easier than ever. We’re excited about everything the new Bug Hunters portal has to offer, including:
More opportunities for interaction and a bit of healthy competition through gamification, per-country leaderboards, awards/badges for certain bugs, and more!
A more functional and aesthetically pleasing leaderboard. We know a lot of you are using your achievements in our VRPs to find jobs (we’re hiring!) and we hope this acts as a useful resource.
A stronger emphasis on learning: bug hunters can improve their skills through the content available in our new Bug Hunter University
Streamlined publication process: we know the value that knowledge sharing brings to our community. That’s why we want to make it easier for you to publish your bug reports.
We now offer swag! The first 20 folks who share this blog post on Twitter and tag @GoogleVRP will receive a gift voucher for swag in their DMs.
Android
The Android VRP doubled its 2020 total payouts in 2021 with nearly $3 million dollars in rewards, and awarded the highest payout in Android VRP history: an exploit chain discovered in Android receiving a reward of $157,000!
Our industry leading prize of $1,500,000 for a compromise of our Titan-M Security chip used in our Pixel device remains unclaimed - for more information on this reward and Android exploit chain rewards, please visit our public rules page.
The program also launched the Android Chipset Security Reward Program (ACSRP), a vulnerability reward program offered by Google in collaboration with manufacturers of certain popular Android chipsets. This private, invite-only program, provides reward and recognition for contributions of security researchers who invest their time and effort into helping make Android devices more secure. In 2021 the ACSRP paid out $296,000 for over 220 valid and unique security reports.
We would like to give a special shoutout to some of our top researchers whose continued hard work keeps Android safe and secure:
Aman Pandey of Bugsmirror Team has skyrocketed to our top researcher last year, submitting 232 vulnerabilities in 2021! Since submitting their first report in 2019, Aman has reported over 280 valid vulnerabilities to the Android VRP and has been a crucial part of making our program so successful.
Yu-Cheng Lin (林禹成) (@AndroBugs) has been another phenomenal researcher for the Android VRP, submitting a whopping 128 valid reports to the program in 2021.
Researcher gzobqq@gmail.com discovered a critical exploit chain in Android (CVE-2021-39698) , receiving the highest payout in Android VRP history of $157,000.
Chrome
This year the Chrome VRP also set some new records – 115 Chrome VRP researchers were rewarded for 333 unique Chrome security bug reports submitted in 2021, totaling $3.3 million in VRP rewards. The contributions not only help us to improve Chrome, but also the web at large by bolstering the security of all browsers based on Chromium.
Of the $3.3 million, $3.1 million was awarded for Chrome Browser security bugs and $250,500 for Chrome OS bugs, including a $45,000 top reward amount for an individual Chrome OS security bug report and $27,000 for an individual Chrome Browser security bug report.
Of these totals, $58,000 was awarded for security issues discovered by fuzzers contributed by VRP researchers to the Chrome Fuzzing program. Each valid report from an externally provided fuzzer received a $1,000 patch bonus, with one fuzzer report receiving a $16,000 reward.
The Chrome VRP would not be able to smash these records over the last year without the efforts of so many exceptional VRP researchers. We’d like to highlight a few researcher achievements made in 2021:
Rory McNamara, a Chrome OS VRP researcher who has been participating in the Chrome VRP for five years, became the highest awarded Chrome VRP researcher of all time. This year he was rewarded for six reports achieving root privilege escalation in Chrome OS, one of which received the highest reward amount achieved for a single Chrome bug report in 2021 at $45,000.
Chrome Browser VRP researcher Leecraso (@leecraso) of 360 Vulnerability Research Institute was the most awarded researcher of 2021, with 18 valid bug reports; a majority of which were for memory corruption vulnerabilities affecting the browser process.
We love when researchers write about their findings (only after we have publicly disclosed the bug, of course)! Chrome Browser VRP researcher Brendon Tiszka wrote an excellent two-part blog series on his discovery and exploitation of a V8 vulnerability, CVE-2021-21225, the analysis and reporting of which earned him a $22,000 VRP reward.
Huge thanks and congratulations to all Chrome VRP researchers that helped us make Chrome and Chrome OS more safe for all users in 2021!.
Google Play
Google Play paid out $550,000 in rewards to over 60 unique security researchers.
The Google Play Security Reward Program also released their Android App Hacking Workshop content and published a blog on their work to empower the next generation of Android Application Security Researchers.
kCTF VRP
In November we expanded our reward amounts for exploits against our kCTF cluster from 5,000-10,000 up to 31,337-50,337 USD. In the last 3 months we were happy to have several participants receive $175,685 USD in rewards. We also extended the timeline of the increased rewards until February 14 (from January 31) which should give everyone a couple more weeks to finalize any almost-working exploits.
GCP VRP Prize
To encourage security researchers to focus on Google Cloud Platform, we initiated the annual GCP VRP Prize in 2019. In March this year, we announced the winners of the 2020 edition of the prize and paid out $313,337 in prizes. Ezequiel Pereira won the top prize of $133,337 for finding an RCE in Google Cloud Deployment Manager. We saw some amazing research on Google Cloud Platform this year too. Stay tuned for the 2021 winners!
Research Grants
Six years ago, the Google VRP launched an experimental Vulnerability Research Grant program to encourage seasoned security researchers to take a detailed and extensive look into the security of Google products and services. And reward them even if there are no vulnerabilities found. Six years later, we are happy to announce that in 2021 we awarded over $200,000 in grants to more than 120 security researchers around the world.
If you are a Google VRP researcher and want to be considered for a Vulnerability Research Grant make sure you opted in on your bughunters profile.
Looking forward
With the launch of the new Bug Hunters portal, we plan to continue improving our platform and listening to you - our researchers - on ways we can improve our platform and Bug Hunter University.
Thank you again for making Google, the Internet, and our users safe and secure! Follow us on @GoogleVRP
Thank you to Adam Bacchus, Dirk Göhmann, Sarah Jacobus, Amy Ressler, Martin Straka, Jan Keller, Jon Bottarini, and Rishika Hooda