6620handout4o
0 likes380 views
This document discusses power attacks on cloud computing infrastructure. It describes how oversubscription of power capacity leaves data centers vulnerable to attacks that generate power spikes. The attacks could be launched by malicious users running intensive workloads on public servers. Experiments show how workloads can be tuned to significantly increase power consumption and potentially trip circuit breakers. Various attack vectors are explored targeting infrastructure as a service (IaaS), platform as a service (PaaS) and software as a service (SaaS). Simulations demonstrate the attacks could cause outages and damage at the data center level if launched at large scale. Mitigations are difficult due to the challenges of predicting and limiting peak power usage.
6620handout4o
- 1. INSE 6620 (Cloud Computing Security and Privacy) Attacks on Cloud Prof. Lingyu Wang 1
- 2. OutlineOutline Co-Residence Attack Power Attack 2 Ristenpart et al., Hey, You, Get Off of My Cloud: Exploring Information Leakage in Third-Party Compute Clouds; Xu et al., Power Attack: An Increasing Threat to Data Centers
- 3. The Threat of Multi-TenancyThe Threat of Multi Tenancy In traditional systems, security goal is usually t “k b d t”to “keep bad guys out” Clouds brings new threats with multi-tenancy: l i l i d d h h h i lMultiple independent users share the same physical infrastructure So an attacker can legitimately be in the sameSo, an attacker can legitimately be in the same physical machine as the target The bad guys are next to you... 3
- 4. What Would the Bad Guys do?What Would the Bad Guys do? Step 1: Find out where the t t i l t dtarget is located St 2 T t b l t d ithStep 2: Try to be co-located with the target in the same (physical) machinemachine Step 2.1: Verify it’s achieved Step 3: Gather information about the target once co-locatedg 4
- 5. “Hey, You, Get Off of My Cloud”Hey, You, Get Off of My Cloud Influential, cited by 872 papers as of 2014 July (G l S h l )(Google Scholar) Media coverage: MIT Technology Review Network World Network World (2) Computer WorldMIT Technology Review, Network World, Network World (2), Computer World, Data Center Knowledge, IT Business Edge, Cloudsecurity.org, Infoworld Attack launched against commercially available ” l” l d (A EC2)”real” cloud (Amazon EC2) Claims up to 40% success in co-residence with target VMtarget VM First work showing concrete threats in cloud 5
- 6. Approach OverviewApproach Overview Map the cloud infrastructure to estimate where the target is Footprinting P t iestimate where the target is located - cartography Use various heuristics to verify Port scanning Discovering vulnerabilitiesy co-residence of two VMs Launch probe VMs trying to be co residence with target VMs vulnerabilities Initial exploitation co-residence with target VMs Exploit cross-VM side-channel leakage to gather information Privilege escalation g g about the target 6
- 7. Threat ModelThreat Model Attacker model Cloud infrastructure provider is trustworthy Cloud insiders are trustworthy Attacker is a malicious non provider affiliated thirdAttacker is a malicious non-provider-affiliated third party who can legitimately use cloud provider's service Victim model Victims are other cloud users that have sensitive information 7
- 8. The Amazon EC2 Dom0Guest1 Guest2 The Amazon EC2 Xen hypervisor Zen Hypervisor Domain0 is used to manage guest images, physical resource provisioning, and access control rights Dom0 routes packages and reports itself as a firstDom0 routes packages and reports itself as a first hop Users may choose to create instance iny 2 regions (United States and Europe) 3 availability zones (for fault tolerance) 5 Linux instance types m1.small, c1.medium, m1.large, m1.xlarge, c1.xlarge 8
- 9. IP Addresses of InstancesIP Addresses of Instances An instance may have a public IP 75.101.210.100, which, from outside the cloud, maps to an external DNS name ec2-75-101-210-100 compute-1 amazonaws comec2-75-101-210-100.compute-1.amazonaws.com And an internal IP and DNS name 10 252 146 5210.252.146.52 domU-12-31-38-00-8D-C6.compute-1.internal Within the cloud, both domain names resolveWithin the cloud, both domain names resolve to the internal IP 75.101.210.100 -> ec2-75-101-210-100.compute- 1.amazonaws.com -> 10.252.146.52 9
- 10. Network ProbingNetwork Probing nmap: TCP connect probes (3-way handshake) hping: TCP SYN traceroutes both nmap/hping targeting ports 80 and 443 wget: retrieve web pages up to 1024B Internal probing from an instance to another Legitimate w.r.t. Amazon policies External probing from outside EC2 Not illegal (port scanning is) Only targeting port 80/443 – with services running – implication on ethical issuesimplication on ethical issues 10
- 11. Step 1: Mapping the CloudStep 1: Mapping the Cloud Hypothesis: The Amazon EC2 internal IP address space is cleanly partitioned between availability zones (likely to make it easy to manage separate network(likely to make it easy to manage separate network connectivity for these zones) Instance types within these zones also show considerable regularityconsiderable regularity Moreover, different accounts exhibit similar placement.p 11
- 12. Mapping the CloudMapping the Cloud 20 instances for each of the 15 zone/type pairs, total 300 Plot of internal IPs against zonesPlot of internal IPs against zones Result: Different availability zones correspond to differentResult: Different availability zones correspond to different statically defined internal IP address ranges. 12
- 13. Mapping the CloudMapping the Cloud 20 instances of each type, from another account, zone 3 Plot of internal IPs in Zone 3 against instance typesPlot of internal IPs in Zone 3 against instance types Result: Same instance types correspond loosely withResult: Same instance types correspond loosely with similar IP address range regions. 13
- 14. Derive IP Address Allocation RulesDerive IP Address Allocation Rules Heuristics to label /24 prefixes with both il bilit d i t tavailability zone and instance type: All IPs from a /16 are from same availability zone A /24 inherits any included sampled instance typeA /24 inherits any included sampled instance type. If multiple instance types, then it is ambiguous A /24 containing a Dom0 IP address only contains/ g y Dom0 IP addresses. We associate to this /24 the type of the Dom0’s associated instance All /24’s between two consecutive Dom0 /24’sAll /24’s between two consecutive Dom0 /24’s inherit the former’s associated type. 10.250.8.0/24 contained Dom0 IPs associated with m1.small instances in 10.250.9.0/24, 10.250.10.0/24 14
- 15. Mapping 6057 EC2 ServersMapping 6057 EC2 Servers 15
- 16. Preventing Cloud CartographyPreventing Cloud Cartography Why preventing? Make following attacks harder Hiding infrastructure/amount of users Wh t k i i ?What makes mapping easier? Static local IPs – changing which may complicate managementmanagement External to internal IP mapping – preventing which can only slow down mapping (timing and tracert are still possible) 16
- 17. Step 2: Determine Co-residenceStep 2: Determine Co residence Network-based co-resident checks: instances lik l id t if th hare likely co-resident if they have: matching Dom0 IP address Dom0: 1st hop from this instance or last hop to victimDom0: 1st hop from this instance, or last hop to victim small packet round-trip times Needs a “warm-up” – 1st probe discarded numerically close internal IP addresses (e.g., within 7) 8 m1 small instances on one machine8 m1.small instances on one machine 17
- 18. Step 2: Determine Co-residenceStep 2: Determine Co residence Verified via a hard-disk-based covert channel All “i t ” i 3All “instances” are in zone 3 Effective false positive rate of ZERO G ith i l t tGo with a simpler test: Close enough internal IPs? If yes then tracert. A single hop (dom0) in between? If yes test passessingle hop (dom0) in between? If yes test passes 18
- 19. Step 3: Exploiting VM PlacementStep 3: Exploiting VM Placement Facts about Amazon placement Same account never has instances on same machine (so 8 instances will be placed on 8 machines)machines) Sequential locality (A stops then B starts, A and B likely co-resident) Parallel locality (A and B under different accounts run at roughly same time likely co-resident) Machines with less instances are more likely beMachines with less instances are more likely be placed to (load balancing) m1.xlarge and c1.xlarge have their own machines 19
- 20. Step 3: Exploiting VM PlacementStep 3: Exploiting VM Placement Strategy 1: Brute-forcing placement 141 out of 1686: a success rate of 8.4% Strategy 2: Abusing Placement Locality Attacker instance-flooding right after the target instances are launched – exploiting parallel locality Observing instance disappearing/reappearingObse g s a ce d sappea g/ eappea g Triggering the creation of new instances (elasticity) 40% success rate flooding after 5 mins 20
- 21. Step 3: Exploiting VM PlacementStep 3: Exploiting VM Placement “Window” for parallel Evidence of sequential l litlocality is quite large locality (Each instance is killed immediately afterimmediately after probing) 21
- 22. Step 4: Information LeakageStep 4: Information Leakage Co-Residency affords the ability to: Denial of Service Estimate victim's work load Extract cryptographic keys via side channelsExtract cryptographic keys via side channels 22
- 23. MitigationsMitigations Co-residence checks: Prevent identification of dom0/hypervisor VM placement: Allow users to control/exclusively use machines Side channel leaks: M th d i tMany methods exist Limitations: impractical (overhead), application- specific, or insufficient protectionspecific, or insufficient protection Also, all of them require to know all possible channels in advance 23
- 24. Amazon's responseAmazon s response Amazon downplays report highlighting l biliti i it l d ivulnerabilities in its cloud service "The side channel techniques presented are based on testing results from a carefully controlled labon testing results from a carefully controlled lab environment with configurations that do not match the actual Amazon EC2 environment." "As the researchers point out, there are a number of factors that would make such an attack significantly more difficult in practice."significantly more difficult in practice. http://www.techworld.com.au/article/324189/amazon_downplays _report_highlighting_vulnerabilities_its_cloud_service 24
- 26. BackgroundBackground The number of servers in data center surged f 24 illi i 2008 t 35 illi i 2012from 24 million in 2008 to 35 million in 2012 Power consumption 56% percent increase V i t d i tiVery expensive to upgrade existing power infrastructures How to add more servers with less cost?How to add more servers with less cost? 26
- 27. Power AttackPower Attack Solution: Oversubscription Place more servers than can be supported by the power infrastructure Assumption: not all servers will reach peakAssumption: not all servers will reach peak consumption (nameplate power ratings) at the same time Leaves data center vulnerable to power attack: Malicious workload that can generate power spikes lti l / k / h l d t ton multiple servers/racks/whole data center Launched as a regular user Causing DoS to both providers and clients byCausing DoS to both providers and clients by triggering the circuit breakers (CBs) 27
- 28. Power Distribution in Data CentersPower Distribution in Data Centers Three tiers: (60-400kV → transformer → 10-20kV → switchgear → 400- 600 ) UPS/PDU600v) → UPS/PDUs (Power distribution units) → racks Ci it b k tCircuit breakers at switchgear, PDUs, and rack-level branch circuitcircuit 28
- 29. OversubscriptionOversubscription Google’s analysis Workload traces collected from real data centers: search, webmail, and MapReduce Peak power reaches 96% of rated capacity at rackPeak power reaches 96% of rated capacity at rack level, but 72% at data center level Oversubscription would allow adding 38% more servers A big assumption: Workloads never reach peak consumption Benign workloads – maybe; malicious ones – no! 29
- 30. Threat ModelThreat Model Target can be rack, PDU, or data centerTarget can be rack, PDU, or data center Running public servers, e.g.,Running public servers, e.g., IaaSIaaS,, PaaSPaaS,, SaasSaas Power oversubscriptionPower oversubscription Power consumption is monitored/managed at rackPower consumption is monitored/managed at rackPower consumption is monitored/managed at rackPower consumption is monitored/managed at rack-- level (machinelevel (machine--level is too expensive)level is too expensive) AdversaryAdversaryAdversaryAdversary Hackers, competitors, cyber crime/cyber warfareHackers, competitors, cyber crime/cyber warfare Regular user with sufficient resources (large numberRegular user with sufficient resources (large numberg ( gg ( g of accounts, workload) and mapping of cloudof accounts, workload) and mapping of cloud Our focus: How to generate power spikes?Our focus: How to generate power spikes? 30 UnderUnder IaaSIaaS,, PaaSPaaS,, SaasSaas??
- 31. Power Attack in PaaSPower Attack in PaaS PaaS: Attacker can run any chosen applications Load balancing Load (utility) balancing ≠ power balancing Attack in two stages Utility reaches 100% (e.g., CPU) Fi t kl d t f th iFine-tune workload to further increase power consumption (remember utility ≠ power) 31
- 32. Single Server TestSingle Server Test Goal: find out how workloads affect power SPECCPU2006 HPC benchmark Results: Different workloads have very different power cost Same CPU (100%), memory ≠ power e g 462 vs 465 462 vs 456e.g., 462 vs 465, 462 vs 456 32
- 33. Single Server TestSingle Server Test HPL benchmark Multiple parameters to adjust Adjust block size NB (how problem is solved) R ltResults: Same workload, same CPU and memory Different power cost under different parametersDifferent power cost under different parameters 33
- 34. Rack Level TestRack Level Test Results: Similar to single machine Attack: Increasing workload to reach utility cap Further increasing power cost by changing workload/tuning parametersworkload/tuning parameters 34
- 35. Damage AssessmentDamage Assessment Overheating One CPU is overheated, resulting in system failure CB tripped in a room with 16 servers out of which only 4 servers are under attack It ill onl get o se in eal o ldIt will only get worse in real world When memory/I/O devices are attacked With better “power proportionality”With better power proportionality 60% power consumed when idle in this case 35
- 36. Power Attack in IaaSPower Attack in IaaS IaaS: More control using VMs; more exposure Attack vectors: Parasite attack – attack from inside R li ti f VMRun applications from VMs Launch DoS attacks on such VMs (more power cost than normal workload) Exploit routine operations Live migration of VMs Launch parasite attack during migrationLaunch parasite attack during migration 36
- 37. Evaluation - Parasite AttackEvaluation Parasite Attack Parasite attack Co-resident with victim Run intensive workload Launch DoS attacks on such VMsLaunch DoS attacks on such VMs Results Normal load: 180wNormal load: 180w Intensive load: 200w DoS: 230w (peak 245w)DoS: 230w (peak 245w) Increase 30% smurf: broadcasting 37
- 38. Evaluation – VM MigrationEvaluation VM Migration Results During migration, both source and dest. experience power spikes (memory copy, NICs, CPUs) Intra-rack and inter-rack (migrating to same server)Intra-rack and inter-rack (migrating to same server) 38
- 39. Power Attack in SaaSPower Attack in SaaS SaaS: Limited control Attack vectors: Specially crafted requests Trigger larger numbers of cache misses Floating point operations Floating point unit (FLU) more power hungry than Arithmetic logic unit (ALU) Divisions rather than add/multiplication 39
- 40. Power Attack in SaaSPower Attack in SaaS RUBiS online shopping benchmark Modified to support: Floating point operations (discount coupons) Cache misses (continuously browsing at random) 30-40% of power spikes 40
- 41. Data Center Level SimulationsData Center Level Simulations Based on configurations of Google data center i L i NC USAin Lenoir, NC, USA “Original” workload based on traces of the data ente nd “Att k” in l de HPC o klo dcenter and “Attack” includes HPC workloads Attacking the “Peak”, “medium”, and “valley” regionsregions 41
- 42. ResultsResults One PDU: 22 min attack trips PDU-level CB Multi-PDU: 4 attacks all trip CB First 3 attacks recovered due to load balancing Last attack causes DoS (only 53% of requests processed) during 58-69hrs 42
- 43. ResultsResults DC level attack iblpossible Larger scale attacks eq i e mo erequire more resources 43
- 44. MitigationMitigation Power capping: limit peak consumption Challenges: 2-minute sampling window enough for attacks; even longer (12 mins) to actually reduce consumptionconsumption Server consolidation (shutdown if not in use) Better power proportionality – more aggressivep p p y gg oversubscription - more vulnerable Challenges: Need to save power Difficult to monitor power iffi l di i i h b d k 44 Difficult to distinguish between users and attacks
- 45. MitigationMitigation Promising solutions Models estimating power consumption of requests and consequently limit them Power balancing instead of load balancingPower balancing instead of load balancing Deploying per-server UPS 45