Exploring emerging technologies in the HPC co-design space

Exploring emerging technologies in the HPC co-design space

• 1.
  Exploring Emerging Technologies inthe HPC Co-Design Space Jeffrey S. Vetter Presented to SC13 DOE Booth Denver 20 Nov 2013 http://ft.ornl.gov  [email protected]
• 2.
  Presentation in anutshell • Our community expects major challenges in HPC as we move to extreme scale – Power, Performance, Resilience, Productivity – Major shifts and uncertainty in architectures, software, applications – Applications will have to change in response to design of processors, memory systems, interconnects, storage • Technologies particularly pertinent to addressing some of these challenges – Heterogeneous computing – Nonvolatile memory • DOE has initiated Codesign Centers that bring together all stakeholders to develop integrated solutions • Aspen is a new approach to model characteristics of applications and emerging architectures – This structure allows easy development, sharing, verification of models – Automated exploration of design spaces 2
• 3.
  Contemporary HPC Architectures Date System Location Comp Comm 2009 Jaguar;Cray XT5 ORNL AMD 6c Seastar2 2.3 7.0 2010 Tianhe-1A NSC Tianjin Intel + NVIDIA Proprietary 4.7 4.0 2010 Nebulae NSCS Shenzhen Intel + NVIDIA IB 2.9 2.6 2010 Tsubame 2 TiTech Intel + NVIDIA IB 2.4 1.4 2011 K Computer RIKEN/Kobe SPARC64 VIIIfx Tofu 10.5 12.7 2012 Titan; Cray XK6 ORNL AMD + NVIDIA Gemini 27 9 2012 Mira; BlueGeneQ ANL SoC Proprietary 10 3.9 2012 Sequoia; BlueGeneQ LLNL SoC Proprietary 20 7.9 2012 Blue Waters; Cray NCSA/UIUC AMD + (partial) NVIDIA Gemini 2013 Stampede TACC Intel + MIC IB 9.5 5 2013 Tianhe-2 NSCC-GZ (Guangzhou) Intel + MIC Proprietary 54 ~20 3 Peak (PF) Power (MW) 11.6
• 4.
  Notional Exascale ArchitectureTargets (From Exascale Arch Report 2009) System attributes 2001 2010 “2015” “2018” 10 Tera 2 Peta 200 Petaflop/sec 1 Exaflop/sec Power ~0.8 MW 6 MW 15 MW 20 MW System memory 0.006 PB 0.3 PB 5 PB 32-64 PB Node performance 0.024 TF 0.125 TF 0.5 TF 7 TF 1 TF 10 TF 25 GB/s 0.1 TB/sec 1 TB/sec 0.4 TB/sec 4 TB/sec System peak Node memory BW Node concurrency 16 12 O(100) O(1,000) O(1,000) O(10,000) System size (nodes) 416 18,700 50,000 5,000 1,000,000 100,000 1.5 GB/s 150 GB/sec 250 GB/sec 2 TB/sec Total Node Interconnect BW MTTI day 1 TB/sec O(1 day) O(1 day) http://science.energy.gov/ascr/news-and-resources/workshops-and-conferences/grand-challenges/
• 5.
  Notional Future Architecture Interconnection Network
• 6.
  Emerging Architectures ‘The rootof all evil’ – Anonymous application scientist 6
• 7.
  • The pastdecade has started the trend away from traditional architectures • Mainly driven by facilities costs and successful (sometimes heroic) application examples • Examples – Cell, GPUs, FPGAs, SoCs, etc • Many open questions – Understand technology challenges – Evaluate and prepare applications – Recognize, prepare, enhance programming models 7 Popular architectures since ~2004 Recent Experimental Computing Systems
• 8.
  Emerging Computing Architectures– Future Possibilities • Heterogeneous processing – Many cores – Fused, configurable memory • Memory – 2.5D and 3D Stacking – New devices (PCRAM, ReRAM) • Interconnects – Collective offload – Scalable topologies • Storage – Active storage – Non-traditional storage architectures (key-value stores) • Improving performance and programmability in face of increasing complexity – Power, resilience HPC (mobile, enterprise, embedded) computer design is more fluid now than in the past two decades. 8
• 9.
  Emerging Computing Architectures– Future Possibilities • Heterogeneous processing – Many cores – Fused, configurable memory • Memory – 3D Stacking – New devices (PCRAM, ReRAM) • Interconnects – Collective offload – Scalable topologies • Storage – Active storage – Non-traditional storage architectures (key-value stores) • Improving performance and programmability in face of increasing complexity – Power, resilience HPC (mobile, enterprise, embedded) computer design is more fluid now than in the past two decades. 9
• 10.
  Heterogeneous Computing You couldnot step twice into the same river. -- Heraclitus
• 11.
  DOE’s “Titan” HybridSystem: Cray XK7 with AMD Opteron and NVIDIA Tesla processors 4,352 ft2 12 SYSTEM SPECIFICATIONS: • Peak performance of 27.1 PF • 24.5 GPU + 2.6 CPU • 18,688 Compute Nodes each with: • 16-Core AMD Opteron CPU • NVIDIA Tesla “K20x” GPU • 32 + 6 GB memory • 512 Service and I/O nodes • 200 Cabinets • 710 TB total system memory • Cray Gemini 3D Torus Interconnect • 8.9 MW peak power
• 12.
  http://keeneland.gatech.edu Keeneland – FullScale System Initial Delivery system installed in Oct 2010 •201 TFLOPS in 7 racks (90 sq ft incl service area) •902 MFLOPS per watt on HPL (#12 on Green500) •Upgraded April 2012 to 255 TFLOPS •Over 200 users, 100 projects using KID Full scale system Keeneland System •792 M2090 GPUs contribute to aggregate system peak of 615 TF Rack (11 Compute Racks) (6 Chassis) S6500 Chassis (4 Nodes) ProLiant SL250 G8 (2CPUs, 3GPUs) M2090 Xeon E5-2670 166 GFLOPS 665 GFLOPS 2327 GFLOPS 32/18 GB 9308 GFLOPS Full PCIeG3 X16 bandwidth to all GPUs 55848 GFLOPS 614450 GFLOPS Mellanox 384p FDR Infiniband Switch Integrated with NICS Datacenter Lustre and XSEDE J.S. Vetter, R. Glassbrook et al., “Keeneland: Bringing heterogeneous GPU computing to the computational science community,” IEEE Computing in Science and Engineering, 13(5):90-5, 2011, http://dx.doi.org/10.1109/MCSE.2011.83.
• 13.
  Opportunity: Dark SiliconWill Make Heterogeneity and Specialization More Relevant Source: ARM
• 14.
  Fused memory hierarchy:AMD Llano Fused GPU better Discrete GPU better K. Spafford, J.S. Meredith, S. Lee, D. Li, P.C. Roth, and J.S. Vetter, “The Tradeoffs of Fused Memory Hierarchies in Heterogeneous Architectures,” in ACM Computing Frontiers (CF). Cagliari, Italy: ACM, 2012. Note: Both SB and Llano are consumer, not server, parts. 15
• 15.
  Applications must usea mix of programming models for these architectures MPI Low overhead Resource contention Locality OpenMP, Pthreads SIMD NUMA OpenACC, CUDA, OpenCL Memory use, coalescing Data orchestration Fine grained parallelism Hardware features
• 16.
  OpenARC: Open AcceleratorResearch Compiler • Open-Sourced, High-Level Intermediate Representation (HIR)-Based, Extensible Compiler Framework. – Perform source-to-source translation from OpenACC C to target accelerator models. • Support full features of OpenACC V1.0 ( + array reductions and function calls) • Support both CUDA and OpenCL as target accelerator models – Provide common runtime APIs for various back-ends – Can be used as a research framework for various study on directive-based accelerator computing. • Built on top of Cetus compiler framework, equipped with various advanced analysis/transformation passes and built-in tuning tools. • OpenARC’s IR provides an AST-like syntactic view of the source program, easy to understand, access, and transform the input program. 17
• 17.
  OpenARC System Architecture InputC OpenACCP rogram OpenACC Preprocessor OpenACC Parser C Parser General Optimizer OpenARC Compiler Host CPU Code A2G Translator GPU-specific Optimizer Device Kernel Code Output Executable Backend Compiler CUDA Driver API OpenARC Runtime API OpenCL Runtime API OpenARC Runtime Other Device-specific Runtime APIs 18
• 18.
  Preliminary Evaluation ofOpenARC Performance of OpenARC and PGI-OpenACC compiler (V13.6) relative to manual CUDA versions (Lower is better.) 19 19
• 19.
  Prototyping Solutions inOpenARC • Issues in Directive-Based Accelerator Programming Models – Directive-based accelerator programming models provide abstraction over diverse architectural details and related low-level programming complexities. – However, too much abstraction puts significant burdens on: • Performance tuning • Debugging • Scaling • Our Approach – Provide an extensible research framework, called OpenARC, to enable various compiler/runtime study on directive-based accelerator computing 20 20
• 20.
  Research Agenda • ResearchIn Progress – Directive-based, interactive program debugging and optimization – Compiler-driven, custom profiling system for directive programming – Directive-based, application-driven cooperative CPU/GPU fault detection – autoASPEN: directive-based, automatic performance modeling – OpenACC-e: OpenACC extension to support accelerator-specific features at high-level 21 21
• 21.
  Research in Progress Optimizationand Interactive Program Verification with OpenARC • Problem – Too much abstraction in directive-based GPU programming! – – • Debuggability – Difficult to diagnose logic errors and performance problems at the directive level Performance Optimization – Difficult to find where and how to optimize Solution – Directive-based, interactive GPU program verification and optimization – – OpenARC – User Runtime Iteratively find where and how to fix/optimize 22 OpenARC compiler: – Generates runtime codes necessary for GPU-kernel verification and memory-transfer verification and optimization. Runtime – Locate trouble-making kernels by comparing execution results at kernel granularity. – Trace the runtime status of CPU-GPU coherence to detect incorrect/missing/redundant memory transfers. Users – Iteratively fix/optimize incorrect kernels/memory transfers based on the runtime feedback and apply to input program.
• 22.
  Future Directions inHeterogeneous Computing • Over the next decade: Heterogeneous computing will continue to increase in importance – Embedding and mobile community have already experienced this trend • Manycore • Hardware features – Transactional memory – Random Number Generators • MC caveat – Scatter/Gather – Wider SIMD/AVX • Programming models and runtime systems • Inform vendors about our • Synergies with BIGDATA, mobile priorities markets, graphics • Top 10 list of features to include • Inform applications teams from application perspective. Now to new features and gather is the time! their requirements 26
• 23.
  Memory Systems The Persistenceof Memory http://www.wikipaintings.org/en/salvador-dali/the-persistence-of-memory-1931
• 24.
  Notional Exascale ArchitectureTargets (From Exascale Arch Report 2009) System attributes 2001 2010 “2015” “2018” 10 Tera 2 Peta 200 Petaflop/sec 1 Exaflop/sec Power ~0.8 MW 6 MW 15 MW 20 MW System memory 0.006 PB 0.3 PB 5 PB 32-64 PB Node performance 0.024 TF 0.125 TF 0.5 TF 7 TF 1 TF 10 TF 25 GB/s 0.1 TB/sec 1 TB/sec 0.4 TB/sec 4 TB/sec System peak Node memory BW Node concurrency 16 12 O(100) O(1,000) O(1,000) O(10,000) System size (nodes) 416 18,700 50,000 5,000 1,000,000 100,000 1.5 GB/s 150 GB/sec 250 GB/sec 2 TB/sec Total Node Interconnect BW MTTI day 1 TB/sec O(1 day) O(1 day) http://science.energy.gov/ascr/news-and-resources/workshops-and-conferences/grand-challenges/
• 25.
  Notional Future NodeArchitecture  NVM to increase memory capacity  Mix of cores to provide different capabilities  Integrated network interface  Very high bandwidth, low latency to onpackage locales
• 26.
  Jeffrey Vetter, ORNL RobertSchreiber, HP Labs Trevor Mudge, University of Michigan Yuan Xie, Penn State University Blackcomb: Hardware-Software Co-design for Non-Volatile Memory in Exascale Systems Objectives http://ft.ornl.gov/trac/blackcomb FWP #ERKJU59  Rearchitect servers and clusters, using nonvolatile memory (NVM) to overcome resilience, energy, and performance walls in exascale computing:  Ultrafast checkpointing to nearby NVM  Reoptimize the memory hierarchy for exascale, using new memory technologies  Replace disk with fast, low-power NVM  Enhance resilience and energy efficiency  Provide added memory capacity Established and Emerging Memory Technologies – A Comparison SRAM DRAM eDRAM NAND Flash PCRAM STTRA M ReRAM (1T1R) ReRAM (Xpoint) N N N Y Y Y Y Y 50-200 4-6 19-26 2-5 4-10 8-40 6-20 1- 4 Read Time (ns) <1 30 5 104 10-50 10 5-10 50 Write Time (ns) <1 50 5 105 100-300 5-20 5-10 10-100 Number of Rewrites 1016 1016 1016 104-105 108-1012 1015 108-1012 106-1010 Read Power Low Low Low High Low Low Low Medium Write Power Low Low Low High High Medium Medium Medium Leakage Refresh Refresh None None None None Sneak Data Retention Cell Size (F2) Power (other than R/W) 30
• 27.
  Tradeoffs in ExascaleMemory Architectures • Understanding the tradeoffs – ECC type, row buffers, DRAM physical page size, bitline length, etc “Optimizing DRAM Architectures for Energy-Efficient, Resilient Exascale Memories,” (to appear) SC13, 2013 31
• 28.
  New hybrid memoryarchitectures: What is the ideal organizations for our applications? B A C Natural separation of applications objects? DRAM 32
• 29.
  Identifying Opportunities forByte-Addressable Non-Volatile Memory in Extreme-Scale Scientific Applications • Problem – Do specific memory workload characteristics of scientific apps map well onto NVRAMs’ features? – Can NVRAM be used as a solution for future Exascale systems? • Solution – Develop a binary instrumentation tool to investigate memory access patterns related to NVRAM – Study realistic DOE applications (Nek5000, S3D, CAM and GTC) at fine granularity • Impact – Identify large amount of commonly existing data structures that can be placed in NVRAM to save energy – Identify many NVRAM-friendly memory access patterns in DOE applications – Received attention from both vendor and apps teams D. Li, J.S. Vetter, G. Marin, C. McCurdy, C. Cira, Z. Liu, and W. Yu, “Identifying Opportunities for Byte-Addressable Non-Volatile Memory in Extreme-Scale Scientific Applications,” in IEEE International Parallel & Distributed Processing Symposium (IPDPS). Shanghai: IEEEE, 2012 33
• 30.
  Measurement Results 34
• 31.
  Rethinking Algorithm-Based Fault Tolerance •Algorithm-based fault tolerance (ABFT) has many attractive characteristics – Can reduce or even eliminate the expensive periodic checkpoint/rollback – Can bring negligible performance loss when deployed in large scale – No modifications from architecture and system software • However – ABFT is completely opaque to any underlying hardware resilience mechanisms – These hardware resilience mechanisms are also unaware of ABFT – Some data structures are over-protected by ABFT and hardware 35 D. Li, C. Zizhong, W. Panruo, and S. Vetter Jeffrey, “Rethinking Algorithm-Based Fault Tolerance with a Cooperative Software-Hardware Approach,” Proc. International Conference for High Performance Computing, Networking, Storage and Analysis (SC13), 2013, pp. (to appear),
• 32.
  We consider ABFTusing a holistic view from both software and hardware • We investigate how to integrate ABFT and hardware-based ECC for main memory • ECC brings energy, performance and storage overhead • The current ECC mechanisms cannot work – There is a significant semantic gap for error detection and location between ECC protection and ABFT • We propose an explicitly-managed ECC by ABFT – A cooperative software-hardware approach – We propose customization of memory resilience mechanisms based on algorithm requirements. 36
• 33.
  System Designs • Architecture –Enable co-existence of multiple ECC – Introduce a set of ECC registers into the memory controller (MC) – MC is in charge of detecting, locating, and reporting errors • Software – The users control which data structures should be protected by which relaxed ECC scheme by ECC control APIs. – ABFT can simplify its verification phase, because hardware and OS can explicitly locate corrupted data 37
• 34.
  Evaluation • We usefour ABFT (FT-DGEMM, FT-Cholesky, FT-CG and FT-HPL) • We save up to 25% for system energy (and up to 40% for dynamic memory energy) with up to 18% performance improvement 38
• 35.
  Future Directions inNext Generation Memory • Next decade will also be exciting for memory technology • New devices – Flash, ReRam, STTRAM will challenge DRAM – Commercial markets already driving transition • New configurations – 2.5D, 3D stacking removes recent JEDEC constraints – Storage paradigms (e.g., key-value) – Opportunities to rethink memory organization • Logic/memory integration – Move compute to data – Programming models 39 Managed by UT-Battelle for the U.S. Department of Energy • Refactor our applications to make use of this new technology • Add HPC programming support for these new technologies • Explore opportunities for improved resilience, power, performance
• 36.
  Co-designing Future Extreme ScaleSystems
• 37.
  Predictive Performance • Empiricalmeasurement is necessary but we must investigate future applications on future architectures using future software stacks Predictions now for 2020 system Bill Harrod, 2012 August ASCAC Meeting 44
• 38.
  Holistic View ofHPC Performance, Resilience, Power, Programmability Applications • • • • • • • • • • • • Materials Climate Fusion National Security Combustion Nuclear Energy Cybersecurity Biology High Energy Physics Energy Storage Photovoltaics National Competitiveness • Usage Scenarios • Ensembles • UQ • Visualization • Analytics 97 Programming Environment • Domain specific • Libraries • Frameworks • Templates • Domain specific languages • Patterns • Autotuners • Platform specific • Languages • Compilers • Interpreters/Scripting • Performance and Correctness Tools • Source code control System Software • • • • • • • • Resource Allocation Scheduling Security Communication Synchronization Filesystems Instrumentation Virtualization Architectures • Processors • Multicore • Graphics Processors • Vector processors • FPGA • DSP • Memory and Storage • Shared (cc, scratchpad) • Distributed • RAM • Storage Class Memory • Disk • Archival • Interconnects • Infiniband • IBM Torrent • Cray Gemini, Aires • BGL/P/Q • 1/10/100 GigE
• 39.
  Examples of DesignSpace questions • Architectures – – – – – – Concurrency Inst/flop rates Working set capacities Access patterns Message sizes and rates Other user defined resources • SIMD • Elementary functions – Sync • Interrelationships and tradeoffs between application requirements and resources – Eg: If node memory is size X, how large will messages be and how many flops will be executed b/w message ops? 45 • Software questions – – – – – – – – – – Algorithmic choice Memory issues Communication bottlenecks Load imbalance Sync and scheduling Power and energy efficiency Resilience Heterogeneity Domain abstractions Correctness
• 40.
  Prediction Techniques Ranked 47
• 41.
  Prediction Techniques Ranked 48
• 42.
  Aspen – DesignGoals • Abstract Scalable Performance Engineering Notation – Create a deployable, extensible, and highly semantic representation for analytical performance models – Design and implement a new language for analytical performance modeling – Use the language to create machine-independent models for important applications and kernels • Models are composable K. Spafford and J.S. Vetter, “Aspen: A Domain Specific Language for Performance Modeling,” in SC12: ACM/IEEE International Conference for High Performance Computing, Networking, Storage, and Analysis, 2012 49
• 43.
  Aspen – DesignGoals (2) • Develop a suite of analysis tools which operate on the models and produce key performance metrics like – available parallelism, memory capacity, arithmetic intensity, and message volume • Not goals for Aspen – Replace simulators by making highly detailed models – Solve all the problems of analytical modeling • Cache policies • Network contention • Constructed models for important apps and mini-apps: MD, UHPC CP 1, Lulesh, 3D FFT, CoMD, VPFFT, … 50
• 44.
  Aspen Model Walkthrough:3DFFT Pencil v. Slab Decomposition Based on earlier analytical models by 1/ Gahvari and 2/ Czechowski 51
• 45.
  3DFFT 52
• 46.
  3DFFT: Slab vs.Pencil Tradeoff Ideal Parallelism • Insights become obvious with Aspen 53
• 47.
  ProblemSizePlanner • n isapproximately 5000 54
• 48.
  Understanding Future Workloads Flops 1e16 3DFFT 1e14 CoMD 1e12 miniMD 1e10 1e8 1e6 Memory Loads 1e4 1e2 1e0 Messages 55 Problems fixed at 100MB Memory per MPI task Stores
• 49.
  Recent Advances inAspen 56
• 50.
  Probabilistic Execution • Example:Monte Carlo step with 93% acceptance rate probability { [0.93] {execute { flops [20] stores[4] } } } • Example: load imbalance probability { [smallodds] { kernel_small(); } else { kernel_large(); } } • Can calculate expected value analytically • Can simulate random behavior stochastically 57
• 51.
  ASPEN + SST/macroIntegration • Walk through an Aspen model as if it were an executing program • use Aspen to calculate flop/load/store run times • but use network simulator to get more accurate communication, synchronization run times • Aspen calculates runtimes for flops, loads, stores resource usage – emits SSTMAC_compute() • Emits faux MPI calls for message resource usage • Early results: assume perfect load balancing  61 Ideal Simulated 101.0% 100.5% Parallel Efficiency – Generate synthetic input to interconnect simulator – create skeleton code to feed interconnect simulator Aspen+SST/macro weak scaling CoMD @ 262k atoms/task 100.0% 99.5% 99.0% 98.5% 98.0% 1 4 16 64 Number of Nodes 256
• 52.
  Summary • Our communityexpects major challenges in HPC as we move to extreme scale – Power, Performance, Resilience, Productivity – Major shifts and uncertainty in architectures, software, applications – Design of processors, memory systems, interconnects, storage • Technologies particularly pertinent to addressing some of these challenges – Heterogeneous computing – Nonvolatile memory • DOE has initiated Codesign Centers that bring together all stakeholders to develop integrated solutions • Aspen is a new approach to model characteristics of applications and architectures – This structure allows easy development, sharing, verification of models 63
• 53.
  Contributors and RecentSponsors • Future Technologies Group: http://ft.ornl.gov – Publications: https://ft.ornl.gov/publications • Department of Energy Office of Science – – – – – – Vancouver Project: https://ft.ornl.gov/trac/vancouver Blackcomb Project: https://ft.ornl.gov/trac/blackcomb ExMatEx Codesign Center: http://codesign.lanl.gov Cesar Codesign Center: http://cesar.mcs.anl.gov/ SciDAC: SUPER, SDAV http://science.energy.gov/ascr/research/scidac/scidac-institutes/ CS Efforts: http://science.energy.gov/ascr/research/computer-science/ • DOE 'Application' offices • National Science Foundation Keeneland Project: http://keeneland.gatech.edu • NVIDIA CUDA Center of Excellence at Georgia Tech • Other sponsors – ORNL LDRD, NIH, AFRL, DoD – DARPA (HPCS, UHPC, AACE) 64
• 54.
  Q&A More info: [email protected] 65
• 55.
  Recent Publications fromFTG (2012-3) [1] F. Ahmad, S. Lee, M. Thottethodi, and T.N. VijayKumar, “MapReduce with Communication Overlap (MaRCO),” Journal of Parallel and Distributed Computing, 2012, http://dx.doi.org/10.1016/j.jpdc.2012.12.012. [2] C. Chen, Y. Chen, and P.C. Roth, “DOSAS: Mitigating the Resource Contention in Active Storage Systems,” in IEEE Cluster 2012, 2012, 10.1109/cluster.2012.66. [3] A. Danalis, P. Luszczek, J. Dongarra, G. Marin, and J.S. Vetter, “BlackjackBench: Portable Hardware Characterization,” SIGMETRICS Performance Evaluation ReviewSIGMETRICS Performance Evaluation Review, 40, 2012, [4] A. Danalis, C. McCurdy, and J.S. Vetter, “Efficient Quality Threshold Clustering for Parallel Architectures,” in IEEE International Parallel & Distributed Processing Symposium (IPDPS). Shanghai: IEEEE, 2012, http://dx.doi.org/10.1109/IPDPS.2012.99. [5] J.M. Dennis, J. Edwards, K.J. Evans, O. Guba, P.H. Lauritzen, A.A. Mirin, A. St-Cyr, M.A. Taylor, and P.H. Worley, “CAM-SE: A scalable spectral element dynamical core for the Community Atmosphere Model,” International Journal of High Performance Computing Applications, 26:74–89, 2012, 10.1177/1094342011428142. [6] J.M. Dennis, M. Vertenstein, P.H. Worley, A.A. Mirin, A.P. Craig, R. Jacob, and S.A. Mickelson, “Computational Performance of Ultra-High-Resolution Capability in the Community Earth System Model,” International Journal of High Performance Computing Applications, 26:5–16, 2012, 10.1177/1094342012436965. [7] K.J. Evans, A.G. Salinger, P.H. Worley, S.F. Price, W.H. Lipscomb, J. Nichols, J.B.W. III, M. Perego, J. Edwards, M. Vertenstein, and J.-F. Lemieux, “A modern solver framework to manage solution algorithm in the Community Earth System Model,” International Journal of High Performance Computing Applications, 26:54–62, 2012, 10.1177/1094342011435159. [8] S. Lee and R. Eigenmann, “OpenMPC: Extended OpenMP for Efficient Programming and Tuning on GPUs,” International Journal of Computational Science and Engineering, 8(1), 2013, [9] S. Lee and J.S. Vetter, “Early Evaluation of Directive-Based GPU Programming Models for Productive Exascale Computing,” in SC12: ACM/IEEE International Conference for High Performance Computing, Networking, Storage, and Analysis. Salt Lake City, Utah, USA: IEEE press, 2012, http://dl.acm.org/citation.cfm?id=2388996.2389028, http://dx.doi.org/10.1109/SC.2012.51. [10] D. Li, B.R. de Supinski, M. Schulz, D.S. Nikolopoulos, and K.W. Cameron, “Strategies for Energy Efficient Resource Management of Hybrid Programming Models,” IEEE Transaction on Parallel and Distributed SystemsIEEE Transaction on Parallel and Distributed Systems, 2013, http://dl.acm.org/citation.cfm?id=2420628.2420808, [11] D. Li, D.S. Nikolopoulos, and K.W. Cameron, “Modeling and Algorithms for Scalable and Energy Efficient Execution on Multicore Systems,” in Scalable Computing: Theory and Practice, U.K. Samee, W. Lizhe et al., Eds.: Wiley & Sons, 2012, [12] D. Li, D.S. Nikolopoulos, K.W. Cameron, B.R. de Supinski, E.A. Leon, and C.-Y. Su, “Model-Based, Memory-Centric Performance and Power Optimization on NUMA Multiprocessors,” in International Symposium on Workload Characterization. San Diego, 2012, http://www.computer.org/csdl/proceedings/iiswc/2012/4531/00/06402921-abs.html [13] D. Li, J.S. Vetter, G. Marin, C. McCurdy, C. Cira, Z. Liu, and W. Yu, “Identifying Opportunities for Byte-Addressable Non-Volatile Memory in ExtremeScale Scientific Applications,” in IEEE International Parallel & Distributed Processing Symposium (IPDPS). Shanghai: IEEEE, 2012, http://dl.acm.org/citation.cfm?id=2358563, http://dx.doi.org/10.1109/IPDPS.2012.89. 77
• 56.
  Recent Publications fromFTG (2012-3) [14] D. Li, J.S. Vetter, and W. Yu, “Classifying Soft Error Vulnerabilities in Extreme-Scale Scientific Applications Using a Binary Instrumentation Tool,” in SC12: ACM/IEEE International Conference for High Performance Computing, Networking, Storage, and Analysis. Salt Lake City, 2012, http://dl.acm.org/citation.cfm?id=2388996.2389074, http://dx.doi.org/10.1109/SC.2012.29. [15] Z. Liu, B. Wang, P. Carpenter, D. Li, J.S. Vetter, and W. Yu, “PCM-Based Durable Write Cache for Fast Disk I/O,” in IEEE International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS). Arlington, Virginia, 2012, http://www.computer.org/csdl/proceedings/mascots/2012/4793/00/4793a451abs.html [16] G. Marin, C. McCurdy, and J.S. Vetter, “Diagnosis and Optimization of Application Prefetching Performance,” in ACM International Conference on Supercomputing (ICS). Euguene, OR: ACM, 2013 [17] J.S. Meredith, S. Ahern, D. Pugmire, and R. Sisneros, “EAVL: The Extreme-scale Analysis and Visualization Library,” in Proceedings of the Eurographics Symposium on Parallel Graphics and Visualization (EGPGV), 2012 [18] J.S. Meredith, R. Sisneros, D. Pugmire, and S. Ahern, “A Distributed Data-Parallel Framework for Analysis and Visualization Algorithm Development,” in Proceedings of the 5th Annual Workshop on General Purpose Processing with Graphics Processing Units. New York, NY, USA: ACM, 2012, pp. 11–9, http://doi.acm.org/10.1145/2159430.2159432, 10.1145/2159430.2159432. [19] A.A. Mirin and P.H. Worley, “Improving the Performance Scalability of the Community Atmosphere Model,” International Journal of High Performance Computing Applications, 26:17–30, 2012, 10.1177/1094342011412630. [20] P.C. Roth, “The Effect of Emerging Architectures on Data Science (and other thoughts),” in 2012 CScADS Workshop on Scientific Data and Analytics for Extreme-scale Computing. Snowbird, UT, 2012, http://cscads.rice.edu/workshops/summer-2012/data-analytics [21] K. Spafford, J.S. Meredith, S. Lee, D. Li, P.C. Roth, and J.S. Vetter, “The Tradeoffs of Fused Memory Hierarchies in Heterogeneous Architectures,” in ACM Computing Frontiers (CF). Cagliari, Italy: ACM, 2012, http://dl.acm.org/citation.cfm?id=2212924, http://dx.doi.org/10.1145/2212908.2212924. [22] K. Spafford and J.S. Vetter, “Aspen: A Domain Specific Language for Performance Modeling,” in SC12: ACM/IEEE International Conference for High Performance Computing, Networking, Storage, and Analysis, 2012, http://dl.acm.org/citation.cfm?id=2388996.2389110, http://dx.doi.org/10.1109/SC.2012.20. [23] C.-Y. Su, D. Li, D.S. Nikolopoulos, M. Grove, K.W. Cameron, and B.R. de Supinski, “Critical Path-Based Thread Placement for NUMA Systems,” ACM SIGMETRICS Performance Evaluation ReviewACM SIGMETRICS Performance Evaluation Review, 40, 2012, http://dl.acm.org/citation.cfm?id=2381056.2381079, [24] V. Tipparaju and J.S. Vetter, “GA-GPU: Extending a Library-based Global Address Space Programming Model for Scalable Heterogeneous Computing Systems,” in ACM Computing Frontiers (CF), 2012, http://dx.doi.org/10.1145/2212908.2212918. [25] J.S. Vetter, Contemporary High Performance Computing: From Petascale Toward Exascale, vol. 1, 1 ed. Boca Raton: Taylor and Francis, 2013, http://j.mp/RrBdPZ, [26] J.S. Vetter, R. Glassbrook, K. Schwan, S. Yalamanchili, M. Horton, A. Gavrilovska, M. Slawinska, J. Dongarra, J. Meredith, P.C. Roth, K. Spafford, S. Tomov, and J. Wynkoop, “Keeneland: Computational Science using Heterogeneous GPU Computing,” in Contemporary High Performance Computing: From Petascale Toward Exascale, vol. 1, CRC Computational Science Series, J.S. Vetter, Ed., 1 ed. Boca Raton: Taylor and Francis, 2013, pp. 900, 78 [27] W. Yu, X. Que, V. Tipparaju, and J.S. Vetter, “HiCOO: Hierarchical cooperation for scalable communication in Global Address Space programming models on Cray XT systems,” Journal of Parallel and Distributed ComputingJournal of Parallel and Distributed Computing, 2012,