Grid Technologies in Disaster Management

Grid Technologies in Disaster Management

• 1.
  Grid Technologies in Disaster Management & Infrastructure Protection Geoffrey Fox Computer Science, Informatics, Physics Pervasive Technology Laboratories Indiana University Bloomington IN 47404 [email_address] http://grids.ucs.indiana.edu/ptliupages/presentations/CIGridGDINmar28-04.ppt http:// www.infomall.org http://www.grid2002.org
• 2.
  Introduction Grid isNOT a mesh in this talk!!!! The Grid represents “Internet Scale Distributed Computing” or the “Managed Internet” Major commercial and research Initiative – IBM HP Oracle Sun SGI Platform We describe how one can use familiar system of systems (Grid of Grids) language for critical infrastructure and emergency response (command and control) Grids We give infrastructure examples from flood and earthquake cases with HPC simulations linked to other Grid resources Discuss GIS and Collaboration Grids
• 3.
  e-Business e-Science andthe Grid e-Business captures an emerging view of corporations as dynamic virtual organizations linking employees, customers and stakeholders across the world. e-Science is the similar vision for scientific research with international participation in large accelerators, satellites or distributed gene analyses. The Grid or CyberInfrastructure integrates the best of the Web, Agents, traditional enterprise software, high performance computing and Peer-to-peer systems to provide the information technology e-infrastructure for e-moreorlessanything . A deluge of data of unprecedented and inevitable size must be managed and understood. People , computers , data and instruments must be linked. On demand assignment of HPC resources, experts, computers, networks and storage resources
• 4.
  e-Defense and e-Crisis Grids support Command and Control and provide Global Situational Awareness Link commanders and frontline troops to themselves and to archival and real-time data; link to what-if simulations Dynamic heterogeneous wired and wireless networks Security and fault tolerance essential System of Systems; Grid of Grids The command and information infrastructure of each ship is a Grid; each fleet is linked together by a Grid; the President is informed by and informs the national defense Grid Crisis Management and Response enabled by a Grid linking sensors, disaster managers, and first responders with decision support Grid supports On-demand HPC Simulations Define and Build DoD relevant Services – Collaboration, Sensors, GIS, Database etc.
• 5.
  Raw (HPC) Resources Middleware Portal Services System Services System Services System Services Application Service Libraries User Services “ Core” Grid Typical Grid Architecture Application Service Application Service Re-use Re-use Application Customization Each service should be able to run independently on separate machines Service Oriented Architectures scale better than traditional Distributed objects Database
• 6.
  A typical Webor Grid Service In principle, services can be in any language (Fortran .. Java .. Perl .. Python) and the interfaces can be method calls, Java RMI Messages, CGI Web invocations, totally compiled away (inlining) The simplest implementations involve XML messages (SOAP) and programs written in net friendly languages like Java and Python Payment Credit Card Warehouse Shipping control WSDL interfaces WSDL interfaces Web Services Web Services Security Catalog Portal Service
• 7.
  Composing Systems Weare familiar with a hierarchy Lines of Code  Methods  Objects  Programs  Packages Grids extend the software hierarchy to larger distributed systems and link it to a corresponding resource hierarchy Overlay and Compose Grids of Grids Methods Services Functional Grids CPUs Clusters Compute Resource Grids MPPs Databases Federated Databases Sensor Sensor Nets Data Resource Grids
• 8.
  Information Grid EnterpriseGrid Compute Grid Campus Grid Teacher Students Dynamic light-weight Peer-to-peer Collaborative Training Grid Composing an Overlay Grid R2 R1
• 9.
  EducationGrid Inservice Teachers Preservice Teachers School of Education Teacher Educator Grids Informal Education (Museum) Grid Student/Parent … Community Grid Science Grids CI Grids Typical Science Grid Service such as Research Database or simulation Transformed by Grid Filter to form suitable for education Learning Management or LMS Grid Publisher Grid Campus or Enterprise Administrative Grid Education as a Grid of Grids Digital Library Grid
• 10.
  Solid Earth ResearchVirtual Observatory (SERVO) NASA funded: JPL, UC Davis, UC Irvine, USC, Brown, Indiana Web/Grid-services and portlet based Problem Solving Environment (PSE) Couples data with simulation, pattern recognition software, and visualization Enable investigators to seamlessly merge multiple data sets and models, and create new queries. Data Spaced-based observational data Ground-based sensor data (GPS, seismicity) Simulation data Published/historical fault measurements Analysis Software Earthquake fault Lithospheric modeling Pattern recognition software International Version iSERVO Australia China and Japan as an APEC (Asia-Pacific Economic Cooperation) Initiative – next meeting Beijing July 2004
• 11.
  Database Database ResearchSimulations Analysis and Visualization Portal Repositories Federated Databases Data Filter Services Crisis Data Streaming Data Sensors SERVOGrid Research Crisis Customization Services From Research to CI/Crisis Cell Phone Mobile Network Geoscience Research and Crisis Grids Other Interdependent CI and Crisis Grids ? Discovery Services GIS
• 12.
  Database Database Analysisand Visualization Portal Repositories Federated Databases Data Filter Services Streaming Data Sensors SERVOGrid Research Education Customization Services From Research to Education Education Grid Computer Farm Geoscience Research and Education Grids Field Trip Data ? Discovery Services Research Simulations GIS
• 13.
  HPC Simulation DataFilter Distributed Filters massage data For simulation Other Grid and Web Services Analysis Control Visualize SERVOGrid (Complexity) Computing Model Grid OGSA-DAI Grid Services This Type of Grid integrates with Parallel computing Multiple HPC facilities but only use one at a time Many simultaneous data sources and sinks Grid Data Assimilation Data Filter Data Filter Data Filter Data Filter
• 14.
  Crisis and CIGrids We first form Critical Infrastructure CI Grids from the “atomic” functional and resource Grids and Services This gives water (flood), gas, postal, agricultural etc. Grids Then we customize, compose and overlay CI with other Grids (such as weather, census data) for Public Health Emergency Response (Command and Control) or Crisis Grids Infrastructure Planning (e.g. transportation planning) Education (schools) and Training (of managers and first responders) Interdependency analyses between the different infrastructures Note need to link to “Cell Phone Grid” for first responders with services that reduce image sizes etc.
• 15.
  What are wetrying to do Center for Disaster and Critical Infrastructure Informatics IUPUI, Indiana, Purdue, openGIS Consortium project to build a generic Crisis/CI Grid infrastructure with a FloodGrid as initial prototype SERVOGrid (Solid Earth Research Virtual Observatory) building Grid to support Earthquake Forecasting GlobalMMCS is an open-source Audio-Video conferencing and collaboration system built totally on Web Services and spanning Access Grid, H323, SIP and WebCAM’s. NaradaBrokering is a Grid message system supporting reliable message delivery, software multicast, firewall friendliness, security. (Grid as managed web) Hope to partner with NISAC (National Infrastructure Simulation and Analysis Center) on core CIGrids
• 16.
  NaradaBrokering Queues WebService A Web Service B Stream Reliable Routed software multi-cast Secure filtered Message delivery Minicomputer Firewall Computer Server PDA Modem Laptop computer Workstation Peers Peers Audio/Video Conferencing Client Audio/Video Conferencing Client NaradaBrokering Broker Network
• 17.
  CI/Crisis Grid ofGrids Services in a CI or Crisis Grid fall into three classes 1) Those that special to Critical Infrastructure or Crises such as biological sensor nets or evacuation planning service 2) Those that are important but can be taken from other Grids such as GIS or collaboration and security 3) Those that come from other Grids and are refactored for crises The simulation is reduced in size The public health database interface is simplified CI/Crisis Grid Grid Resource Filter CI/Crisis Grid View of generic Grid Resource
• 18.
  Critical Infrastructure CIGridsFlood CIGrid Gas CIGrid … … Gas Services and Filters Security Workflow Messaging Notification Metadata Data Access/Storage Registry Physical Network Compute Sensors Visualization Collaboration openGIS Services Flood Services and Filters Electricity CIGrid
• 19.
  openGIS Grid SemanticsNote GIS (Geographical Information System) Grid at heart of all these Grids Geography Markup Language (GML) is an XML encoding for the specification of the geometry and properties of geographic features. GML utilizes the OpenGIS Abstract Specification geometry model which has been harmonized with the ISO geospatial geometry model. We are building CI specific ontologies in terms of GML to define faults, satellites etc. http://ripvanwinkle.ucs.indiana.edu:4780/examples/download/schema/ Styled Layer Descriptor (SLD) specifies the format of a map-styling language for portraying the output of Web Map Servers, Web Feature Servers and Web Coverage Servers etc. SLD will enable different communities in the Emergency Response area to develop a set of customized portrayal rules that best fit their mission requirements. This becomes the specification of portals to different composite Grids Sensor Markup Language (SensorML) defines the information model for discovering, querying and controlling Web-resident sensors. Observations & Measurements (O&M) defines the information model for observations that are returned from the CrisisGrid sensors.
• 20.
  GIS Grid ServicesI Web Feature Service (WFS) supports the query and discovery of geographic features delivering GML representations of simple geospatial features in response to queries from HTTP clients. WFS can access geographic features including critical infrastructure features, incident locations, and flood-related geographic features including inundation areas, watershed boundaries, and demographic feature. Web Coverage Service (WCS) supports the query and discovery of digital geospatial information such as digital elevation models, imagery, orthophotography, weather coverages (such as predicted rainfall, air pressure, wind speed and direction), and any other space-varying flood-related phenomena. Web Map Service (WMS) uses a SLD portrayal to generate "pictures" of georeferenced feature or coverage data. WMS will provide a means to portray geographic information independent of the underlying data model (WFS or WCS). Coverage Portrayal Service (CPS) defines a standard interface for producing visual pictures from coverage data typically accessed via WCS with a SLD portrayal.
• 21.
  GIS Grid ServicesII Web Terrain Service (WTS) augments WMS with advanced visualization including 3D terrains. Catalog Service - Web Profile (CS-W) is a catalog service that will be built on a general Grid metadata service Sensor Collection Service (SCS) fetches observations from a sensor or group of sensors and will be integrated with research on Grid sensor services Sensor Planning Service (SPS) assists in 'collection feasibility plans' and to process collection requests for a sensor or group of sensors. Web Notification Service (WNS) will be replaced by standard Grid notification service
• 22.
  Grid Portals Portalsneed to provide services for security, customization, layout, rendering Jetspeed and GridSphere are two well known portals Commercial systems like IBM WebSphere similar These Portals are very important as they encourage “ component ” model for user interfaces and so this fits service model so every service can be packaged with its (document fragment) user interface P ortlets are good as they support they mixing and matching of services for a particular composed/overlay Grid to allow this to be easily reflected in user interface New JSR168 portlet Java Standard NSF Middleware Initiative NMI funding collection of open source Grid portlets
• 23.
  The OGCE ComputingGrid Portal Provides Portlets for Management of user proxy certificates Remote file Management via Grid FTP News/Message systems for collaborations Grid Event/Logging service Access to OGSA services Access to directory services Specialized Application Factory access Distributed Workflow Access to Metadata Index tools User searchable index Real Time Collaboration Audio/Video Conferencing Download from open source http://www.ogce.org
• 24.
  Example Capability: FileManagement Grid FTP portlet– Allow User to manage remote file spaces Uses stored proxy for authentication Upload and download files Third party file transfer Request that GridFTP server A send a file to GridFTP server B Does not involve traffic through portal server Portal Server User GridFTP Server A GridFTP Server B GridFTP Service 1 of many Portlets Jetspeed
• 25.
  Watershed Entry Page:Select a County FloodGrid Portal
• 26.
  Watershed Boundaries inBlack Open source GIS MapServer presented as a simple portlet
• 27.
  Data Download PortalThis is where you can download ArcView compatible data files for your watershed. Some of this gets hooked to the runoff model in our connected crisis grid model.
• 28.
  Flood Grid ParallelSimulations Sensors GIS Data Archives provide terrain, elevation, and land use data Grid Workflow Links Data, Sensors, Model
• 29.
  SERVOGrid HPC ApplicationsCodes range from simple “rough estimate” codes to parallel, high performance applications. Disloc : handles multiple arbitrarily dipping dislocations (faults) in an elastic half-space. Simplex : inverts surface geodetic displacements for fault parameters using simulated annealing downhill residual minimization. GeoFEST : Three-dimensional viscoelastic finite element model for calculating nodal displacements and tractions. Allows for realistic fault geometry and characteristics, material properties, and body forces. Virtual California : Program to simulate interactions between vertical strike-slip faults using an elastic layer over a viscoelastic half-space RDAHMM : Time series analysis program based on Hidden Markov Modeling. Produces feature vectors and probabilities for transitioning from one class to another. PARK : Boundary element program to calculate fault slip velocity history based on fault frictional properties; a model for unstable slip on a single earthquake fault. PDPC : Phase Dynamics Probability Change Preprocessors, mesh generators Visualization tools: RIVA , GMT
• 30.
  QuakeSim Portal ShotsSERVOGrid linking HPC, Data and Visualization SAR Data
• 31.
  Run Finley –Simulation from Australia
• 32.
  Global-MMCS 2.0 XGSPMCU We are building an open source protocol independent Web Service “MCU” which will scale to an arbitrary number of users and provide integrated thousands of simultaneous users collaboration services. We will deploy it globally with first release end of May 2004. The function of A/V media server will be distributed using NaradaBrokering architecture. Media Servers mix and convert A/V streams Open XGSP MCU based on the following open source projects openh323 is basis of H323 Gateway NIST SIP stack is basis of SIP Gateway NaradaBrokering is open source messaging from Indiana Java Media Framework basis of Media Servers
• 33.
  XGSP Web ServiceMCU Architecture Gateways convert to uniform XGSP Messaging High Performance (RTP) and XML/SOAP and .. Use Multiple Media servers to scale to many codecs and many versions of audio/video mixing NB Scales as distributed Web Services NaradaBrokering SIP H323 Access Grid Native XGSP Admire Media Servers Filters Session Server XGSP-based Control NaradaBrokering All Messaging
• 34.
  A/V Collaboration SystemsGlobalMMCS Grid federates existing A/V Protocols H323 H.323 is defined as an umbrella standard specifying the components to be used within an H.323-based environment. SIP The Session Initiation Protocol (SIP) defines how to establish, maintain and terminate Internet sessions including multimedia conferences Access Grid enhanced Mbone A/V tools ( VIC, RAT ) Internet 2 network ( Multicast support )
• 35.
  Grid Services inGlobalMMCS I Audio Mixing The audio mixer creates a mixed audio stream from all the audio streams in the session Video Mixing Video mixing makes the unicast users watch the pictures of multiple participants in a meeting through one video stream Video Thumbnail visualize the VS set in the session, embedded into the control panel of each endpoint, which Image grabbers capture video streams and save them as static JPEG files. All the media processing components can be distributed among the pool of the media servers connected to NaradaBrokering infrastructures. This generalizes to a HPC farm of “stream servers” doing image processing etc. Perhaps need 100 node cluster to support 10,000 simultaneous A/V streams
• 36.
  H.323, SIP GatewayServers, A/V Session Server H.323 and SIP gateway transform their protocol specific messages into XGSP signaling messages so that H.323 and SIP A/V endpoints could communicate with the XGSP A/V session server The session server implements session management logics creating/destroying A/V sessions allowing endpoints to join/leave session Allowing users to make audio/video selection, managing A/V application components
• 37.
  0 10 2030 40 50 60 0 200 400 600 800 1000 1200 1400 1600 1800 2000 Delay (Milliseconds) Packet Number Average delays per packet for 50 video-clients NaradaBrokering Avg=2.23 ms, JMF Avg=3.08 ms NaradaBrokering-RTP JMF-RTP
• 38.
  Polycom, Access Gridand RealVideo views of multiple streams using GlobalMMCS A/V Web Service
• 39.
  Unicast AG Portlet
• 40.
  WS DisplayWS Viewer Master WS Display WS Viewer NaradaBrokering Message Bus Collaboration as a WS Set up Session with GlobalMMCS defining participants Shared Output Port Collaboration Other Participants Text Chat Whiteboard GIS Services etc. are AUTOMATICALLY Collaborative if Built as a Grid Services WS Display WS Viewer Application or Content source WSDL Web Service F I U O F I R O
• 41.
  Integration of PDA,Cell phone and Desktop Grid Access