“Building consistent and highly available distributed systems with Apache Ignite and GridGain.”
1 like153 views
The document discusses the capabilities of the GridGain and Apache Ignite in-memory computing platforms, emphasizing their durability, strong consistency, and high availability. It covers topics such as cache backups, persistence, data center replication, and disaster recovery solutions. Additionally, it addresses the CAP theorem and outlines features like dynamic scaling and external storage options.
“Building consistent and highly available distributed systems with Apache Ignite and GridGain.”
- 1. © 2018 GridGain Systems, Inc. Building Consistent and Highly Available Distributed Systems with Apache Ignite and GridGain Akmal Chaudhri Technology Evangelist GridGain
- 2. © 2018 GridGain Systems, Inc. • What is Ignite/GridGain? • Ignite/GridGain and CAP Theorem • High Availability in Ignite/GridGain • Cache Backups • Persistence • Data Center Replication • Data Snapshots (+Demo) • Q&A Agenda
- 3. © 2018 GridGain Systems, Inc. the in-memory computing platform that is durable, strongly consistent and highly available with powerful SQL, key-value and processing APIs
- 4. © 2018 GridGain Systems, Inc. GridGain In-Memory Computing Platform Security&Auditing Monitoring&Management DataSnapshots&Recovery Memory-Centric Storage Ignite Native Persistence (Flash, SSD, Intel 3D XPoint) Third-Party Persistence (RDBMS, HDFS, NoSQL) SQL Transactions Compute Services MLStreamingKey/Value IoTFinancial Services Pharma & Healthcare E-CommerceTravel & Logistics Telco DataCenterReplication
- 5. © 2018 GridGain Systems, Inc. Data Grid JCache Transactions Compute SQL RDBMS NoSQL HDFS Server Node Distributed Key-Value Store Dynamic Scaling Distributed partitioned hash map ACID TransactionJCache & SQL Server Node Server Node 3rd party storage caching DURABLE MEMORY DURABLE MEMORY DURABLE MEMORY
- 6. © 2018 GridGain Systems, Inc. Distributed SQL JDBC ODBC SQL API Java .NET C++ BI SELECT, UPDATE, INSERT, MERGE, DELETE, CREATE and ALTER DDL, DML Support Cross-platform Compatibility Indexes in RAM or Disk Dynamic Scaling Server Node Server NodeServer Node Apache Ignite Cluster DURABLE MEMORY DURABLE MEMORY DURABLE MEMORY Tools
- 7. © 2018 GridGain Systems, Inc. GridGain and CAP Theorem • CAP • Consistency (C) • Availability (A) • Partition Tolerance (P) • Impossible: CAP • Most of distributed systems are CP or AP • GridGain • Strongly CP • But still highly available?
- 8. © 2018 GridGain Systems, Inc. Cache Backups Ignite Node A C D B Ignite Node B A C D Ignite Node C D B A Ignite Node D B A C
- 9. © 2018 GridGain Systems, Inc. Rack Safety: Problem Ignite Node Switch Ignite Node Ignite Node Switch Ignite Node A C B A C D D B Partition A is lost! Client
- 10. © 2018 GridGain Systems, Inc. Rack Safety: Solution • Assign attribute to each node: • Implement affinity backup filter:
- 11. © 2018 GridGain Systems, Inc. Rack Safety: Solved! Ignite Node Switch Ignite Node Ignite Node Switch Ignite Node A D B C C A D B No partitions lost Client
- 12. © 2018 GridGain Systems, Inc. Persistence Ignite NodeIgnite NodeIgnite Node Ignite Cluster RDBMS NoSQL HDFS
- 13. © 2018 GridGain Systems, Inc. Durable Memory Off-heap Removes noticeable GC pauses Automatic Defragmentation Stores Superset of Data Predictable memory consumption Fully Transactional (Write-Ahead Log) DURABLE MEMORY DURABLE MEMORY DURABLE MEMORY Server Node Server Node Server Node Ignite Cluster Instantaneous Restarts
- 14. © 2018 GridGain Systems, Inc. Ignite Native Persistence 1. Update RAM 2. Persist Write-Ahead Log Partition File 1 3. Ack 4. Checkpointing Partition File N Server Node
- 15. © 2018 GridGain Systems, Inc. GridGain Data Center Replication Active-Passive Replication for Disaster Recovery Split-Brain Resolution Active-Active Bi-Directional Replication DC1 DC2 DC3 DC5 DC4
- 16. © 2018 GridGain Systems, Inc. Data Snapshots & Recovery External Store Backup Restore Full & Incremental Backups Managed Backup Schedules GridGain Cluster DURABLE MEMORY DURABLE MEMORY DURABLE MEMORY Server Node Server Node Server Node Data & Indexes
- 17. © 2018 GridGain Systems, Inc. Any Questions? Thank you for joining us. Follow the conversation. http://ignite.apache.org @apacheignite @akmalchaudhri