An Introduction to Apache Geode (incubating)

2© Copyright 2015 Pivotal. All rights reserved. 2© Copyright 2015 Pivotal. All rights reserved.
Open Sourcing
GemFire
An Introduction to Apache Geode (incubating)
The open source distributed in-memory database
Anthony Baker
abaker@apache.org
@metatype
Dan Smith
dsmith@pivotal.io

3© Copyright 2015 Pivotal. All rights reserved.
Topics
 GemFire history, architecture, and use cases
 Geode, the open source core of GemFire
 Example application and demo
 Technical deep dive

It’s all about the
DATAand how you use it

2004 2008 2014
• Massive increase in data
volumes
• Falling margins per
transaction
• Increasing cost of IT
maintenance
• Need for elasticity in
systems
• Financial Services
Providers (every major
Wall Street bank)
• Department of Defense
• Real Time response needs
• Time to market constraints
• Need for flexible data
models across enterprise
• Distributed development
• Persistence + In-memory
• Global data visibility needs
• Fast Ingest needs for data
• Need to allow devices to
hook into enterprise data
• Always on
• Largest travel Portal
• Airlines
• Trade clearing
• Online gambling
• Largest Telcos
• Large mfrers
• Largest Payroll processor
• Auto insurance giants
• Largest rail systems on
earth
Hybrid Transactional
/Analytics grids
Our GemFire Journey Over The Years

Apps at Scale Have Unique Needs
Pivotal GemFire is the distributed, in-memory database for
apps that need:
1. Elastic scale-out performance
2. High performance database capabilities in distributed systems
3. Mission critical availability and resiliency
4. Flexibility for developers to create unique applications
5. Easy administration of distributed data grids

1. Elastic Scale-Out Performance
Nodes
Ops /
SecLinear scalability
Elastic capacity +/-
Latency-minimizing
data distribution
China Railway
Corporation
“The system is operating with solid
performance and uptime. Now, we have a
reliable, economically sound production
system that supports record volumes and
has room to grow”
Dr. Jiansheng Zhu, Vice Director of China
Academy of Railway Sciences
• 4.5 million ticket purchases & 20
million users per day.
• Spikes of 15,000 tickets sold per
minute, 40,000 visits per second.

2. High Performance Database Capabilities
Performance-
optimized
persistence
Configurable
consistency Partitioned Replicated Disabled
Distributed &
continuous queries
Distributed transactions, indexing, archival
Newedge
“Our global deployment of Pivotal
GemFire’s distributed cache gives me a
single version of the trade – resolving
hard-to-test-for synchronization issues
that exist within any globally distributed
business application architecture”
Michael Benillouche, Global Head of Data
Management

3. Mission Critical Availability and Resiliency
Cluster to cluster
WAN connectivity
Cluster resilience
& failover
XX Gire
“We can track and collect money at our
4,000+ kiosks and branches – even without
a reliable Internet connection. GemFire
provides the core data grid and a significant
amount of related functionality to help us
handle this unreliable network problem”
Gustavo Valdez, Chief of Architecture and
Development
• 19 million payment transactions per
month
• 4000+ points of sale with intermittent
Internet connectivity

4. Flexibility for Developers to Create Unique
Apps
• Data Structures:
– User-defined classes
– Documents (JSON)
• Native language support:
– Java, C++, C#
• API’s
– Java: Hashmap
– Memcache
– Spring Data GemFire
– C++ Serialization API’s
• Embedded query authoring
– Object Query Language (OQL)
• Publish & subscribe framework for
continous query & reliable
asynchronous event queues
• App-server embedded functionality:
– Web app session state caching
– L2 Hibernate

5. Easy Administration of Distributed Data
Grids
• Auto tuning of distributed computing resources to optimize
performance
• Cluster monitoring dashboard
– Cluster and node status & performance
• Offline performance statistics analysis tool
– View historical logs and events to diagnose performance and resource bottlenecks
• Command-line tool for taking action on clusters and nodes

What makes it fast?
 Design center is RAM not HDD
 Really demanding customers
 Avoid and minimize, particularly on the critical read/write paths:
– Network hops, copying data, contention, distributed locking, disk seeks, garbage
 Lots and lots of testing
– Establish and monitor performance baselines
– Distributed systems testing is difficult!

Horizontal Scaling for GemFire (Geode) Reads
With Consistent Latency and CPU
• Scaled from 256 clients and 2 servers to 1280 clients and 10 servers
• Partitioned region with redundancy and 1K data size
0
2
4
6
8
10
12
14
16
18
0
1
2
3
4
5
6
2 4 6 8 10
Speedup
Server Hosts
speedup
latency (ms)
CPU %

GemFire (Geode) 3.5-4.5X Faster Than Cassandra
for YCSB

Roadmap
 HDFS persistence
 Off-heap storage
 Lucene indexes
 Spark integration
 Cloud Foundry service
…and other ideas from the Geode community!

Use cases and patterns

“Low touch” Usage Patterns
Simple template for TCServer, TC, App servers
Shared nothing persistence, Global session state
HTTP Session management
Set Cache in hibernate.cfg.xml
Support for query and entity caching
Hibernate L2 Cache plugin
Servers understand the memcached wire protocol
Use any memcached client
Memcached protocol
<bean id="cacheManager"
class="org.springframework.data.gemfire.support.GemfireCacheManager"Spring Cache Abstraction

Application Patterns
 Caching for speed and scale
– Read-through, Write-through, Write-behind
 Geode as the OLTP system of record
– Data in-memory for low latency, on disk for durability
 Parallel compute engine
 Real-time analytics

Development Patterns
 Configure the cluster programmatically or declaratively
using cache.xml, gfsh, or SpringDataGemFire
<cache>
<region name="turbineSensorData" refid="PARTITION_PERSISTENT">
<partition-attributes redundant-copies="1" total-num-buckets="43"/>
</region>
</cache>

 Write key-value data into a Region using { create | put |
putAll | remove }
– The value can be flat data, nested objects, JSON, …
Region sensorData = cache.getRegion("TurbineSensorData");
SensorKey key = new SensorKey(31415926, "2013-05-19T19:22Z"); // turbineId, timestamp
sensorData.put(key, new TurbineReading()
.setAmbientTemp(75)
.setOperatingTemp(80)
.setWindDirection(0)
.setWindSpeed(30)
.setPowerOutput(5000)
.setRPM(5));

 Read values from a Region by key using { get | getAll }
TurbineReading data = sensorData.get(new SensorKey(31415926,"2013-05-
19T19:22Z"));
 Query values using OQL
// finds all sensor readings for the given turbine
SELECT * from /TurbineSensorData.entrySet WHERE key.turbineId = 31415926
// finds all sensor readings where the operating temp exceeds a threshold
SELECT * from /TurbineSensorData.entrySet WHERE value.operatingTemp > 120
 Apply indexes to optimize queries

 Execute functions to operate on local data in parallel
 Respond to updates using CacheListeners and Events
 Automatic redundancy, partitioning, distribution, consistency,
network partition detection & recovery, load balancing, …

Apache Geode
(incubating)
The open source core of GemFire

Why OSS? Why Now? Why Apache?
 Open Source Software is fundamentally changing buying patterns
– Developers have to endorse product selection (No longer CIO handshake)
– Community endorsement is key to product visibility
– Open source credentials attract the best developers
– Vendor credibility directly tied to street credibility of product
 Align with the tides of history
– Customers increasingly asking to participate in product development
– Allow product development to happen with full transparency
 Apache is where you go to build Open Source street cred
– Transparent, meritocracy which puts developers in charge
– Roman keeps shouting “Apache!” every few hours

Geode Will Be A Significant Apache Project
 Over a 1000 person years invested into cutting edge R&D
 Thousands of production customers in very demanding verticals
 Cutting edge use cases that have shaped product thinking
 Tens of thousands of distributed , scaled up tests that can randomize
every aspect of the product
 A core technology team that has stayed together since founding
 Performance differentiators that are baked into every aspect of the
product

Geode or GemFire?
 Geode is a project, GemFire is a product
 We donated everything but the kitchen sink*
 More code drops imminent; going forward all development
happens OSS-style (“The Apache Way”)
* Multi-site WAN replication, continuous queries, and native (C/C++) client driver

X
Apache

Geode Demo

Post Region
Partitioned
People Region
Partitioned
Social Network
Person
Name: String
Description:String
Post
Id: PostId(name, date)
Text: String

Partition put
Client
Server 1
Server 2
Server 3
Bucket 1
Bucket 1
Bucket 2
Bucket 2
#(LOL)=1
Put LOL

Partition put
Client
Server 1
Server 2
Server 3
LOL
LOL
Bucket 2
Bucket 2
Replicate
To Secondary

“User” Use Case – Save Objects
public interface PersonRepository extends CrudRepository<Person, String> {
}
@Autowired
PersonRepository people;
public static void main(String[] args) {
people.save(new Person(name));
posts.save(new Post(new PostId(name, date), text));
}
Nested Objects,
Compound Keys

“User” Use Case – Save Objects
public interface PersonRepository extends CrudRepository<Person, String> {
}
@Autowired
PersonRepository people;
public static void main(String[] args) {
people.save(new Person(name));
posts.save(new Post(new PostId(name, date), text));
}
Automatically Serialized
With PDX

Configuration
<gfe:partitioned-region id="people" copies="1"/>
<bean id="pdxSerializer”
class="com.gemstone.gemfire.pdx.ReflectionBasedAutoSerializer">
<constructor-arg value="io.pivotal.happysocial.model.*"/>
</bean>
<gfe:cache pdx-serializer-ref="pdxSerializer"/>

Data Analyst – Determine Sentiment
 Find all of the posts for a user
 Analyze their content

First try – Just use a Query
public interface PostRepository extends
GemfireRepository<Post, PostId> {
@Query("select * from /posts where id.person=$1")
public Collection<Post> findPosts(String personName);
}
Collection<Post> posts = postRepository.findPosts(personName);
String sentiment = sentimentAnalyzer.analyze(posts);

First try – Just use a Query
public interface PostRepository extends
GemfireRepository<Post, PostId> {
@Query("select * from /posts where id.person=$1")
public Collection<Post> findPosts(String personName);
}
Collection<Post> posts = postRepository.findPosts(personName);
Query Nested Objects

Use an Index
<gfe:index
id="postAuthor"
expression="id.person"
from="/posts”
/>

Still could be more efficient
Client
Server 1
Server 2
Server 3
Joe: LOL!!
Joe: LOL!!
EJ: arrg
Maya: Hii
Jess: sup
Jess: ok
Hitting multiple
Nodes
Bringing too much
Data to the client

Colocate the data
Client
Server 1
Server 2
Server 3
Joe: LOL!! Joe: LOL!!
EJ: arrgMaya: Hii
Jess: sup Jess: ok
<gfe:partitioned-region id="posts" copies="1"
colocated-with="people”>
<gfe:partition-resolver ref="partitionResolver"/>
</gfe:partitioned-region>

Send behavior to data
Client
Server 1
Server 2
Server 3
Joe: LOL!! Joe: LOL!!
EJ: arrgMaya: Hii
Jess: sup Jess: ok
Execution function
getSentiment
On Joe, Jess
Execute on Joe
Execute on Jess

Sample Function – Client Side
@Component
@OnRegion(region = "posts")
public interface FunctionClient {
public List<SentimentResult> getSentiment(@Filter Set<String> people);
}

Sample Function – Server Side
@GemfireFunction(HA=true)
public SentimentResult getSentiment(Region<PostId, Post> localPosts,
@Filter Set<String> personNames)
throws Exception {
String personName = personNames.iterator().next();
Collection<Post> posts = localPosts.query("id.person='" personName + "'");
return new SentimentResult(sentiment, personName);
}

throws Exception {
}

Demo

Design
• Persistence
• PDX Serialization

Persistence – Design Goals
 High availability
 Low latency, high throughput
 Durability
 Minimal impact to in memory speed
– Minimize IO operations
– Minimize locking
– Minimize deserialization

Persistence – Shared Nothing
Server 3Server 2
Bucket 3
*primary*
Bucket 1
secondary
Bucket 3
secondary
Bucket 2
*primary*
Server 1
Bucket 2
secondary
Bucket 1
*primary*

Server 1 Server 3Server 2
Bucket 2
secondary
Bucket 1
*primary*
Bucket 3
*primary*
Bucket 1
secondary
Bucket 3
secondary
Bucket 2
*primary*
What if a server
crashes?

Bucket 2
secondary
Bucket 1
*primary*
Bucket 3
*primary*
Bucket 1
*primary*
Bucket 3
secondary
Bucket 2
*primary*
Bucket 2
secondary
Primary
Failover
Restore
Redundancy
Bucket 1
secondary

Bucket 2
secondary
Bucket 1
*primary*
Bucket 3
*primary*
Bucket 1
*primary*
Bucket 3
secondary
Bucket 2
*primary*
Bucket 2
secondary
Bucket 1
secondary
What if they all crash?

Server 3Server 2
Bucket 3
*primary*
Bucket 1
*primary*
Bucket 3
secondary
Bucket 2
*primary*
Bucket 2
secondary
Bucket 1
secondary
Server 1
Bucket 2
secondary
Bucket 1
*primary*
Compare Persistent View
On Restart

Server 1
Bucket 2
secondary
Bucket 1
*primary*
Server 3Server 2
Bucket 3
*primary*
Bucket 1
secondary
Bucket 3
secondary
Bucket 2
*primary*
Bucket 2
secondary
Bucket 1
secondary
Recover Latest
Data &
Rebalance

Modify
k1->v5
Create
k6->v6
Create
k1->v1
Create
k2->v2
Modify
k1->v3
Create
k4->v4
Modify
k1->v5
Create
k6->v6
Member
1
Persistence - Operation Logs
Put k6->v6
Oplog2.crf
Oplog1.crf
Append to
operation log

Modify
k1->v5
Create
k6->v6
Create
k1->v1
Modify
k1->v3
Member
1
Persistence - Compaction
Oplog2.crf
Oplog1.crf
Create
k2->v2
Create
k4->v4
Copy live
data forward

Modify
k1->v5
Create
k6->v6
Persistence - Compaction
Create
k2->v2
Create
k4->v4
Oplog2.crf
Member
1
Modify
k4->v7Oplog3.crf
Put k4->v7

Design
• Persistence

Fixed or flexible schema?
id name age pet_id
{
id : 1,
name : “Fred”,
age : 42,
pet : {
name : “Barney”,
type : “dino”
}
}
OR

But how to serialize data?
http://blog.pivotal.io/pivotal/products/data-serialization-how-to-run-multiple-big-data-apps-at-once-with-gemfire

Efficient for queries
SELECT p.name from /Person p WHERE p.pet.type = “dino”
{
id : 1,
name : “Fred”,
age : 42,
pet : {
name : “Barney”,
type : “dino”
}
}
single field
deserialization

Easy to use
 Access from Java, C#, C++, JSON
 Domain objects not required
 Automatic type definition
– No IDL compiler or schema required
– No hand-coded read/write methods

Distributed type registry
Member A Member B
Distributed Type Definitions
Person p1 = …
region.put(“Fred”, p1);

Member A Member B
Person p1 = …
automatic
definition

Member A Member B
Person p1 = …
automatic
definition
replicate serialized data
containing typeId

Schema evolution
PDX provides forwards and backwards
compatibility, no code required
Member A Member B
v
1
v
2
Application
#2
Application
#1
v2 objects preserve data
from missing fields
v1 objects use default values
to fill in new fields

Questions?
 http://geode.incubator.apache.org
 dev@geode.incubator.apache.org
 user@geode.incubator.apache.org
 http://github.com/apache/incubator-geode

Bonus Content

Design
• Persistence
• Asynchronous Events
• Client Subscriptions

Asynchronous Events – Design Goals
 High availability
 Low latency, high throughput
 Deliver events to a receiver without impacting the write path

Member 3
Member 1
Serial Queues
Member 2
LOL!!put
Primary Queue
Secondary Queue
Enqueue
sup
LOL!!
sup
sup
LOL!!
Replicate

Member 3
Member 1
Member 2
Serial Queues
sup
LOL!!
sup
LOL!!
AsyncEventListener
{
processBatch()
}
LOL!!
sup
Dispatch Events
from Primary

put
Secondary Queue
(Partition 1)
LOL!!
sup
sup
LOL!!
Primary Queue
(Partition 2)
Parallel Queues
Primary Queue
(Partition 1)
LOL!!
sup
sup
LOL!!
Word
Word

LOL!!
sup
sup
LOL!!
Parallel Queues
LOL!!
sup
sup
LOL!!
Word
Word
AsyncEventListener
{
processBatch()
}
AsyncEventListener
{
processBatch()
}
Parallel
Dispatch

Design
• Persistence
• Asynchronous Events
• Client Subscriptions

“There are only two hard things in Computer Science: cache
invalidation, naming things, and off-by-one errors.”
– the internet

Client driver
 Intelligent – understands data distribution for single hop
network access
 Caching – can be configured to locally cache data for even
faster access
 Events – registerInterest() in keys to receive push
notifications

Client subscriptions
 Useful for refreshing client cache, pushing events (“topics”)
to multiple clients
 Highly available & scalable via in-memory replicated queues
 Events are ordered, at-least-once delivery
 Durable subscriptions, conflation optional

Member A Member B
region.registerInterest(“Fred”);
Client
Client

Member A Member B
Person p1 = …
Client
Client

An Introduction to Apache Geode (incubating)

More Related Content

What's hot(20)

Viewers also liked(16)

Similar to An Introduction to Apache Geode (incubating)(20)

Recently uploaded(20)

An Introduction to Apache Geode (incubating)