An intro to Neo4j and some use cases (JFokus 2011)

An intro to Neo4j and some use cases (JFokus 2011)

• 1.
  Neo4j - or, why graph dbs are teh AWESOME!!!11 oh, and also... USE CASES!!! #neo4j Emil Eifrem @emileifrem Neo Technology [email protected]
• 2.
  What's the plan? An intro to graph databases and Neo4j Brief review of some use cases Conclusion and Q&A
• 3.
  Graph DBs & Neo4jintro
• 4.
  The Graph DBmodel: representation Core abstractions: name = “Emil” Nodes age = 29 sex = “yes” Relationships between nodes Properties on both 1 2 type = KNOWS time = 4 years 3 type = car vendor = “SAAB” model = “95 Aero”
• 5.
  Example: The Matrix name = “The Architect” name = “Morpheus” rank = “Captain” name = “Thomas Anderson” occupation = “Total badass” 42 age = 29 disclosure = public KNOWS KNOWS CODED_BY KNO 1 7 3 W S KN 13 S OW name = “Cypher” KNOW S last name = “Reagan” name = “Agent Smith” disclosure = secret version = 1.0b age = 3 days age = 6 months language = C++ 2 name = “Trinity”
• 6.
  Code (1): Buildinga node space GraphDatabaseService graphDb = ... // Get factory // Create Thomas 'Neo' Anderson Node mrAnderson = graphDb.createNode(); mrAnderson.setProperty( "name", "Thomas Anderson" ); mrAnderson.setProperty( "age", 29 ); // Create Morpheus Node morpheus = graphDb.createNode(); morpheus.setProperty( "name", "Morpheus" ); morpheus.setProperty( "rank", "Captain" ); morpheus.setProperty( "occupation", "Total bad ass" ); // Create a relationship representing that they know each other mrAnderson.createRelationshipTo( morpheus, RelTypes.KNOWS ); // ...create Trinity, Cypher, Agent Smith, Architect similarly
• 7.
  Code (1): Buildinga node space GraphDatabaseService graphDb = ... // Get factory Transaction tx = graphDb.beginTx(); // Create Thomas 'Neo' Anderson Node mrAnderson = graphDb.createNode(); mrAnderson.setProperty( "name", "Thomas Anderson" ); mrAnderson.setProperty( "age", 29 ); // Create Morpheus Node morpheus = graphDb.createNode(); morpheus.setProperty( "name", "Morpheus" ); morpheus.setProperty( "rank", "Captain" ); morpheus.setProperty( "occupation", "Total bad ass" ); // Create a relationship representing that they know each other mrAnderson.createRelationshipTo( morpheus, RelTypes.KNOWS ); // ...create Trinity, Cypher, Agent Smith, Architect similarly tx.commit();
• 8.
  Code (1b): Defning RelationshipTypes i // In package org.neo4j.graphdb public interface RelationshipType { String name(); } // In package org.yourdomain.yourapp // Example on how to roll dynamic RelationshipTypes class MyDynamicRelType implements RelationshipType { private final String name; MyDynamicRelType( String name ){ this.name = name; } public String name() { return this.name; } } // Example on how to kick it, static-RelationshipType-like enum MyStaticRelTypes implements RelationshipType { KNOWS, WORKS_FOR, }
• 9.
  Whiteboard friendly owns Björn Big Car build drives DayCare
• 10.
  The Graph DBmodel: traversal Traverser framework for high-performance traversing name = “Emil” age = 31 across the node space sex = “yes” 1 2 type = KNOWS time = 4 years 3 type = car vendor = “SAAB” model = “95 Aero”
• 11.
  Example: Mr Anderson’sfriends name = “The Architect” name = “Morpheus” rank = “Captain” name = “Thomas Anderson” occupation = “Total badass” 42 age = 29 disclosure = public KNOWS KNOWS CODED_BY KNO 1 7 3 W S KN 13 S OW name = “Cypher” KNOW S last name = “Reagan” name = “Agent Smith” disclosure = secret version = 1.0b age = 3 days age = 6 months language = C++ 2 name = “Trinity”
• 12.
  Code (2): Traversinga node space // Instantiate a traverser that returns Mr Anderson's friends Traverser friendsTraverser = mrAnderson.traverse( Traverser.Order.BREADTH_FIRST, StopEvaluator.END_OF_GRAPH, ReturnableEvaluator.ALL_BUT_START_NODE, RelTypes.KNOWS, Direction.OUTGOING ); // Traverse the node space and print out the result System.out.println( "Mr Anderson's friends:" ); for ( Node friend : friendsTraverser ) { System.out.printf( "At depth %d => %s%n", friendsTraverser.currentPosition().getDepth(), friend.getProperty( "name" ) ); }
• 13.
  name = “TheArchitect” name = “Morpheus” rank = “Captain” name = “Thomas Anderson” occupation = “Total badass” 42 age = 29 disclosure = public KNOWS KNOWS CODED_BY KNO 1 7 3 W S KN 13 S OW name = “Cypher” KNOW S last name = “Reagan” name = “Agent Smith” disclosure = secret version = 1.0b age = 3 days age = 6 months language = C++ 2 name = “Trinity” $ bin/start-neo-example Mr Anderson's friends: At depth 1 => Morpheus friendsTraverser = mrAnderson.traverse( Traverser.Order.BREADTH_FIRST, At depth 1 => Trinity StopEvaluator.END_OF_GRAPH, At depth 2 => Cypher ReturnableEvaluator.ALL_BUT_START_NODE, RelTypes.KNOWS, At depth 3 => Agent Smith Direction.OUTGOING ); $
• 14.
  Example: Friends inlove? name = “The Architect” name = “Morpheus” rank = “Captain” name = “Thomas Anderson” occupation = “Total badass” 42 age = 29 disclosure = public KNOWS KNOWS CODED_BY KNO 1 7 3 W S KN 13 S KNOW OW name = “Cypher” S last name = “Reagan” name = “Agent Smith” LO disclosure = secret version = 1.0b VE age = 6 months language = C++ S 2 name = “Trinity”
• 15.
  Code (3a): Customtraverser // Create a traverser that returns all “friends in love” Traverser loveTraverser = mrAnderson.traverse( Traverser.Order.BREADTH_FIRST, StopEvaluator.END_OF_GRAPH, new ReturnableEvaluator() { public boolean isReturnableNode( TraversalPosition pos ) { return pos.currentNode().hasRelationship( RelTypes.LOVES, Direction.OUTGOING ); } }, RelTypes.KNOWS, Direction.OUTGOING );
• 16.
  Code (3a): Customtraverser // Traverse the node space and print out the result System.out.println( "Who’s a lover?" ); for ( Node person : loveTraverser ) { System.out.printf( "At depth %d => %s%n", loveTraverser.currentPosition().getDepth(), person.getProperty( "name" ) ); }
• 17.
  name = “TheArchitect” name = “Morpheus” rank = “Captain” name = “Thomas Anderson” occupation = “Total badass” 42 age = 29 disclosure = public KNOWS KNOWS KNO CODED_BY 1 7 3 W S KN 13 S name = “Cypher” KNOW OW S last name = “Reagan” name = “Agent Smith” LO disclosure = secret version = 1.0b VE age = 6 months language = C++ S 2 name = “Trinity” $ bin/start-neo-example new ReturnableEvaluator() Who’s a lover? { public boolean isReturnableNode( TraversalPosition pos) At depth 1 => Trinity { $ return pos.currentNode(). hasRelationship( RelTypes.LOVES, Direction.OUTGOING ); } },
• 18.
  Bonus code: domainmodel How do you implement your domain model? Use the delegator pattern, i.e. every domain entity wraps a Neo4j primitive: // In package org.yourdomain.yourapp class PersonImpl implements Person { private final Node underlyingNode; PersonImpl( Node node ){ this.underlyingNode = node; } public String getName() { return (String) this.underlyingNode.getProperty( "name" ); } public void setName( String name ) { this.underlyingNode.setProperty( "name", name ); } }
• 19.
  Domain layer frameworks Qi4j (www.qi4j.org) Framework for doing DDD in pure Java5 Def nes Entities / Associations / Properties i Sound familiar? Nodes / Rel’s / Properties! Neo4j is an “EntityStore” backend Jo4neo (http://code.google.com/p/jo4neo) Annotation driven Weaves Neo4j-backed persistence into domain objects at runtime S pring D a ta G ra ph (http://www.springsource.org/spring-data) Collaboration with SpringSource Annotation driven, AspectJ based
• 20.
  Neo4j system characteristics Disk-based Native graph storage engine with custom binary on-disk format Transactional JTA/JTS, XA, 2PC, Tx recovery, deadlock detection, MVCC, etc Scales up Many billions of nodes/rels/props on single JVM Robust 7+ years in 24/7 production
• 21.
  M'kay... but whatare some use cases?
• 22.
  Social data (customer:e.g. EU largest social network) name = “Marcy Runkle” name = “Hank” last_name = “Moody” name = “Mike” age = 42 42 age = 29 S W NO disclosure = public K KNOWS KNOWS KNO 1 7 3 W S KN 13 S OW name = “Charlie” KNOW S last_name = “Runkle” name = “Dani” last_name = “California” age = 3 days age = 27 2 name = “Karen”
• 23.
  Just a socialgraph?
• 24.
  Spatial data (customer:large telecom company) name = ... name = “The Tavern” lat = 1295238237 name = “Omni Hotel” long = 234823492 42 lat = 3492848 long = 283823423 length = 7 miles AD RO ROAD ROAD ROO 1 7 3 OAD RO 13 AD name = ... ROAD lat, long = ... name = “Swedland” lat = 23410349 length = 3 miles long = 2342348852 2 name = ...
• 25.
  Social? Spatial? …Social AND spatial!
• 26.
  Social AND spatialdata (customer: LBS) name = “Peter” name = “The Tavern” lat = 1295238237 name = “Omni Hotel” long = 234823492 42 lat = 3492848 S W NO long = 283823423 weight = 10 K ROAD LIKES SIBL 1 7 3 IN G RO 13 AD name = “Emil” ROAD beer_qual = expert name = “Maria” age = 30 length = 3 miles beer_qual = non-existant 2 name = ...
• 27.
  Financial data (customer:international bank) name = ... name = “The Tavern” lat = 1295238237 long = 234823492 42 name = “Mr Godfather” AW karma = veeeery-low HDR cash = more-than-you amount = $1000 IT W OWNS TRANSFER WIT 1 7 3 HDR AW 13 S FE R DE name = “Emil” P OS cash = always-too-li'l IT TRAN title = “ATM @ Wall St” id = 230918484233 amount = $1000 cash_left = 384204 2 name = ...
• 28.
  Cute. But whatabout performance? name = ... name = “The Tavern” lat = 1295238237 long = 234823492 42 name = “Mr Godfather” AW karma = veeeery-low HDR cash = more-than-you amount = $1000 IT W WITHDRAW TRANSFER WIT 1 7 3 HDR AW 13 S FE R DE name = “Emil” P OS cash = always-too-li'l IT TRAN title = “ATM @ Wall St” id = 230918484233 amount = $1000 cash_left = 384204 2 name = ... pathExists(a, b, 5) # nodes query time Relational database (MySQL) 1 000 2 000 ms Graph database (Neo4j) 1 000 2 ms Graph database (Neo4j) 1 000 000 2 ms
• 29.
  How ego areyou? (aka other impls?) Franz’ A lle g roG ra ph (http://agraph.franz.com) Proprietary, Lisp, RDF-oriented but real graphdb Sones g ra phD B (http://sones.com) .NET, open source version available Twitter's Flo c k D B (http://github.com/twitter/flockdb) Twitter's graph database for large and shallow graphs Sparsity Technologies D ex (http://sparsitytechnologies.com) New on the scene, spun out of Spanish university Objectivity I nfiniteG ra ph (http://infinitegraph.com) Graph db on-top-of OODB
• 30.
  Conclusion Graphs &&Neo4j => teh awesome! Available NOW under AGPLv3 / commercial license AGPLv3: “if you’re open source, we’re open source” If you have proprietary software? Must buy a commercial license But the first one is free! Download http://neo4j.org Feedback http://lists.neo4j.org
• 32.
  T he Neo 4j tea m is hiring ! Do you see the Matrix? Apply now.
• 33.
  Questions? Image credit: lost again! Sorry :(
• 34.
  http://neotechnology.com