Big Data Science with H2O in R

H2O – The Open Source Math Engine
Big Data Science
with H2O in R

4/23/13
H2O –
Open Source Math
& Machine Learning
for Big Data
Anqi Fu, August 2013

Universe is sparse. Life is messy.
Data is sparse & messy.
- Lao Tzu

Introduction to Big Data
• There are about as many bits of information in our digital
universe as there are stars in our actual universe.
• The process to decode the human genome took 10 years.
It can now be done in a week.
• Big data means more than “lots of data”

H2O – The Open Source Math Engine
Better
Predictions
Same Interface

Installation
1. Install and run H2O
• Command line: java –Xmx2g –jar h2o.jar
• Pull up http://localhost:54321 in browser
2. Install the R package
• install.packages(c(“RCurl”, “rjson”, “bitops”))
• install.packages(“Path/To/Package/ h2o_1.2.3.tar.gz", repos = NULL,
type = "source")
3. In R console, type library(h2o)
• demo(package=“h2o”)
• demo(h2o.glm)
Replace this!

Always have H2O running first!

Basic R Script
1. Tell R where H2O is running:
localH2O = new(“H2OClient”, ip=“127.0.0.1”, port=54321)
2. Check connection:
h2o.checkClient(localH2O)
3. Pass H2OClient as parameter to import:
h2o.importFile(localH2O, path=“Path/To/Data”, …)

Overview of Objects
• H2OClient: ip=character, port=numeric
• H2OParsedData: h2o=H2OClient, key=character
• H2OGLMModel: key=character, data=H2OParsedData,
model=list(coefficients, deviance, aic, etc)
Example: myModel@model$coefficients
H2O
key=“prostate.hex”
key=“airlines.hex”

Overview of Methods
Standard R H2O
read.csv, read.table, etc h2o.importFile, h2o.importURL
summary summary (limited to data only)
glm, glmnet h2o.glm(y, x, data, family, nfolds,
alpha, lambda)
kmeans h2o.kmeans(data, centers, cols,
iter.max)
randomForest, cforest h2o.randomForest(y, x_ignore,
data, ntree, depth, classwt)

Demo 1: Basic GLM in H2O through R

Demo 1: Prostate Cancer Data
• Prostate cancer data set from Ohio State University
Comprehensive Cancer Center
• N = 380 patients, ages ranging from 43-79
• Goal: Predict presence of tumor from baseline exam of
patient (age, race, PSA, total gleason score, etc)

Prostate Cancer
Data:
y = CAPSULE
0 = no tumor
1 = tumor
x = PSA
(prostate-specific antigen)

Prostate Cancer
Logistic Regression Fit
Family: Binomial, Link: Logit
Data:
y = CAPSULE
0 = no tumor
1 = tumor
x = PSA
(prostate-specific antigen)
Goal:
Estimate probability
CAPSULE = 1

GLM Parameters
• y = response variable
• x = predictor variables (vector)
• family = binomial (default link = logit)
• data = H2OParsedData object
• nfolds = cross-validation
• lambda = weight on penalty factor
• alpha = elastic net mixing parameter
• alpha = 0 is ridge penalty (L2 norm)
• alpha = 1 is lasso penalty (L1 norm)

Under the Hood: Hacking R for H2O

Under the Hood
REST API
Data
(JSON)
Import
Parse
H2O
Data Scientist,
Analyst, etc

GLM Code Snippet
• Create an object to represent model
setClass("H2OGLMModel", representation(key="character",
data="H2OParsedData", model="list"))
• Declare new method for algorithm
setGeneric("h2o.glm", function(x, y, data, family, nfolds = 10, alpha
= 0.5, lambda = 1.0e-5) { standardGeneric("h2o.glm") })
Name Slots
Parameter Initial Value

GLM Code Snippet
setMethod("h2o.glm", signature(x="character", y="character",
data="H2OParsedData", …), function(x, y, data, …) {
• Send parameters to GLM.json page  GLM job started
res = h2o.__remoteSend(data@h2o, h2o.__PAGE_GLM, key
= data@key, y = y, x = paste(x, sep="", collapse=","), …)
• Keep polling and wait until job completed
while(h2o.__poll(data@h2o,
res$response$redirect_request_args$job) != -1) { Sys.sleep(1) }
• Query Inspect.json page with GLM model key to get results
res = h2o.__remoteSend(data@h2o, h2o.__PAGE_INSPECT,
key=res$destination_key)
http://cran.r-project.org/doc/contrib/Genolini-S4tutorialV0-5en.pdf

Demo 2: Data Munging and Remote H2O

Demo 2: Airlines Data
• Airlines data set 1987-2013 from RITA (25%)
• Goal: Predict if flight’s arrival will be delayed
• Examine slices of data directly
head(airlines.hex, n = 10); tail(airlines.hex)
summary(airlines.hex$DepTime)
• Take a subset of data to play with in R
airlines.small = as.data.frame(airlines.hex[1:1000,])
glm(IsArrDelayed ~ Dest + Origin, family = binomial, data =
airlines.small)

http://www.transtats.bts.gov/Fields.asp?Table_ID=236

Connecting to H2O Remotely
• Your slip of paper contains IP/port of your assigned cluster
• Point R to remote H2O client
remoteH2O = new(“H2OClient”, ip = “192.168.1.161”, port = 54321)
• All data operations occur on cluster
h2o.importFile(remoteH2O, path =
“Path/On/Remote/Server/To/Data”, …)
• Objects/methods operate just like before!

Roadmap
• Long-term Goal: Full H2O/R Integration
• Subset col by name/index: df[,c(1,2)]; df[,”name”]
• Add/Remove cols: df[,-c(1,2)]; df[,3] = df[,2] + 1
• Filter rows: df[df$cName < 5,]
• Combine data frames by row/col: rbind, cbind
• Apply functions: tapply, sapply, lapply
• Support for R libraries (plyr, ggplot2, etc)
• More Algorithms: GBM, PCA, Neural Networks

4/23/13
Questions and
Suggestions?

Big Data Science with H2O in R

More Related Content

What's hot

Viewers also liked

Similar to Big Data Science with H2O in R

Recently uploaded

Big Data Science with H2O in R

Editor's Notes