Are API Services Taking Over All the Interesting Data Science Problems?

Are API services
taking over all the interesting
data science problems?

Very short “hello world” examples
https://github.com/kylepolich/apis-talk

What is an API?
Application Programming Interface
“a set of subroutine definitions, protocols, and
tools for building application software”
- Wikipedia

APIs described in this talk
I’m speaking about a subset of this definition.
My use is the colloquial use.
Client
Client
Server
request
response
request
response

Example: Wikipedia pageview API
>>> from mwviews.api import PageviewsClient
>>> p = PageviewsClient()
>>> response = p.article_views('en.wikipedia', ['Machine_Learning'],
start='20170101', end='20170107')
>>> print(response)
defaultdict(<type 'dict'>,
{datetime.datetime(2017, 1, 4, 0, 0): {'Machine_Learning': 59},
datetime.datetime(2017, 1, 6, 0, 0): {'Machine_Learning': 54},
datetime.datetime(2017, 1, 3, 0, 0): {'Machine_Learning': 66}})

Some quick technical mentions
PROTOCOLS
HTTP
SERIALIZATION
JSON, XML
AUTHENTICATION
OAUTH
METHODS
GET, POST, PUT, DELETE

What’s great about APIs
● Separation of concerns
● Defined service contracts
● Common abstraction layer for operational monitoring
● Versioning and change management
● Buffering, load testing
● Useful tooling for common formats: buffering, load testing, caching, rate
limiting
● Minimize software a data scientist has to write
● Isolated A/B tests

Build your project as an API from day one
● Makes seamless integration
● Change management advantages
● Ownership of entire service
● All benefits on previous slide

Example: Spam Filtering
● Keyword lists were a failure
● Machine learning solved the problem
● Generalized, robust solutions
● Commoditization
● Federation
● Domain specific challenges still remaining

Newer Frontiers for APIs
Computer vision
● Facial recognition
● Object identification
● Logo detection
● Flag inappropriate content
Natural Language Processing
● Sentiment analysis
● Name entity recognition
● Summarization
● Translation
Personalization and Targeting
● Adtech
● Recommender systems
● Content optimization
● Lead scoring
Misc
● Speech to text
● Fraud detection
● Generalized ML tasks
● ???

Build, Buy, Acquire
● How long will it take your to build an adequate internal
version?
● What is the cost of expected API usage vs. time cost of
building it?
● Is the API likely to improve in the future faster than your can
improve your system?
● Are there any strategic advantages to an internal build?
● What happens if that API disappears?
● Can you use ensembling?

Ensembling
● A term used in many ways
● Leverage multiple models, perhaps built by different algorithms
● Stacking is a special case - Divide dataset, builds layer of models, then one
model on top those
● Random forest is a popular example

Two stage system
Internal
Model
3rd party
service API
Feature #1
Feature #2
Feature #3
Feature #4
Feature #5
Sensitive feature #6
Domain specific #7
Other feature #8
First stage output
Output

Two stage system
Internal
Model
3rd party
service API
Feature #1
Feature #2
Feature #3 x1
Feature #4
Feature #5
x2
x3
x4
Output

Facial Recognition
Try available options like Kairos, Microsoft Cognitive Services, Lambda Labs,
Google Cloud Vision API
Use libraries like OpenCV, Caffe, etc.
Read papers and build from scratch
Conduct research and innovate

Facial Recognition
Try available options like Kairos, Microsoft Cognitive Services, Lambda Labs,
Google Cloud Vision API
Use libraries like OpenCV, Caffe, etc.
Read papers and build from scratch
Conduct research and innovate
IMO, Facial recognition
is now a commodity

Disclaimer
Although I feature some commercial solutions as examples…
● These are not necessarily endorsements
● I’m not employed or compensated by any of these companies
● The services I mention range from adequate to great
● It’s your job to determine the best solution for your needs

Microsoft Cognitive Services
● Many services in
text, audio, video,
and image tasks
● You can probably
get $200 to try it
out

Microsoft Cognitive Services - Faces API response
[{"faceId":"1a228680-7582-47ac-bc7b-029536e95468",
"faceRectangle":{"top":145,"left":466,"width":67,"height":67},
"faceAttributes":{"age":45.0}},
{"faceId":"d0ee8e65-07fd-4674-9210-d05816205839",
"faceAttributes":{"age":33.8}},
{"faceId":"92696f9e-1f94-4a79-be30-4ff88539d0bb",
"faceAttributes":{"age":70.1}}]

● A source to find free and commercial solutions
● A place to exhibit your work
algorithmia.com market.mashape.com
Algorithmic marketplaces

{
"src":"This weekend, I am flying to Dallas
and presenting a talk on API trends.",
"format":"tree", "language":"english"
}
https://algorithmia.com/algorithms/deeplearning/Parsey

NOUN++NN ROOT
+-- This DET++DT det
+-- flying VERB++VBG acl:relcl
+-- I PRON++PRP nsubj
+-- am AUX++VBP aux
+-- Dallas PROPN++NNP nmod
| +-- to ADP++IN case
+-- and CONJ++CC cc
+-- presenting VERB++VBG conj
+-- talk NOUN++NN dobj
+-- a DET++DT det
+-- trends. NOUN++NN nmod
+-- on ADP++IN case
+-- API NOUN++NN compound

indico
“Machine learning for human use”
● Sentiment
● Text tagging
● Language detection
● Political leaning
● Name Entity Recognition
● … several others

Honorable mentions
● Google Cloud Platform - Large variety of services
● IBM Watson - Language, vision, speech, and data
● Wit.ai - Natural language processing
● Diffbot - crawl and extract data from the web
● Many, many more

Data APIs vs Data Science Services
Data APIs - extend the features available
Data Science Services - provide a model for
interpreting your features

3rd party data sources can...
● Match on email address
● Provide demographic and psychographic data points
● Up to date content (news, local businesses)
● etc.

Disclaimer #2
● I started OpenHouse
● OpenHouse is a free service
● We are a group of volunteers
● Everything is open source

OpenHouse
http://gallery.openhouseproject.co

OpenHouse
Direct access:
http://api.openhouseproject.co/api/property/?min_price=0&max_price=5000000&m
in_bedrooms=0&max_bedrooms=8&min_bathrooms=0&max_bathrooms=7&min_
building_size=100&max_building_size=4000&close_to=(159.17225089355944,33.
7,-118.2)

Microservice architecture
When a large system is constructed from simpler
components called microservices. Each performs a
small, specific function. Microservices communicate
with each other to fulfill higher level objectives.
Process an order:
● Order creation microservice
● Credit card validation service
● Inventory microservice
● Email service
● Fraud detection service
● Personalization service
● Advertising targeting service
● Discount offering service

Will I become obsolete?
Probably not.
● Adapt
● Learn methods over tools
● Always be looking at new tools
● Sometimes innovation reduces demand for certain
skills, usually slowly, and almost always creating a new
demand elsewhere
● Learn to recognize what problems will exist in a world
in which your problem is solved

Are API Services Taking Over All the Interesting Data Science Problems?

Conclusions about 3rd party services
● They’re easily swapped with something else later
● Think about quick wins by tying things together
● Build your project service oriented from day 0
● Consider how your service could be generalized
● Keep an eye out for tools you can use but wouldn’t have
invested in developing

Thank you
@DataSkeptic
https://github.com/kylepolich/apis-talk

Are API Services Taking Over All the Interesting Data Science Problems?

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Are API Services Taking Over All the Interesting Data Science Problems? (20)

More from IDEAS - Int'l Data Engineering and Science Association (20)

Recently uploaded (20)

Are API Services Taking Over All the Interesting Data Science Problems?