Geek camp
0 likes392 views
This document provides an introduction and overview of Hadoop. It discusses the brief history of Hadoop, including its origins from Google papers in 2005 and promotion by Yahoo since 2006. It then discusses why Hadoop is useful for big data applications that are petabyte in scale, scalable, robust, and secure. Specific use cases like analytics, reporting, filtering and machine learning on log files, user behavior data, and other structured or unstructured data sources are covered. Finally, it outlines the Hadoop ecosystem and tools like native Java APIs, Pig, Hive, and streaming options for other languages.
Geek camp
- 1. Intro to Hadoop Jaideep Dhok
- 2. Hi! ● I work at ● Involved with Hadoop for 2+ years
- 3. Outline
- 4. Brief History of Hadoop ● 2005 - ● Inspired by the GFS and MapReduce papers published by Google. ● Promoted heavily by Yahoo! Since 2006 ● Today, the defacto standard in 'Big Data' computing
- 5. The Buzz
- 6. Why? ● 'Big Data' ● How big? - petabyte scale ● Scalable ● Robust ● Secure!
- 7. Scalability
- 8. When To Use It ● Can you use Hadoop to do X? ● Is your problem 'embarassingly' parallel? ● Workflow? – Dependent/Independent Tasks ● Data/CPU intensive? ● Can you use Hadoop to do X in the Clouds? ● Depends where your data is
- 9. Why To Use It? ● Ad hoc analysis ● Semi/structured data – Log files – Text – CSV, XML, anything really – RDBMS – NoSQL!
- 10. Use Cases ● Analytics ● User behavior ● Reporting ● Filtering ● Machine Learning ● Just storing your data
- 11. Just From The Logs ● Suppose you run a web-site ● User breakdown by browsers ● Location ● Understanding user session – How long do they use it? – Who are the active users? – What part of my app they use the most? – What part of my app is user X's fav?
- 12. Tools ● Native Hadoop APIs – Java ● Streaming – Perl, Python, Ruby, any language as long it has support for 'stdin' and 'stdout' ● Pig ● HIVE ● Pipes – C and C++
- 13. Ecosystem
- 14. Don't Wait ● Hadoop ● hadoop.apache.org ● Cloudera tutorials on Hadoop ● Books
- 15. Questions?