Big Data.pptx
- 2. UNIT III BIG DATA & INDUSTRY 4.0 What Is the Fourth Industrial Revolution The original industrial revolution took place in the 18th and 19th centuries and involved innovations such as steam engines and mechanical production. The second industrial revolution came along toward the end of the 19th century and just before World War I. It included advancements such as the telegraph and industrial sewing machines. Fast forward to the late 1970s, and we have the third industrial revolution — a period that’s still ongoing and brought along things such as the internet and smartphones. Since the buildup to the fourth industrial revolution is still happening, the idea of it is a bit abstract. However, it will involve a future in which artificial intelligence allows machines of all types to communicate with and learn from each other, an idea that could potentially have a huge impact on production.
- 3. What is Data? The quantities, characters, or symbols on which operations are performed by a computer, which may be stored and transmitted in the form of electrical signals and recorded on magnetic, optical, or mechanical recording media. S.No DATA INFORMATION 1 Data is unorganised raw facts that need processing without which it is seemingly random and useless to humans Information is a processed, organised data presented in a given context and is useful to humans. 2 Data is an individual unit that contains raw material which does not carry any specific meaning. Information is a group of data that collectively carry a logical meaning. 3 Data doesn’t depend on information. Information depends on data. 4 It is measured in bits and bytes. Information is measured in meaningful units like time, quantity, etc. 5 Data is never suited to the specific needs of a designer. Information is specific to the expectations and requirements because all the irrelevant facts and figures are removed, during the transformation process. 6 An example of data is a student’s test score The average score of a class is the information derived from the given data.
- 4. TYPES OF BIG DATA Structured Unstructured Semi-structured Metadata 1. Structured Any data that can be stored, accessed and processed in the form of fixed format is termed as a 'structured' data. Over the period of time, talent in computer science has achieved greater success in developing techniques for working with such kind of data (where the format is well known in advance) and also deriving value out of it. However, nowadays, we are foreseeing issues when a size of such data grows to a huge extent, typical sizes are being in the rage of multiple zettabytes. 2. Unstructured Any data with unknown form or the structure is classified as unstructured data. In addition to the size being huge, un-structured data poses multiple challenges in terms of its processing for deriving value out of it. A typical example of unstructured data is a heterogeneous data source containing a combination of simple text files, images, videos etc. Now day organizations have wealth of data available with them but unfortunately, they don't know how to derive value out of it since this data is in its raw form or unstructured format.
- 5. 3. Semi-structured Data Semi-structured data is a form of structured data that does not conform with the formal structure of data models associated with relational databases or other forms of data tables, but nonetheless contain tags or other markers to separate semantic elements and enforce hierarchies of records and fields within the data. Therefore, it is also known as self- describing structure. Examples of semi-structured data include JSON and XML are forms of semi-structured data. The reason that this third category exists (between structured and unstructured data) is because semi-structured data is considerably easier to analyse than unstructured data. Many Big Data solutions and tools have the ability to ‘read’ and process either JSON or XML. This reduces the complexity to analyse structured data, compared to unstructured data. 4. Metadata – Data about Data A last category of data type is metadata. From a technical point of view, this is not a separate data structure, but it is one of the most important elements for Big Data analysis and big data solutions. Metadata is data about data. It provides additional information about a specific set of data. for example: In a set of photographs, metadata could describe when and where the photos were taken. The metadata then provides fields for dates and locations which, by themselves, can be considered structured data. Because of this reason, metadata is frequently used by Big Data solutions for initial analysis.
- 6. What is Big Data? Big Data is a collection of data that is huge in volume, yet growing exponentially with time. It is a data with so large size and complexity that none of traditional data management tools can store it or process it efficiently. Big data is also a data but with huge size. Following are some of the Big Data examples- The New York Stock Exchange generates about one terabyte of new trade data per day. BSE & NSE, Social Media: The statistic shows that 500+terabytes of new data get ingested into the databases of social media site Facebook, every day. This data is mainly generated in terms of photo and video uploads, message exchanges, putting comments etc. A single Jet engine can generate 10+terabytes of data in 30 minutes of flight time. With many thousand flights per day, generation of data reaches up to many Petabytes. Definition: Big data is different than "Business Intelligence" and "data mining" in terms of data volumens, number of transactions and number of data sources are very big and complex. Hence Big data require special methods and technologies in order to draw insight out of data.
- 7. CHARACTERISTIC FEATURES OF BIG DATA (i) Volume – The name Big Data itself is related to a size which is enormous. Size of data plays a very crucial role in determining value out of data. Also, whether a particular data can actually be considered as a Big Data or not, is dependent upon the volume of data. Hence, 'Volume' is one characteristic which needs to be considered while dealing with Big Data. (ii) Variety – The next aspect of Big Data is its variety. Variety refers to heterogeneous sources and the nature of data, both structured and unstructured. During earlier days, spreadsheets and databases were the only sources of data considered by most of the applications. Nowadays, data in the form of emails, photos, videos, monitoring devices, PDFs, audio, etc. are also being considered in the analysis applications. This variety of unstructured data poses certain issues for storage, mining and analyzing data. (iii) Velocity – The term 'velocity' refers to the speed of generation of data. How fast the data is generated and processed to meet the demands, determines real potential in the data. Big Data Velocity deals with the speed at which data flows in from sources like business processes, application logs, networks, and social media sites, sensors, Mobile devices, etc. The flow of data is massive and continuous. (iv) Variability – This refers to the inconsistency which can be shown by the data at times, thus hampering the process of being able to handle and manage the data effectively. Example: The major big data landscapes are twitter, facebook and youtube. Moreover big data volume is increasing day by day due to creation of new websites, emails, registration of domains, tweets etc
- 8. Benefits or Advantages of Big Data Following are the benefits or advantages of Big Data: ➨Big data analysis derives innovative solutions. Big data analysis helps in understanding and targeting customers. It helps in optimizing business processes. ➨It helps in improving science and research. ➨It improves healthcare and public health with availability of record of patients. ➨It helps in financial tradings, sports, polling, security/law enforcement etc. ➨Any one can access vast information via surveys and deliver anaswer of any query. ➨Every second additions are made. ➨One platform carry unlimited information.
- 9. Drawbacks or disadvantages of Big Data Following are the drawbacks or disadvantages of Big Data: ➨Traditional storage can cost lot of money to store big data. ➨Lots of big data is unstructured. ➨Big data analysis violates principles of privacy. ➨It can be used for manipulation of customer records. ➨It may increase social stratification. ➨Big data analysis is not useful in short run. It needs to be analyzed for longer duration to leverage its benefits. ➨Big data analysis results are misleading sometimes. ➨Speedy updates in big data can mismatch real figures.
- 10. WHAT ROLE BIG DATA WILL PLAY In fact, some say big data is Industry 4.0. In manufacturing, for example, improvements and efficiencies in the analysis of big data are expected to bring billions of dollars to the industry over the next five years. Others look at it as an equation in which artificial intelligence plus big data equals the fourth industrial revolution. On one hand, you can see the possibility of job losses as autonomous machines take over tasks that humans have handled for years. On the other, there could be a slew of new jobs created when it comes to harnessing the power of data and using it in a meaningful way.
- 11. BEYOND THE MANUFACTURING INDUSTRY Throughout history, industrial revolutions have often been judged by their impact on the production and manufacturing of goods and products. That’s no different with the looming Industry 4.0, but it will affect many other industries as well. In financial services, for example. In this field, experts view big data as “the new electricity” — the power source driving change in the way that steam, actual electricity and digital technology did before it. In one example, a company in Chile uses big data and machine learning to predict the likelihood individual customers will be able to repay loans. If you look back 20 or 30 years, it took groups of human employees time and effort to determine your credit score. Now, using information such as automotive credit history, utility bills and census data, combined with predictive machine learning, that process can be almost instantaneous.
- 12. BIG DATA, AI AND THE INTERNET OF THINGS Part of the fourth industrial revolution is the manner in which all types of machines and devices interact, communicate and learn from each other. At this early stage, it’s similar to the Internet of Things, or IoT — the concept in which everyday objects such as cars, refrigerators, TVs, ovens and home security systems are all connected to the internet. Add on top of that a layer of artificial intelligence that saves time by making decisions for you, and you see how these products and ideas can make your life easier and even create new business opportunities. As one IBM analyst puts it, the thing that AI and IoT have in common is the use and interpretation of big data. Companies that invest in all three areas — AI, IoT and big data — stand a good chance at becoming leaders and innovators in the fourth industrial revolution.
- 13. WHY SMALL BUSINESSES SHOULD BE USING BIG DATA Big businesses aren’t the only ones who can make data-driven decisions using big data these days. Small businesses can reap the benefits, too. Analyzing all the online and offline information that one can helps to grow their business. Big data is defined as very large datasets that can be analyzed computationally to reveal patterns, trends, and associations – especially in connection with human behavior and interactions. A big data revolution has arrived with the growth of the Internet, wireless networks, smartphones, social media and other technology. Organizations who discuss using big data usually have the resources to hire research forms and data scientists to do the work for them. But, if one knows where to look, small businesses can finally step up to the plate and utilize big data, themselves.
- 14. 7 BENEFITS TO USING BIG DATA FOR SMALL BUSINESSES 1. Using big data cuts your costs 2. Using big data increases the efficiency 3. Using big data improves your pricing 4. The small companies can compete with big businesses 5. Allows the comapnies to focus on local preferences 6. Using big data helps the small companies to increase sales and loyalty 7. Using big data ensures you hire the right employees
- 15. HOW BUSINESSES CAN ANALYZE BIG DATA In order to analyze big data, we need to first identify the issues that need solutions or answers. Then, attempt to identify the answer to our question and ask ourself, ‘how can we get the data to solve it?’ or ‘what can big data do for our business?’ Your big data solutions need to be user-friendly, match what you had in mind for pricing, and flexible enough to serve your business both now and in the future. Research what the most reliable tool is for the problem we need to solve. For example, if you want to launch more effective promotions and marketing campaigns, you can use Canopy Labs, which predicts customer behavior and sales trends. There are many tools out there that are inexpensive or even free that you can use. Google has user-friendly tools like Google Adwords and Google BigQuery. Administering a survey is simple and cheap using tools, such as SurveyMonkey and Doodle.