Download as PDF, PPTX
Uploaded byDerek Graham
Adventures in Azure Machine Learning from NE Bytes
The document discusses Azure Machine Learning, a cloud-based service by Microsoft that allows users to analyze data, identify patterns, and make predictions using ready-made and custom algorithms. It highlights the tools available for building machine learning models, the challenges of auto-scaling resources, and the intricacies of real-time data analysis. Additionally, the document notes the importance of data quality and offers resources for further learning in data science and machine learning.
More Related Content
Similar to Adventures in Azure Machine Learning from NE Bytes
![Governance, Deployment & Methodologies for Agentic Automation [2/3]](https://cdn.slidesharecdn.com/ss_thumbnails/agenticcommunityseries-day2-251112135038-a386476d-thumbnail.jpg?width=640&height=640&fit=bounds)
Adventures in Azure Machine Learning from NE Bytes
- 1.
- 2. derek graham! principal developer@sage! special product responsibility for Azure bits
- 5.
- 6. Machine learning is nottelling computers what to do but letting them learn from examples or past experience
- 8. Which means you can… •Analyse historic or current data • Find patterns and trends • Make predictions about future events
- 13.
- 18. Azure Machine Learning • A"new" cloud-based service from Microsoft • Integrates with existing Cloud technologies • Use ready-made algorithms • Program custom algorithms tuned to your problem • You can evaluate it for free
- 19.
- 22. • Browser based •Drag n Drop • Flowchart-y • Example data sets • Use R or Python • Excellent intro wizard ML Studio
- 24. • Import data •Filter and aggregate data • Create machine learning models • Run experiments • Publish finished model Provides tools to:
- 26. The Learning Process •Define a problem you want to solve • Design a solution • Experiment! ! • Identify your data • Train the model with the data • Evaluate against expected results (speed and accuracy) • Adapt data or algorithm (or both) • Repeat ! • Save the best model • Publish • Run with live data
- 27.
- 28. Imagine… • "Business" Software •Azure hosted • PaaS • Multi-tenanted
- 29. Open-ended Workflow • MondayMorning Login • Friday Reports • In between? • Weekends? • Holidays?
- 30. Balancing • User demand •User experience • Compute resources • Cost
- 31. Scaling • Instances auto-scalebased on the CPU% metric using Azure’s standard scaling model. • Azure standard scaling is slow • Once auto scaler notices we need more capacity, the demand has often disappeared! • Not a good user experience
- 33. Experiment • Customer useis not regular... • ...but, is it predictable?
- 34. Hackathon! • Can webuild a better autoscaler? • Spin-up before high demand • Tear-down when idle • Better Cost vs UX
- 36. Requirements • What will"we" need on a given date or time? • Do "we" need to take action now to compensate for what will happen in 20 minutes time? • Number of instances • Predicted CPU
- 37. Best Predictor ofDemand? • Sessions? • Instance Memory Use? • Instance CPU?
- 39. Table Storage Diagnostics •Too slow • Purging • ML queries all or nothing • ML Data Reader stops after 4GB • GB !!!! • ML times-out after ~3 Hours
- 40.
- 41. Event Hubs • Applicationlog sink • Low overhead • Highly scalable • Time-based • Disposable
- 43. Neural Net Experiments ! •Feed Forward NN • Written using R libraries • Good predictor for 10-20 minute window • Too inaccurate after that • Best compromise between precision and speed • Recurrent NN better at forecasting • RNN execution time too long • Need to reduce data to optimal subset
- 44. Stream Analytics ! • Real-timedata analysis • Fast • Sql-like syntax • Range of inputs and outputs • Interesting development
- 50. Anomalies • Dev Processis painful • Syntax Errors • “Test” Import Behaviour • Starting and Stopping and Starting and Stopping
- 52.
- 53.
- 54. Publish… • ReST Webservice • Client Worker Role • Management Service API
- 55. …& Be Damned •Too much data crashes model • Fine in ML Studio • 500 • Out of memory?
- 57.
- 59.
- 62.
- 63. Bugs • We werepushing the environment quite hard • YMMV • ML studio has bugs • Parallel tasks !Parallel • ML portal missing functionality preventing it being production ready
- 64. #DevOps • Sharing modelsis "public" - Gallery • No export support • No support (yet) for model deployment • Still Drag n Drop • PowerShell for EventHubs and Stream Analytics
- 65. Machine Learning • ParallelR processing library would help • Finding an appropriate solution often requires a data science specialist • Solution is only as good as your data • You may need to compromise on accuracy for speed • Cost • Hosting • Each call to the service
- 66.
- 67. E-Book • Microsoft AzureEssentials: Azure Machine Learning • Download from: https://mva.microsoft.com/ebooks
- 68. Titanic • Jennifer Marsman- https:// blogs.msdn.microsoft.com/jennifer/2016/02/19/ using-azure-machine-learning-to-predict-who-will- survive-the-titanic/ • Data Science! https://www.kaggle.com/ • Amy Nicholson @AmyKateNicho https:// blogs.technet.microsoft.com/amykatenicho/
- 69. #DevOps • https://azure.microsoft.com/en-gb/documentation/articles/event- hubs-programming-guide/ • https://azure.microsoft.com/en-gb/documentation/articles/service- bus-event-hubs-manage-with-ps/ •https://azure.microsoft.com/en-us/documentation/articles/stream- analytics-dotnet-management-sdk/ • https://azure.microsoft.com/en-us/documentation/articles/stream- analytics-monitor-and-manage-jobs-use-powershell/
- 70.