From the course: Enterprise AI Development with GitHub Models and Azure
Join today to access over 24,900 courses taught by industry experts.
Monitoring your solution in production
From the course: Enterprise AI Development with GitHub Models and Azure
Monitoring your solution in production
- [Instructor] There are a couple of important things to monitor when you use any generative AI solution in a production environment. You can start by looking at the utilization of your Azure resources. You have configured a specific scaling unit for the model deployment in Azure Open AI, as well as for the AI search resource. You need to monitor if those provision scale units are sufficient for your end users. If not, you can adjust your usage or scale out the resources. Next to adjust the throughput for the resources, also, be aware of the rate limits for those services. If you have more users, it is likely that you need to scale the resources to meet those needs. Since you are using models deployed on Azure OpenAI, you are using content safety filters as well. If you're users ask questions that do not match these filters, you need to be aware and monitor how often this happens. These filters are there to protect you and your business from legal repercussions. Next up is making sure…
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.