Hello everyone, welcome in this talk I am going to share some strategies and tools that can help sres in adopting AI tools in various aspects of their work. SAris core responsibilities are generally defined as improving reliability, scalability, performance, and response time while reducing cost. A good SRE generally automates themselves out of a job, meaning they shouldnt have to do the same task again. Here are a few key responsibilities that a typical sres would have and where AI can help. Alerting and monitoring systems and configurations are a big part of what sres do. Various AI tools and methodologies can be used for efficient filtering, monitoring, and anomaly detection. AI can also help with faster incident response by assisting with analysis and summarization. More and more tools are using AI and machine learning models for forecasting and predictive analysis that can be used for capacity planning. AI, especially generative AI, can also be helpful in reducing toil for many tasks. Let's take a look at these areas in a bit more detail. Supervised learning, a type of machine learning, can be useful in more efficient filtering and prioritization of alerts. By training models on historical incident data, AI can learn to distinguish between genuine and false alert, effectively reducing noise and enhancing the focus on critical issues. Additionally, these models can assess that the severity of incidents by recognizing patterns that correlates with major disruptions. This capability allows sres to prioritize the response more effectively, ensuring that the most damaging issues are addressed promptly while minimizing the distractions of false alarms. This approach not only improves response time, but also helps in the allocation of resources where they are needed the most. Traditional static thresholds for system alerts often either lead to a flood of notification during peak load or miss early signs of critical issues during lower activity period. By employing AI driven forecasting model, dynamic thresholds can be set that adjust based on predicted changes in the system behavior or load. This method uses historical data to forecast future states and adjust alert sensitivity accordingly, enhancing the systems ability to preemptively flag potential issues before they escalate. Dynamic thresholds help maintain system stability can preemptively manage potential bottlenecks or failures, thus allowing for more proactive rather than reactive management strategies. By continuously monitoring system metrics and logs, AI algorithms can detect anomalies that mean indicate emerging issues, often before they are apparent to human observers. Capability enables sres to quickly identify and mitigate problems, often automating the response to standard issues without human intervention. For instance, an AI system can detect an unusual increase in response time for service and can trigger an auto scaling action or restart the service without needing direct human oversight. This not only speeds up the resolution process, but also helps in maintaining high availability and performance consistency across services. Tools such as Grafana offer powerful capabilities for dynamic alerting and metric forecasting that are essential for modern sari practices. Grafana integrates machine learning tools that can analyze the time series data to forecast future trends and automatically adjust alert threshold based on these predictions. These integrations allow sres to visualize complex data and receive alerts that are both timely and contextually relevant, reducing the overhead of manual threshold management. By leveraging such tools, sres can enhance their monitoring system with automated intelligence, making it easier to handle large scale systems efficiently. An effective strategy for anomaly detection starts with establishing a baseline for normal operation matrix using statistical methods to identify numerical outliers. This involves calculating statistical parameters such as mean, median, etcetera to understand typical system behavior. Once the baseline is set, any significant deviation from these matrix can be flagged as an outlier. This initial setup is crucial as it sets the stage for more sophisticated analysis and helps in quickly spotting anomalies that falls outside of the expected pattern, thus enabling timely interventions to mitigate potential issues. Advanced techniques such as principal component analysis, PCA, or density based spatial clustering of applications commonly known as TB scan, are employed to further enhance the detection of anomalies and understand group behavior within the system. Data. PCA reduces the dimensionality of the data, highlighting the most significant variations and patterns that are often obscured in high dimensional data. TP scan, on the other hand, is excels in identifying clusters of similar data points in the data set and distinguishing these from points that do not belong in any cluster. These techniques together allow not only to pinpoint unusual activity, but also categorize them into meaningful groups for easier analysis and troubleshooting. There are several tools available that indicates machine learning capabilities specifically designed for anomaly detection. Machine learning toolkit Kafanas machine learning outlier detection Davis AI Datadogs Bitsai each offer unique features to automate and enhance the detection of anomalies. Splunk provides a robust platform for real time data analysis, while Grafana focuses on visualizing trends and patterns that deviate from the norm. David Ci, part of Dynatrace, and datadog bits leverage AI to provide predictive insights and automated root cause analysis as well. These tools significantly reduce the manual effort involved in monitoring and diagnosing systems, allowing sres to focus on strategic tasks and productive system management. Generative AI can be transformative how SR is managed and responsible incidents by providing efficient civilization tools. These AI models are trained to extract key information from vast amount of incident data, such as logs, metrics, and user reports, condensing them into concise summaries. This capability is crucial during high pressure situations where quick understanding and actions are required. By automating the summarization process, SRE teams can swiftly grasp the scope and impact of an incident, enabling faster decision making and allocation of resources to address the most critical aspects. First, tools like DevOps guru leverages machine learning to automate root cause analysis, significantly speeding up the identification and resolution of the issues. The AI power service continuously analyzes system data to detect abnormal behaviors and provide contextual insights by pinpointing the likely cause of operational problem. It offers recommendations of fixing issues before they escalate, thereby minimizing downtime and improving system reliability. By automating these complex processes, sres can address the underlying cause of incident more quickly and with greater accuracy, which is crucial for maintaining high service levels and customer satisfaction. Grafana offers features called Incident auto summary, which is designed to help team quickly comprehend the details of an incident without sifting through overwhelming data. Manually uses algorithms to highlight significant changes and patterns that led to the incident, providing a summarized view of events in a clear and digestible format. Such summaries are invaluable for postmortem analysis and for keeping stakeholders informed. By automating the creation of incident summaries, Grafana helps ensuring that all team members have a consistent understanding of each incident, which is essential for effective communication and collaboration during a crisis situation. Capacity planning in the context of SRE involves forecasting future demand on system resources and adjusting the infrastructure to handle these demands efficiently. By utilizing predictive analysis, SREs can anticipate periods of high demand, such as seasonal spikes or promotional events, and proactively scale their system to meet these needs. This approach helps in avoiding performance bottlenecks and ensure user experience by maintaining optimal service levels. Predictive analysis tool analyze historical data and user trends to model future scenarios, enabling organizations to make informed decisions about when and how to scale resources. The integration of ML with traditional load testing methods forms the backbone of what is called as machine learning assisted system performance and capacity planning. This involves. This innovative approach allows for more accurate and dynamic capacity planning by predicting how new software changes or user growth will affect the system. Machine learning algorithm analyzes past performance data and simulates various load scenarios to identify potential capacity issues before they occur in real environment. This proactive technology helps in optimizing resource allocation, reducing cost, and enhancing overall system efficiency by ensuring that the infrastructure is neither underutilized or overwhelmed. Several advanced tools offer machine learning capabilities to facilitate predictive analysis in capacity planning. Splunk's machine learning toolkit allows users to create custom models tailored to the specific system dynamics, enabling precise forecasting of system needs. Dynatrace provides a real time analytics and automates automatic baselining, which are crucial for detecting deviations in system performance and predicting feature. State datadog leverages these capabilities with a predictive monitoring feature that forecasts the potential outage and scaling. Need can empower to harness the power of AI, making more accurate predictions about system demands, significantly improving the effectiveness of capacity planning strategies. Generative AI is reshaping infrastructure as a core practices by offering AI driven assistant that help automate and optimize the creation of and management of infrastructure setup. An example of this is Pulumi, which integrates with an AI assistant to help developers and sres generate tests manage IAC script more efficiently. These AI assistants can suggest best practices, identify potential errors in code, and even recommend optimizations. This reduces the cognitive load on engineers and accelerates development cycle by enabling quicker iteration and deployments, thereby significantly reducing manual toil in infrastructure management. Machine learning is increasingly being utilized to enhance testing process in software development. Tools like Mabel testimony leverages AI to automatically create, execute and maintain testing suits. These platforms use machine learning to understand the application behavior over time, allowing them to adapt test dynamically and as the application evolves. These capabilities minimizes the need for manual test maintenance, which is often time consuming and prone to error. By automating these aspects, machine learning not only speeds of the testing process, but also enhances its accuracy reliability, freeing up the benefit for more complex and creative tasks. Managing and reducing technical debt is a crucial part of maintaining the health of health and scalability of software. Tools like sonar codeclimate provide automated code reviews, services that helps developer identify and fix quality issues that contributes to technical needs such as code smells, bulk security vulnerabilities, etcetera. By integrating these tools in the CI CD pipeline, teams can ensure continuous code quality checks, which helps prevent the accumulation of technical debt over time. Moreover, these tools offer actionable insight and metrics that aid in decision making regarding refactoring efforts, ultimately ensuring that the code base remains clean, efficient, and maintainable. Adapting to using AI tools, of course, comes with many challenges as well. AI technologies often require a level of expertise that may not exist within the current team. This lack of skills can be a significant barrier to the successful implementation of and uses of AI tools. Invest in training existing staff and consider hiring new team member with unnecessary skill sets. There are plenty of resources and courses available online to train your team on AI tools and technologies. AI tools rely on quality of data to deliver reliable insights. Data might be unclean, unstructured, or siloed across different systems, which can hinder the effectiveness of AI. Implement robust data management and governance practices to ensure your data is clean, structured and accessible. AI tools itself can also be used to assist in data cleansing and structuring process. People can also be resistant to new technologies, especially when it comes to something as transformative as AI. Resistance can slow down or even halt the transition to AI based SRE practices. Develop a strong change management strategy, which could involve regular communication about the benefits and importance of the change. Provide training and support, and gradually integrate AI tools into existing workflows to allow for us for the transition. AI tools needs to work in tandem with existing IT systems, tools and processes. Integration issues can prevent from AI tools from accessing necessary data or functioning as expected. Plan integration process carefully ensuring that the selected AI tools are compatible with your existing system. Where possible, choose AI solutions that offer flexible APIs and integration options. Implementing AI tools can also require a substantial investment, including purchasing or subscribing to the tool, integrating them into your existing system and training stock. Calculate the return on investment before implementation to understand the potential benefit and saving the AI tool can provide in the long run. Look for scalable solution that allows you to start small and increase your investment as you c value. That's all for this session. Thank you for tuning in. Enjoy the conference.