Today I'm going to talk about generative AI in development processes and dev tools for this. My name is Bogdan Snisar, 14 years of software engineering experience, had different roles, software lead, engineer, CTO, remarkable companies. It's a Revolut, Wix, currently partner at Overview. It's consulting for LLM and machine learning research. And development of products, co founder and CTO for RedRay Space. Fashion search and discovery platform for finding clothes. Here is my links, feel free to follow. Okay, let's start. Before diving into the details, I want to just highlight what actually, highlight and group projects into several areas. it's a taxonomy of usage of AI for system operation and software development lifecycle. there is a four groups that I find out reasonable here. Conventional systems with conventional SDLC. This is where most current software lives. Standard systems, like traditional inventory management systems using standard development processes without any AI. Some legacy. Conventional systems with AI augmented as DLC. Emerging area where tools like GitHub Copilot or something similar, used by developers. Could be just any project and this is the area where I want to concentrate further during this talk. another area, it's actually some products where people developed products. Exactly, AI systems itself, yeah, augmented systems with conventional SDLC kind of idea of having, ML based product in production, but using the same as DLC as usually, and just the combination of everything. When we develop in AI product and using AI tools for SDLC, as we see as DLC, it's a part It's common for everything and where AI development's really interesting for me. And I want to share my experience from what I used and what looks reasonable at this current moment. What's SDLC itself? It's a process that contains from different phases. It's not only coding. It starts from requirement analysis. Just, we need to plan what we do. going to implement, then development process, then testing, integration testing of it, deployment into the production, and maintenance of it. This is the SDLC overall. interesting to see what actual researchers are trying to solve. Where are the current trends and what's happening? There is an interesting paper that Summarize, it's pretty, pretty fresh, that summarize the whole number of white papers and researchers that happening across the whole internet. They find out that majority of things concentrated on software development itself. Different angles of it, I will cover a couple of them, but there is a developments in software maintenance requirements, engineering, software design. Software management looks like in the SDLC process. There is a two emerging parts. It's a coding and maintenance. However, in the proportion, I believe maintenance, it's a bigger because we have a lot of already written software that requires maintenance. And it's interesting that problems mostly concentrated on generation. Not problems, but tasks that develop, tasks that researchers try to achieve. It's a generation of code, generation of different things, recommendation and some classification things. Let's think about what EDL product or what's actually some people envision as a dream about software development. It's a fully automated SDLC. That's a set of AI agents that's responsible for. operating independently and make the SDLC autonomous. Starting from role assignments, requirements engineering, development, testing, deployments, and all another things. Currently looks like we can use LLMs during SDLC as a helpful tools. To reduce human errors on, or make our work easier in some way. moreover, there is some, of course, rumors about autonomous software engineer as a service. For example, Divine or open sourced SWE agent. This is a set of AI agents that can reflect software engineer. Or they try to do that and according to the test, it's not works so well. in reality, this agent is just a dream right now. However, we see that this chance will continue. And who knows what happens in 2030s. instead of trying to replicate the whole software engineer as an agent, there's a reasonable to have it splitted on several functions and maybe to automate or create helpful tools for part of the SDLC. First thing that comes to me, it's a planning. There is a requirements engineering phase where we need to gather requirements and understand. There's an interesting quote. If my job was coding solutions to problems with rigorously defined requirements, it would be concerning. It's about that people are pretty bad in expressing what they want and like developers say. I think opposite. LLMs are really good in understanding of some unstructured requirements into some structured way. this is really good thing for requirements engineering. It can be used to interpret unstructured requirements into the text specifications or even user stories in reality. LLMs can help requirement engineering to highlight unknowns. It's actually an important thing. It can navigate to some gaps into the requirements that actually are not so visible. Or it's hard to understand that these gaps exist just because LLMs has some generic knowledge and it could be used to fulfill those gaps or identify them and use this as a knowledge for follow up questions during the requirement phase. And another interesting point is to classify functional and non functional requirements from different sources. From, for example, user reviews. On the platforms. It's actually interesting thing that LLMs can be used to identify requirements based on the user feedback. And just create already fully refined or halfly refined tickets that already ready for grooming phase or planning phase. Or at least to create them into the developers pipeline. So requirements engineering, it's an interesting thing. Another, of course, part is the development, how LLMs used in software development. There is a lot of directions where research happening right now. It's a code completion, summarization, search, content, code understanding, actually pretty important program synthesis, API recommendations, API kind of generation, also interesting and I guess here it's emerging. problem exists, method name generation, et cetera. That's nice. Moreover, there's only two hard things in computer science, according to the quote, caching validation and naming things. And naming things already not a big problem, according to the LLMs. What tools I'm personally find out interesting? It's a replit, for example. in reality, it's reflect of AI agents that It tries to build some ready to use solution. it works only for Python and JavaScript. Majority of LLMs are well trained on these things. And it's really bad actually for some rare languages or complex languages like Rust. What is this tool good for? It's for prototyping for something to show as an idea, as a concept. It has a lot of nice features. Standard use cases like authorization, authentication, access to the database. And it's time saving for fast prototyping. And it's pretty crappy for specific use cases. When you try to do something non specific, it's a bit problematic. Cursor. It's actually not a platform, it's a tool. It's a kind of a VS Code fork that had a lot of recent developments. With code completion, assistance for code, we can refine and ask it to do something, to modify code at different angles, and it's pretty nice, actually. However, it's not so good in working with big projects, so my current activity, when I copy paste something to the cursor, work with it, and then copy paste back to the IDE or VS Code. The reason, because I can do the same with chat gpt, but with chat gpt I need to all every time write some specific prompts. Here it's a tool that likes design to work with code. So in some way useful. that's mostly my main usages of LLMs in software development. There is a GitHub co pilot, but I cover it in the next phase. What about maintenance? There is actually a really good fit for LLMs. Program Repair, Detection, Review, Debugging, Bug Reproduction, it's an interesting direction. Hope there will be some interesting products there. Logging, another interesting idea that we can automatically augment software code with logging. Because usually software developers put logging randomly, without any kind of a system. Either it's some problematic place or there was a bug or something like this If the LLM can make it automatically and put it with the understanding of the context, it's really nice time saving for maintenance especially BugJar, also a nice thing Actually good fit for LLMs another interesting thing for maintenance. It's a search how to find Something across the whole part of the project, it's gives ability right now to merge and join a lot of piece of information together and make it as a useful information to make, to help, to actually, to solve something. It could be specs, Jira code, test cases, bug reports, documentation, et cetera. Just something can give you the answers based on it. There is a couple of products that working with this. I actually work on the GitHub Copilot workspace, but there is Atlassian Aroma, not used, but it's interesting to see, Naboo AI, and GitHub Copilot. In some way it's interesting, right now it's mostly good for finding, however, GitHub Copilot tries even to give you ability to add something. For example, U Express, you have a specification, then you want to add some specific details, and this thing can generate code based on it. This part works, let's say, extremely well. Okay with small classes for specific cases. It's a bit problematic. It's problematic Yes, because it can lose the context or don't understand the whole context of the problem clearly But the ability to merge information really nice Another thing of maintenance, it's working with pull requests. Actually, this is a nice thing It's a GitHub marketplace, CodeRabbit, AI, it's nice, actually, it helps to analyze code and makes pull requests review really good. Because sometimes it can find what I missed, sometimes I find what this tool missed, and this is a good combo in reality. Just because it's just a real feeling of Copilot. Another part of maintenance, it's working with issues. There is a, just example, GitHub Marketplace 2, dosu bot. It can work with issue requests and it tries to resolve this issue as answering to the question. sometimes it makes a good job, in getting answers, especially with analyzing the code base and resolving some issues. issues without human interaction Pretty nice actually the idea good Another real direction where it works for maintenance. It's a documentation Usually developers dislike write documentation because they becomes outdated pretty easily However, with LLAMPs, I guess it's a good direction. I just don't use any tools except chat gpt, but it's nice thing, especially for code comments and reflecting readme and documentation with code. So we just changing the code and documentation reflects it without our interaction, interactions, maybe just only with some supervisory. However, no tools here. Another really nice thing, it's tests. That's a really important part of SDLC, and here the LLMs are really powerful. And see a really big value of them. it's a test generation, verification, test automation, a lot of things actually. why it's so good? Because, edge cases, That's what I think, and edge cases that LLM can generate is quite bigger. So it's really nice that it can generate more edge cases than I usually do. And that's really helpful in this case, because it also helps navigate some gaps in understanding. Proven clear value. Works not only for unit tests, but works for Selenium tests. It can be used, especially right now, as I can understand, a visual part, so it can be used for writing some Selenium specifications that gives you ability to test application end to end and to reflect it. Someone can be called that's TDD not so aligned with his developments, but I would say even It's current capabilities, this LLM tools can at least cover and increase test coverage for code that make it more maintainable. It can detect changes and improve overall code quality. Another good example, it's a paper by Meta about automated unit test improvement using large language model. They create the pipeline that's responsible to improve test coverage. Just as we discussed previous slide, the idea is that it generate test case test cases, not the unit test, but actually try to understand what to test to fulfill the ideas or like specifications for tests. Then it's doing some prep testing of it. Then generate tests. Runs builds, and when this build passes, means this is already successful number of tests. Then post processing is improving of code coverage. And here we are, we have a ready to create the pull request. Really nice paper that says that Meta itself internally tries to use it, uses for different cases. what we can summarize here. It's interesting that for code writing, it's not so bad. It really helps for boilerplate and repetitive code tasks. it's good for changes to end user code, actually. When it's not about complex things. When it's just for some small part of it, for some class, it can be useful. It's really nice for test and especially for test data generation, because sometimes it's really hard to generate some test data or come up with it. You need to think about it. here is really proven value. It's nice for assistance in unfamiliar tasks. It's another interesting part when LLMs can be used. For example, this CSS for HTML generated. Even though that you don't know CSS, like CSS for back end developers. that's nice. I tried even with ChatGPT. It works really, it works. After some number of iterations, I can manage to write some specific cases that I am not familiar with. And it's really good for integrated search over a data lake. It's a really emerging thing, especially for big projects, because In big projects, the issue of complexity between different parts is really big. It's not good for changing a complex code. Yeah. And it's not good for refactoring. Because refactoring requires changing with understanding of the big context. for maintainability of complex legacy code, it's not so workable. And of course it's not good for generating a complex code or like code for multiple cases or even multiple classes when they're involved in this. In a nutshell, it works great right now for unit tests, for testing, let's say. It works great for some prototyping. And it's not so good for maintenance of the legacy or big projects. Yes, we heard about recent talks that, for example, Amazon just save a lot of hours for migrating from one version Java to another, but in reality, I think it's just minor code changes because migrating from one version Java to another, not so complex. Just the language really famous for their backward compatibility. So this is a kind of a summary that I see reasonable right now. What else? What interesting side effects this thing creates? One side effect I see it's code churn by years. There is a article that, yeah, this article will use some extrapolation, but in reality it says that code churn increases after the LLM cycle. boom started, there is a couple of comments and, the idea is that, practical consequence of LLMs, that, it works well for experienced developers, but for newcomers, it becomes a bit problematic because, LLMs generate some code, but you need to understand, is it okay or not, how to modify it. And it requires experience to work with it. And usually it generates something that's not really nice, or it's nice, but partly. and it means if we really follow the LLMs without any supervisory, It creates a legacy code quicker than people usually do. this legacy code will be problematic for next maintainability, the maintenance of it. Debugging and fixing this code. it means LLAMP is good for generating code, but this code has a higher effort on maintainability. And this is a problem that actually could be solved. And where I guess some researchers will try to solve because new code generation, it's nice, but maintainability of it, it's another important part because, it's a really big part of overall lifecycle of the project. This is an interesting outcome that I see even with my code. that's, here is the emerging thing. However, it's all my story right now about the usage of LLMs and things for DevTools. So thank you for your time and I hope it was interesting to listen.