077: LLMs, with great power comes great responsibility
Hello, Gophers. I'm Dominic StPierre. Unfortunately, Morten is not here this week. He's sick. And we have a guest, especially a returning one.
Dominic:Rams, you were there in episode 42. Thank you very much for returning.
Ramesh:Thank you, Dom. So, Dominic, 40 two is a favorite number for all the geeks. Right? So it was a coincidence that you had me on 42, and we had a lovely discussion on gatekeeping and stuff like that. And again, I really appreciate you for giving me another opportunity and having me.
Ramesh:Hopefully, we'll have some good discussion. Would have loved to have Morten over as well.
Dominic:Totally. Because, yeah, Morten is a little bit he's more using LLMs than me. So we we will talk about LLMs today. So, basically, Rams Morten contacted me and wanted to to give his experiences and and, you know, just talk a little bit because well, maybe we have let let ourselves, I don't know, maybe it felt like we are again not against L and M's and whatnot, but we are not always positive. Let's let's leave it at that.
Ramesh:Yes. That's that's a very good way to put it. Yeah. So the takeaway, at least for me, and again, I could be wrong, is that the message you leave when you guys discuss about LLMs is it's not a very positive experience. And my purpose of trying to have this discussion and coming on the podcast is to try and, you know, share my experiences and hopefully the listeners will take back that, yes, there are limitations.
Ramesh:Yes, it's not, you know, good or 100% of the yellowing the code and things like that. But there are very good use cases where LLMs and agent decoding is very, very helpful.
Dominic:Totally. Totally. I think I think we will have a a good discussion for sure. Maybe the it's it's easier to have the the bad experiences surface up. Maybe that's why you might have felt a little bit like like you know, we are not and and I I will talk for more than a little bit because he he he built a he built a lot of this project with with LLMs actually.
Dominic:And, you know, I I think I think we are not completely negative about that. It's not that. But it's true that for for us and especially the the last episode that just came came in this morning, actually, we we we both had very very similar experiences, lastly, to to look at the code that was generated for us and have to yeah. That that is still real. And I want I want to hear you about first of all, let's let's start by, let me ask you exactly, you know, what are you using LLMs for at the moment?
Ramesh:Great question. Yeah. So I started using LLMs almost a year ago or even earlier than that. That was during my previous job. And then we were exploring and all of these agentic coding, I mean, it has been moving extremely fast, and they were just new on the scenes and things like that, right?
Ramesh:So what I am using LLM based, agentic based coding is exclusively in the current job, which I started very recently, is to build proof of concepts to understand existing code bases, to figure out and plan, if I have an idea, how do I bring that to fruition, right? So does it, what I'm thinking, does it really work, and what are the holes in it and things like that? So I usually, so mainly POC work, but in the previous job, we had been extensively using it for real production work. My current job does not have, unfortunately doesn't have any Go code. Previous job, we were exclusively a Go shop.
Ramesh:And there were a couple instances that I can share where we had positive experience and other people had some negative experience.
Dominic:That's interesting. Yeah. I'm I'm I'm interested to unwrap a little bit about that. Because from my point of view being a SaaS builder myself, a solopreneur, call it what you want, I don't have currently a lot of experiences, seeing what what is happening exactly inside enterprises, inside companies at the moment. What's happening exactly at the team level?
Dominic:One question that I have is mainly, did the the usage of LLMs kind of put a toll on on the shoulder of seniors, developers to now have to drink from the fire hose. Let let's be real because, you know, there's a lot of output that that can be now generated. So did it add this impact at all where you were, you know, regarding the the Go code and whatnot? You know, how how did it affected the team?
Ramesh:Yes. So it does produce a lot more PRs that need to be reviewed and things like that. But I push back a little bit saying, as a senior developer, most of the times, you are responsible for reviewing other people's code, because either you grew up with this code base, and you're more intimately familiar with it, and as you onboard juniors, they're going to be slapping code and producing tons of PRs, and things like that, or maybe a gigantic PR. So the pressure of review still exists. But again, going back to what you said, yes, there is an increase in the volume of the PRs, It does put an extra pressure, but it is nothing different than in a large organization having several different architects and several teams with up and coming junior developers and also mid level developers.
Dominic:But is the have have you felt the quality of the PRs to be a little bit low, especially at the beginning since you are saying because I'm personally using Cloud Code only since, I would say, you know, October 2025. Before that, never used LLMs to generate anything. So I'm curious to hear was, you know, have you have you seen a good improvement, you know, compared to when it started and how it started actually at your company. So who who decided at some point, you know what? Let's try that.
Dominic:Was that on a small team to create some experiment? How did it arrive at the company?
Ramesh:Right. So the previous company was a very small team, a handful of developers, about six to eight developers in different domains. But the code base was developed over several years. And then the pressure to be market leading and put out features is what led us to start exploring AI based, agentic based coding. So we started almost a year and a half ago when CoPilot was coming up, Claude was coming up.
Ramesh:So a couple of us took one thing upon each of us, meaning I looked at Cloud Code, my colleague looked at Copilot, someone else looked at ChatGPT, and we started using it for about two or three weeks. But all of this was mainly driven by our CTO and our need to be nimble and get more features out. The good thing about that is almost all of us were very familiar with the code base, so we could go and argue with these agentic tools, and then also detect issues immediately. Does that answer your question?
Dominic:That's that's interesting. Was was there any friction from one or two, you know, software engineers that did not really like that that new flow because if I understand correctly what you're saying is that at some point, it was decided that to stay productive and whatnot, you know, this this could this could be used to improve the velocity of of the teams and whatnot. So was there any any, you know, reaction that was not very positive to that?
Ramesh:Oh, definitely yes. And in fact, I was initially not sold on this GitHub Copilot at all. For a long time, almost two or three months, when couple of other colleagues were actively using GitHub Copilot, I had not even taken advantage of the license that was available to us. Again, this was the beginning days, the agents and the code built, they were all sloppy. And most of my experience was within an IDE and with a small chat window, and it was producing garbage.
Ramesh:I completely agree with that sentiment, right? So yes. But these tools became better and better, and we also learned that when the CLIs became available, there was something about this mode of agentic. Although it's the same model that is available within the IDE versus the CLI, there was something different about the CLI that produced slightly better results, at least for me. And then finally, I was sold on it.
Ramesh:And I signed up to use Cloud Code, and ultimately, the entire organization moved to Cloud Code. We canceled all our GitHub Copilot licenses.
Dominic:Interesting. Yeah. Okay. So that that that was another aspect of my question before. So you you did see some kind of improvement as as, you know, time passes because we all agree that today's it's it's it's pretty capable what we have in on our hand, the models that we that we are not able to use, but it was not like that one year ago.
Ramesh:Oh, absolutely, yeah, one year ago. So there were a couple of different cases where I actively used Cloud Code as agent based development. So on my personal front, I am working on a couple mobile apps. So I'm not a Swift developer, so I was building an iOS application. And I didn't have access to Cloud Code CLI.
Ramesh:So I used the desktop application. And then when I started building it, it got me up to speed to up to a level. Then it got stuck on and off by one error, which it could never resolve. And I, not being a Swift programmer, had a difficulty in trying to understand. It was like, okay, I got this thing that accelerated my velocity, but I'm now stuck, and I have no way of trying to understand.
Ramesh:So there's always a balance. So if you use these tools in languages and things that you're familiar with, the acceleration that you're gonna gain is phenomenal.
Dominic:Right. Yeah. I recognize what you are seeing exactly. I I we interviewed Andy Williams from the Find Toolkit. It's it's a Mhmm.
Dominic:It's a GUI. Yeah. It's a it's a framework to build GUI. And I tried to help contributing to with the accessibility aspect of of the toolkit. And to be frank, I I don't have a huge knowledge of all the APIs that that you need to know to have an accessible GUI.
Dominic:So I generate I I I had the cloud, you know, help me with that. And at some point, I was like, you know, in between c code and and Go and the c go and whatnot, and I was like, wow. What what am I doing here? I I don't understand anything now.
Ramesh:Right.
Dominic:So I I I I see what you are what you're what you mean by the fact that if you are using it in in a place to learn let's let's face it, it's it's not that good to help you learn new stuff.
Ramesh:Wanna push back a little bit on that one. So meaning so let me ask you a question. What do you mean it is not good to help you learn?
Dominic:Because you don't know what to ask. When you don't know a topic, it's very, very hard to ask the right questions. And those model expects you to ask good question or at least be, be a little bit knowledgeable to extract the right answer that you are looking for. And you don't know if what they are going to tell you is going to be good because you don't know the topic. So to me, you know, you put the finger right right on where it should be, where it's really really helpful when you know this the subject that especially, you know, the language that you're working on on the platform and whatnot.
Dominic:But when you don't really know it might not be the right way to learn.
Ramesh:So let me give you a couple of examples. So in my previous company, we had a junior engineer who was trying to use Cloud Code to understand the code base, and they often found that they were going down rapid holes, right? But then we had other senior engineers in trying to fix production problems, and we were pretty much very productive in taking error logs directly from production logs, feeding it to Claude, and also being very focused on, okay, this is the area where things are happening. Here's the logs. Here's what I think, and then help me debug this.
Ramesh:And within a couple minutes we arrived at solutions. Now there was another engineer who, extremely smart, but they were not familiar with the code base. As an experiment, they wanted to solve the same problem, and they were again led down rabbit holes, Okay, but I see what you're saying like, okay, so if you're not familiar with a lot of things, it might be leading you down rabbit holes and things like that that can lead you completely astray and waste your time. But if you step back and start from defining the problem, almost like rubber ducking, right? So you define the problem, this is what I'm trying to do, this is what I'm trying to solve, okay, what are your ideas and things like that.
Ramesh:If you approach it from that perspective, would you still think it's not that helpful?
Dominic:I think it would be helpful in in that in that case for sure. What I what I mean by learning is that, let's say, I never write an Ascal line of my life, and I would like to use the LLM to help me build something and hopefully learn, you know, learn Ascle in in So of course, I will ask good software engineering questions and whatnot and I will I will But I won't learn as good regarding Askel than if I I were to do it myself more. So the the amount of things that it will it will output, and also, I might I might ask questions. It's it's all about the unknowns, you know. You When you don't know something, it's very hard to ask about something that you don't know.
Ramesh:But what is the focus of your learning? Are you trying to learn Haskell as a language? Or are you trying to use Haskell to solve a computing problem that you're having?
Dominic:No. I I mean, if you if you want to learn something new. So so not be be because I I understand what the the the scenario that you that you are talking about, you know. If you have if you are in a new code base and there is some bug fix to do, if you are going to explain the situation correctly, you know, maybe you know, you you have you have a good chance that the LLM will fix it if you explain that correctly even if you don't know the code base. So that's not what I'm talking about learning.
Dominic:I'm really talking about, let's say, I want to learn I don't know, you know, advanced mathematic, topics, for example. Mhmm. So since I don't know them, I don't I don't even know what to ask for. So they are not they are not like intelligent tutor, if you if you you know what I mean by that. So they they they so what what I'm trying to say is that when you don't know a subject, it's pretty, pretty hard to to get, you know, a very, very helpful response.
Ramesh:I think I see where you're coming from. But my slight pushback would be, again, if you're learning if you're trying to learn advanced mathematics, right, so so you would you wouldn't just go and say, I wanna learn advanced mathematics. So first you would start figuring out, okay, what do you mean by advanced mathematics? It is calculus, it is something else, it is just the Pythagoras theorem or something like that. So when you define that, and then also you're like, why do you want to learn this advanced mathematics?
Ramesh:So it may not be able to completely teach you, but at least it's going to guide you and then give you enough initial steps, maybe a lesson plan. You still have to go do the work, right? Unless you do the work and put pencil to paper, or pen to paper and things like that, you probably are not going to grok all the mathematical concepts. But it definitely is a tool where you can ask it some questions, and and it's totally new. Right?
Ramesh:So you're learning advanced mathematical concepts. You can start with what is this, help me define it, and things like that at some point. Yeah. That's how I view it.
Dominic:No. Yeah. Maybe maybe my example are just are just not very good because I I I did not really prepare for for that scenario specifically. But but, yeah, let let's just let's just say that Yes. For for sure.
Dominic:When you know what you need and when you know how to explain yourself and when you have a way much better chance than up you know, to obtain something of value compared to when you don't know something, you cannot even tell if it's going if it's the truth or not. So that that is that is mostly where I was going with that.
Ramesh:Got it. Okay. So it could be trying to fool you because it's always trying to please you and
Dominic:that's Exactly. Exactly. That that yeah.
Ramesh:Okay. So, yeah, I I get that. Yeah. I will give you that.
Dominic:Yes. Mhmm. But, again, if you if you know and and this is this is this is something that I started to to talk the I don't I don't recall when, but with Morten the other day, I was saying it's very rare for me to use Claude without going to plan mode. You know, for for me, it's always like plan mode. I I go into crazy detail, give a very precise code example and whatnot.
Dominic:And after that, I really carefully read the plan. And I often don't let the first plan, you know, go with it. I I I continue to chat and things like that. So I think I think the efforts that that you are putting is greatly, going to, influence what what you're getting as a result for sure.
Ramesh:Right. So even if if it let's let's say we didn't have access to any of these. Right? So so when you start building a system, you would still sit down and diagram, try to think through in your mind about, okay, here are different systems, this is what I wanted done, and things like that. You have a general idea.
Ramesh:You would start there and then you would go write main. Go or something like that and slowly build upon it. So the planning, I'm thinking the LLMs kind of force you to get into a planning mode because you know that spitting out code is now not the bottleneck, but getting the design is correct.
Dominic:Right.
Ramesh:Right? Yeah, so I'm also building a game. I'm not a game developer. I'm just a straight application developer. So I started using this engine called Godot.
Ramesh:It has a scripting language. It's very similar to c sharp. Yeah. So Yeah.
Dominic:I know. I know it. Yeah. I know it. I I I was using the the Python version of the script before they adopted the the c sharp version before.
Ramesh:Ah, that is great. And I have made tremendous progress just by prompting and then looking at some of the code very little. I'm almost, if you will, YOLO ing this one. And then so far, it has made good progress. I'm not saying it is perfect, there are bugs, I need to fine tune it.
Ramesh:But what I found was if you work with these tools in a focused manner, right? So my game has about six levels. I start with level one, I want just this one thing in there. Let's go build it, and then it starts building it. And again, I went through the tutorial, tried to build something.
Ramesh:When you're building a game, there are so many different things that you need to take care of. You're not even familiar if you're an application developer, right? It was just overwhelming me. But using Cloud Code, I got so much momentum on this game that I'm now excited to continue working on this one.
Dominic:But how do you know yeah. Yeah. I hear you on that. The problem I have with everything that I I did so far with with Cloud Code is that at some point, it's it's working. All is good.
Dominic:When I start to look at the code, very very often I have an n plus one issues. It's doing something in the loop. So in your case of your of your game, you know, are you really looking at what they are they they are doing in the update and the draw loops and whatnot because often it's not going to be what you would have written yourself. And and this is the experience I have so far with those LLMs is that of course it's working. But when you are really looking at the code, it's sometimes, it's not very good.
Dominic:It's just not performant. It's not going to to scale or there there is a lot of issues regarding Yeah. It it's it's doing a lot of things in in in repetition, for example, in a for loops. Just take the example what I was doing. Yeah.
Dominic:Let let let let's say let's say just just importing something. It was it was going to I've I wanted to have some kind of import process. So let's say a CSV import. And in the CSV loop, it was it was doing a SQL query instead of doing that before and things like that. So my point is
Ramesh:Mhmm.
Dominic:It's working, but, you know, it's not it's not always the optimal code.
Ramesh:Agreed. But you got to a point where you had a semblance of program that was processing the CSV, right? So if you were to hand code all of that, would you be bogged down by premature optimization and things like that? So that's where I come from saying, okay, we have a problem, let's generically solve this first. Yes, there are places you need to pay attention to.
Ramesh:I mean, why write, I mean, is SQL in a loop? Yes, that is something that you got to pay close attention to. But you had a working program within about five minutes, and now you can start looking at the program deeply and trying to understand where the optimizations could come from, and then also create instructions so that the next time you're doing something similar, it doesn't fall into that trap.
Dominic:Yeah. Yeah. Sometimes not. Sometimes yes. So so that's that's the thing.
Dominic:And and this is this is where we we started to discuss Morten and me. So yeah. So I agree on that. Mhmm. But sometimes the amount of of time that we save might not be beca especially when when you don't do the the the, you know, the extreme decent review right away.
Dominic:And you let it cook for a couple of days, and now you you're just happy because now you have the the sentiment of, oh, it's going well. I am I am, you know, advancing in my project and whatnot. And at some point, you say, okay, that this is this is the time to look at what what we have now. And now you have so much things to do that, you know, this this phase is extremely not fun. So, you know, arguably, it's probably just a workflow issues, you know, maybe maybe the optimal workflow is to is to have have it write some code, immediately check what what we have and now do do a kind of back and forth.
Dominic:But that to me is is very hard to to do. Because personally, I would I would prefer to focus. I feel like since since I'm using Cloud Code
Ramesh:Mhmm.
Dominic:I I'm never I'm never in in that spot where I was before when I was writing a software system where I was deeply focused, deeply concentrated on what I was doing, and just writing code at at a, you know, at a decent pace. But, you know, this this feeling is is is I'm missing that a lot. So so so yeah. May maybe maybe it's my workflow. Maybe I I I I should really, you know, be be more careful and just review everything.
Dominic:But, again, I have difficulties to do that.
Ramesh:So, yeah, let me try to parse that a little bit. Yeah. It could be the workflow. Or are you saying you're having a little bit hard time letting go of the joy that you get out of cranking out code?
Dominic:Yeah. Yeah. That that is for sure a little bit, but, yeah, a lot, in fact.
Ramesh:So should we elevate that to, okay, I know code can be cranked out by this tool. Now, am I getting more joy because I'm solving the problem and I'm also trying to look at how quickly I can produce good code, right? I'm not saying LLM is going to produce the absolute best. Even human beings, right? So I write some code this week, and next week I come back from vacation.
Ramesh:I'm going to look at the same code and say, who the heck wrote this code, right? Even human beings are sometimes nondeterministic. I mean, says LLMs are nondeterministic. Right? Yes.
Ramesh:It is. But even human beings are.
Dominic:Yes. Yeah. Yeah. A little bit.
Ramesh:But Okay.
Dominic:But it's for me, it was kind of rare that I especially for Go. You know, especially for Go, I I What? I I just revived my old static back end. It's it's been one and a half year. I returned into the project and everything, you know, everything seems to be good.
Dominic:There's a lot of things that I I would I would like to clean up and whatnot, but it's never as bad as when the LLM generated something that I did not like. I I I I just exited a huge three or four days refactoring Mhmm. Of of a of a Go server that I I would I did not wanted to to write it myself, but, you know, if I knew that I would have done this this three, four days refactoring, I would have written that, to be frank. So in a in a sense, I I don't care about the TypeScript code that I need to generate at the moment. Mhmm.
Dominic:But when it when it comes to Go, I have a huge diff difficulties. Even even though, you know, I'm trying to have a a good a good cloudcloud.md file and and, you know, trying to let it know my style and whatnot. So far, I haven't got great result with Go. And I was curious to to hear you you're you were saying that at your new company, it's not Go. So have you seen a difference between Go and where you are at the moment, which I don't know what what the language is, but have you seen the difference?
Ramesh:So I'm really surprised that LLMs can produce good Go code because Go is such a simple language, right? And then you do do while versus while, next, or for loops, or things like that in various languages. So in my current place, we do not use Go, but we use C, and Node. Js. None of these things I have had extreme experience in recent.
Ramesh:I was a dot net developer before joining the company that did Go, so did Go exclusively for four years. But I'm seeing that it does produce a lot of slob and things like that, but I can catch it even though I'm not a node developer or a Python developer or a c sharp developer.
Dominic:Okay. What what is what is your workflow exactly? How do you how do you attack different different aspects? So let's say let's start with the Mhmm. Well, while fixing fixing bugs, I found I found that the LLMs are pretty good at it.
Dominic:If you describe the situation precisely, you know, that I haven't seen much of code that I would not have written myself. When it's very very simple and very, you know, it's when you're able to explain it clearly, I haven't seen anything, you know, out of the ordinary. For a new feature, though, I would I would be curious to to hear what is your process.
Ramesh:So there are a couple of different projects that I did, and it was highly accelerated. So wanted to build a so I work in the health care space. So there is a lot of different kinds of documents that are produced within the health care domain. And these could be PDFs, XMLs, and text files and things like that. And I wanted to build a system that would take all of these from different sources and do a comparison of source A has 20% data, but source B has 80% data, so source B is better kind of thing.
Ramesh:That's the answer that I wanted to get to. Now to build a system, I sat down and then got into a plan mode first. So I started describing, here's what I want. This is what the input looks like. I want to ultimately be able to compare sources of data.
Ramesh:So I went back and forth, then the LLM suggested that we've got to standardize on one format. So we take all the different inputs and put it into one format. And then did some research and settled on a government standard that everything would be converted to, So once I had that plan working, then I asked Claude Code to give me an overview of what are the different assets it was going to create, what language was it going to create. I was partial to Go, and I said, can we use Go? And it came back and said, no, Python makes better sense because Go's XML processing is limited and things like that.
Ramesh:So that's how I started with very minimal stuff and built upon that, starting with, okay, I accept this document, what do I do next? I want to process it, break it down, and I had to develop all of this code and peek at the code once in a while just to, because I'm not a Python developer, but the general sense of the flow and what the program looked like makes sense to me at least. And then this is still an idea that we are trying to evaluate. Can we accomplish this? And all of this, we could do that within about two weeks.
Ramesh:So this takes documents, it breaks it down, it converts into a standard format, it then creates Parquet files, and then it also slaps iceberg on top of it. And there are 15 to 16 containers, all of this running locally. Right? So I could do all of that within two weeks. So that's the acceleration that Cloud Code gave me.
Dominic:It took what? Like a one hour, two hours with with Cloud Code?
Ramesh:One hour, two hours? The entire project took about
Dominic:two weeks. It took two weeks. Okay.
Ramesh:Two weeks.
Dominic:For Cloud Code? Wow. I'm I'm surprised. Okay. That much?
Ramesh:Yeah. Because there's a lot of back and forth. Okay. Yeah. Yeah.
Ramesh:It's it's a complicated system. So you're doing a lot of work. You're doing PDF processing. You're trying to download models and then have it process the PDF, and then PDF processing is notorious. Yeah.
Dominic:How many how many time would it have taken you, you think? You know, roughly, like one month, one month and then half, two
Ramesh:months? Much, much longer. First thing is at least six months.
Dominic:Yeah. Wow. Yes. Okay.
Ramesh:Yes. So even six months to even validate the idea, when you look at it from a person with a budget, that's a lot of money for them, right? Six months and then you don't even know whether something is viable or not, versus in two weeks we got something, we then can start interrogating this architecture. Okay, it is 80% there. We can now spend time to really try and productionalize this.
Dominic:Okay. But what it means to productionalize this, does it means now you you would you would take more time to build it?
Ramesh:A little more time, yes, yes. Okay. So even if you had built the prototype in six months, right, getting to production was not at the end of the six months. I mean, know most organizations, you start with the POC and the POC ends up in production. But
Dominic:Yeah. We've all done that.
Ramesh:Right.
Dominic:Oh, that's interesting. Okay. So so, yeah, it's it's way more than than a than a feature. We are we are talking about a a full system in a sense.
Ramesh:Okay. Yes.
Dominic:Okay. Okay. Okay.
Ramesh:Yes.
Dominic:I'm I'm using plan mode for maybe maybe I'm I'm doing that wrong. I don't know. I'm I'm I'm usually cutting down very, small piece of of a functionality. Let's say I have a let let's let's take something like a CRM, for example. Let's say I I want to add a a CRM module, a simple a simple CRM.
Dominic:So I would I would probably cut that into one or two plan
Ramesh:Mhmm.
Dominic:That I I would I would I would use the plan mode to do very very specific things. And and so am I using the plan mode wrongly? Is is it is it there to to because from what I I understand from what you said, you you had one plan for for the entire two weeks of work.
Ramesh:Oh, one plan that was constantly being interrogated and fine tuned.
Dominic:Okay. Okay. Okay. Interesting. Okay.
Dominic:I I'm I'm using plan for very, very smaller smaller piece, like something that would have taken me maybe four hours of work. Mhmm. This this is a plan for me now.
Ramesh:Got it. So another example was there is a feature that we are trying to build across couple distributed systems. Actually, different systems that all need to talk to each other. And I wanted to try and figure out how to add this little feature into the lowest system so it could percolate up to all the way up to the other systems. And remember, I'm new to the organization.
Ramesh:I have no idea any of this code. Half of it is in node, half of it is in dot net. What I was able to accomplish with Cloud Code was get the source code and start from, so these are all event based distributed services. So you start pulling the thread from one end. You know here's the starting event, something else happens, something next happens after that.
Ramesh:You slowly interrogate the code, and I was able to understand everything that was going on and put together that feature pretty quickly.
Dominic:Yeah. Yeah. That's interesting, for sure. Was the company already used L and M's or everyone else is using that as well?
Ramesh:Unfortunately, we are still in the nascent stages at this point. Not everyone is using it. A lot of the senior folks and architects are using it. It is being still evaluated. And especially being in the healthcare space and working with PII and personal identifiable information, health information, got to be a little more careful.
Ramesh:We are very careful not to submit any PHI to LLMs.
Dominic:Yeah, sure. Yes. Sure. That's interesting. Yeah.
Dominic:Do you have any tips? Maybe, you know, I I don't know. Any any anything that you have seen so far that works really well?
Ramesh:So here's a surprising thing that I discovered personally. I've heard a lot of people use Gemini and Codex and couple of other models. But personally, for me, the Claude Sonnet models and the couple of Claude models, these have been the only ones that I got very good results with. And the other surprising thing was hooking up Claude models with GitHub Copilot agent versus Claude Code agent. There also is some difference.
Ramesh:I got better success with the Claude code agent than with the GitHub Copilot. The tip is, if I may call that, use the plan more exclusively and don't be seduced by the fact that it's gonna quickly get you something out the door. Right? So you ask it to I wanna add a web page, and it it shall have these two fields. It can do that in about five minutes.
Ramesh:But sit up front and then have a plan. Okay. What is the purpose? Understand the business requirements. Then you will come up with a much, much better solution.
Dominic:Yeah. Totally. Yeah. Totally. I have you have you had some difficulties with I I have see I've seen it I'm I'm also using Sonnet.
Dominic:Right? I I I I don't have good result with Opus myself. I tried it a couple of times. Mhmm. I don't know I don't know why exactly, but it it seems to it seems to be way overkill for the way that I I currently use these tools, which is, like I was saying, I'm used to to chunk the the work that I need to do very, very small, very tiny things, and I feel like Opus has difficulties to understand what I what I want.
Ramesh:The what I found was and then I don't use Opus because it is 3x compared That to
Dominic:as well.
Ramesh:That as well. But I've used it in very limited cases when I'm trying to understand design problem. Definitely not for cogeneration. For cogeneration, I just slip back into Sonnet. One other thing that I found really interesting was you start with a little prompt and then slip on to HaiQ, the HaiQ model, and ask it to improve the prompt.
Ramesh:And then switch to the Sonnet model and give it the prompt.
Dominic:Interesting.
Ramesh:Yeah. And so at my previous company, we also used CodeRabbit AI for all of our code reviews. So it was hooked up into our GitHub actions and every PR that was submitted, CodeRabbit would do a review. And, that was pretty good. There was a really interesting problem, with something to do with an array.
Ramesh:I mean, this is all Go code, array within a loop, and it was able to find that niche kind of problem. Yeah.
Dominic:So is it is it what you what you refer to when you say that you are now using the the the cloud model in GitHub Action? Was it to review PR and whatnot? What are you using it for?
Ramesh:So, yeah, CodeRabbit AI was a tool that has its own model, and it was purpose built for code reviews. So every PR, it goes through the code and does a code review and it also learns so we can teach it. So it's going to come back and say, you should not write a sequel and such and such a thing. Especially in Go, it might suggest, hey, why don't you use an ORM, Right? So in in the Go domain, we don't like using ORMs.
Ramesh:And then we all know, you know, the ORMs may have a purpose, but anyways. So we teach it saying, you know, in this domain, we don't use ORMs. We use SQL, blah blah blah kind of thing. Yep.
Dominic:But that was at at your last Yes. Company. So now now you you you you said that you replaced the GitHub Copilot for the GitHub Action with with the cloud, you know, one of the model. What was was it also to do code review?
Ramesh:Kind of. Yes. Yes.
Dominic:Or or it's the feature that that now you can assign I I think you can assign GitHub issue to an LLM now, and it's supposed to try and do that?
Ramesh:That is true, but we do not use GitHub for our source code in my new place. We got internally hosted GitLab and a couple other Azure DevOps and things like that. It's a mix and match of several things.
Dominic:Okay. Okay. Well, at least it's not the Team Foundation server, which is
Ramesh:Oh, no. Some point, people must have moved on before my time.
Dominic:Totally.
Ramesh:Yes.
Dominic:Oh, that's that's interesting. So so yeah. So, again, we we might have sound a little bit negative, but it it's not really that. It's just that I I don't know. We we are still trying to see exactly where it fits in our day to day workflow.
Dominic:And and sometimes, you know, for generating tests, for example, I I I've seen it deleted some code. I I've seen it done really strange things. I'm I'm but, you know, everyone is talking about that. So I'm that to me is is not very out of the ordinary, and I'm I'm expect I'm expect that, you know, it's it's still it's still just a tool at some at some point. But and I see that it is true when you start to use it.
Dominic:Like I was saying, before October 2525, I was not using LLMs myself at all. And it's true. I I've seen it especially because I I don't I don't want to build any kind of of UI and front end code myself anymore.
Ramesh:Right.
Dominic:So for me, it was it was that at first. And at some point, I say, you know what? I will I will try to to let it let it go and generate a a quick Go server. But, yeah, may maybe I was not really, you know, careful enough or not giving as much attention and things like that. So so so again, I understand that we do have our, you know, fault in in this equation, but it it's not always roses, for example.
Ramesh:Oh, definitely, definitely. Yeah. Any tool for that matter, right, it's not always roses. People say you have to use Rust, and then it's going to solve all problems. And that's not the case, right?
Ramesh:Yeah. And, or use Go or Java or any of these dogmatic proclamations are always wrong. But what I wanted people to take away is this is a useful tool. This is a tool that really you should be spending some time trying to understand the limitations and trying where to use it, and it definitely is going to accelerate a lot of things. Yeah.
Ramesh:Yeah. Unit test is going to be a little bit, tricky. And and, like you pointed out, I did find certain cases where it would just write something and then mock up stuff, and then the whole point was to pass the unit test.
Dominic:Yeah. Yeah. Yeah. I I I would love to have a more how how can I say that in English? I would I would love to have the LLMs, you know, challenge me way more than what it is at the moment.
Dominic:Not always be agreeing with what I'm saying and and and be a little bit more opinionated.
Ramesh:Right. Yes. Now just to flip that a little bit, can you critic yourself? And and you know that the LLM is always gonna agree with you, and it's trying to please you. Right?
Ramesh:So if you go in with that premise, can you then ask it to be critical?
Dominic:I don't know. May probably not. I I would probably not like that. So I would like that and I would not like that. So I don't know.
Dominic:Maybe maybe one one last point before we we wrap up. So Yes. I'm a little bit concerned about what happens now when teams start to use LLMs in a way more and now we also have the review process that is also using LLMs. So who's going to who's going to be responsible when something is bad because let let's say, you know, the Amazon website was was down. I don't know when whether it was it today or yesterday for six hours Mhmm.
Dominic:Because of code that that that was sent that that was written by the LLMs. Notepad on Windows had this crazy, you know, c v c v Mhmm. Security hole in in in where it was using the the ShellExecute and whatnot. So I mean, those those those bigger organization, those companies that I was in the impression that was doing a lot of code reviews before code gets in production, what happens now that it seems like, you know, there there's going to be a a huge problem.
Ramesh:Oh, definitely. Yeah. I'm not gonna disagree with that one, especially. But what I fear more is leaders seeing this as the new hammer and getting rid of people without also grooming younger folks. So these tools are very powerful in the hands of senior people like you who have experienced all kinds of systems who have hand coded a lot of things, right?
Ramesh:So you can quickly catch them. But the junior folks, if you give them this one, they're going to go down rabbit holes and put something together. But we do not have a system where we need to teach the younger folks up and coming because that's the fear that I have, which is these management now looking at this immediately saying, no, we don't need junior folks, and there's no training program to bring these folks in an apprenticeship model or something like that. And yes, if you start producing tons of codes, things will slip through. If you have a large team because you want to deploy something within the next six months, you're still going to produce some crap code.
Dominic:Yeah. Absolutely. Totally. But, yeah, with with the amount of money that they are saving now, they they should they should leave the the junior developers with with seniors way more longer. Let's say, I don't know, six months for example that like like you said, the apprentice model could could be reinvented or re you know, it was it was like that way way I'm not talking about software engineering.
Dominic:I'm talking about, you know, in in in the the eighteen hundreds and whatnot. The the way that you were learning your craft was to to be with someone that that was doing that for a long time. So maybe we should go there at some point.
Ramesh:I I completely agree with you. So what I was thinking was a senior person sitting with an LLM, trying to solve a problem, but also working with the junior person. Yeah. Or let pair programming probably should be brought back. So in which we have three people now, right, a junior, a senior, and the LLM.
Dominic:Yeah. At least that way, you know, at least that way we, you know, you know, there will be someone to replace us, replace us at some point because there will be, there will be a huge problem.
Ramesh:Yes.
Dominic:Right. So that that's interesting. So so, yeah, not an easy tool. Not not as easy as as, you know, the the big v VCs want us to to think that that is going to to be the the end of all problems. So that is that is another another thing that I have a huge difficulties with.
Dominic:It's all the the messaging around around what it is. It it's it's refreshing to see more and more people, you know, going the other the other way of the slope. You know, we are returning the the return of the balance here. I don't know how to say that, but, you know, we are seeing some more and more people that just say, oh, you know what? You know, this this is good, but it's not, you know, it's not going to replace any software engineer soon.
Ramesh:Absolutely. But my fear is the leaders don't think that.
Dominic:No. Yeah. But the yeah. There there's there's a lot of problem that the leaders are also thinking. It's that's that
Ramesh:Cool.
Dominic:Just stopping to eat all the the revenue of the company and just dispatch that to to the lower level could be could be nice as well.
Ramesh:Right.
Dominic:Awesome. But we cannot we cannot solve solve it all here. But but, yeah, again, this is this is interesting, and I I I think if if anyone is that that is listening isn't using that yet, well, yeah. May maybe maybe try to to jump. If you had the bad experiences before, those things are changing from month to month.
Dominic:And again, myself, I I have seen that the the smaller the the smaller thing that I'm asking, the better result I have.
Ramesh:Yes. Focused work, it's absolutely great. So one thing I wanted to ask you was you in episode seven and six that came out recently, right? You said you let the LLM cook for a couple of days, and then you have to now throw away all the things that it did. So Yep.
Ramesh:Was it literally running for several days?
Dominic:No. No. No. No.
Ramesh:You just had a lost context.
Dominic:No. It just mean no. No. Hey. No.
Dominic:It just means that I was not rigorous enough to carefully check at the code that it generated. And Ah. And and like you said, as soon as there is a couple of patterns that that get inserted, the LLMs will will replicate the patterns. It's normal. It's it's the way it works.
Dominic:So if I have one tip myself to get to give is that don't, you know, at all costs, don't let it introduce patterns that you don't want because, it will it will be it will be a lot to clean up.
Ramesh:Yes. It accelerates you, but it again doesn't take away your responsibility of the code. Ultimately, it is a tool. You are responsible. And if you YOLO it, it's going to come back and bite you.
Ramesh:But if you put in good workflows where you introduce the right, maybe review step points and things like that, checkpoints, or maybe it's time for us to write the unit test. I don't know.
Dominic:Yeah. Probably. Yeah. Unit test seems to be golden because if if you if you think there was a a lot of controversy regarding the the claim that Claude built a C or C plus plus compiler. Don't recall exactly, but the the the test was all there.
Dominic:Mhmm. So maybe the test now is is worth a lot.
Ramesh:Absolutely. Yep. So maybe we we all become testers test writers, test creators.
Dominic:Yeah. I I I was enjoying writing code myself. This is so I I I I told Morten that as well. So at at some point, I I let I let Cloud Code wrote a a Gyxtra algorithm to to to find, you know, for path fan finding and whatnot. So I I was doing we we play board game, basically, and and we play a board game that I wanted to play, and I cannot see the board game anymore.
Dominic:Right. So I I needed I needed some way to to calculate distance between points and I would have loved to write it myself after the fact. So so now now when when, you know, I I will still keep those things that I don't know. You know, the things that I don't know that I Mhmm. I want to to learn, I I don't think I will let the LLM write write them.
Dominic:For me, it's it's it's still where the fun is of all of this. So I I have a a huge time to let it go. I feel like I'm learning way less now.
Ramesh:Just because of the LLMs? That's interesting.
Dominic:Yeah. I'm I'm not I'm not reading the documentation as much as I did. I'm not the fact that I'm not writing code, I I'm I'm losing a little bit of of muscle memory that I had the other day. I I was not remembering exactly, oh, what what what is what is the the way of of printing an error? You know, I I I was like, wait.
Dominic:It should be string. Right? No. It's error. But it was gone.
Dominic:It was gone from memory. I'm getting old. I'm getting very old now. But but the point is Mhmm. It's true that I I have seen it.
Dominic:I have seen it. I have felt it. There there was a couple of skills that evaded me, and I was like, oh, no. No. No.
Dominic:No. No. No. That is that is not happening.
Ramesh:That that is true. But again, it depends on where do you want that dopamine hit coming from. Is is it cranking out this code or solving problems? And that's what it's gonna be.
Dominic:Well it's both for me but I want to I still want to write code especially if I never wrote it. You know for for the for the code that I I've been written you know since since like ten years I don't I don't care. I don't care. Let's go. Write it for me.
Dominic:Thank you. Thank you so much. I like you a lot. But anything that really interests me that I'm doing it for myself, I'm not sure I would I would I would prefer to let it let it even even though I would have the impression that, okay, I had this problem and now this problem is fixed.
Ramesh:Mhmm.
Dominic:I'm not sure I'm not sure it's as good for me When when you talk about dopamine, I'm I'm not sure I'm getting as much dopamine, much dope, if you if you will, as writing it myself.
Ramesh:I mean, yes. You can go to IKEA and pick up a couple chairs, or you go buy the wood and then hand craft it, of course that is going to give you more satisfaction. Yes. But if your goal is to create this one beautiful chair, yes, you got to hand craft this. But if your goal is to have 15 chairs so you're because you're having a party and things like that, go get the chairs from IKEA.
Dominic:I I I would build mine and and make made it build the the 49 office. Correct. Yes. Yeah. That that is that is where I am at the moment.
Dominic:So Absolutely. Cool. Yeah. But but but again, I'm I'm with you, and I'm not as negative that that I might sound. And maybe negative negativity is is easier to to say on on the podcast.
Dominic:It's, you know, it's pretty obvious that we are using it. So obviously, it's not it's not all bad.
Ramesh:Right.
Dominic:But, yeah, let's let's be real, and this this thing is working. This thing is doing a lot, and it's going way faster than me, for sure. There's no question there.
Ramesh:No doubt there. Yes. Dominic, thank you so much for your time.
Dominic:Thank And yeah, let's keep in touch.
Ramesh:Sure. Have a great day. And, again, once again, thank you for giving me this opportunity. Alright. See you later.
Ramesh:Bye.
Creators and Guests
