go podcast()

Dominic: 00:01

Hello, gophers. I'm Dominique Sempial, and you're listening to Go Podcast. Today, I'm joined by Jakub Yarosz, and we talk about DevOps, Security, and Testing. Hello, gophers. So I'm joined today by Jacob Yarosz.

Dominic: 00:22

And I hope I say your name correctly and, Perfect. That's good. Yeah. So thank you very much for accepting, this invitation.

Jakub: 00:33

It's my pleasure. Thank you, Dominique, for inviting me for the podcast.

Dominic: 00:36

Very cool. So as always, the first thing that I'm ask that I ask, to guests is to, you know, introduce themselves. So whatever information you want to disclose, so we know you a little bit better. It could be what what I typically like to hear is is, you know, where are you coming from in terms of software engineering? You know, what what was your path?

Dominic: 00:59

How did you, you know, came across Go? And, you know, what are you doing at the moment? Where are you working? So any anything that you want to say us, it it will it will be very appreciated.

Jakub: 01:12

So my name is Jacob. I'm based in Ireland right now, so in city Drogheda. It's 50 kilometers north of Dublin, and in my path to software engineering is quite interesting because my background is actually the electronics and industrial automation so this this that's my my education was in this in this direction. So before moving to Ireland Twenty Years ago I was mainly involved in the IT and mainly electronics and automation systems working back in Poland many many years ago. So then when I moved to Ireland I was mainly involved in hardware business at the very beginning and later on transitioned to the test automation and software engineering slowly sorted.

Jakub: 02:01

In terms of Go actually my first, my first time with Grow was like good seven or eight years ago when before I joined the Salesforce almost seven years ago. At that time, I was working for the company called Axway and we were automating the VMware setup in our physical data center and then with a colleague of mine we actually decided hey let's use Go to create the API wrapper. Right? Because before this main language that I used to work was was Python and Ruby. So we were doing most of the stuff with the Python and Ansible at that time, and then it was good opportunity to actually try the Go.

Jakub: 02:51

So it was my first, first time. And then and then when I moved to Salesforce that I spent four years, I would say that 80% of time I was writing the Go, mainly around the system security tasks. So command line applications, different parsers, small services, but there's still remaining, like, 30% was Python. But it was the interesting time, when the Python three was, basically, going more and more popular, and it was the final call for migration from the Python two to Python three and then with many teams we were facing dilemma, right? Shall we start porting our old Python libraries and Python applications to Python three and in most of the cases actually the Python become Go.

Jakub: 03:51

So that was like a real real, multi team projects that we were porting certain applications from Python to directly to Go instead of moving to Python three.

Dominic: 04:04

This is very interesting. I, yeah, I've heard that a lot. I I must admit, there is a lot to impact, before we we we we start. I I really like, a couple of things that I I want more information. When you say hardware, I mean, I'm very curious.

Dominic: 04:20

What what are we talking about here?

Jakub: 04:22

In terms of Salesforce, so directly, for example, the interacting with the iLO interfaces. So provisioning, booting up machines, provisioning machines. Interesting. Before before this, it was mainly operation with the with, data centers.

Dominic: 04:41

Okay. So so this this is what you were saying before you you started as a software engineer. You were mainly in oh.

Jakub: 04:48

Yes. Mainly mainly hardware. Mainly hardware and small writing, a little bit of the scripts in Python but mostly was the electronics mainly, writing the automation systems, a little bit of PLC programming and automation system for the electronic scales. So I used to work in Poland in the company that we were designing and the installing the different electronic systems mainly for the measuring the weight of the different devices like production line in Coca Cola, for example, in which. So we're installing different systems and then integrating with the existing IT infrastructure, mainly networking, in order to gather the data and the readings from these electronic scales to the main databases of different companies.

Dominic: 05:37

Very cool.

Jakub: 05:39

Yes. Very, very inter very interesting. Very interesting.

Dominic: 05:42

What made you what made you, you know, kind of switch to, to more more software, you know, writing more code?

Jakub: 05:50

That was actually the the the funniest story is because I moved to Ireland. I came for the six months English course, and the six months become twenty years at this moment. So so after It's

Dominic: 06:05

a long course.

Jakub: 06:06

Yes. Exactly. Exactly. So so I decided after this half a year not to come back to Poland, but actually start my new chapter of my life back, twenty years ago in Ireland. And they joined the company at that time, the startup, actually they were producing the huge laser machines for cutting some components, mainly it was around the noses for the ink printers at that time.

Jakub: 06:36

And it was Irish startup very successful. We were sending these machines around the world for the different companies that they were producing these ink jets, ink machines, and it was very interesting from this this point of view that I was working mainly with hardware but the software components were very very big part of the entire system. So interaction between the hardware and the software that's really what sparked my huge interest and then gradually because some different opportunities As far as I remember at that time, I started moving more into actually the software, programming different, test automation components for these robots that they were responsible for the lasers that they were cutting these devices.

Dominic: 07:24

Wow. Yeah. That sounds that sounds very, very cool. So, yeah, I can I can imagine that at some point if you're, you know, working a lot with with the hardware and and the software is controlling, the hardware, it it's it's kind of intriguing a little bit to see, you know?

Jakub: 07:41

Yes.

Dominic: 07:41

Yeah. I can I can understand that?

Jakub: 07:43

Yes. And it's very, very interesting from the point of view that you can basically see immediately the results of your work. So you are programming something, and immediately you see how, for example, robot behaves of it behaves correctly or not. So it's it's really cool.

Dominic: 08:01

Yeah. The the there's no line that is printed to the console. Now it's a laser that shoot shoots at something, which which is nice.

Jakub: 08:07

Yes. Yeah. Exactly. Exactly.

Dominic: 08:09

No danger. No thing.

Jakub: 08:10

Or some camera, for example, inspecting the wafers.

Dominic: 08:15

Nice.

Jakub: 08:16

This this kind of stuff. Yeah. Back then was visual basic, basically. That was

Dominic: 08:21

Oh, yeah. The good old the good old VBS.

Jakub: 08:24

Yeah. Visual basic and before this visual basic for applications.

Dominic: 08:28

Oh, yes. Yeah. Yeah. This is this is where I started as well, but, ouch. I I don't really miss those those those days, you know, that it it it was still interesting, but the dependencies, I remember, were were kind of crazy in that environment.

Jakub: 08:44

Oh, yes. Yes. I still remember, like, moving a little bit between the teams when I was responsible for actually producing the software and builds and we are using this, don't remember right now, this set was some Norton application to basically create this distribution of software. We are burning x amount of CDs that were attached and distributed later on to the to customers. So it was quite, quite interesting but comparing nowadays with the with current systems, it was quite, labor's work.

Dominic: 09:22

Oh, yeah. Yeah. Yeah. Totally. Yeah.

Dominic: 09:24

When when people are complaining about NPM, because an NPM on the on the JavaScript ecosystem is is kind of notoriously hard around dependencies. But it I mean, it's it's nothing compared to what we what we had to do, like, in in, you know, her 1998, '90 '9, February. It was, it was it was a little bit more difficult. And like you were saying, if you were burning CDs and and, for for some reason, the the the there needed to be an update quickly after that. I mean, wow.

Jakub: 09:56

It was it was really manual work at that time, I remember. I I recall one time that it was some very quickly patch discovered in some c plus plus application that was driving some robot, and we had to quickly, with with colleagues, re burn, multiple numbers, many numbers of the CDs and then distribute around the world. Yeah. Customers could actually use this new software.

Dominic: 10:21

So I don't know.

Jakub: 10:22

There was a shipping company was involved, so nothing no Internet.

Dominic: 10:27

The good old days.

Jakub: 10:29

Yes. Yes.

Dominic: 10:29

Yes. So okay. So let's turn a little bit. So you you wanted to talk about Go and security. I'm I'm I'm very interested, into that.

Dominic: 10:39

So why Go exactly? What what makes Go a good choice, you think, for attacking security problem, for example? And what are you doing, with Go and security, actually?

Jakub: 10:52

So the back in Salesforce, we were writing the different, different applications for, for example, validating the AMI roles in AWS or the setting up the Kerberos or different security systems. So different system needs to be glued together and provisioned and that Go at that time, I mean Go fit the bill very very nicely comparing with Python at that time for the one specific reason. We were able to produce one single binary and we didn't have problem with dependencies that we used to have. Because imagine, like, hundreds of security people that they were involved in writing Python libraries, then different environments, different virtual environments, and we frequently spent a lot of time actually fighting these dependencies. Yeah.

Jakub: 11:50

Here we were able to produce one single binary, distribute this binary, create quick installation, inform the of the rpm and then the software was easy to you easy to use and ready to use without without problem of of dependencies. And as as well different, later on, back in my current role that I work right now for the Nginx and I focus mainly on security, As well, apart from the our main product, which is the NGINX ingress controller that is % written in Go, we are using as well the Go for writing different security checks, be this on the pipelines, for example, or for so called DevSecOps, tasks.

Dominic: 12:41

Right. Yeah. I've I've I've seen that a lot, lately to to be frank. Was that was that was that an acronym that I was seeing a lot? Like, I don't know, three, maybe four years ago, but but now these days, it seems like security is is exploding everywhere with with good reason for sure.

Jakub: 12:59

Oh, yes. Yes. Yes. Yes. Definitely, like, mainly the software supply chain attacks.

Jakub: 13:04

So this is the area that we are focusing the same like back in my previous company. So having systems that would be able to filter and validate these different dependencies that are coming to the final product, it's crucial and especially that these different security attacks evolve very quickly. It's not even about some DDoS attacks but actually getting some malware back to the system and then distribute the system among the different software and get the access to, for example, Kubernetes clusters.

Dominic: 13:42

Right. Yeah. Yeah. That's a a totally new, a little bit more involving than than just DDoS attack. I I I have some question about I'm not sure I understood correctly what the module that you're working on.

Dominic: 13:57

So so you're working at, you know, for for an an NGINX module, if I understand correctly. So what this module intercepts request and, I don't know, validates that they are good. Is it is it a key akin to what a web application firewall is in a in a sense?

Jakub: 14:17

No. We are not working on the NGINX module itself, we are working on the NGINX Ingress controller. So this is the piece of software that accepts the traffic from the outside and then distributes the traffic inside the cluster. Oh. So under underneath of the ingress controllers, they are the physical real applications.

Jakub: 14:40

So it could be NGINX or HAProxy or many different companies. They, configure their products to meet the requirements, Kubernetes requirements of ingress controller. So there are certain rules that this software supposed to meet and then you can use different software in order to configure under the hood to get the traffic from outside and then distribute traffic to the services inside the Kubernetes cluster. So we are working as well in our offering. There is the WAF module, but this is completely developed by other by other the team.

Jakub: 15:23

Mhmm. But we are incorporating, these capabilities in, by configuring, our our product in order to allow customers to specify, let's say, different rules in terms of the filtering traffic and verifying if there are no, certain attacks.

Dominic: 15:46

Right.

Jakub: 15:47

But no. It's not it's, we are my team is is not working on this product. We are integrating this product inside, the ingress controller.

Dominic: 15:58

Okay. Got it. Was there any discussion at some point? So why Go for this? Was c ever in the in the balance or may maybe Rust?

Dominic: 16:06

I don't know.

Jakub: 16:09

Probably, when I joined the project three years ago, that was already this project was around three or four years old.

Dominic: 16:15

Okay.

Jakub: 16:15

And I would say because of Kubernetes is written in Go, that was the natural choice to go with Go.

Dominic: 16:23

Yeah. Sure. Sure. What was have you have you seen any any pitfall, any challenges that that you have faced, so far with with, with, you know, with this, because I I I must I must admit that I guess that this thing needs to scale at some point. Because if it if it's dispatching the the the traffic, then it it needs to end all of it first.

Jakub: 16:48

The like a high level challenge, I would say, not only related to the NGINX and to this product that we are working on, but I encountered in the previous company, is the challenge of bringing applications that they are, they were born, let's say fifteen, twenty, ten years ago before this Kubernetes and the cloud environments they flourished and the workflows that basically use this old software needs to be adjusted to this very dynamic environments. Environments by dynamic I mean that scale up and down very very quickly. So it's different approach to, let's say, configuring, a different approach to manage, let's say, the cache between the different instances. So it's like a constant, regardless of the product, is the constant the headache, how to fit installation, for example, of the product that, let's say, ten years ago we were using VMs or bare metal bare metal servers. We could have some high high availability system configured on the Linux level, say, having, I don't know, x number of machines, and then handle traffic in this setup versus moving this different configuration that was done in the physical world to pod levels, put everything inside containers and make sure that more or less the same mechanism work right now in the cloud native environments.

Dominic: 18:46

Right.

Jakub: 18:47

So this is this is challenge. I remember like old Java application, not in this company, but old old Java application, as well the API gateway that the this application were shipped to customers as appliances. And suddenly customers started asking can we run this in AWS? Can we run this in the Kubernetes cluster? And then because the nature of these applications that they were, let's say, installed manually, they were reading in many stages different configuration files, they were not so easy automatically, let's say, scale up and down.

Jakub: 19:22

That was a huge challenge to put them and fit them inside the docker images and make sure that actually they behave in the cloud native environment correctly. Not only threat the docker image as the another VM or the clone of of your server.

Dominic: 19:45

Right. I and I guess also the challenge is to not have to touch any line of code of the of those legacy application because let's face it, almost nobody wants to touch them anymore.

Jakub: 19:56

Yes. Yes. Exactly. But even though even though the the, way how these applications are configured sometimes has the huge impact on the workflow or the amount of work that would be required to adjust them to, let's say, dynamic environments running in Kubernetes.

Dominic: 20:15

Right. Yeah. Yeah. It's it makes sense when you think about that. I mean, the it it was I I I kind of live something like that because, at first at first, let's say, that the system was was on prem even, and now the lift and shift is is kind of very, very hard to accomplish, to be to be frank.

Jakub: 20:35

As long as, let's say, this lift and shift is done in the way that you are not changing, any setup, but the nature of of of Kubernetes is basically to be dynamic and and allow users to scale applications up in order to, let's say, fulfill some traffic requirements or the amount of load that is coming in and then automatically scale down. So this is, like, truly dynamic way of of handling. Yeah. Environments. And on the other hand, they allow customers to, in theory, save money of not using the amount of resources that they would potentially be using with this application having the, let's say, physical servers.

Dominic: 21:22

Right. Right. So you you are using Go for automation. Can you can you give us some example of what, I don't know, what what you find comfortable, to use Go for for automating, I guess, tools and task?

Jakub: 21:40

So we are we are using the Go for writing the ingress controllers. So, basically, this is the I would describe this as the application that, takes, Kubernetes native, the objects, and then, performs different validation, and then at the very end assemble the final Nginx configuration file from these pieces of the Kubernetes native objects. So this is quite interesting, interesting the area where you would need to deal with the native objects, apply special validation, and, on the other hand, make sure that all rules that Nginx itself require from the configuration files, not only syntax but the business rules between, let's say, different virtual servers or different other components, are met. Otherwise, we would be getting error because the NGINX basically wouldn't start correctly. So this is the main, main, main challenging point not only for NGINX but for every single company that writes and develops the ingress controllers based on specification from the Kubernetes and then getting the specification to basically create the pieces of configuration for the product, either be this NGINX or HAProxy or any other web server that or the or the proxy that is responsible for getting the traffic, apply certain rules, and then forward this traffic back to the cluster inside the cluster.

Dominic: 23:25

I would imagine that this is kind of very hard to test. I mean, how do you how do you ensure you were you were talking about security earlier. Mhmm. I mean, is is is is this, a challenge to test this entire system?

Jakub: 23:41

There are there are challenges. Standard, of course, when when you write the Go code, the standard is that you have your own tests written in Go. They can be on the very different letters. So I'm talking right now about this standard unit test for the, let's say, your different methods or functions just to make sure that everything is working correctly. But as well, the main challenge is that in many cases in order to simulate the behavior of the Kubernetes, we would need to write extensive mocks.

Jakub: 24:24

There are some frameworks written by the community to simulate certain behavior. But at the end, when I look through the amount of work required in order to mimic some behavior, sometimes it's easier to write the high level tests using, for example, the test containers that we spin up, like, one one node cluster and then simulate the user interaction through the API.

Dominic: 24:57

Right.

Jakub: 24:58

And then and then verify in if certain components are created. So I found that this in many cases, it's much less work to do this. You interact with the real cluster and then perform the validation of correctly created objects, for example, of behavior, versus having very very extensive the setup of preconditions and then ending up with a lot of mocks that could behave differently based on configuration. So we have like, I would say, two levels of testing: normal unit test for configuration for validators and then the higher level tests where we spin up the cluster and then perform the tests. And then another third layer that we have, like a final integration test, we have the Python framework written to simulate as well the user interactions.

Dominic: 26:00

Interesting. So when when you say the Python framework, is it it's because there there's two languages to interact with?

Jakub: 26:08

That's the historical reason. When I joined company, it was already already this framework was in place. I know that the different different teams and different companies that they work on with the Go and Kubernetes, the ingress controller, they have both the layer of tests written in in Go. This this project, the test containers, it it helps a lot because it's, basically, simulates. You can quickly, simulate creating the you can create the real real one node cluster and then simulate the user actions, with very few lines of the go code.

Dominic: 26:53

Right. Yeah. Yeah. I I I I can imagine that it's, it's it's pretty it's more easy to do that that way. I I can see that.

Jakub: 27:03

Mhmm.

Dominic: 27:04

Do you have the lot to learn to, when when you say you you're there since, it's, since three years, so were you already all familiar with with with everything, everything that you're working on at the moment? The well, was there a lot to learn?

Jakub: 27:21

It it it it is still. There are some core dark corners of the of of of Kubernetes, that basically it's still require, to learn. And especially that, it's not only software itself, but the new features that they are being added to Kubernetes that customers are asking or customers are asking how to use the product in their setup. So there is there is no way you achieve the the

Dominic: 27:49

Right.

Jakub: 27:49

Some some stage that you would say, I mastered this. So this requires still constant refactoring or the making sure that these new features, they are coming to Kubernetes, which is basically released every three months and is very, very dynamic environment. We are still, basically are up to speed and are able not to even only write the write our Go code and adjust to the changes, but as well to answer our customers' questions.

Dominic: 28:25

Yeah. Are you using, TDD to to install those new feature? How how are you handling the, the development of that?

Jakub: 28:34

Yes. So myself coming from the QA and test automation background for years, for me, actually, the reading test first is the perfect way and and way to go to learn about the system. When I when I see the clear tests and clearly describe the action preconditions, I don't usually need the documentation. Assuming that these tests are written in the way that you can figure out what kind of the behavior of the system developer, for example, wanted to test or document behavior by by by using certain certain tests. So myself, I use test driven development.

Jakub: 29:21

So first, I write I think about, okay, what what kind of the behavior I want to implement? Then I write the test, then I write code. And this is like we know the cycle red green refactor. Yep. So if I don't know what I really want from the system, for me, it's quite impossible to imagine how I can start writing something unless I'm, like, hacking a little bit then checking on the, let's say, some, let's say, developing main function or some functions and hacking a little bit.

Jakub: 29:56

But when I have like a broad idea what I really want, I need to be able to express this behavior in in a test. I need to know what the system should be in the beginning of the state, what my, let's say, functionality developed should change in the system, and I need to know how the system should behave after this change. So knowing these answers to these three questions, it actually it's all what it takes to write failing tests at the very beginning, then fulfill this test with the writing software that meets this criteria and then see this this test pass. And it's like a small step by step cycle that allows the me to model behavior, making sure that behavior is is is, covered with tests and then move to the next step.

Dominic: 30:51

Yeah. I've I found this is the challenge, with with any any kind of test, especially with TDD to just knowing, like you like you are saying, knowing what the system is you know, what it should do exactly and how should it behave. Did you know, this is the challenge and something sometimes it's not clear at all and you're starting to to do your test. And at some point, you you realize that, wow, this behavior is not going to work. So so now you need to change all the tests as well.

Dominic: 31:23

So it's it's it's a pretty it's a pretty hard thing to do right. I I believe when when when you know exact I I was doing my last episode on Montes. So, when when you're knowing exactly what you need to do, this this is this is great. This is perfect. But, yes, exactly.

Dominic: 31:39

When when there's a doubt or anything that is unclear about the system, are you finding that doing TDD help you refine your initial thoughts or your initial understanding of the system or what you need to build? Or is it helping you at all?

Jakub: 31:58

Definitely. Definitely. And the problem that you mentioned that, let's say you change mind or you figure out that you will need to change a little bit behavior and then you end up, let's say, for half a day changing your failing test. So this could be the reason that many people maybe, they focus not on the testing behavior, be this let's say in go in Goland it would be exported function or exported methods, but a lot of, maybe effort is put into writing this very small test for per function each, not exported functions. So I usually start, let's say, when I start writing some function or thinking about some function, even not exported, I write a test, then when at some point this small not exported function would be a part of some exported behavior, I usually deleted the small test as soon as I have covered this with the larger test that checks the behavior of the system, that is, trigger this behavior by exported method or exported function.

Jakub: 33:17

So this this case my tests evolve when I'm basically not keeping the small test that tests the separate functions, but I keep later on only this test that executes and give me the answer if the behavior of the system that I could trigger using some public API for my package through the exported function is correct.

Dominic: 33:47

I like that a lot. I must admit. This is, this is pretty good, pretty interesting. Have you have you seen that, you know, when you are deleting those tests, are you are you is it helping you shaping the, the architecture of your code at some point? But but, you know, making sure that you're keeping your function short and things like that, is is it helping you in in designing the code at all?

Jakub: 34:16

It helps, but I must, say that when I do this transition from the, let's say, when I remove small tests for these not exported functions, I am thinking about what kind of the table tests I will create for exported function, and I keep an eye every time on the test coverage. Did, for example, this behavior that was at the very beginning exposed by the small function is covered with my, let's say, table tests that I use to inject certain data to this exported function? So then when I see, okay, this this is covered, I can remove the small test because, there's no point to keep this the same the same test for the same behavior. And just in case even to, as you mentioned, if something changed, I would need to have, let's say, 20 small functions that, anyway, that they they have to test that already larger, larger tests for the large exported function covers. So this is like constant, refactoring of the code and thinking about, do I have the test coverage?

Jakub: 35:33

Do I how this function should, just in case, fail? Under what kind of the input data? And in the same time, thinking what kind of tests are already obsolete, and I can remove them because behavior is already tested in my different test.

Dominic: 35:53

I like that. Is is your testing mentality is is from a long time ago? Because, we were talking about VB at some point. There there were no tests. There there was not even a a test suite coming with Visual Basic at the at the time.

Dominic: 36:10

So when when when have you have you started to

Jakub: 36:14

That was actually a funniest story. The testing for testing testing mentality, I would say, started when I was, like, maybe eight or nine years old, when my father bought me the first electronics, like blocks for playing with electronics.

Dominic: 36:28

Nice.

Jakub: 36:29

And, I basically mixed up batteries, and instead of connecting three volts, I believe I connected nine volts. And, at that time, system basically crashed in terms of this, my this, one of the circuits, got burnt. I only felt a little bit of smell of the burnt burning. And and, unfortunately, it was not something that could be replaced very easily because it was the imported at that time from the that was the time before before, actually, we were behind the iron curtain during the communist time. So it was quite tricky to get this particular circuit, but my father managed managed this.

Jakub: 37:14

So, since that time, I I always had the basically hard lesson always to put the multimeter to figure out if I really need to this particular voltage to this particular places and then transferring from the hardware, of course, to software. Right? If I really have something that is, that is useful to test particular behavior. So this this mindset comes from interacting with hardware that you can damage something Yes. And it's not so easy repairable, like, let's say, piece of, piece of software that you can change couple of lines.

Dominic: 37:55

Yeah. I I like that. I like this story a lot. I I imagine a couple of soft software engineers that do not really do tests. And if their codes were to burn at some point, it's because there's a bug in there.

Dominic: 38:08

So not only there's a bug in the system, but the the source code is is is damaging. I mean, that would be

Jakub: 38:14

That that was common bug I remember back in the zedX spectrum when I actually attended first the programming course with basics. I remember the issue that first we needed to connect the power supply to zedX Spectrum, then the to the powers to the socket. Mhmm. Otherwise, that could be a sparkle that basically would burn the motherboard, and I managed to do this once.

Dominic: 38:41

So

Jakub: 38:42

yeah. Yeah. Yes. Yes.

Dominic: 38:43

You are you are pretty dangerous as a as a boy. Right? Yes. Yeah. That that that was very funny.

Dominic: 38:51

What about let's say let's discuss that because, I mean, I I find that the test subject very interesting, to to me, I thought, at least. What about when you are jumping into some code and there's not much test? Is is is you know, what what is your reaction then? Because let's let's face it. Everyone is saying that everywhere on the Internet.

Dominic: 39:12

It's all it's always you always read the sentence like, oh, you know, I started working on that code base and nobody really trust the the test suit. It's it's a little bit less, less and less, true these days, but it's it's still happening. So what what are you doing?

Jakub: 39:32

That's unfortunate situation, but, my first thought is you are a little bit paralyzed when you try to change something, and I would say I remember this time, but then back then working with Python. So, basically, I spent countless of days with the PDB Python debugger to figure out how the values of particular variables change. And then, based on this discoveries, we were with a couple of colleagues trying to write the tests for this particular behaviour. But, in general, it's very risky business start refactoring or changing this particular software when we don't have test coverage. So I would first actually start thinking about writing some higher level tests to wrap larger behavior and then started experimenting and reading through the code, start noticing maybe certain patterns and then extracting this behavior into separate tests and then separate functions.

Jakub: 40:49

Right. Other other otherwise, otherwise, it's it's hard a little bit. Yeah. I'm just looking through the history. I remember one this kind of problem that we start started doing some refactoring back then with the Python code many, many years ago.

Jakub: 41:12

And it happens that, for whatever reason, the order of the some values in XML files, as far as I remember, changed, and it broke for some functionality. But I it shouldn't interior. Right? Because when the XML was basically parsed, into the data, it it shouldn't matter. But at that time I remember some problems related to this.

Jakub: 41:44

And then the nightmares and time wasted spent on this refactoring and trying to figure out why something is not working and trying to change something without the test. And it was always like, praying time.

Dominic: 42:00

Right.

Jakub: 42:01

Which is not which is not a great approach.

Dominic: 42:03

No. For sure. Have you have you noticed that, in the in the Go teams that that you you were part of the the test were kind of, you know, at least minimally always, required for any new development and things like that?

Jakub: 42:22

Maybe not. We have very, very, very good, I would say, the test coverage. The problem that I see with tests, not only in my company, right, but in the in the different code bases, is that people who write this test, they don't use descriptive names. What kind of the behavior is tested? Everything is okay until we have in our mind this piece of functionality that we are working on.

Jakub: 42:56

But even though next week, I would need to if I don't describe the tests in the, let's say, some centers that would describe behavior, even though the name of the test would be longer, But instead of saying test a b c or test name of the function s b c and then I have, for example, 20 or 30 different inputs, some of them make the test, pass, some of them make the test fail. When someone new is jumping to the codebase and especially when it's new to the domain, there is no way that he could figure out, okay, what this data input, why this data input to make this test fail, make this function behave incorrectly. And then, it involves digging into documentation, which frequently is not up to date, Mhmm. Talk talking with people, assuming that people who wrote this are still on the team. If they are already left either company or they are moved to the different team, then, it becomes the problem with creating the mental model of the functionality of the system.

Jakub: 44:06

That's the reason I'm big supporter of idea of writing the comments, but not comments or documentation, documents for the functions and for methods not to describe what the function does because everybody can read who writes, let's say, Go code, but when there is some underlying context, describe why, for example, certain approach were used, to give the broader context. So then, I don't need frequently official documentation for the code because usually these kind of quirky details are not documented. But when I read the code and I see this needs to be done this way because of this kind of bug somewhere, right? So, you can spot these even commands into the standard library in Go. It gives you the ability to create very fast the mental model why something is done.

Jakub: 45:08

So the answer, in my opinion, for the question why it's more important than what we are doing.

Dominic: 45:16

Yeah. Totally. Yeah. Totally. This this is this is a sane a sane approach to, to to have, yeah, in in in the comments.

Dominic: 45:24

So you you are going to say?

Jakub: 45:28

Yes. So this this as well is in situations that, you have some functions that take, let's say, the as arguments, some structs. And certain in many times these structs are quite large. And, for example, this function supposed to fail in your test suite because of some invalid value of the one field in your struct. You have, let's say, 30 fields in this struct.

Jakub: 45:55

If you don't have documentation what makes this struct incorrect and why, it's quite hard to figure out. So then, when I write the command that, for example, the value, I don't know, let's say five twenty kilobytes when I put here in the configuration versus 200 to 56, this is incorrect because of something. For me, when I write this kind of software, it would be a matter to spend additional minutes to add one line. For the new colleagues that they are coming to the codebase or for the external contributors to the open source software. This is like godsend, I would say, at least for myself.

Jakub: 46:35

When I read code with the information why something is done, what's the reason, then it becomes much, much clearer and allows to basically focus on on your work, which is, let's say, develop your functionality or fix the bug much, much faster.

Dominic: 46:53

Oh, yeah. Yeah. Totally. And you don't do that inside a function. So let's say let's say in in its implementation itself, there's something, you know, very particular about about something.

Dominic: 47:06

So you are not tending to put an explanation come a comment there, related to, you know, either the business logic or something something related to the domain. So you you are not doing any any in implementation comment?

Jakub: 47:24

Inside the function sometimes, yes. But the we have in Go great parser for our documentation and our functions. We can embed the links to some, let's say, the the external external sources, so generated documentation for the go package it's really brilliant and we can really use all these features that is actually documented in on the go dev page how this different this doc strings should be structured and what kind of information we can add them And I would say that spending additional few minutes is basically very, very valuable for for other developers.

Dominic: 48:09

Oh, yeah.

Jakub: 48:09

Or potential contributors, especially for the open source process.

Dominic: 48:14

Yeah. Oh, yeah. Totally. Have have you have you have difficulties balancing between the pressure of releasing something and, you know, the need to write either this documentation or or the test itself. So how do you balance that?

Jakub: 48:34

I focus main focus is on quality. Right? I don't like skipping We are all under the pressure, and I was couple of times in the history of working in different companies, up to the one point that we made basically within one entering pressing one enter key, I managed to, delete one database. Thanks god it was test database because we were under pressure. I've seen wiping out, 1.6 thousands of the virtual machines with CentOS in one company Wow.

Jakub: 49:21

Because of the lack of, basically, tests and the pressure. And then it resulted with the multiple teams, with our colleagues in Hyderabad working twenty four hours trying to, basically, resume this this this VMs because, because we didn't have certain tests or tests were not done for the puppet code that pushed changes. That reminds me one joke that the DevOps is, like, let's break the servers 1,000 at a time. So I remember this. We were this kind of jokes in the community.

Dominic: 50:03

Nice.

Jakub: 50:04

So this is real, sometimes it's real problems because, quality is the the the paramount. Right? So it's very easy very easy to, make the very very mistake. Okay. Simple.

Jakub: 50:22

I wouldn't say simple. Some mistake that's, otherwise could be could be avoided because of the stress of the pushing the pressure that we need to, let's say, fulfill certain the timeline or the requirements, but then possible the disaster that could happen it's much larger. So it's the blessing if the teams have, directors and managers who actually support this approach towards quality. And it reminds me the one of the training that I attended years ago in Dublin in person. The uncle Bob was, here, with a clean coat and his approach towards the professionalism and approach towards the engineering.

Jakub: 51:10

And it's very famous, his quote is over the Internet that QA team should find nothing, and another, we will not ship shit.

Dominic: 51:23

Nice. I like that.

Jakub: 51:24

So under any circumstances pressure, we should not, let's say, ship the software when we feel that, we are not sure about about about untested some behavior or some corners that are not verified.

Dominic: 51:42

Yeah. Yeah. AI will will help us clean everything else.

Jakub: 51:48

Yes. Yes. Yes. That's that's that's another very, very, very, very hot topic. Right?

Jakub: 51:54

On the other hand, yes, unless it will create the future arms of so called, hundreds hundred times ninja developers who will be able to fix all these different bugs created automatically Yeah. By AI, so they are different different, views. Yeah.

Dominic: 52:16

Yeah. Yeah. Totally. So yeah. Maybe maybe in, maybe in conclusion, I I'd like to hear your thoughts about, you were mentioning that you were finding that potentially Go could be a good first language or or maybe maybe not.

Dominic: 52:33

I I'm not sure about about your opinion about that. So

Jakub: 52:36

It can be. It can be. It can be. Could be. The matter is to find the proper, approach to introduce to the language, not even to the language, but to the idea of automating certain tasks.

Jakub: 52:57

Because this is basically this is software for we need to we are using software to automate some some tasks, some processes. So if this is done correctly from the education perspective, I wouldn't, say that Go is not, for example, it's it's would be worse or from Python or the or or something like this because I I seen on Internet this kind of the the discussions. But I would say, yes. Definitely definitely can be. And, there are many books, that they are that they proof that can be with the proper approach, like the books from the John Aranda about the the for the love of go.

Jakub: 53:43

Brilliant approach for people who even they are not tech savvy, but approach his education approach basically allows people to create the mental model in their head and connect the physical world with actually this, a little bit less, world of the of writing software, which is the less physical. We can't touch this. So sometimes it's harder to understand

Dominic: 54:13

Yeah.

Jakub: 54:14

Harder to figure out and imagine, right, what's what's going on. So if we use the proper analogies and proper, proper descriptions, that would allow to create this mental model in the heads of the newcomers to to to programming, all obstacles can be can be can be, basically, removed.

Dominic: 54:39

Right. Yeah. Yeah. I hear you on that for sure. It's, it's it's it's a different it's a difficult, difficult topic.

Dominic: 54:47

I feel like yeah. I mean, may may yeah. Maybe the language is is almost not really a matter of of concerns for for newcomers, especially these days. I mean Mhmm. Learning a new language never have been so easy, where where we are at the moment.

Dominic: 55:07

So, yeah, the the concept and having a solid a solid basis is is way more, more valuable.

Jakub: 55:14

And regardless of the age. So I would say if someone is in fifties or sixties still is working and it requires his work, some automation approaches, there is no reason that this person can't, for example, to figure out and learn how to do this versus someone in the elementary school that, let's say, is tinkering with the raspberry pi or with Arduino from the age of five or six. And and and using some, let's say, scratch environment, to figure out even some basic algorithmic thinking.

Dominic: 55:48

Oh, yeah. Totally. Totally.

Jakub: 55:52

So definitely definitely, Go, I would say, it's it's it's it's would be suitable as a first programming language. Definitely.

Dominic: 56:00

Yeah. Yeah. I really I really like the, you know, reading reading others code in Go for me, especially as a blind person, it's it's it's so standardized and I don't know. It just feels it it feel, how can I say that? Like, I'm I'm I've been here before.

Dominic: 56:21

We know if if I enter into someone else code, it it's like, oh, yeah. You know what? I I'm comfortable here. I know what's going on. I understand what's going on.

Dominic: 56:28

There's no there's no magic. That that's what I like about Go.

Jakub: 56:32

You touch this very important point that we don't need to spend in Go time, I wouldn't say arguing, but discussing. Okay. Shall I use, for example, this naming or this space or how to structure this code. Yep. We have one Go fmt font, and, there is no problem with communication and and the reading, right, and figuring out that someone would wrote similar functionality in a different way and you need to spend time to trying to figure out what this another person tried to convey through this code.

Jakub: 57:12

As you say, you just look at the code, read the code, and you have immediately feeling what is, what is what is doing.

Dominic: 57:22

Yeah. Oh, yeah. Totally. It it does not does not really do that in in other languages that I I I've came across, to be frank. There there's always well, yeah.

Dominic: 57:33

One can say that Go might not be as expressive as as something else, but, yes, on the other hand, it it really helps read readability for the code.

Jakub: 57:43

Yeah. Simplicity and readability. Easy to easy easy to understand. Easy to, basically, understand what's going on and in a standard way. Yeah.

Jakub: 57:56

You log in to the standard library, then you look into some different different packages, and and functionality is expressed almost in the same way.

Dominic: 58:07

Yeah. What what are you looking for? Let's let's have a closing thoughts here. What are you looking for for the future? Is there is there any anything that that is coming to go or that you would like to see coming?

Dominic: 58:18

What what what are you looking for?

Jakub: 58:22

I I don't have to be on a such desires. I current version of I of Go has everything that I really need to describe and express the behavior of the system that I'm working with. In my personal project as well, I don't I have what I have in Go, and I'm really happy, and I rather concentrate on, digging deeper into, let's say, fundamentals or the other technologies that they are coming, like the web assembly, for example, versus arguing, oh, do I have generic, and how it's or do I don't have generics, and this kind of discussions? For me, to be honest, doesn't really matter. As long as I can write and read, clean code and solve the business problems or my problems in this in this particular language in the simpler way?

Jakub: 59:25

I don't have any, you know, desires. What should come first?

Dominic: 59:29

Oh, yeah. I'm I'm totally with you on that, and I've said that earlier in this podcast. I will say that again. In my opinion, even even the cadence that they are releasing and they are I I would slow that down a little bit. Mhmm.

Dominic: 59:42

Mhmm. I would I would do not more than one update per year and just do some security update and things like that. We we don't need we don't need that language to go to go that fast. It's not c sharp. It's not Java with yeah.

Jakub: 59:55

And not Python as well as well.

Dominic: 59:57

Yeah. Totally.

Jakub: 59:58

Yeah. Especially, nowadays, I I have opportunity to dig deeper into the Python and basically, many, many constructs I need to look up the documentation because I'm just what is this?

Dominic: 01:00:11

Oh, yeah. Oh, yeah. I hear you. I'm I'm I'm a little bit on Python these days as well for monetary reason, but, but, yeah, it's, it's interesting sometimes. Thank you so much, Jakub.

Dominic: 01:00:25

So do you have any any, I don't know, resources or closing, message that you want to, to say to the listeners?

Jakub: 01:00:34

Don't be afraid to dig into the standard library and learn from there as well, especially the constructs. For the newcomers, definitely John's, books.

Dominic: 01:00:45

Oh, yeah.

Jakub: 01:00:47

That's that's really, really, I would say, even a godsend. This this approach, the education approach, and spend at least fifteen, twenty minutes daily reading the Go code in standard library. We read not enough, versus writing. So reading the standard library, the code, it's really, really good source of, of learning learning material.

Dominic: 01:01:13

Yeah. I agree. Totally. So go, go clone the, the the Go, the Go repo and

Jakub: 01:01:18

Yes.

Dominic: 01:01:18

Pick a package, per week and and read that.

Jakub: 01:01:22

Go is written in Go. Exactly. It's easy to clone the repository from the GitHub. Go to the src, we have packages, everything. We can read, we can see the patterns, how this certain constructs the approach.

Jakub: 01:01:37

And what is the interesting address to closing the remark? When we look how the standard library is constructed and how the Go developers approach the problems, then we can when we start using the same patterns in our Go code, when we are running or the debugging code or even reading, doesn't matter what ID or editor we are using, then we don't need to mentally switch between the different ways of the approaching certain constructs. When I write Go way of solving the problems, then when I go to the standard library and I need to read documentation for certain function, this, basically, the code is the same. If I see the Go code written more like JavaScript, right, it it would require already mental mental, a little bit, the juggling

Dominic: 01:02:33

Right.

Jakub: 01:02:34

In the in the head. So read read standard library.

Dominic: 01:02:38

Absolutely. Totally. That's a good advice.

Jakub: 01:02:40

People don't need to understand. It's a matter of getting familiar with syntax.

Dominic: 01:02:46

Yeah. Absolutely. Alright. Thank you, thank you so much. It was fun.

Jakub: 01:02:51

Thank you very much, Dominique, for having me. It was great fun.

Dominic: 01:02:54

Alright. Let's talk later. Alright. That's it for this week. I would really appreciate if you can talk or share about this podcast, it's always helpful.

Dominic: 01:03:09

Also, another way to support this show is by purchasing my course, there's always a link in the show notes. So on that, see you next week.