go podcast()

Dominic: 00:01

Hello there, I'm Dominic Saint Pierre and you're listening to Go Podcast. So today I will talk about using shim over APIs to, prevent breaking changes. Hello Gophers. So we are back and I'm doing a solo episode this week. I wanted to talk about, something interesting.

Dominic: 00:27

But, yeah, beef before we we go there, make sure that you are making your, your updates. So there's, you know, there's, some some security patches, that that has been released, in in the latest Go version. So, for 1 22, there's 1 22 dot 1 and there there's one for 121 as well. So make sure you are updating because there's, there's some security vulnerabilities that, that, you know, that you could prevent by just upgrading, which is, which is always a the easiest way to protect yourself. So, yeah, I was, I I was thinking a little bit lately about, services that stopped, you know, offering offering their their service.

Dominic: 01:16

I'm I'm not I'm not talking about microservices here. I'm I'm talking about, let's say, maybe it's a software as a service or maybe it's a it's a it's a public API, whatever. So let me, you know, let me tell you a quick story, about, you know, Twilio actually did that to me. So Twilio is this is this, telephony service, but, as a company, I was I was using their fax service at some point to send and receive faxes for a credit bureau. And as you might imagine, there was, there was a lot of faxes.

Dominic: 01:53

So you don't talk to bank via API. Even even in these days, you talk to bank, with, with faxes back and forth. So, that that's the way to go. And at some point, I mean, they sent you, Twilio send send an email to all their, fax customers and says something like, well, you know what? We are not going to provide this service anymore.

Dominic: 02:18

So, thank you and sorry. And, please have a look at this, at this other company, where where you you should be able to continue, you know, doing your your your fax, services. The problem with that, while it's true that the other company is is offering, you know, globally the same the same service, you know, their their API is is not is not the same at all. So you still you still have some rewrites to do. And usually when you are outsourcing something to a 3rd party API, you kind of, you know, you kind of hope that this this is in the pocket for a long time and you don't, don't really want to touch that, that part, as often as, you know, internal stuff, for example, and things that, I don't know, that matters more more for the company that you are working with.

Dominic: 03:15

So, I mean, it's it's always, it's always a sad sad part. And I was, I was starting to think that, you know, even even, even when when we are doing some kind of rewrite internally or there's, there's a huge breaking change that is coming up. How much, you know, how much the the the dependent service needs to to rewrite themselves, you know, just to, just to continue to, to be compatible. So I was, I was starting to, to think a little bit. Wouldn't wouldn't that be nice if, you know, when we are doing some kind of breaking change, there could be there could be a a long period of time where the old API would still will still continue to work, just by, you know, just by wrapping the, the new one.

Dominic: 04:13

And at some point, I mean, the team could have I don't know. Maybe, you know, there there should be there should be some kind of, deadline for that because other otherwise, no nobody will will ever, you know, write the write, you know, connect to the new API and and change and and implement the breaking change and whatnot. But, you know, the point is, if the team that is going to break something because some you know, sometimes you still need to break something and there will be breaking changes all over the place. It's it's it's, you know, it's the nature of things. But maybe maybe it could be the responsibility for that team to provide some kind of shim on top of the new API.

Dominic: 04:57

So they could just have an extra level that continue to, you know, to offer the same contract as the previous one. And internally, that that shim is is calling the new, the new API by doing you know, it could could it could be, you know, it could do a lot of things. Could change the data, the data type and back and forth and things like that to to honor the new API. But the point is, I mean, from from an external point of view, and it's true as well for my situation. So if Twilio would have, you know, built something to wrap the the API of the of the competitor that that they were, you know, supposedly sending all their customers to would have been a superb, customer experience in my opinion.

Dominic: 05:52

So I understand that they would have, you know, they would they would have needed to invest some time themselves, but at the end of the day, they are closing their services. I mean, it's it's it's just, it's just a little bit less crude than just telling people, well, you know what? You will you will you will need to to rewrite this part of your application and, you will need now to, so even even though this rewrite is kind of inevitable, just having just having a temporary shim or at least, you know, it could could have been the the third party, you know, could have been the the third party as well. The, the competitor that that that says to themselves, well, you know what? Twilio will will send us, I don't know, tens of thousands of customers all all at once.

Dominic: 06:42

You know what? It could be nice if we would have a shim on top of our current applicate, our current API that would, you know, replicate exactly the API of of, of Twilio in that case. And now, you know, all of their customers would just, you know, it would it would be well, first of all, it would be a super no brainer to switch there. Nobody would have even think, one minute about about doing that or not. Could have been, way way quicker than than having to have all the teams, inside all of of the Twilio customers to to do their rewrite.

Dominic: 07:23

So, I mean, the there there there were a a a crazy business opportunity in here. But I I think it's the same, when when we think about internal, I don't know, you know, in in internal services inside companies. So either either the the team that is responsible for this this big breaking changes offer a shim shim on top of their API. Or or, you know, some some other team in in the company could could build that, and now they could share that. And just just like that, then yes.

Dominic: 08:00

You know, it's it's still it's still like I I understand that it's probably a technical depth to have this this kind of shim, being called. But sometimes, I mean, you know, responding to breaking changes is is not really high in priority for for a business in in terms of and and especially for smaller businesses. You know, I'm I'm coming from a more of a startup world where, yeah, unfortunately, the the first priority is not always the technical debt, I would say. So, I mean, you still you still need to, you still need to pay your I don't know, your 5 or 10 employees on the next month. And if you, if you need to introduce a huge breaking changes like that, and, you know, lots, lots of small pieces needs to change and you are stopping all all the development for, I don't know, 2, 3, 6 months, whatever.

Dominic: 09:00

You know, that could really that could really hurt the the business and and the employees. So this is not the same reality in in bigger organization. I understand that, you know, when when they're breaking change, you know, when when when some breaking changes major breaking changes occur, you know, they they they have probably the budget and and the time to, to do that. But I I think I think having a shim like that could be could be just the answer for most of the most of the situation. You know, we we see that a lot in the client side world where, you know, at at some point, some some functionalities were not really handled by all the browsers.

Dominic: 09:48

And, you know, there there there were like shim like that that was built to, I don't know. You know, to to just mimic or replicate what what the browser would would have done if if it would have supported that API, for example. So I I again, it's it seems to be it seems to be very simple and then probably, you know, may maybe maybe people are already all doing that. I don't know. I haven't seen that a lot.

Dominic: 10:15

And to be frank, this, this Twilio thing happened to me in, I think it was 2021 or something like that. So I mean, it's it's still, very recent. And, you know, none none of the party suggested that. So, I mean, again, I I I I I could not, I could not stop to believe that if if the the competitor that that Twilio was sending all the customers would have done that, it would have been just, just great and easy. Imagine how much how much love they would they would have get day 1 from all the Twilio customer.

Dominic: 10:55

Now you you are tell you you are told, well, in, you know, in in 6 months, you, you will not be able to use, the Twilio API for faxes. But we, you know, we have this, this third party company that is ready to to take your, your load for, and and you don't need to change anything in in your system. I mean, this is this is just I class I would say. And this is, you know, this the kind of developer experience that, that or I would say I would say like engineering experience that we are kind of expecting these days from companies that that make money from, you know, by selling their their API or internally as well. So if you are responsible for some some kind of API internally, then I mean, it's it's kind of your job to, to, to make the engineering process, you know, just just smooth and interesting.

Dominic: 11:57

And and when you think about that, how much time how much time would it would it take to write a shim on top of a of an API that that that will that will get, discarded? I mean, yeah. In the in the case of of Twilio, I would I would I would I would dare say that there there were no more than 5 or 6, you know, endpoint, function and and things like that. It's it's all about the data usually. So it's it's not it's not really the the time to to point, to point the some some kind of client to a new URL and call and call a different route and whatnot.

Dominic: 12:37

It's it's not that. So sometimes it's just, you know, all the the data structure difference from one version to the other. That is that is mostly, mostly, longer to do that. And it could be it could be internal in the code as well. So it could be, if if we think about Go, for example, So may maybe you add the model, the, the the old API with with some go structure and now, you know, the shim could be just to have a a temporary structure that that, you know, would accept the the new the new format.

Dominic: 13:14

And and now you you just have some kind of converter to to take the the old structure and and model it to to the new one. So so again, it could be done as well internally. And this this could be this could be a library that it is that is, like, you know, part part of the of the client if if there is such a thing. So if if there is a client, or, you know, you know, across the company that is built to call this API, well, you know, the the shim could be could be directly there. So it's not, you know, it's it's it's not, re required to have that on the, on the server itself.

Dominic: 13:52

It could be on the client as well. It's totally fine. I mean, it's probably, it's probably better, that way in fact. But, you know, sometimes there there's multiple, there's multiple teams with multiple languages. So now all the clients should have that.

Dominic: 14:05

So, I mean, it it depends. It's it's, it's something that is debatable. The the fact that the fact is that, you know, if we if we if we start to to think about breaking changes like that, where there is a period of time where we will have both version, running not running together because the the old API, the old server, the thing that that has the breaking changes, you know, should be taken down. But but again, yes, there is some way to run to run both in parallel. And that could be that could be very elegant for for all the teams.

Dominic: 14:43

And at some point, I mean, it's it's part of the I don't know. I mean, it's it's part of the backlog and would it would it get fixed at some point? Well, I don't know. Maybe not. But, yeah.

Dominic: 14:56

I I can see I can see some problem about that where, where now a shim a shim is built for another shim and now it's it's just, multiple multiple layers of transforming the data to to prevent breaking change. I don't know. It it it's a double edged sword, for exam for example, for instance. But, I don't know. In in in terms of external services.

Dominic: 15:23

So if your company is providing an external services. So instead of just having a V2 for your API that introduced some some breaking changes, could be nice to at least investigate the fact, of of of having some kind of converter from the v one to the v two. So your your, you know, the client of your API does not need to, always rewrite their their their themselves. You know, companies, we we all we all have other things to do especially in the startup world. And and yes, I mean, this is this is not not cool when things like that happen.

Dominic: 16:01

So I mean, it's it's a quick it's a quick thought. I don't I don't know if if it makes any sense, if it makes you maybe maybe you have some, like, experience, experiences that is similar to that. And maybe you already implemented that. I I haven't seen that very often to be frank, and I don't really know why because I I mean, I don't know. On on paper and and on this, 15 minutes, you know, rambling or podcast.

Dominic: 16:30

I mean, it's it seems to be it seems to be interesting. But, but yes it comes with, with some disadvantages like like everything. So again, yeah. That's that's it for for this week. It was a quick episode.

Dominic: 16:44

We will have a guest next week and on that, wish you good coding and have fun. I mean, this is, this is something we should, we should always remember ourselves. We are doing that, you know, mostly because it's it's probably the the the best job in the world. So, keep at it and have a have a great week.