go podcast()

Dominic: 00:02

Hello there. I'm Dominic Sibial, and you're listening to Go podcast. Today, I'm joined by Peter Straumann, and he is the creator of Ghost, a headless browser. Before we dive in into the interview, I'd like to remember everyone that we are on the Gopher's channel called Go Podcasts, so in one word. So, again, I mean, I I won't shut up with it until there is, you know, way more people than that in there.

Dominic: 00:38

I know that, we have a lot of listeners, so maybe you're not on the gopher's Slack. I don't know. If you are, then it could be nice if you can join, this channel. I'd like to start taking, you know, voice question, maybe voice, question to guests that are coming to the show or just question in general. So just trying to make the podcast a little bit more dynamic and things like that.

Dominic: 01:04

So if you want to join, we are in Go Podcast, the channel in the Gopher Slack. So the website for the podcast is gopodcast.dev. So in case, there is a link in there to, to join the channel as well. If you want to listen to previous episodes and things like that, it's all there. All the links are there to, you know, to listen the show in your favorite, podcast listener tool.

Dominic: 01:38

And on that, this is the interview. Hello, gophers. So I'm joined by Peter Stroman today. Thank you very much for accepting the invitation.

Peter: 01:50

Yeah. Thank you for having me.

Dominic: 01:53

So, Peter, you are working on a very, very cool and intriguing open source project. But before we dive into that, I I'd like you know, I'm always asking this question to everyone here. If you can, you know, present yourself, introduce yourself, any anything that you want to to send us as information so we know you a little bit better. So I I'm mostly interested, you know, if you can focus on, you know, you know, technical stuff, of course. But, you know, how how did you come, cross Go and things like that?

Dominic: 02:25

And if you if you have, you know, done other languages and things like that. So anything that you want to to tell us.

Peter: 02:32

Yeah. Sure. Sure. Yeah. I've, yeah, I'm I'm I'm getting pretty old, and I've been, like, in the, software industry for more than, like, twenty five years.

Peter: 02:41

And before that, I actually started coding. I think I was about 10 years old when my older brother got a, like, an Amstrad CPC. I wasn't really allowed near it, but I could read the instruction manual, and that was mostly a description of the basic programming language. And that's that's the first time I learned programming, and I think that, like, intrigued me. And, yeah, then I started working in 1997, part time while completing my studies and and it's been I've been, like, full time software developer ever since.

Peter: 03:16

Been doing freelance work, for the most of that time and, like, I've been, like, doing a lot of web applications and, like, some years ago, I decided this is kind of, like, my specialty. This is what I want to, like, do and, like, excel at. So yeah.

Dominic: 03:37

Nice. So when you say you you say freelance almost all this time. This is this is impressive. I mean, so is is there a reason for that?

Peter: 03:49

Yeah. It's yeah. Actually actually, it's a reason that I didn't actually learn until about, like, a year and a half ago. So I got diagnosed with ADHD.

Dominic: 03:59

Okay.

Peter: 03:59

And that made me realize why sticking on to sort of like a long employment really wasn't a good fit for me because I fry when I get new, like, new challenges. I need I need things to be challenging. And, like, when you stay the same place for too long, it gets, like, like, the daily routine. And and then I just, like, my brain starts shutting down. But I yeah.

Peter: 04:30

So I didn't actually know that until, like, a year and a half ago.

Dominic: 04:35

So did did that change, anything in your other other than that? So did that did that change anything? Is it have you have you understood the more things that that you were, doing that, you know, explain that and things like that?

Peter: 04:48

Yeah. Yeah. Most certainly. I've for example, I've had a few, like, roles with some kind of, like, management responsibility, like scrum master roles. And while I know exactly what I should do, I just, like, couldn't really get around to do those kind of things because it was more fun to actually write code.

Peter: 05:08

And so so, like, it wasn't, like, mentally a difficult thing to to do those things, but, like, of of what you call intellectually difficult. But yeah. Now I know why I just couldn't like, why I was poor in those roles and excelled in in actual software development.

Dominic: 05:28

Yeah. This is interesting. This is interesting. My my wife do does, have that. So I I hear a lot of what you are saying, to be frank.

Dominic: 05:38

So so what about what about freelance then? Have have you have you seen this this field evolved a lot in all those years? Because, I I've I've been doing, freelance myself for the last, I would say, maybe seventeen years now. But before that, I was a I was a nine to five employees.

Peter: 05:55

Yeah. I I think I think first of all, it depends a lot on the market, like, where you really are. Mhmm. But I think it's been roughly the same, but there's been, of course, been been ups and downs. So again, during the, like, finance crisis in, like, 02/2009, a lot of the companies that use freelancers are financial institutions.

Peter: 06:27

And so they, of course, they cancel a lot of contracts that made a lot of freelancers competing for the same job, so it is kind of difficult. I was a little bit fortunate sitting on a longer project during those years. And I've also seen that, like, in the recent, like, the recent half year, like, since since summer, it's also been, pretty difficult to get something at least here here in in Denmark where where I live. But it's also I mean, in some I think in, like, Sweden and Germany, like, my two neighboring, countries, firing employees is, like, legally extremely difficult, and that makes it more likely for companies to use freelancers if they don't really know what what how many they they need because Yeah. That's that's a legal, reason to to use them.

Dominic: 07:25

Yeah. Yeah. I I can see that. So would you would you say that being a freelancer made you learn more programming languages?

Peter: 07:34

Most probably because I've been, like, doing a lot of, many, many different, things. Most of the time it's been been .net, but but there's been, yeah, c plus plus VB scripts, Visual Basic, f sharp, Go, obviously. And in the recent years, a lot of JavaScript, focusing on Right. Web development, which becomes, of course, more and more JavaScript heavy.

Dominic: 08:06

Yeah. The the inevitable JavaScript.

Peter: 08:09

Yeah. And then then they're the ones that I learned for fun like OCaml, which I think is a a beautiful language. So but Yeah. But I haven't had paid work using OCaml.

Dominic: 08:22

Right. Yeah. Yeah. Yeah. I really like, the ML family myself.

Dominic: 08:26

I I I learned that or at least my first experience was with, with Elm actually in '19, in 2037. So and I discovered that and I was like, wow. This this is this is really nice. So I always wanted to to try, you know, Haskell and things like that. Never.

Dominic: 08:45

The the tooling for me was was already too much. Just installing the things and the, the amount of packages that that do install, like, natively on your system, that was that was way, way, way too much that I could accept from a from a language to to try it out. But once I discovered Elm, wow. I I was like, this this is this is so interesting. And, yeah, the functional programming is is really nice.

Dominic: 09:10

I I I all I I compared the tooling of Elm with Cure at some point, which which I posted the on both Reddit, the Elm Reddit

Peter: 09:20

and the

Dominic: 09:20

Cure, and then was not really well received because but I I was mainly talking about about the tooling, you know, because Elm also had a formatter and had the the the compiler is extremely fast. There there's there's, there's testing baked in. So I was I was mainly, you know, drawing a a comparison bit between the way that you are working day to day with Go is is very similar to what you would do in Elm. And I I was finding that very, refreshing compared to, typical, you know, React and and TypeScript, kind of project. Yeah.

Dominic: 10:01

So TDD, this this is yeah. This is interesting. I did I did a I did a an episode recently about that. I'm I'm struggling personally, not not with TDD, but but with testing in general. I inevitably find my test suite, I don't like it at some point and something like that.

Dominic: 10:22

I guess it's just a a mentality. I'm I'm probably not good at at that yet, even though I've I've been trying to do that for a long time. But this this seems to be, something that you, that you are, you know, mastering.

Peter: 10:36

Yeah. I hope I am, but I've, yeah. So I've been been practicing TDD since, about 02/2010. And yeah. And and I've yeah, it's it's a skill that I, like, I've learned over many years.

Peter: 10:55

And, like, with the thing I did back in 02/2010, I wouldn't even call it TDD today, even though it still gave benefits. But I've I've had a few, like, moments. The first was actually that project. I realized that, like, it was a a project with many layers, c sharp code with a, like, a a thick, client, a WPF application. So you had sort of like a domain layer in the in the application, that presentation layer, and a communication layer talking to a back end that had a communication layer, domain layer, database layer, and then you had the database tables.

Peter: 11:33

And sometimes you needed to make, like, a feature required changes in all of these layers. And, like, one day after been working on the project for a long time and I made changes to all of these layers driven by test, I found something very remarkable happened, and that was it worked the first time. I was like, woah.

Dominic: 11:54

Nice.

Peter: 11:54

And then I realized how much time I saved by not having to debug the application because, like, debugging through all of this, and there was, like, no hot code reloading or anything like that. So every time I made a change to the code, I need to restart the application, navigate manually to the window where you have the the issue and try again. And and, like, that was, like, debugging and finding bugs was a lot of time wasted. So so just by, like, the thing working the first time, it was, like, extreme amount of time I saved on that. And then there was a lot of cases where it didn't work the first time, but finding the issue was pretty trivial.

Peter: 12:33

Like, the WPF layer wasn't, like, statically checked. So so it might, so it could be a matter of, like, just poor bindings and change the, text in the in the, like, WPF, files would would be enough.

Dominic: 12:50

Oh, yeah. That's

Peter: 12:51

So that so that's the first time where I really saw, like, okay. This isn't something that is just providing a benefit. Like, six months from now, I I received the benefit upfront.

Dominic: 13:02

Oh, yeah. Yeah. I can see that. I, I I was not remembering, I I did I did I did WPF in in another life, and I was not remembering that, yeah, the binding is is not actually, is not actually the type checking or at least, you can you can write it's it's just string at the end of the day. Right?

Peter: 13:20

Yeah. Yeah.

Dominic: 13:21

Yeah. So how would you compare, that to Go? Are are you finding that it's it's a little bit easier or at least a little bit more comfortable to do TDD in in Go?

Peter: 13:33

In in many ways. First of all, like, .net, the first iterations weren't really built for for testing in mind. I did a lot of, like, web applications and you had this what did they call it? The web forms where the system would construct the objects for you. So you didn't even have any control of how things got created.

Peter: 13:52

So just in that sense that that I just the industry is move towards promoting testable code is, of course, a huge help. But one thing that that, for example, in Go, specifically for web applications is you have your web application is represented by like a single function. You have in the HTTP package, you have the the handler interface, which just has a single function, the server HTTP. So when I test like my HTTP server, I don't actually have to make HTTP requests. I can just call a function.

Peter: 14:28

So it's I can test the entire HTTP layer without, like, just like any other component. I can do dependency injection if I want, like replace dependencies. And that's like makes it so much simpler too when you have and and it's also going back to OCaml. You have, like, the dream framework for OCaml. You have the same thing there.

Peter: 14:52

You have, like, your HTTP server is just a single function so you can call your function in tests. A few minor minor differences, but, like, the overall principle of just, like, calling a function to test your HTTP is so much simpler.

Dominic: 15:08

Yeah. So I I think it will be a good time to introduce the project now. So the the project is called Ghost, and I will I will let you introduce it.

Peter: 15:16

Yeah. So Ghost is, a headless browser. So it acts like a browser, and it's written in Go and it's obviously intended to test Go projects. And so you can create an instance and connect it to your HTTP server. And again, you just connect it to your, like, functions.

Peter: 15:39

So you don't need to actually start the, the server on a TCP port, and then it can, like, consume HTML and execute JavaScript, using a built in v eight engine. And then from your Go test code, you're able to inspect the, the DOM as it is. So you can see, like, if I click a button, does it have the desired effect? You can fill out forms, sum submit them, and, yeah, verify things work as they as they should. And again, you can replace dependencies in your web application.

Peter: 16:26

And you have, like, each browser has a has its own v eight instance. So you can run the test in parallel, like, in in full isolation. And the idea started really because I like, I saw the Primagen's course in the HTMX, and after I've done a lot of React projects, I felt that HTMX was actually a, like, really, really nice way of writing, like, interactive applications because React does bring a lot of complexity. So you can make really nice user interfaces in React, but you also bring a lot of complexity compared to just server side rendering. And HTMX can create a lot of the, like, nice user interface, but in a much simpler way.

Peter: 17:24

Yeah. So so I think that, like, more than 90% of all React projects could be done much simpler using HTMLX. And, and then if you have really complicated UI, you can always put, like, a React component inside, like, an HTMLX skeleton. So so it's really only if you need a true progressive web app, then HTMLX is out of the question.

Dominic: 17:50

Yeah. It's pretty it's pretty nice. HTMLX is is I I always come, combined it with with Alpine GS myself. So when you need to to have a on page, interaction and whatnot, you know, Alpine is also very, very simple. Just just a couple of attributes, and and that's fine.

Dominic: 18:10

So the question is, Peter, my my I mean, why why are you building a browser? This this is kind of crazy.

Peter: 18:17

Yeah. So yeah. It's it started like so so it combines so so I wanted to test the HTMLX, and I wanted to to do it in a TDD way. I started browsing, like searching for what does people use, and people use, like, actual browsers, like, browser automation. And in my experience, that is, like, brings in a significant overhead, and you don't tend to write the test first.

Peter: 18:46

So you actually don't get the benefit of TDD, which is really like it sets up a a fast feedback loop. So I really wanted to have something headless, in in the in the the programming language you're using. And so it really started as a, like, an experiment. Can I do this? And so I first started trying to see, okay, I got two major issues I need to solve to see if this is possible.

Peter: 19:15

Can I pass HTML, and can I embed a V eight engine? And I know that Go can inspect with C code, V eight, C plus plus It should probably be possible. And so first, I start looking at like, a parser, and it's something I've done before. Though HTML is slightly different than normal parsers because you have to provide a result even if the input is not valid. Like, if your source code is not valid, your compiler will say error.

Peter: 19:47

But if your HTML is not valid, you still need to use a, like a a DOM. I decided to not deal with that to begin with, just see kinda like decode a, like, a very simple, HTML. I got to that point, and then I say, okay, let me get, like, can I get v eight integration? There was actually already a project that provided this for Go. So that's a v eight Go project.

Peter: 20:13

Yeah. So I found v eight Go and, so got this, embedded into to the code and and then said, okay, here's a script tag. And when I encountered the script tag, can I execute the script? And I managed to get that far. And, okay, this is pretty cool.

Peter: 20:33

I can, like, fetch HTML, and again, by just calling the your your Go handler function and pass the script. Then the next thing is, okay, can I then say, okay, I have a script source? Can I download the script and execute it? Oh, yeah. I can do that.

Peter: 20:52

And I started doing stuff like, okay, what kind of DOM does the script see? Because it needs to, like, happen at the correct time doing, like, DOM construction. And and sort of like, okay, I got these basic things to work, and I posted in the the Golang subreddit, and I got, like, a massive amount of, of of attention and feedback. And people like, oh, this is extremely cool.

Dominic: 21:20

And Yeah.

Peter: 21:20

So I was like, okay. Let's try continue and see see where where it leads us. And yeah. So it became a little bit of a challenge. I continued and saying, okay, let me get a I think it by the time I posted it, I had an HD mix embedded.

Peter: 21:39

Like, it could load h t m x, and I think maybe it's just like without any errors. And I could react to, like, it emits an event when it's loaded completely. I think that's where I sort of like, okay, let me publish this. And then I started looking into interacting with elements and, say, okay. I clicked something.

Peter: 22:01

Can this trigger an an action? Like, h t m x fetches new content, swaps the content. And okay. So I've got that working. Okay.

Peter: 22:12

Let me see. Can I get forms working? So you have something like, yeah, part big part of many applications, and I got that working. And, with the HMX support, so actually, you can click links in in the, in the browser, but it doesn't like, my code doesn't actually yet follow like, HTTP redirects, but, but the HTMLX like, the HTMLX code works.

Dominic: 22:38

Yeah. That's impressive. So would would you say that it's it's it's lighter than using something like Chrome d p. I don't know if you know Chrome d p. It's, you know, it's akin to Puppeteer in in the JavaScript world where it basically just connect to to the Chrome, dev, dev server.

Peter: 22:57

Yeah. Because everything is like it's just a, like, a function call. You don't have any out of process communication. You're just calling Go functions.

Dominic: 23:07

Yeah. And the v and v eight, yeah, so you were mentioning that you are currently using a project that, like, what, is embedding the the normal v eight, in inside inside a Go, a Go package?

Peter: 23:21

Yes. That is correct. Although, I did need to make some extensions to it because there's a lot of, like, features in v eight that that was necessary for my use case that that wasn't actually in the v eight package. So it is a long time since I programmed in c plus plus but, so that was a lot of, yeah.

Dominic: 23:41

Just admit that you were missing it. Right?

Peter: 23:45

I I I always, like, described, C plus plus a bit like I would say driving a sports car. You can go pretty fast, but it you can definitely also feel the suspension working.

Dominic: 23:58

That's good. I I haven't wrote any any c plus plus since since school, like, way, way, way back. I never had

Peter: 24:06

this job. But yeah. But one of the things, for example, like, the v eight is is really a c plus plus library because the way you do resource management with objects on stack, that's sort of like you need to c plus plus at least in my experience. And that's something that that v eight relies a lot on. So

Dominic: 24:23

So is is it a a recent version of v eight? Like, is it is it able to to run, let's say, can I can I write a let keyword and it will it will understand what it is?

Peter: 24:35

Yeah. I'm so glad you asked, actually, because it that can allow me to give a big shout out to another GitHub user called Tommy. So so the original project is created by GitHub user, Rockchap, and then GitHub user, Tommy, actually took this project and updated updated it with latest v eight versions and has made a GitHub pipeline that pulls latest v eight from, the Chromium sources. So so that's actually pretty awesome. And then as I said, I've done a lot of work then extending the functionality with, like, features that aren't v eight but weren't exposed in the Go project.

Peter: 25:15

So, yeah, they did Tommy did an amazing work on that as well. And and the original project isn't being maintained at the moment. So but but all my changes, I'm working with Tommy to get them into his branch.

Dominic: 25:28

Very nice. The power of open source.

Peter: 25:32

Yeah. Yeah. That's like that's that I'm also glad you brought that up because that's, like, one thing I've seen is, like, a major change in, like, in my time is, like, when I started, you purchased, like, software development tools and you purchased Yeah. Like, component libraries and stuff like that. But, like, with open source, like, it's changed so much, and it's, like, it's changed for the better.

Peter: 25:58

It's like, I owe so much to open source, and and it's, like, huge difference it's it's made to the the software development, like, community, industry. Yeah.

Dominic: 26:10

I I remember, when when I started, I started with, with VB, actually, on a on a Windows desktop application, and there were some component that we bought. I I don't exactly recall the name of the of the company, but at some point, the company just closed. And now, in a couple of years, later, you you it was kind of very difficult to find those, those DLL anymore.

Peter: 26:34

Yeah. What do you do when then, like, there's a new Windows version and these components no longer work?

Dominic: 26:40

Exactly. Exactly. Yeah. Yeah. So it's crazy.

Peter: 26:42

So yeah. Yeah. Truly, like, the power of open source, it's, like, amazing. I I think in my mind still, like, open source is a much more important, change to the software industry than AI is.

Dominic: 26:56

Yeah. Well, yeah. The the yeah. The problem with AI is that we're still not really sure exactly what kind of problem it will bring.

Peter: 27:06

Yeah. So

Dominic: 27:07

But, that's another another story, I guess.

Peter: 27:11

So but yeah.

Dominic: 27:13

So about about the this v v eight then, is it, is it is it is it, like, possible to have your library, I guess, in a in a CI pipeline and just, just run the the test there?

Peter: 27:29

Yeah. Sure. Definitely. So first of all, the, the Ghost itself runs, like, tests in a pipeline where, like, many of the tests set up and, like, HTTP handler themselves delivering content that is relevant for the feature being tested. So it's doing it itself.

Dominic: 27:58

Nice. So would you would you so I I was I was intrigued to hear you say that you you had built a parser. I I was I was in the impression that there was something in the in the standard library because I I guess that for the the templates that that there is already something.

Peter: 28:16

Yeah. That that's that's actually true. I started building a parser, and I've I was facing some sort of, like, how should I design this? How should I design that? Again, wrote a question in the Golang separated, and someone mentioned that there is the the x slash net slash HTML package that actually has a parser.

Peter: 28:38

Yeah. And and so I actually quickly adopted that.

Dominic: 28:42

Nice.

Peter: 28:42

It for a lot of reasons. For example, scripts needs to be executed at the correct time during DOM tree construction. And they don't execute scripts of

Dominic: 29:04

course. Right.

Peter: 29:04

But but but the thing I have right now is I do it, like, in two steps. I use their package to, pass it into their structure, and then I just iterate through their structure to construct my own structure. And then they have dealt with all the rules of how to handle invalid HTML. So, like, so I know didn't actually have to deal with that problem. It is handled correctly now.

Peter: 29:29

And that's, so that's pretty nice. And there are other things that, for example, the HTML template element, the children in the HTML doesn't actually become children of the element, which is something I actually didn't know. But

Dominic: 29:46

I'm so I'm sorry. I'm I'm not sure I understand that.

Peter: 29:48

So is that yeah. So there's a specific element called the template, like HTML element. And when you have it in the HTML, like, text, everything you have inside actually becomes, like, children of a document fragment that is on the template element. So there's also there are other places where the, like, the structure of the HTML text doesn't, like, doesn't map to the DOM one to

Dominic: 30:14

one. Oh, interesting.

Peter: 30:16

So which is also why I need my my own, like, parsing layer because these roots are not in the in the, in the library that I use. But it sim in it simplified the HTML parsing significantly and allowed me to then focus on on, like, moving on and solving the problem. And if somebody wants to contribute and and make a dedicated parser, that that eliminates this two step parsing, I wouldn't mind it, but it's, like, not a priority to, to optimize that at the moment. Yeah.

Dominic: 30:49

I would I would guess it does not really take, that that that long of a time. So I I would I would imagine that you are downloading so let's say let's say I have a script tag in in my HTML, and and it's it's a it's a an SRC, URL. So I would I would imagine that you are downloading that in between those two parsing phases?

Peter: 31:12

Yes. That is correct. I would say during the second phase, like, when I construct my DOM from, like, iterating the other structure. So when I get to the script tag and say, okay, I mounted into the DOM, I check, okay, what's what is the source? And then I, like, download it.

Peter: 31:33

But again, downloading is just calling. I assume you have an HTTP server that serves it. So I'm just calling the Go function. I'm actually not, like, doing a TCP request. So

Dominic: 31:44

Yeah. Yeah. Yeah.

Peter: 31:45

And and and then it's being executed at that time. And if it's a script tag with content, like you have the script inside the script tag, it's also being executed when mounted.

Dominic: 31:58

Nice. Yeah. I can I can imagine this is, this is a this is a major step because there there's a lot of discussion around the HTML template package that well, you know, they they are they are not, type checked and whatnot? So this can be this can be great because now not only well, with with with normal test, you can also test your template. That's true.

Dominic: 32:23

But now you can test your interaction and also, some server side rendering that that you would do from submitting a form. This is kind of very, very interesting.

Peter: 32:33

But I but I would add that that if you are sort of a if you have, like, a pure server side rendering, I wouldn't recommend going this route. I would I would probably just go for making HTTP request and verify, like, pass the HTML you get back and verify it has the correct shape. But this is like specifically intended for the HMX where you the behavior of your application becomes like a big choreography of you have the attributes and certain elements. They trigger some certain endpoints on your server which are designed to be called by HTMX and then provide something with specific HTTP response headers that are intended to have a specific effect when the HMX passes it. So so so in order to test this in a way where you actually see the the produced effect, that's sort of that's where I would recommend using this.

Peter: 33:25

I would say it's probably overkill if you just, like, deliver static HTML content.

Dominic: 33:30

Right. Oh, yeah. Yeah. I understand what you are saying. So any any challenges that that, you know, surprising aspect that you have, you needed to overcome so far?

Peter: 33:44

Yeah. The, yeah, the first thing I was sort of like is the implementing the DOM itself because there is a specification I need to, like, follow that specification. And that is a very object oriented, like, specification. You have, like, parent, children, circular references, and you have elements overriding behavior. For example, I just I mentioned the the template that has, like, specific specific rules for how it works.

Peter: 34:17

And Go isn't an object oriented language in in that regard. So while you have embedding where you can say, oh, we can have an object and I can embed it into another one which can, like, then have the same methods. But, I might override some in my, like, the the the one that is embedding the object. If I then call into a function on the embedded object, it doesn't know anything about being embedded. You don't have this, like, override Yeah.

Peter: 34:48

Functionality. So that was how sort of, like, a a challenge figuring that out. And, like, there are many I'm not totally happy with how it works. What I've done right now is whenever I create an object, I call the sort of like embedded node passing like an interface representation of it. You can say it's outer self to the embedded node.

Peter: 35:20

So when, for example, you iterate through children, it doesn't return the sort of embedded node object, but it actually returns the, a a reference to the embedder Mhmm. Being the specific HTML element implementation

Dominic: 35:40

Right.

Peter: 35:40

Might be. And I'm kind of wondering if a strategy pattern would have been better, but that that provides some other challenges because all the different elements have different functions that need to be implemented. So but it's it's working. And but, yeah, every single specialized specialized element needs to call this, needs to call this function. Otherwise, it doesn't work.

Dominic: 36:07

Yeah. Yeah. That's an interesting this part, I mean, because because of the of the Google construct, it's, it's difficult to express yourself like you would do maybe in in c sharp, for example. I can I can

Peter: 36:19

Yeah?

Dominic: 36:20

Yeah.

Peter: 36:21

And and and also, like, I mentioned, OCaml. I was kind of thinking, like, how would it be to make this an OCaml? I think it would be like a nightmare to implement this, this kind of behavior in OCaml.

Dominic: 36:36

Oh, really?

Peter: 36:37

Yeah. Because you did, like, this isn't like you don't even have this the same object notation, like the pure functional programming. I mean, OCaml isn't pure, but but being idiomatic in OCaml, I've I've I think it could be, like, even more difficult. But yeah. But but I've would say that's probably one of the main challenges.

Peter: 36:58

Like, you are modeling a an API, a specification that just is, like, so object oriented, like, mid nineteen nineties, like, how we design things when I started in the industry.

Dominic: 37:14

Do do you see Ghost as being used for something else? Is it is it, you know, could it could it be if you extend your vision for the future, is it is it only for testing?

Peter: 37:28

I mean, of course, you can use it for other purposes. It is intended for testing, but, I mean, you have a headless browser. There are some memory issues at this right now, that if you keep it alive for a long time, like each browser instance, it will leak memory. So that's not a problem for the incentives. But, and some of it is in my code.

Peter: 37:55

Some of it, I think, is actually down in the V eight Go, that it's it's holding on to objects, that it's not really releasing. Mhmm. But I've been actually deferring dealing with that problem because I see the weak pointers, being a tool I can help solve the the memory issues. So I've been waiting for a go one twenty four to come out, which just released. So

Dominic: 38:21

Yeah. Nice. Yeah. Because I I I was imagining maybe, you know, maybe some automation that that would not want to use the, you know, the full Chrome browser

Peter: 38:35

for example. Yeah. I mean, definitely, you can use it for all kinds of automation. And if we get the memories you solve, you can even use it for sort of like long running processes. But,

Dominic: 38:46

interesting. So what

Peter: 38:49

you But but but again, so but but but you've sort of like you can also ask is it the right tool for the job because it is sort of a built to to fit inside a Go context where you are Yeah. Exercising your Go code and might want to replay some dependencies in your Go code while writing the test. So so, so, yeah, you can have definitely have, like, much faster execution because everything is, in process. Everything is just function calls compared to to having browser automation. But is that, is that beneficial for, like, automation?

Dominic: 39:28

Yeah. Might not might not be. I I was just curious if, you know, if at some

Peter: 39:31

But I mean, it's a, it's like, of course, you can use it for other purposes than than what it was built for. Eventually, it should probably also be able to handle React. I doubt it would work today in the current state. But

Dominic: 39:47

Because of the module system, I would I would say?

Peter: 39:50

No. Mostly because I basically have just added support port for just enough to get these features of h t m x running. Yeah. So, yeah. As a so at at at this I'm I'm building sort of like a a a like example application now on the side.

Peter: 40:07

And so the features and need for that application will very much dictate, like, how things are prioritized unless I get, like, feedback from from other users, of course.

Dominic: 40:20

Right. Yeah. Yeah. Yeah. Yeah.

Dominic: 40:22

So this is this is entering, the, you know, the life cycle of an open source project. Now it's because I I I've seen the project on on Reddit as well. And, yes, the re the reaction is, seems to be, seems to be very positive so far.

Peter: 40:39

Yeah. So I hope it will gain traction. And, because I I mean, one of the reasons why I continued with this is because I truly believe this can be a very, very helpful tool.

Dominic: 40:53

Oh, yeah. Yeah. So have you have you used it yourself in in production? So that that that is why you are building this, this this project on the side, I would imagine.

Peter: 41:03

I I would not call it production, but it is it's I I had an idea for an application, which I started and, and which is like then okay. I I want to have a headless browser. And I think that the, Ghost will be, have a higher chance of becoming a success than the website idea I had, which is my why I'm more focused on on Ghost and and spend so much time on it. But I at least it's it's it is kind of a concrete idea that can help drive, like, actual necessary features, so it doesn't become, like, just theoretic what would be, helpful.

Dominic: 41:48

And and do you think this is the part that we are leaving a little bit, like the HTML interaction in in in testing in general?

Peter: 42:02

That's a good question. So I would when I've been doing React applications, it's been a lot like testing interactions and building tests that would end interact with the components as a user would. I mean, one of the, like, big inspirations, for me, like, you have the JS DOM obviously. And there's also testing library, which seems to be also the sort of the way to test right now. And that promotes like interacting with the application in the way that the user also would.

Peter: 42:40

It promotes writing also more accessible applications. So say I need to fill out an input field. So instead of saying, okay, it has some ID attribute, you go in and say find a text field that has a label, email for example. And it did the library doesn't like it can be a label element associated with input using the for attribute. It can be like an aria label, attribute on the input, and can be like aria labeled by referencing another element.

Peter: 43:17

But you have it it it promotes good practices to make sure you have bring the context to the, the input, fields. And, yeah, that's what I've been, like, practicing a lot for testing react React applications. But something I don't see so much in, like, in other contexts where, like, test suites using browser automation, it becomes very technical in in how they're described. They're described through element IDs and and and so on rather than describing actual user interaction.

Dominic: 44:01

Yeah. Yeah. Yeah. Yeah. I I yeah.

Dominic: 44:04

That that that to me is is, you know, close to my heart, I would say, because that that that is the first thing that I've I've I was thinking myself. When I saw your your your project, I was, like, I was I was wondering if it could be something that I I would use to ensure that a, you know, a a page is accessible or not. So so so from a from a screen reader, user point of view, just just what you described is is extremely important. Just just having a the basic I I would call that the basic having having the, you know, the four or or the the area. But but it seems, yeah, it's it's it's a challenge to to test for, for accessibility, I think.

Peter: 44:53

Yeah. And it's it's also it's it's an area that I see just, like, very much, it's it isn't getting the focus that that it it should have. Building accessible applications doesn't really need to take more work, but it requires some knowledge. And I, like, personally will spend time, like, researching what is sort of, like, the the best way to do it. But then I have learned, then I know the next time.

Peter: 45:18

And it's it's mostly just a matter of knowing which attributes to set to make sure that, like, the different elements exist in the correct context. That so, like, a visual, like, a person with normal vision looks at the page. You have, like, a two dimensional view of the page. You can easily see that, like, bring everything into context, but, like, having just these, these small attributes behind the scenes that bring this context is overlooked. And also, like, other other properties, like, people who, cannot use a mouse, for example, force to use a keyboard.

Peter: 45:57

Make sure you have, like, focus indicators around what what is currently in focus, contrast ratio for, like, those that don't need screen readers but still have limited vision, the use of color for, like, colorblind.

Dominic: 46:14

Oh, yeah. It's a it's a huge topic.

Peter: 46:15

Yeah. Yeah. So there's a lot, and it's it's not particularly difficult to to do, but but people, like, I I generally find that most are unaware of these, challenges. And and I actually just became aware that that, like, here in in Europe, we have new EU legislation, coming into force this summer, June 28. Like, most information system needs to adhere to to accessibility standards.

Peter: 46:44

Again, that that includes, like, b to b websites, software as a service, mobile applications, like, most information systems.

Dominic: 46:55

Yeah. That that will that will be that will be major. I I I must admit. I hope that we will have that at some point. Yeah.

Peter: 47:02

So here,

Dominic: 47:03

but yes. Yeah. Europe Europe is is always quicker to like like with the GDPR, for example, this this is an another example. Yeah. I I I was testing just, you know, just to return to, accessibility is hard.

Dominic: 47:19

I was testing, Claude for the first time the other day, the Entropic, AI. And and it's it's terribly not accessible. So once you once you finish to enter your prompt, well, first of all, you are not even announced that the thing has has a response. And and this is this is so easy to do. It's one attribute.

Dominic: 47:41

It's called aria it's either aria announce or aria polite. So but but that is that is minimal. I would I would be able to live without that. But now you need to navigate to the response, and they don't even have a good way for me to to start from, the input box and go to the to the response. I need to have some crazy work around.

Dominic: 48:04

Just adding a a header there would, you know, with my question would would have worked for me. Mhmm. So sometimes it's it's not even the property of of elements, but the just the elements themselves.

Peter: 48:18

Yeah.

Dominic: 48:19

Just put headers everywhere. When you have when you have a a loop, you're you're looping something. Let's say you have a list of comments in in a page, just have the title of the of the comment b and h four, for example. Mhmm. That will do, you know, 90% of of what a a screen reader user needs to to do to navigate quickly to, to that.

Peter: 48:43

Yeah. So so, actually, one of the, one of the so in my this, like, example projects, it's also where I'm kinda, like, building different helpers to to help, like, write test. And one of them is, like, get the title of the page, which looks for one h one element, like, one heading one because there should be one and exactly one, for example.

Dominic: 49:04

Yeah.

Peter: 49:05

And will make the test fail if if you have zero or many.

Dominic: 49:11

Yeah. Yeah. Exactly. So is is there am I did I read that correctly? But I I think on the issues, on GitHub, you you were there there's something like you would want to build, you know, a Go v eight from scratch.

Peter: 49:32

No. No. No. Not so much. No.

Peter: 49:36

No. But that's another thing. Actually, like, starting using v eight was a, a mistake in some point because I didn't, do my research. There's, like, there is a, there's a pure Go JavaScript engine, called Goja.

Dominic: 49:54

Yeah.

Peter: 49:54

Which I am sort of like on the side adding, support for as an alternate alternative to v eight. But but no, it was more like, okay, I, it could be fun to write your own JavaScript engine, but, that would take quite a long time be before we would then have a working browser. But but, but it would be nice to have a pure Go solution. And, yeah, I didn't know that Go to existed. And so but but now that I have v eight, it will definitely stay because you have you have a JavaScript engine where you can be sure that it's being updated with the latest JavaScript features.

Peter: 50:36

So whatever you whatever you use in your project, v eight will support it.

Dominic: 50:40

I I was using Gosha in in static back in my open source project. I I did, three years ago, to to run some kind of server side function. It it was it was a a back end as a service platform to, Mhmm. But, but, yeah, I I well, it it might have changed. But at the time, the, you know, the JavaScript that this thing was able to, to execute was ECMA ECMAScript five, not not anything earlier than that.

Dominic: 51:09

So I won't I I'm I'm unsure if HTMX would have worked with with it.

Peter: 51:15

Yeah. That's a good question. That's also what they sort of still mention on the readme file. So which is like ECMAScript five is actually the, version that never got, like, off officially published. So I would also say ECMAScript five would mean that probably, like, ES six modules wouldn't work because they were introduced in ECMAScript six.

Peter: 51:42

Yeah. But but I don't know. I'm I'm continuing to add support, and then let's see if HMX is going to to work when I get it, when I get the the Go gen implementation up to date with, v eight.

Dominic: 51:58

Yeah. Sure. So that that is great. I mean, it's a it's a nice project. So do you do you need any any help?

Dominic: 52:06

What what are you looking for, in terms of, you know, community and whatnot? Is it is it something that that you you want to to build further?

Peter: 52:16

Yeah. I would I would love to see this, this, like, becoming a, like, a mature product that you can actually use, a library. And, yeah, definitely, I could use help. Like, specifically, like, if there are people who have experience building browser engines, that would that would, help a lot. I mean, I've done web applications for twenty years, and I'm surprised at how many things I didn't know, like, how many things that didn't work when I started doing this.

Peter: 52:48

Like, so, yeah, that's definitely I would love to get other contribute contributors. And, yeah, I've had a lot of, like, spare time because I wasn't like, the freelance market has been, like, pretty poor, which means I've spent a lot of time doing this. Like, if I could live live off contributions, like, open source contributions, and then, like, dedicating my time to this, it would be, like, pretty amazing. But it's, it's not in the near future that that would happen. Yeah.

Peter: 53:21

So at the, so so I would need to take some paid jobs, and then obviously, the productivity on Ghost would go down.

Dominic: 53:30

Yeah. Sure. What what I've seen, with, you know, people that that are able to to sustain themselves with their their project is, well, often, it's it's because there are one or two companies that that are sponsoring them for, you know, something substantial. Yeah. Because it's pretty, pretty hard to to squeeze some, some, you know, monthly recurring revenue from, from users of of a library I I I found.

Peter: 54:01

Yeah. If anybody from Google is sitting out there, hey, they Yeah. This is going to add value to Go, the programming language.

Dominic: 54:10

Oh, yeah. I'm sure they are listening as well. Yeah.

Peter: 54:12

Yeah. Well, you can and maybe we're like a a a a large enough donation can may persuade me to only use v eight perhaps.

Dominic: 54:20

Oh. Oh. The message is the message is sent.

Peter: 54:24

Yeah. So

Dominic: 54:26

So, yeah. So, yeah. Thank you very much. It was great. So do you have any any closing, closing thoughts in here?

Dominic: 54:32

Any any resources? Any anything that you want to say?

Peter: 54:35

No. I don't I don't think so at the moment. So

Dominic: 54:40

Perfect. So alright. So it was, it was great. So thank you. And maybe, maybe we can, redo an episode later when when there's, some milestone that you have reached and things like that.

Peter: 54:51

Yeah. Sure. If you want to do something, like, specifically, on TDD testing and stuff like that, I'd be happy to to also

Dominic: 55:00

Oh, yeah.

Peter: 55:01

Participate in that.

Dominic: 55:02

Yeah. I would like that. Alright, Peter. Thank you.

Peter: 55:05

Yeah. Thank you. It was nice being here.

Dominic: 55:12

Alright. That's it for this week. I would really appreciate if you can talk or share about this podcast. It it's always helpful. Also, another way to support this show is by purchasing my course, there is always a link in the show notes.

Dominic: 55:28

So on that, see you next week.