054: Datastar with Delaney Gillilan
Hello there, I'm Dominic Saint Pierre and you're listening to Go Podcast. Today I'm joined by Delaney Gilliland and we talk about Datastar. If you are on the Gopher's Slack community, I would appreciate if you can join the Go podcast channel. So it's in one word. We are there.
Dom:I'm not sure, you know, where where are the listeners, of the show because, you know, the stats show that we, we are a lot of people and, yeah, the channel is, is a little bit quiet. So if you are there, I would appreciate if you can join. I'm also considering, the idea of maybe taking a Patreon for the podcast and see if, you know, because it's it's a lot of times, a lot of efforts. And, and, yeah, I'm I'm looking at the option at the moment. So on that, I will leave you on with the interview with Delaney.
Dom:Hello, gophers. So I'm super happy to receive a special guest today. So actually, he's the creator of, DataStore. And I I will I will let you introduce yourself. So I I'm, you know, I typically like to, to ask things like, what what is your path, you know, technology wise?
Dom:How did that did you came across Go at some point and exactly that's why anything that you want to, to let the listeners, know about you, it will be it will be great.
Delaney:Yeah. So I'm Delaney Gilliland. I'm the creator of DataStar, and I've been using Go for probably about ten years. I had a project that was in Node, and I basically it was a long running task and it was running out of memory and having all kinds of issues. So ended up saying, hey, I'm gonna try this Go thing.
Delaney:And it was, I think, 30 times less memory, and it was 12 times faster. And it was before I even started to learn how to optimize it or anything. I I just fell in love immediately, and the code was shorter, ironically. So I've been doing it for quite a while. I'm currently at Synadia, who are the is the company behind NATS.
Delaney:A lot of Go developers use that. And I ended up making a lot of real time dashboards and other kinds of things with it. And, yeah, the tools for the front end just were not there. So, that started the journey, which then became Datastar.
Dom:That that is great. So it yeah. It's it's very similar to my, my story with with Go, actually. I'm a little bit curious now to hear, about this this node. So what was that in a time where, you know, promises was not really a thing, and now you had some some kind of callback hell and things like that?
Delaney:Yeah. That's exactly right. It was Yeah. I was using the old there was I think it was waterfall. I forget the name of that project.
Delaney:But, yeah, basically, it was called back hell all the way down. Yeah. It was rough.
Dom:Oh, yeah. Yeah. I hear you. I hear you real on that. I was I was building a SAS back then.
Dom:It was called LeadFuse. And and, yes, at at some point, it was not ironically, I was the CTO and, you know, Node was not really my decision. And at some point, I woke up and one day I I said, you know what? I'm rewriting this thing and go. And boom.
Delaney:Well, the funny thing is the whole node thing was like, hey. We finally can have a single language to do the front end and back end.
Dom:Yeah.
Delaney:This is gonna be magic. Like, it's just gonna work. And the irony is that now the project I work on now allows me to be in one language, and it's amazing. Like, it's really great.
Dom:Right. Be because because I guess you are using, like, Go for for to to do front end. Am I mistaken?
Delaney:Yeah. So, basically, what DataStar is, for those that don't know, it spun off from HTMX. Basically, there is a bunch of things inside HTMX that I, didn't agree with. I wanted to make things that were more pluggable and make things type safe and all these things around, how you create plugins and stuff. Like, there's extensions in HTMX, but, like, I wanted to go further.
Delaney:So every single thing was a plugin. And so long story short, I I went off and did that. And the cool thing is is that the plug in system has, built in fine grain reactive signals so that you don't have to put in Alpine and HD Max or Hyperscript and try to make all these things work together. It's one cohesive answer. And the kinda cool thing is that it's actually smaller than either Alpine or HD Max, even though it replaces both of them.
Dom:Yeah. That that that's what I was I was going after because I'm a huge user of HD Max and Alpine myself. I've been doing that for a long time now because at at some point, it's pretty pretty comfortable. So I'm I'm curious to hear a little bit about what exactly are we talking about the HDMX when you say that there were there were some, some vision that you had that that might not align with what what it is, or or what are we talking exactly?
Delaney:Sure. So I got into, doing HMX a few years ago and got right into it with Carson going, hey. Like, this doesn't do enough. And to simplify it really is that HTMLX is a great library that's trying to solve the problems of the early two thousands. It's basically trying to extend, HTML with just a little bit extra to be able to put fragments on the screen.
Delaney:Right? But the problem is is is that that's a very MPA way of thinking, and that's great. And most apps are closer to MPAs than the SPA single page app style. But there's still a lot of reactivity that happens on the page, and so it's not a complete solution. So it's funny.
Delaney:I see these SPA and MPA advocates fighting with each other, and I'm like, I think both of you are wrong. I do think I think I do think the MPA is a lot closer, but but there's still reactivity on the page. There's still things that are in the control of the user. So you need that fine grained reactivity. You need that stuff.
Delaney:And the problem is things like Alpine and HD Max have a completely different idea of what the main loop is and where things are. Like, for example, event handling is completely wrapped over each other. Right? Like, Alpine and HMX completely disagree on things. So if you make a unified set of plug ins that are aware of each other, now you can do things in less code.
Delaney:It's actually more efficient, and it's smaller.
Dom:Yeah. Yeah. I hear I hear you on that. This is this is an interesting. So so SPA.
Dom:So MPM multi page application. Right? The the Mhmm. The old school. Right?
Dom:Yeah. Right. Yeah. I I saw I saw your your project a couple of months ago. Someone someone replied, in in on Reddit.
Dom:I I don't recall exactly. I I haven't heard of of it yet. And I have looked at it, and I immediately said to myself, wow. Okay. Okay.
Dom:So to me, to my current situation at the moment, you know, just combining Alpine h HTML seems to be to be a good value. But, you know, fundamentally, what what is the the differences? So am am I mistaking that you are, like, using a lot of of data dash and and things like that? So there's no, you know, the, the at click, for example, for to to capture some some unclick event in Alpine. So you you all do that via the, the data attribute.
Dom:Right?
Delaney:Yeah. So the whole name of it is literally there's something built into the HTML spec called data dash star attributes. And what it means is that you can put on your own, keys and values. It's basically a map of string to string for any element. And when I first saw that speckle years ago, I was like, I don't get it.
Delaney:Why do I need this? Like, I already have all my state in the front end. But once you can basically send down these, little pieces like data dash signal, data dash on click, on debounce, data dash class, all these different things. Now I can hook into and listen to that with some a little bit of shim of JavaScript and basically build out these fine grain reactive expressions. So expressions are not something that's built into JavaScript out of the box.
Delaney:And what I mean by that is that when you want this thing to run whenever one of these, signals is updated. So you're the code that you write is more like a formula in an Excel spreadsheet. And the the thing that's different is that, for example, Alpine's using these object proxies, which are really not the slow it basically it's in it's an older style of doing things. And we're using fine grain reactive signals. We're using this the the lowest level thing, and we actually generate, the functions the same way as Alpine, but we do it in a unified way where all the plugins can work together.
Dom:Yeah. Okay. I think we will need to unpack a little bit about where I'm I'm not 100% sure. I I I get it. So what is what is the the low level react, reactive way?
Dom:I'm I'm not sure I understand that.
Delaney:So, for example, it's very similar to what you're used to doing with, like, x or the data x. It's the same kind of style, but what's happening is that it's everything is a plug in in a way that like, Alpine has a very, you know, fixed set, and then you can add extensions. And the way they've written that is there's a very big core. Our core is, like, less than four k of like, it's the actual engine.ts is, like, 250 lines. So we have we're we're much more built around the idea that you can pick and choose.
Delaney:In fact, if you go to the data store website, you can go to the bundler page and actually pick your own bundle, and it can be down for to, like, 5.5 kilobytes all the way up to including every single plugin that does everything Alpine does and everything HDMX does is less than 15 k. It's, like, 14.4 right now.
Dom:Wow. Okay. Okay. Okay. I I get it.
Dom:I get the the the general idea there. And when you said, you know, now I'm able to work in one language. So were you implying that, you know, HTML is is a thing and now you you can, you know, more focus on on Go, basically?
Delaney:Yeah. So one of the things that people kinda gravitate towards DataStar because one thing that makes us very different because the the interactivity does feel a lot like Alpine, in a way. But the interactivity part is where we I think we do it better than anybody else. And what I mean by that is everything in Datastar with the built in plugins, again, you could change this if you wanted, uses SSE, which is, server sent events. And the thing is people didn't realize for a long time that server sent events are a basically a superclass of what you can do with a regular return of HTML.
Delaney:So with a server sent event, you can send down zero, one, or infinite number of chunks down to this to the user, which means that you can change your thing from a crud app and just use service and events, and it feels exactly the same as you're used to, or you can send down a million events. And I have, videos of me doing, like, hundreds of millions of, data updates into a web browser, and it's, like, a nonissue. In fact, one of the projects I'm working on at work, I'm literally doing a hundred million keys and watching it update in real time.
Dom:Well okay. Okay. Wait wait a minute. So so does that means that so if it's using SSC, so that means that I do as a developer, I do have something on the back end that allows me to broadcast, you know, events and things like that to certain browsers and things like that?
Delaney:So it really depends on what you wanna do. Obviously, working on NATS, we do a lot of, messaging systems. So, like, I use NATS as an event bus for a lot of stuff. But you can do just like you're used to doing in HTMLX today, you can do a request reply and be done. But you can decide how long that stays open.
Delaney:So for example, let's say you have a post that you're sending up to your server and you're like, hey. I have five things that need to happen before I can make sure that this person's at, like let's say you're booking a flight or something like that. Right? You can keep that post open and start instead of waiting till the very end to send a response. So, like, in in HTML, right, you wait till you send a response at the end.
Delaney:Right? You can start sending a response immediately, like, saying, hey. I'm working on this. Hey. I'm talking to the service.
Delaney:Hey. There's three other jobs happening. Here's your result. So you can do that on a post and actually get a real time response. Once you start adding this stuff in, it becomes so handy in every place that you usually would think, well, I'm gonna not do that.
Delaney:But once you get rid of polling, you can do all kinds of really interesting things. So you can set up an event bus and actually send commands and then pull them back down and make them completely independent. One of the examples is on the front page right now. There's a to do app. If you open that up in two different tabs, you'll see that it's a perfectly in sync app, no matter which one you make changes to.
Delaney:And that's because it's using an internal event bus and all that stuff, and it it it works fine.
Dom:Okay. And this this plumbing, if I can say that, is is per request. So you can decide so let's say let's say on your page, you might have multiple places where you are going to do a post and expect a response at some point. Some of those things can be can be served by a server side event, for example?
Delaney:Yeah. So the the one thing that's different if you're coming from HT Max, if you know that well, I think that the the way that, the out of bounds stuff works is just not correct. And so, for example, you know how you have to put h x target or h x select and HX bars and all these stuff. Right? Like, in DataStore, you don't do any of that.
Delaney:Like, that's all goes away. It it just a % goes away. And what happens is is that when you are sending the server sent events down, there's only five different event types. There's merge fragment, merge signal, remove fragment, remove signal, and execute JavaScript. Like, that's the entire SDKs that you see.
Delaney:But the thing is is that when you say I'm going to merge this fragment, if it has an ID and there's already an ID on the page, it automatically does ideomorph out of the box. That's also another thing. We include ideomorph in that 14 k. Like so you automatically can update any part of your page in that stream at any time, and it'll do the right thing and morph it and not lose focus, not changing out it handles move before all these, like, things that are coming in HD max, we already have support for.
Dom:Okay. Okay. Okay. Okay. So there there's there this is a I I won't say huge, but this is a major differences with difference with, with HTMX then, I I I assume.
Delaney:Yeah. No. It it makes a huge difference in practice. I guarantee you, if you take any app, even if you're doing doing regular request response and polling, it will be smaller in data star. It'll be less code for you to write.
Delaney:And also, as soon as you decide to make it streaming or anything else, there's no code changes. It's just about how much data you're sending and when you're sending it. So it becomes a back end issue, has nothing to do with the front end anymore.
Dom:And for yeah. And from the back end point of view, it is just standard s SSC. There there's not there's nothing on the back end that that is is related to Datasaur. Right?
Delaney:No. The only thing that's nice is that, because we only have five functions that you have to implement, we have SDKs in over 12 languages right now. And I am the one behind the go one because that's original I that's what I use it for. But what's nice is I have stuff built in to handle, rendering directly from temple into SSC events and also from GoStar, which is my other it's kind of a fluent type safe version of components. So and then you can do raw strings and all these other things.
Delaney:So the thing is that that stuff's just built in, and the entire website, for goat people is one single, binary host on Flight IO on the free server. So you get to see real time streaming events with millisecond or microsecond timings, and it's on, like, a free tier server.
Dom:That's cool. That that's yeah. Okay. That sounds cool. I I received a Marcus from component last week.
Dom:It was Mhmm. The second time. So, that episode is not posted yet, but I I was mentioning, I I was talking with him about Alpine and things like that. So so I was not very aware of GhostR as well. So when you say it's, so basically, you you have built something on on top of component to make sure that it's it's now it's now possible to to call the it's easier to, to structure the the data on the HTML elements, I guess.
Delaney:Yeah. So I love what it's funny. I actually started Ghostart because, Adrian from Temple was not, didn't even have a website. He didn't have a lot of things. I think some of the first issues were mine kinda saying, hey.
Delaney:I I want these things. And the component stuff is really good, but it was it's too there's too much any in there. Like, you can put stuff in Yeah. The avoid start. Like, you basically add children to an input, all these kinds of things that are, like, easy to mess up.
Delaney:So I I wanted I loved the component interface. So what I did is I actually took the entire HTML spec and wrote it out by hand in protobuf and then made a code generator so that I can actually code generate every single valid conditional state that basically allows you to basically do components, but to do it in a fluent interface that's type safe.
Dom:Interesting. Okay.
Delaney:And I still recommend people use Temple from, like, 99% of things. But if there's if something is very dynamic and you still want it to be type safe, I think GoStar is an interesting choice.
Dom:Yeah. Yeah. Yeah. Exactly. So for for me, Temple is is all about, difficulties with with my screen readers and things.
Dom:I I received, yeah, the, Adam, I think the the the author of anyways, that that's a different story, but, yeah, pretty pretty cool product. So so what what about what about migration? So let's say let's say I have a an existing project at the moment, and, I've tried Datastore. And now I'm I'm I'm super happy, and I I would like to to migrate my current project. Is it is it have have you have you done that?
Dom:Is it difficult? Any anything that that we should know about?
Delaney:Yeah. The only thing I will say is that it we have a lot of people go through the process of unlearning because there's a lot of things that you kinda get forced into when you think about because the problem is you have the SBA and the MPA style. And when you can actually do something that's just truly just about hypermedia and about what can I show on this page, which is a resource? Right? Like, what can I do on this page?
Delaney:It's the most interactive, most interesting things. It changes the way you think about things. So a direct port of a HDMI site, it's gonna be fine. Like, it'll work great. But it the real value is and when you make things bigger and faster, here's a perfect example.
Delaney:Like, we have the there's an HX podcast, by this guy named Lazarus, and he wanted to try DataStar, and he literally wrote Google Wave. That was his, like, hello world project. Right? Like, that's the kind of thing you you just wouldn't think about doing in HTML as, like, a real time thing where you can see people typing and going back and forth, and he was doing, like, updates to a SQLite database in every fifty milliseconds. These kinds of things you would just never think about.
Delaney:So part of it is, like, come in with fresh eyes. The same way that people came in from a SPA going to HTMX, I think there's almost the same difference between HTMX to Datastar. That's it. But it's it can a % do all the things that a spa can do feature wise and same with HD max. It's just it's a different beast in some ways.
Dom:Right. And and before you started to decide to write this this library, what have you have you tested the a couple of things like the, you know, all those live views, library out there, either with, I don't know. I don't know what what it's called. The one with with rails at the outwire or something like that. So so those things that, yeah, seems to be, I I would say the other way, compared to HMX, but but it's still kind of accomplished the the the same kind of functionality.
Dom:So what was your your thinking about those?
Delaney:Yeah. I know. It's it's it's a great in fact, my background, before doing, the stuff that I'm doing now, I was I had, like, top secret clearance. I was working on a bunch of military stuff and had done a ton of WebSocket work, like, a ton of work socket work. So when I saw the the live view stuff, I said, hey.
Delaney:I want that in Go. And I actually there was a project called Go Go Racer, which is literally me making that work in Go. And the problem is is that, a, WebSockets are a nightmare for actually deploying things. The nice thing about SSC is it's just a normal HTTP fetch request. That's it.
Delaney:There's nothing special about it. It just works over every firewall, doesn't have a special handshake, doesn't have any extra cost. It's literally the cost of, like, one extra header if you don't set any headers. Like, that's all it costs. So the problem with live view is it's actually trying to diff and keep state on the client and server and trying to it's a basically, it's a CRDT problem.
Delaney:It's trying to do a conflict resolution and it's trying to diff these things. The thing that we found is that if you just send down the the big chunks of updates, the things like broadly and gzip and z standard compress it so well. You you're basically getting the diff costs anyway. Like, it doesn't make a difference. And I will say the other thing is is because now you're doing things in instead of doing a polling style, you're actually pushing as you need to.
Delaney:You can do back pressure and all these other things. So you're using a thousand times less networking to start.
Dom:Right. And and and this is the back end that decides also what what's happened with the data that it's sending, if I understand correctly, with with with your five, you know, different events that you are pushing from from the server. So, basically, if you want to happen at the end of a list, for example, this is the server that drives that.
Delaney:Right. And that's the thing that's it becomes really hard in HDMx because, basically, when you this is where my issue kinda comes with HDMx. This is, like, the fundamental issue is that I think a lot of your back end ends up leaking into your templates. Mhmm. So, like, you where you what you're gonna target on the next update, what what variables are you the thing is is that you're putting stuff in the user's hands that they shouldn't have.
Delaney:Like, this back end decides when to update, when to send things, what's valid, and especially becomes more so when you have extra services that are going on or something else that's outside of their control. Like, if you're just doing a CRUD app to a to do list, okay, that's not that bad. But as soon as you start doing collaborative to do list, you're gonna getting updates from multiple people working on it or someone else's emails you. Like, all these things become there's a certain scale where it gets really hard to do that, and now you have to try to do weird polling things where you're updating the polling speeds in each of your templates. And that just seems, like, crazy as opposed to just push data when it's new.
Dom:Right. So what what happened exactly? So when when you when you decided to, you know, to jump into writing this library. So I'll I'll how did it went at at this at this point in time? And where are we in in, you know, what year exactly and things like that?
Dom:So what what what happened there?
Delaney:Sure. So I in fact, the irony is that I was trying to work on SSE and some other things, with HDMX. And I basically said to Carson, hey. I've read through all your code. I've read through all the Alpine stuff.
Delaney:I think we need to move towards, using TypeScript and, like, making this more defined, because you there's a lot of overlap in places. So I actually there's a I think it's rule seven b in the HTML community of, like, you have if you're gonna say something, you have to go build it. So I I literally did. I ported, HTMx to TypeScript, and I found, like, 700 errors. And a lot of it was just things like like making assumptions on on elements and a bunch of you know, like, a lot of it was that some of the type checking.
Delaney:I basically got it down to about 300 errors. And then basically, it was like, yeah. We're not gonna fix this. The good news is that the latest versions, they've actually done more work to try to use JS doc and all these other things. But the thing is that the the core implementation is still, like, one big 5,000 line thing that, you know, doesn't do all the things I wanted to do.
Delaney:So I basically said, hey. I wanna make this work. And this literally was called not H t m X 2.
Dom:Like, you
Delaney:can probably find it in my repos. And so it was gonna be my idea of what H t m X 2 was gonna look like. And they said, yeah. This is a bridge too far. It's not gonna work.
Delaney:And I'm like, okay. Cool. And, like, I will say, Carson has been Carson behind the guy behind h t max has been nothing but supportive. Like, the bumps that I've gotten on Twitter or whatever, like, people knowing about it, he's the first person to say, like, hey. Like, you should go look at this.
Delaney:Because HTMLX is solving a problem, and it's a library, not a framework. And it's doing a great job at trying to fix what was wrong with the browser years ago. I don't think it's the right answer for most things today, but I understand why he built it.
Dom:Right. Oh, yeah. Yeah. Exactly. I mean, this this is a this is a life of open source project as well.
Dom:You know, at some point, you it's it's it's normal to, you know, idea change and things like that. So but what where are we in in the in the time in the timeline of life at this point?
Delaney:So that was roughly, like, eighteen to I think I, actually made it public around eighteen months ago. And I did my first talk about it maybe, about six months ago. So it's just been slowly growing. And the thing is I use it actively for doing work stuff. So it's now in version one, beta one beta one beta nine today will be the I think it's the very last beta, and we are already we're at v one.
Delaney:All the things that we've been changing has been around ideomorph and, like, some extra features that people have been adding. What's been nice is that even though we're on beta nine of v one, there's nothing that couldn't have been a point release after v one launched, if that makes sense. So we're I feel like we're really confident we'll be releasing it probably in the next couple weeks. And, yeah, it's done. Like, there's not much else to do with it to solve problem.
Delaney:Yeah.
Dom:And I and I guess you would you, you know, yeah, that that that's it. There's not much to add after after the fact. You know? There there's not much breaking changes that that are expected in those types of of things.
Delaney:Yeah. If anything, it'll it hopefully will get smaller. Like, if signals become part of the TC 39 standard, then that that it gets smaller in that way. If the like, we have a view transition API and some movie for stuff that's not in all browsers, once that stuff comes in, it's gonna get smaller. So the irony is that in most places, it'll get smaller.
Delaney:I do think that there'll be some interesting plugins that other people do later on, but the core engine of what it does is basically it allows you to write hypermedia in a way that is not gonna change. Right? Like, it's built into the browser. So we're just basically extending the the abilities of the browser to really do dynamic hypermedia.
Dom:Yeah. Sounds good. I completely missed it, to be to be completely frank. I haven't seen it. I I was I was browsing the, the GitHub repo a couple, you know, just before we enter into this call.
Dom:And, so so Go is one of the language in there. So I've in in my mind, that that kind of of library would was kind of TypeScript, you know, most of it. So why why there is so much Go in in your repo?
Delaney:Well, the thing is the entire site is written in Go. And in fact, we don't use node modules or NPM at all. We don't use Vite or Vite for any of the front end stuff. I we actually use the the website. It's actually a go ES build compiler of the the, TypeScript.
Delaney:So it actually lie when you change the bundler, it's actually live building you a custom version of DataStar on the fly. The compiler is built into the into the website. So so the entire website is written in go because that's what I tend to use all 50 to 60 examples plus like more even even more tests, are all written in go. So, yeah, I think just go is a great combination of, a solid run time and everything is channels and push based in Go routines. It's what I think that the future is gonna be for doing web development, because it has all the right features to to move quickly and yet still be very performant.
Dom:Right. Have you found that the adoption is because because let let's face it. The the Google community is is pretty fond of HDMX at the moment. Mhmm. So what is, you know, what is the reaction so far?
Dom:And have you have you seen a lot of adoption in in other community, for example?
Delaney:I what's weird is we're getting a ton of, movement in the, like, Elixir, Erlang closure. Yeah. Because the thing is it basically what I'm the dirty secret is is that DataStar is a functional reactive programming dream because all the things that they were trying to do with React, around tree component update and all that, I it's native in in, DataStar. We do, like, immutable hash structures and all these things to to make things fast out of the box that are not the norm in this space. So the the irony is that DataStar is actually, like, an endofunter in the truest sense of the word that allows you to actually do, like, all the monoid stuff that they talk about that they wrap, react, or try to make happen.
Delaney:DataStar just does it natively, and we're way faster.
Dom:Oh, you said you said the m word I open. I haven't I haven't lost too many listeners in here.
Delaney:Yeah. Sorry. I the oh, and then the thing is that being a GoDev doesn't mean that you don't take good ideas. Right? Like, the idea the the the thought process that, hey.
Delaney:The I can render the back end and do whatever I want and everyone in every front end sees that as a projection. That's a good idea. Like, we should all be doing that.
Dom:Yeah. So so what is what is the strategy from here? So how are you? Because it's I mean, it's it's still it's still a giant desk task, I I would imagine. So what is what is your vision for for the project in in the short term, maybe midterm?
Delaney:Yeah. That's a good question. I think that the community's been growing. Like, we had it started up a, Discord server only a few months ago, and we already have, like, 500 people there. It's around 1,200 stars now on, on GitHub, which is good because, a few months ago, it was, like, like, 300.
Delaney:So
Dom:Nice.
Delaney:Yeah. So it it's getting adoption. The thing is I don't care if it's the most used framework. And if you're happy with the HMX, great. But I think that, for me at least, is that I want people to know about it.
Delaney:I don't care for the biggest or adoption. I I really don't care. I use it for my active job, so I don't need it to be a big thing. It's it's it's more that, like, hey. This is a really good idea, and you should try it and prove me wrong.
Delaney:Like, see that if you have a better way, I wanna know about it. But so far, I've not seen a better way. And the thing is, if you're coming from like, if you love HTMX, but you say, like, man, I really wish I could do, like, streaming stuff or I want to be able to make it it's actually smaller than HTMLX, and it's less to learn. And if you like HTMLX, you you'll probably like it, but, like, give it a fair shake because it's it is different. But with the SDKs, it's actually less code.
Delaney:So
Dom:you mean the the back end SDKs?
Delaney:Yeah. Because they already has the back end SDK. You have these little helper functions to just make it a little bit easier for you. And it's it's less code to write. It's faster.
Delaney:It's smaller. Like, but also the thing is is that if you don't wanna use it, I I would actually be really curious to see why because it's it does everything that these other tools do, and it's less code for you as the user.
Dom:Yeah. Yeah. Habits, maybe.
Delaney:Yeah. Well, and then the thing is that's kinda interesting. You have a valid point of, like, I think that my, my part of the project is slowly kinda atrophying because I did the part I did the hard part of getting, you know, like the plugins stuff in place, making it so that all the core plugins are there. Like, now it's about community and people trying it out and seeing what works for them, what doesn't work because it already works for me. I'm using it every day.
Delaney:I dog food it every day. So, like, I love it. I we the people that are on the Discord server seem to love it too in every different language. So I think Go is the ultimate version of this, of a runtime to be able to do this kind of stuff as far as I can see. So any other Go developer, like, please come try it.
Delaney:Tell me I'm wrong. But, man, it is so much easier than every other tool that I've used.
Dom:Yeah. There there's there's a lot of movement. Let let's I've I've also started, you know, doing gold, like, ten years ago, whatever. And I don't know if it's just me. There there seems to be a lot of things going on with, I I won't say the front end, but let's say let's say the HTML part of of building a web application in Go.
Dom:And and and it's it's it's it's interesting to see that, you know, it's not that's only just, back end API server that that can be built there. And and, yes, it's I also think that it's pretty comfortable to to be on.
Delaney:Yeah. Well, and it has this beautiful mixing of both being low level and being able to do high high level things, especially with the cogent things like temple and all that. Like, I really do think it's the fastest way to build applications and to build them in a performant way. In fact, there was a little while ago, there's a pot or a streamer called Theo that did a thing so said five he made five frameworks and one app. And, someone said, hey.
Delaney:I think again, one of them was go. And so if you go watch this video of Theo, I think he did go really dirty. He's like, oh, this is so hard, so complicated, so everything. And so I took his example, after trying to talk to him about it and basically made it my version was actually the smaller than the the optimal version they did in their own stack. It was smaller.
Delaney:It was, like, an order of magnitude less memory, and it was faster by quite a bit. And it did, like, all these other things, ended start ended streaming. And, yeah, then I never heard a response after that. So, like, you know, some of these like, this is where I'm not very good at the social games because I don't play nice because I'm like, hey. Are you ever gonna respond?
Delaney:So yeah. Like, this is where like, I'm not as good at being, like, a person that builds up a community. I think Carson's done the best job anyone could ever do in that. And that's just not my my general personality. So I will slowly fade off into the desert, on a long enough timeline and let, like, more social people, get involved.
Delaney:But, yeah, like, I know that it's, like, one of the fastest ways to do this stuff. And about the only other way that you can make it faster is to rewrite some of the back end stuff and dig in Odin, but you don't have the closures. You don't have the nice, like, runtime and all that stuff. So I just think Go is the best tool for doing this kind of stuff.
Dom:Yeah. The tooling as well, I I must say. That that's my that's my problem with Zig myself. Mhmm. Or even Ross as well.
Dom:I'll although, you know, Ross isn't isn't that terrible, but the the language is another beast. But I can understand, you know, let's let's return to this five example. I can I can understand, you know, someone that is not really used to build web application in Go? You have to admit that it's pretty different. Let's say let's say from Django or Rails or any kind any kind of of, you know, web oriented, framework like that, the experience is extremely different.
Delaney:Oh, for sure. But I also think that you're much closer to the actual wire. In fact, one of the things that was kinda funny when we had people that are working on the dot net SDKs, for DataStar, we were having a really hard time finding out, like, where the actual, like, bytes from the request came from. Like, how do you get to the actual raw stuff? It's all
Dom:these Right.
Delaney:Like, there's, like, 20 layers between you and, like, what happened on the wire. It was really funny. Whereas I think Go is the right is the right amount of thing because you can build up to be do higher order components and doing all these kinds of ideas, but you don't have to start there. You learn what's actually happening over the wire, and it's the one thing that's really nice about Go is that it's it takes more to write than it just does to read. Like, reading it is wonderful.
Delaney:Writing it is a little bit of a slog, but with the tools like Copilot and all that stuff, you can get if error auto completed for you nowadays. Like, that is just a nonissue, for Go. So I I love the fact that, yeah, it's a little bit more to write, but, man, I know exactly what's going on at every line.
Dom:Oh, yeah. Oh, yeah. I've said it before. I will say it, you know, the the if error for me is is just good. I mean, I I like that.
Delaney:Yeah. Errors are values, people. Like Oh, yeah. Like, that's fine.
Dom:I was worry I was working on a on a Python code base recently. And I mean I mean, this this is this is insane. This is crazy, the amount of things that you need to to be careful with with a with a try catch and things like that. So do you do you see a lot of how can I say that? To to not not be weird or things like that.
Dom:So there's a lot of react out there. How do you, you know, how is there is there any any enticing, thing for them to to try a solution like Datastore?
Delaney:Yeah. So I have, a little while back, I did a a talk where I was showing updating the I was doing a a real time three d game engine inside of the browser, and I was updating every frame with DataStar. Like, using, like, straight web comp or using web components and driving those with DataStar. And I was doing thousands of elements on page, and I was doing them at 44 frames per second. And it just works.
Delaney:And the the the timings are, like, in like, to render a page, it was, like, a millisecond. So DataStar is capable of doing some crazy things. In fact, there's a a thing called DBMon, which was, like, the first React demo that said no one else can compete with us on this demo showing it off. And it's on our website. It's called DB one.
Delaney:And even coming from the back end to the front end, I'm doing that in, in, like, two to three hundred microseconds. Like, it's just the things that people say that you have to have a spa for, it's just not true anymore. It just it's in it's an invalid position. And I think that that's where the the the fight is happening right now is that it's SPA versus MPA, and Datastore can do any feature that either one of those things can do out of the box, like, with no extra tooling. So in less than for a onetime 15 k cost or less, you can you can be in the language of your choice.
Delaney:And we have a lot of people, like, for example, in TypeScript. We have a ton of people that use that in their back end using, like, Hano on, Bun. So they're really using Zig, and they're using a layer on top of it. And Right. They're they're getting great numbers.
Delaney:So, like, I think it's it's a lot of times, especially with the new like, I like to react back in the, like, zero dot six days or, like, that was that was, like, a decade ago now. But the thing is it was a library for doing, like, diffing of stuff because it is a good way to think about the problem. The irony is that when you have things like Temple, you basically have JSX in your back end, and now you can, like, take the data directly from your database and put it into your response. One of the things that I love showing people is that a lot of times, Carson talks about locality of behavior. Right?
Delaney:Like, getting things close. Well, there's nothing closer than data star because you can literally say, hey. I'm gonna fill in this table. You start your SSC event in every row that you pull from SQLite or something. You literally start sending that event immediately, like that row.
Delaney:So you can literally take directly from the database directly into your template, and there's nothing faster and it feels basically instant. So instead of the thing with JSON and React, you're waiting for the entire JSON response to come back. Right? Like, you're waiting for that. Whereas Datastar, you've already loaded half the table before that, like, basically, before the person even realizes that the table's loaded.
Dom:You're talking about a yeah. Okay. Yeah. I I think I understand what you mean. So the the the table the table rows that need to refresh when you when you scroll?
Dom:Is it am I
Delaney:no. Like, when you're first rendering the table, like, let's say you have a a data grid, and you are trying to get that onto the page. In a React way, you would say, hey. I need to make a call to the back end. It had to be some JSON, and then I, you know, do all the VDOM stuff to render out the grid.
Delaney:Right? Yeah. In DataStar, you say, hey. I need a data table. It starts sending down the the rows as it's pulling it from the database.
Delaney:So instead of waiting till the end to send you back JSON, it's already sending you the rows as it's getting them.
Dom:Because of the SSE?
Delaney:Mhmm. So you can start doing that immediately. So the request response time is, like, as fast as your physical connection can be.
Dom:But your your your data query is still not streaming. So how do you how do you send the first record?
Delaney:So I tend to use, I I use a lot of event sourcing. So I end up using SQLite locally, for most things. Like, SQLite can handle millions of rows and all these things. And so, like, I can be streaming directly because it's within process. So I can have my database actually directly have all this information immediately.
Dom:Interesting.
Delaney:Like, we're talking mic nano to microseconds for most of these things to start up and start sending data back.
Dom:Yeah. Okay. Who's responsible for closing this connection? When when is it is it like the the GoEndler that one once it's it's finished, it's it's it's just closing the the SSC connection?
Delaney:Yeah. That's a great question. So you can the nice part with the SSC is you can control it from both sides. So if you're in your regular handler, it's just a normal handler that you have in, you know, the the reader writer, handler. And if you return, you've closed the connection.
Delaney:If you do if if it's a r dot, context dot done. Right? So if you're just selecting on that, if that goes if that gets triggered that channel gets triggered, then you're done or they're done. So it doesn't you you actually can see you can find out if they closed it or if you closed it. And the nice part with SSC is if they somehow lose connection, like, if your server goes down, SSC has built into the spec.
Delaney:It automatically goes and tries to reconnect for you. So one of the things DataStar has built in is the ability to do retries, exponential back offs, all these kinds of things. So if your server dies and comes back up, they'll just get back to where they were in the request. The other thing that's nice about that too is that we actually do things automatically for you that won't happen in regular SSC where if you tab over to another page, it automatically stop listening. And if you go back, it'll turn it back on.
Delaney:And you can obviously turn that on off and on, but that can save battery life and all these other ideas too. So you can have these real time streaming things that automatically save battery when you're not looking at them.
Dom:I like it. But let's say okay. Let let me test let me test that a little bit. So let's say I would like to replicate a a single page, Meaning that, I will I will need some router at some point to catch when I click on the link. Let's say we have let's say we have a a menu, an app bar on the top.
Dom:And I I would also you know, let me let me know if it's a good idea or it's terrible, but, you know, could we have some kind of, SSE connection in at at the top of of the HTML structure that that is always on? So let's say we are building a a live chat, for example. That that could be that could be a good example. So we we could have we could have some conversation on the left. Let's say we have a nav on the left, but our connection to the server is, you know, it's it's not going to close or reopen each time we we we switch to a new conversation.
Delaney:Yeah. Are you familiar with CQRS? Do you know that acronym?
Dom:Yes and no. I mean Okay. I I
Delaney:know. This is gonna be really easy. It's gonna be easier than, like, you just almost you're you're you're literally almost there day one. So the CQRS is command query responsibility segregation. And all that's saying is that you separate your rights from your reads.
Delaney:And in this situation, this is the way I actually do data star. But, like, for some people, it's a bridge too far because they're just used to pulling and, you know, normal web stuff. But what I do is I load the page and I basically for I have an endpoint. And I basically say, if it's the first time you've if it's not a data star connection, render the whole page. Right?
Delaney:Like, you render that initial kind of shell. And then on that shell, there's a data dash on load that does the same URL up. And because it has a header request built in, I now know it's a data star event. Now I can start saying I'm gonna stream updates as the page happens. And then I also during that same time, I'm actually registering into an event bus so that anything that changes anywhere on the page automatically gets sent down and rerenders events.
Delaney:So I use NETs obviously for work, and one of the things that we have built in is a key value store. But the nice thing about it compared to, like, Redis or something, it's actually has a push based washer watcher interface, which means as someone updates a key, you can have someone that's just listening to that not pulling, but get pushed a message of what got updated. So now I can listen to, like, a hundred different keys throughout, like, basically merge that together, throttle that, and then send down a an event, which means for a single page, I can have thousands of events or millions of events happening and basically keep up to date real time with one route. Nice. So, yeah, all the reads are coming in.
Delaney:So your view of the page is a projection. Like, it is a view. So that's a read. And the rights so when you want to do something on the page, you're sending off a command. So every one of those links on your page, you can end up being basically a command saying, hey.
Delaney:I want you to do this. I want you to update this. I want to change this, you know, increment this button, whatever the thing you're trying to do is. And now those become commands that go off to the server. And whether they succeed or failed, you're automatically gonna get that data sent back to you through the update channel.
Dom:That sounds pretty cool.
Delaney:Well, we the for example, on the to do do the to do on the front end website, that's exactly how it works. And the thing is, it's maybe a hundred lines of code.
Dom:Yeah. Yeah. What what would you say about people that that say things like, you know, there's way too much attribute on on the h l ML, element now?
Delaney:So the thing is is that normally, I would say I I, in some ways, I kind of agree because, like, for example, I think h x, h t max has a million h x tags now. There's, like, 40 or so 40 or 50 of them now, and they're just growing and growing. And the thing is is with Datastar, I tend to use two to three for an entire project. There's a lot that's there that you don't need, and you don't like, they're there if you need them. So there's things like for, like, doing dynamic classes for animation and all that stuff.
Delaney:But most of the time, you need, like, data on click, data on key down, some, like, look asides. Like, there's not that many things that you actually need to do most pages. So I guess my thing is that, yes, I agree. But, like, I don't know how to make it any less than what I did and still make it interactive. The thing is is that I will tell you almost 95 to 99% of all websites can be done with zero front end JavaScript or JavaScript unless you're doing a crazy three d engine thing or, like, some really complicated chart thing.
Delaney:But even in that case, you'd end up writing a web component and then driving that with data star, still driving with the the elements. So, like, I don't think there should be any user JS for, like, normal state for almost any page. And I'll take anyone's, if you anyone says that that that's not possible. I will take their, take the challenge. Because the one of the examples that I have been kind of thinking about lately is I made a I made a submission to the Big Sky Conference this year, and I want to do a real time multiplayer version of instead of the if you've seen, like or you've heard about the million checkbox website.
Dom:Oh, yeah.
Delaney:Yeah. Yeah. I wanna make a a a million cell shareable, spreadsheet that's updating in real time and say, okay. Now do this in React.
Dom:To compare? What what Yeah.
Delaney:Yeah. Just like because when people talk about well, people talk about, like, well, you know, the hypermedia thing can't work for certain problems. Like, okay. Tell me one that it can't work for because that's, like, that's the kind of thing that would be very hard to do. And and the thing, especially when you do a multiplayer version.
Delaney:Right?
Dom:Right.
Delaney:Let alone time travel and all these other things being able to go back in history and all that stuff. Like, these are things that are just I don't think are possible. And I think there's a lot of things that have been happening in, like, React, especially with, like, the back end and the front end and how the RC RC or the React compiler, like, how they're trying to make it do the front end and the back end. Like, they're trying to solve a problem that's fixed with a 15 k shim. Like, it's it's it's a it's a it's a weird thing, man, honestly.
Dom:But even if you yeah. Even even if it's SSE though, you still because you were you were saying that you are cutting on a lot of complexity, compared to WebSocket, but you you still, you know, you still need to it's still WebSocket at the end of the day. Right?
Delaney:It's not WebSockets. No. No. No. No.
Delaney:It's not at all WebSockets.
Dom:It's not.
Delaney:It's just a that's the thing that people keep not realizing. A if you do a if you do a request to a fragment in in issue max, right, and you get back the h t the text slash HTML. Right? So in your headers, literally, it's text HTML and then the fragment.
Dom:Mhmm.
Delaney:The SSE is literally just a text format that says it'll say event colon data star dash merge dash events. And then on the next line, it'll say data colon and then fragments, and then it'll have that same fragment. That's it. And then a difference in the header instead of being text slash HTML is text slash event stream. Like, that's it.
Delaney:Those are that's literally all the differences.
Dom:Yeah. But that that there's a TCP connection that that that need to to to keep alive or still open at
Delaney:some point. Exactly how your HTML works. When you go and send a request to, HTMLX, it's holding that connection open until the response is closed. The exact same thing happens in days are literally the exact same thing. You're just saying with the with the chunks encoding, you're just saying, hey.
Delaney:When you see two new lines, send it off to the user. That's it. That's literally the only difference. So just like when you're doing a form and you're sending up a, a big video or something, you're actually doing chunked response or chunked encoded. Right.
Delaney:It's the literally exact same thing, but in reverse, You're just sending down the text things and saying, hey. This buffer is good. Flush it. That's it. That's literally all the differences.
Delaney:So there is no WebSocket. There is none of the handshakes. There is none no sticky sessions. There's none of that. It's literally just a regular fetch.
Delaney:That's one of those things that people don't like, that's why what DataStar is doing to me is magic because it is just as simple as HTMX, but now you have the superpower. So instead of just sending down a fragment, here's I can send down different events. Also, if you get a disconnection, it automatically does reconnects. It automatically tells you, like, where you were in the responses. All these kinds of things, you just get for free.
Dom:Yeah. But who who is responsible for resuming this this this this process?
Delaney:I mean Great question. DataStar is. We just handle it for you. It's just automatic. The the browser is automatically doing exponential back off and retries and all these things for you to get back to the connection.
Delaney:Now that's different if you decide to you severed it from the server, it's not gonna re try to reconnect. But if if they lose their connection, they're gonna reconnect you automatically.
Dom:That is so interesting.
Delaney:Yeah. So it's like fetch on steroids because not only are you getting back the fragments, but you can also get back signals. So there's times where let's say your your DOM is in one of the examples that's on the website is called Bad Apple, which is the there's a Bad Apple black and white video that's basically a people use as a kind of a just a demo for different video filters and stuff. And we actually have a live streaming ASCII version of it coming from that that's a real time system that's on that $5 or the free tier fly.io server. So it's on a basically a garbage box somewhere in the cloud, and you're able to watch a real time video streaming of ASCII, and it's like no no amount of resources.
Delaney:And it's and it's a regular fetch. It's just a normal fetch.
Dom:That yeah. Okay. Okay. I'm I'm I I will need to to dig a little bit more than that.
Delaney:Yeah. No. It sounds it sounds it sounds like that's one of the things I've I've kinda actually had a bit of an issue with is that people don't believe me that it does everything that you can do in React, everything that you can do in HDMX, everything you can do in Alpine, and it's smaller than any of them alone.
Dom:Yeah. That that's pretty strong, I I must admit. I'm, I'm very, very curious.
Delaney:Yeah. I mean, that's that's more where my thing is that I want there to be a bigger community of people that are just, like, on the page of, like, what are as soon as you do it, I guarantee, as soon as you give it a a honest shake, you're gonna go, what have I been doing? Like, why didn't my thing that the only thing that I'm mad about is I didn't think about this five years ago. Like, I didn't I I was I should have known about this. Like, when SSE came out, I was like, well, that's just, like, a text format.
Delaney:I want binary and other kinds of things. Like, this isn't good enough for me. But then you realize, like, HTML is like this. And with the the compressions and all the things that's happening, it's actually pretty damn small. And then, like, when I saw web components, web components didn't make any sense to me because you already have all your state in your Yeah.
Delaney:Reactor, your view, or anything like that. But now if you have a if you basically I can make a web component, and then any other framework from any other back end now can use this web component, and we can drive it with Datastar, which means I can make something. And now the people there, the closure people, or the electric people, or the go people, all are agreeing on how to use these components. So now you're building a true community for the web as opposed to what we have now.
Dom:Right. Pretty pretty exciting. Very, very nice. So, so yeah. I think, do do you have a any any closing thoughts here?
Dom:Because I I I think we will need to wrap up. But, but, yeah, this sounds great. I'm wondering if if you have anything that that we haven't covered.
Delaney:No. I I I would just want people to to give it a shot and come join the Discord. We're we're pretty friendly, especially to people that, like, actually are trying to make things better. Let us know what you think. And, yeah, the v one is, like, literally around the corner unless someone raises something really big.
Delaney:It's done. So, it'll be exciting to see what people can build with it, and I would highly recommend trying it out.
Dom:Great. Thank you. Thank you, Delaney. It was, it was a it was not nice talking to you. And, maybe, maybe, we can do we can do a version two.
Dom:I I like I like to retalk to guests while once I tried their their thing. So, I mean, if if you're up to, it would be nice.
Delaney:Oh, I would love to. I would really want to see what other Go developers think of this. I tend to, like, be very big advocates. So, I'm happy to help you in any way possible.
Dom:Very cool. Thank you.
Delaney:Alright. Thank you, everyone.
Dom:Alright. That's it for this week. I would really appreciate if you can talk or share about this podcast, it's always helpful. Also another way to support this show is by purchasing my course, there is always a link in the show notes. So on that, see you next week.
Creators and Guests
