localfirst.fm

A podcast about local-first software development

Listen

Conference

#22 – Paul Butler: Jamsocket

The guest of this episodeis Paul Butler, founder of Jamsocket and creator of the Y-Sweet project. This conversation will delve into building vs buying a sync engine and explore the various projects behind Jamsocket including Plane, Y-Sweet and ForeverVM.

Mentioned in podcast:


Links:

Thank you to ElectricSQL and Jazz for supporting the podcast.

Transcript

Intro
00:00There's database.
00:01sync engines and then there's document sync engines.
00:04So for database sync engine, I think of things like Linear, like things
00:07where you have some relational model data, you probably don't
00:11want the client to have all of it.
00:12You kind of have, the client storing some subset of a database
00:16for each, for each account.
00:18Maybe you're sharing this data across multiple people in that account.
00:22On the document sync side you're sending all of the data down to the client.
00:27The unit of data that gets synced is in memory size on the browser.
00:31You're not dealing with like a terabyte of data here.
00:33you're not taking a subset of it.
00:34You're synchronizing the entire document.
00:36This would be kind of things like Figma or Google Docs, where there's a full local
00:41copy of some self standing piece of data.
00:45Welcome to the localfirst.fm podcast.
00:47I'm your host, Johannes Schickling and I'm a web developer, a startup founder, and
00:51love the craft of software engineering.
00:53For the past few years, I've been on a journey to build a modern, high quality
00:57music app using web technologies, and in doing so, I've been falling down the
01:01rabbit hole of local-first software.
01:04This podcast is your invitation to join me on that journey.
01:07In this episode, I'm speaking to Paul Butler, founder of Jamsocket,
01:11and creator of the Y-Sweet Project.
01:14In this conversation, we talk about building versus buying a sync engine
01:17and explore the various projects behind Jamsocket, including Plane, Y-Sweet,
01:22and Forever VM Before getting started, also a big thank you to ElectricSQL
01:28and Jazz for supporting this podcast.
01:30And now my interview with Paul.
01:33Hey, Paul, so nice to have you on the podcast.
01:35How are you doing?
01:37I'm good.
01:37Thank you, Johannes.
01:38I'm excited to be here and been listening since the beginning.
01:41Thank you so much.
01:42The two of us had the pleasure to meet in person at last year's
01:45local-first conf, and I'm hoping to see you there again this year.
01:49So for those in the audience.
01:51Who don't know who you are, would you mind introducing yourself?
01:54Sure.
01:55I'm Paul Butler.
01:56I'm a co founder of a company called Jamsocket.
01:58the kind of one line pitch is it's like a Lambda, but for WebSockets.
02:03Yeah.
02:04I've been looking a little bit into Jamsocket and it's
02:06like, looks really fascinating.
02:08And I also want to hear more about The origin story where it's coming from,
02:13since by now we got more and more sort of like infrastructural options.
02:17Obviously there's like CloudFlare with their primitives,
02:21CloudFlare workers, et cetera.
02:22And I think with Jamsocket, you provide a really powerful alternative.
02:27For high scale applications.
02:30So yeah, before we go into more into depth, what Jamsocket is and what
02:35it offers, would you mind sharing a bit more of the origin story,
02:38how you ended up working on it?
Jamsocket Origin Story
02:41Sure.
02:41Yeah.
02:41so I started, with my co founder started the company about three years ago.
02:45prior to that, I was working in finance and I was doing a lot of building a lot of
02:49internal tools for myself, my team, that.
02:52We're dealing with midsize amounts of data, so talking about like single digit,
02:56double digit, gigabytes of data, not anything that was out of realm of putting
03:02in RAM for a desktop application, but I realized that as soon as people wanted
03:06these things to be delivered through the browser, there was really nowhere
03:09to put that data, that I couldn't really load that over the Internet into the
03:13browser, Chrome would just give up, didn't really make sense to load that
03:17into kind of a flask server or something like that, because the web stack is
03:21kind of built for these servers to not consume a lot of memory for each user
03:26of the application, things like that.
03:28So I really wanted this sort of neutral location and a way for the, I almost
03:33think of it as a way for a browser.
03:35Based application to spin up a server side sub process that
03:39just belongs to that browser tab.
03:40And that when you close that browser tab, that server side process also goes away.
03:45So that was essentially the origin story of Jamsocket.
03:48That makes a lot of sense, but could you motivate a little bit more?
03:52What kind of application I should imagine there?
03:54I've never worked in finance and I think the average web
03:57developer has sort of like.
04:00A list of like 50 Airbnb, items that they want to render.
04:05And then there's like pagination and all of that easily fits like in a
04:09JSON array that you can fetch into like, a single, CRUD, REST API call.
04:16But when you say like single digits, double digit, gigabytes of data, what
04:20sort of data are we dealing with here?
04:23And if you want it to transfer it over the wire into the browser,
04:27what would that even look like?
04:28Would that be sort of like one big JSON blob or can't, yeah, maybe
04:33you can motivate that a bit more.
04:35Yeah, so, one of the motivating examples at the time was, we would run like a
04:40simulation, a back test simulation.
04:41So we have some model that we hypothesize is predictive of of stock returns,
04:47run it back in time and generate a bunch of data could be say on every
04:52five minute increment or even more fine grained than that, over petabytes
04:55and petabytes of, past market data.
04:58and then we get back some gigantic time series data, number of time
05:03series, maybe you have profit and loss over time and record of all
05:07the trades and everything like that.
05:08so we have like massive, not massive, large, like, Gigabyte,
05:14multi gigabyte of time series data, likely in something like Parquet.
05:18that tend to be the best format for that type of thing.
05:21and so over the wire, ideally it would be Parquet or Arrow.
05:25Got it.
05:25And over that sort of data you want to do like.
05:28Based on the user's input through the UI, driving some sort of queries, some sort
05:34of like accumulations to make sense of like what the data is trying to tell us.
05:40Yeah.
05:41It was things like, maybe I want to be able to drill down in the data, in the
05:44client, be able to kind of go from this high level overview of data to kind of.
05:51Looking at specific trades, specific stocks, things like that.
05:55Got it.
05:55And sort of your insight and way to deal with that, fundamental problem where
06:01like ideally you could just like move that big data blob over in front of like
06:07your, browser that you're looking at and then like happily query and compute
06:12away, but that wasn't feasible because like Chrome or another browser has.
06:17Certain limits and the way how you want it to like, cope with that
06:22is to say like, okay, we're going to have like a little companion.
06:25Each browser session has a little companion on some beefy server, which
06:30holds all of that data in memory.
06:33And then there's some sort of like.
06:34Real time wire protocol that helps you do that, like still so fast that
06:41it's, sort of proxying to being local.
06:44Yeah, exactly that.
06:45I kind of think of it as like a somewhere along the spectrum where
06:48you have like there's thin client.
06:50Sort of setups where the server does everything and the client's really
06:53just a dumb display all the way to a full fledged browser based app that
06:59where everything's happening in the browser that can happen in the browser.
07:02I think there's some middle ground where you get.
07:05Next frame latency on almost everything, but maybe in the background
07:11it needs to query some server data and load that in and maybe it
07:14can even approximate client side.
07:16what that next frame will look like, but it's able to sort of do
07:18that in the background as well.
07:20Got it.
07:20So as you face those problems that has led you to get so interested in the
07:26problem that you started to dedicate your next chapter in life to that.
07:31And that led to you building a technology called Plane.
07:35And that was then also the foundation for Jamsocket.
07:38So can you explain a bit more what Plane does and then how it
Plane
07:45Yeah, so, and just to kind of continue on the story.
07:48So my co founder, Taylor was at data dog and had kind of faced
07:51some, some similar problems.
07:52So we got together in 2022.
07:55and the first thing we started working on was, yeah, what became
07:58Plane, which is, it's open source.
08:01and it's a, I think of it as kind of the way that we spin up those processes.
08:06It's kind of the orchestration plane essentially for that type of application.
08:11so what it's responsible for is you kind of give it a pool of computers.
08:16you tell it you want to start a specific process and it will find where on those
08:20on that pool of computers to start that.
08:22But it will also give that process a secure web accessible URL.
08:28so it can give it a host name and a kind of a password, essentially,
08:32then anything on the web, anything on the public Internet.
08:35That can access the web, can use that URL to send and receive
08:39messages from that process.
08:42And as long as there's at least one open connection to that
08:45process, Plane will keep it alive.
08:47And then as soon as there's no more processes, Plane
08:50will start a countdown timer.
08:51And if nothing reconnects, it'll shut that process off.
08:54Got it.
08:55So in terms of use cases, you've clearly motivated that original use case that you
09:01had while working, at a financial company.
09:04are those also the kind of use cases that, you know, mostly face when talking to
09:09people that are interested in Jamsocket or is there a wider set of applications
09:14that Jamsocket is trying to serve.
09:17yeah, largely not that actually hasn't, we haven't seen that many
09:20use cases of kind of wanting to just modify massive data sets or deal with
09:25massive data sets in the browser.
09:27But one of the things we quickly realized was that the infrastructure we were
09:31building had a lot of parallels to how Figma did things, how Google Docs
09:35did things, how a lot of these kind of collaborative applications did things.
09:39And so we decided to kind of lean into the sync engine hosting side of things.
09:45Got it.
09:45And when I'm looking at your, website, among a few other
09:49companies, it looks like Rayon is built also on, on top of Jamsocket.
09:54So I've happened to have seen their launch, I think, a while back.
09:57if I recall correctly, it was sort of like a really interesting Figma esque,
10:03application, I think for architects.
10:06And, yeah, maybe you can share a little bit more about their specific scenario,
Rayon
10:15Yeah, Rayon's one of my favorite use cases.
10:17cause we've really grown with them.
10:19They've been using us since we started the company, essentially.
10:22they were one of the first users on the platform and we've seen them kind of grow
10:26as they launched and, and everything.
10:27essentially the way that they're using us is that we are the data backend for
10:32these documents while they're open.
10:34So I open a document, you open a document, they'll start a server
10:39on Jamsocket for that document.
10:41And as I make edits.
10:43They get pushed up to that server.
10:45They get sent back down to you.
10:46And, that back end is also what's storing the data on S3.
10:51So even if it's single player mode, that Jamsocket server still sits between their
10:56end user and the source of truth on the data source, the durable data source.
11:02Okay.
11:02So you mentioning S3 and a durable data source.
11:06maybe we can take a step back and motivate if someone wants to
11:11build their own, little version of something like Jamsocket or Plane.
11:16how would that look like?
11:17So there seems to be something pretty beefy in the middle that
11:22holds the necessary data in memory.
11:25And as the name, as in memory suggests, that's pretty volatile.
11:30So if someone stumbles over a power cord, that data might be gone.
11:34And, that's also why it needs to stay some more, more durable, something like S3.
11:39So can you walk us through like the rough architecture?
11:43And what were sort of like the insides and deliberate trade offs that went into it?
11:49Yeah, I mean, a common pattern that I see people use with Jamsocket is that the
11:53source of truth for application data will kind of shift as the application is used.
11:58So at rest, the source of truth of the application data is In durable storage
12:04somewhere, usually S3, something like that, where you might want to, might
12:09not want to write to that like 60 times a second, but you want it to
12:13persist when that document is open.
12:16Then that source of truth effectively of that document is in memory on Jamsocket.
12:21And the nice thing about that is it's, you know, it's memory.
12:23You can write to it very frequently.
12:25You can write to it 100 times a second if you want to more than that.
12:29and that can then be synced down to all of the connected clients and then.
12:33In some sort of loop or, you could have some sort of write ahead log, but
12:37as changes are made to that document, you are then durably persisting them.
12:41some people really care about that being really low latency.
12:44I think in general, unless it's really a bad thing for users to lose
12:47like 5 seconds of data that just batching everything up into writing
12:53just the edits every five seconds.
12:54Something like that is pretty reasonable.
12:56or you can, you know, what a lot of people do is they just say 60 seconds is fine.
13:00I'm just going to write the entire document over what existed there before
13:04every 60 seconds because the outage, you know, a server just failing out of the
13:10blue is actually pretty rare these days.
13:12Got it.
13:13So if we compared to, a technology like Cloudflare durable objects, with
13:19Cloudflare workers, that's a particularly distinct programming model where it
13:25kind of gives you kind of Best of both worlds in that regard that you
13:28only pay for the CPU cycles where you actually want the CPU to do things.
13:34And otherwise it can hibernate while still keeping a web socket connection alive, for
13:39example, or keep like some memory alive or rehydrated from some persistent storage.
13:46is that sort of like a useful parallel way to think about
13:50the programming model and also.
13:53can I implement any sort of free web socket messages or request handlers, or is
13:59there a more pre specified API, something like Redis, how I interact with data from
14:07a client to the server and vice versa?
14:09Yeah, good question.
14:10I agree that like, I think durable objects is probably the closest kind of
14:14parallel product out there right now.
14:17when we started this up, durable objects, wasn't really a big thing and had may have
14:22existed, but had a lot of limitations.
14:24like, I think we, we came at things from a very different angle, but kind of
14:27landed in a similar architectural space.
14:30in terms of the servers though, we just, Really host anything that's HTTP.
14:35So, when I talk about it as being for WebSocket servers, I think that
14:38we kind of came at it at an angle of we want this to be the right
14:42model for hosting WebSocket servers.
14:44But, and we do, you know, we sit on the connection.
14:48So we.
14:49Work well with WebSockets where there's a long lived connection,
14:52because then we know not to terminate the server with HTTP requests.
14:56We have to rely a little bit more on heuristics.
14:58we've got that WebSocket connection open.
15:00really, just anything could be Socket.IO could be, your own WebSocket protocol.
15:05we essentially just take a container from our customers that
15:07will serve HTTP on port 8080.
15:10And we expose that to the, the outside web through a proxy that we wrote.
15:14Got it.
15:14So in the specific case of Rayon, did they build their own from scratch sync engine?
15:20Did they leverage any specific off the shelf technology, something like Yjs?
15:26given that Jamsocket advertises as the platform where you build your
15:31own sync engine on top of, maybe you can walk us through by this example.
15:36how I should think about that.
15:37Yeah.
15:38they're one of a number of customers who have kind of built their own
15:41sync engine on top of Jamsocket.
15:44there's not like an SDK that you need to adopt or anything like
15:46that on, on the server side.
15:48It's, you're just writing a web server.
15:51but one of the things that's specific about this model is that.
15:54You are guaranteed by the infrastructure that only one, at most one server is
16:00running per document or however, you want to fragment your kind of space of things,
16:06but, in their case, it's per document.
16:08And so, yeah, you get that guarantee from the system, and then it becomes much
16:13easier to implement your own sync engine.
16:14But we, at least at the Jamsocket level, are not opinionated about how
16:19you actually go about implementing that.
16:21but then you mentioned Y-Sweet.
16:22Yeah, we.
16:23So Rayon does not use Y Suite, but, some of our customers use Y Suite, which is a
16:27Yjs backend that we wrote that we provide.
16:29That's a much more opinionated path if they want to take that.
16:32Got it.
16:33Yeah.
16:33I want to learn a lot more about Y Suite in a moment as well.
16:37But given that you've already mentioned those 2 paths of Y Suite, which is a
16:42off the shelf technology that you're building that basis, on top of Yjs,
16:47which is a very well known, CRDT implementation, probably the most, common
16:53and, longest technology that's out there.
16:55so that being an example for an off the shelf technology.
16:59Rayon, which has built their own sync engine.
17:03you've probably seen many, many, decisions being made where people
17:07choose to use an off the shelf technology or choose to build their own.
17:11which sort of advice would you give to people who are thinking
17:14whether they should buy, or, as an alternative to buying is like
17:19adopting an off the shelf technology.
Building vs Adopting a technology
17:22Yeah, I think that where.
17:25It kind of comes down to for the kind of build versus off the shelf is whether
17:29you want to have business logic live in the sync engine on the server side.
17:34so where I think you generally don't need that is if you want to just
17:38think text documents, things like that, where CRDTs are probably the
17:43best way to do it right now, at least the most off the shelf way to do it.
17:48You can do your own way, but it's sort of a research problem.
17:51where, on the other hand, I think if you have a very simple data
17:54model, but you want to do atomic transactions, you want to have, kind
17:57of an event sourcing type approach.
17:59you want to be able to do things like trees with reparenting and
18:04and some of that Ends up being that you're working against the CRDT.
18:08and in those cases, I think it makes more sense to implement
18:12your own business logic.
18:13The other thing that we see is if maybe you want some change to trigger
18:16some action server side, you want actions to have some side effect.
18:19You want to, maybe some piece of data changes and you want
18:21to insert that into a queue.
18:23So it becomes really nice to have some server side code that
18:26Reacts to changes to the document.
18:28that's another place that we find.
18:30Building your own tends to be really nice because you can just
18:34have that be one server that's responsible both for the sync and,
18:38and for triggering some side effect.
18:40Right so maybe to linger a little bit on that specific point, I think
18:45with, local-first software you have, in this scenario where you build your
18:50own sync engine, you have kind of two, approaches, how to deal with that.
18:54And also for the off the shelf approach, if you use something like Yjs.
18:58so if you build your own, you can basically just wherever you handle the
19:02messages, you can, possibly inspect the messages and see, okay, this
19:06seems to be like a user signup event.
19:09And so here let's send out that confirmation email or something like that.
19:15but another approach could also be that you basically have a server
19:19side client instance that listens to the same sync messages and you.
19:25Based on the state that you have, on that server side client, you
19:30could then basically React to that.
19:33have you thoughts on one approach versus the other?
19:37Maybe, one is like a lot more, expensive to run or, more, complex to model.
19:44What thoughts do you have on the different approaches here?
19:47I think that where I've tended to see this breakdown because we've, we've seen
19:50it both ways and we've seen, we've seen customers do it both ways is that if it's.
19:56Purely just sort of Reacting to a side effect.
19:58And it's something that you want to that your model of it is that it's
20:02like a server triggered type of thing.
20:05Like, if it's that, you know, that send email example, send
20:07some sort of notification.
20:09I think that that makes more sense to just do in the server, just in
20:13terms of architectural complexity.
20:15you could certainly listen for the events.
20:16And if there's architectural reasons that that makes sense for you, I don't see
20:20any problems with it, but where I think that the server being a client can make a
20:26lot of sense is like AI integration type things, where you want the server in this
20:32case that, you know, it's code running on the server, but it, your application
20:36should just treat it like another client.
20:38This is something like maybe an agent's going out and modifying
20:40a document based on some prompt.
20:42Then I think it does make sense if you want to run it through the same
20:46kind of code paths that a user edit would go through, then it makes
20:51sense to, to kind of treat that as a distinct client of the data.
20:55Got it.
20:55So to dig a little bit more and towards that, server as a client, when I'm
21:03thinking more about like a browser client, or like using my, phone,
21:08or there's like a concrete point in time where I'm starting a session.
21:14I'm opening a tab.
21:15I'm opening an app.
21:16I'm doing things afterwards.
21:18Like I'm closing it.
21:19So there's like a concrete start stop.
21:22Maybe there's like some background stuff, but, let's pretend there's just
21:26like a clear start, stop 30 seconds.
21:29And that's it.
21:30how should I think about that in a server context?
21:34let's say I'm trying to offer that to a thousand customers.
21:38Would I have a thousand separate, like, but let's go crazy.
21:42Let's say we have a thousand VMs, one per customer.
21:47that strikes me as very expensive.
21:49So what is like a useful programming model, like a useful deployment model.
21:54To, deploy those sort of server side clients.
21:57so the way that Jamsocket does this is that we run a process
22:01for every service, essentially.
22:04So when, when you and I are connected to a document, we're running a
22:07process, not a full fledged VM, but it's, using some Cisco interception
22:11through something called G visor.
22:12So it's a little bit more secure than sort of just.
22:14Yeah.
22:15Containerized workloads.
22:16so the nice thing about that is that processes are pretty good at giving
22:20resources back to the system when they're, when they're not actively in use.
22:24So we've seen is that when you want the server to kind of first Okay.
22:29Class of interactions where it's sort of definitely want it to
22:32be processed by the service.
22:34in those cases, it makes sense to run directly in the
22:36sync engine when it comes to.
22:38multiple clients, we tend to see those run off of Jamsocket.
22:42So these are running on an end user server talking to Jamsocket and the
22:47pattern that I've seen their work is that client will maybe trigger something
22:54directly through like a web endpoint on that remote server that's not running
22:58on Jamsocket, that server will then talk to Jamsocket to say, fetch some data or,
23:03connect and sort of trigger something.
23:05so it might synchronize data, but it's not, a long live client.
23:08It's kind of a client that spun up based on a specific action.
23:12That's usually triggered by the client.
23:14Got it.
23:14That makes a lot of sense.
23:15So instead of like being super long running, and that's times
23:19and for each possible instance, you make it more event based.
23:25So, let's say there is a new sync message that you want to React
23:29to, or there's like some other.
23:31maybe like a webhook that's coming in from Stripe and then, so you, you do your
23:37thing as a response to the event and, then you go, yield again, back to the runtime.
23:44and I think a model that also comes to mind that could fit
23:47really well together here.
23:49Is, our durable long running workflows, something like Temporal.
23:54And there's also other options as well, I think could work really well
23:57together here that you have a workflow.
24:00That's essentially a participant in a sync system where it's
24:05just a long running workflow.
24:07It's just like another client happens to live on a server and not in a browser.
24:12yeah, I'm, I'm really excited to see more folks explore this since I think it will.
24:17open the door for a whole bunch of different application topologies, really.
24:22One of the things, things that we found with Y Suite is that, we had people ask
24:26for, like, I want a Python client to this.
24:28And it was for exactly that reason.
24:30Like they want to run some server side code that interacts with a document.
24:34same with the node on the node side.
24:37We support kind of the built in WebSocket client in the browser, but
24:40we also support a shimmed in WebSocket client so that you can run it in Node.
24:45Very cool.
24:45Yeah, I'm really looking forward to like, whether it's Python or well,
24:50I'm a native person in JavaScript and JavaScript has this amazing, Aspect to
24:55it that supposedly runs everywhere and we're getting more and more there with
25:00like ESM now, being really, the default.
25:03And, I'm really excited about bringing the same business logic, the same code
25:09to all sorts of different platforms.
25:11And I think sync engines are.
25:14or like a huge lever that gets us closer towards that since like, otherwise
25:19we can have, the code there, but if we don't have the data there, that
25:24is only good for so many use cases.
25:27So maybe.
25:28transitioning towards Y-Sweet, what you've already mentioned.
25:32before we get into what Y-Sweet is, can you share more about
Y-Sweet
25:42Yeah.
25:42so we'd already been working on Jamsocket for a while by the time we started Y-Sweet
25:46and we sort of started to see for one thing, You know, we thought from the
25:50get go that, well, people are going to want to write their own sync engines.
25:53one of the things we saw was that a lot of people were sort of using Yjs
25:57and other CRDTs and running those on Jamsocket and finding advantages, even
26:03though they don't need the authoritative kind of model of Jamsocket that they were
26:08still finding advantages to having that.
26:10so we started thinking like, what would a Yjs server kind of built
26:14to run on Jamsocket look like?
26:16And one of the things that.
26:17It's nice if we're, you know, running a lot of a process is that
26:20it's really memory lightweight.
26:22So we wrote Y-Sweet in Rust and it's pretty memory efficient.
26:26another thing that we became really opinionated about is that you shouldn't
26:30really start document data in a database.
26:32I think it's just a bad fit.
26:33I think with something like a, you know, if you're building something
26:37like Figma, like Figma uses S3.
26:40As where they store the document, they store the document metadata in Postgres
26:44and started to see a lot of use cases of like patterns like that, because if
26:49you're writing the document each document that's open many times a minute, If
26:54you're using a Postgres database, that Postgres database is in the bottleneck.
26:57Every, every edit is coming through that.
26:59Whereas S3 is a more distributed kind of file system where if you
27:03have a server that is the authority of what's in a document at that
27:07given point in time, it can just write to S3 and you can horizontally
27:11scale that out as much as you want.
27:13So we kind of became opinionated about, okay, that should be rust.
27:16It should be lightweight.
27:17It should write to S3.
27:18and it should be, Okay.
27:20As simple as possible to just use, like, I really like software like Caddy, where
27:25it is, which is web server written in Go, if people aren't familiar with it, where
27:28you like that has really sane defaults.
27:31It's somewhat opinionated about just doing things right.
27:34You don't have to
27:35fantastic.
27:36It even gives you like, SL certificates that work locally works with tail scales.
27:41Fantastic.
27:42Definitely check it out.
27:43If you're not using it yet.
27:45Yeah, so Caddy just like simplifies so much and just like does things right.
27:50And so we wanted to build a piece of software that was felt like
27:52that to use, that it was, we wanted something that you could use in a
27:56CICD process and it would be the same API as if you were using it at scale,
28:01horizontally scaled out on the cluster.
28:03so it was like, because the other thing, I mean, the things that we were thinking
28:06about at the time were like, what would an open source document sync engine
28:11look like, if we were to write it from scratch and we kind of kept landing on,
28:17it would look something like, you know, pretty close to Yjs, even if we didn't
28:22have the distributed constraints of Yjs.
28:25So we're like, well, Yjs exists.
28:28It has great community.
28:30Great people involved with it.
28:32this looks like what we would want to build.
28:34So let's just build a sync engine around this.
28:37Got it.
28:37In terms of the, behavior or like what makes it a little bit more like Caddy
28:43in terms of opinionated, but like, very well motivated opinions baked
28:48into it, if you compare it to the Yjs.
28:51Default server, any sort of thing that stands out there where you lean a little
28:56bit more heavy into some opinions?
28:59Yeah, I mean, I think the default Yjs server is built to be very modular
29:03and suit a bunch of use cases The Yjs community in general embraces this
29:08idea of providers where a provider.
29:10So Yjs itself is just a data structure and then providers are what will synchronize
29:15it to another client or synchronize it to a database or things like that.
29:19the kind of official way to do things in the.
29:22Yjs world is to kind of compose a bunch of providers together.
29:26so you might have an index db provider on the client, synchronizing the index db.
29:30You might have a web socket provider, synchronizing to other clients.
29:34And then you might have a database provider on the server.
29:37We wanted to just have a single stack that was kind of our opinionated stack.
29:41So we have an index db implementation on the client.
29:45We have our s3 storage, which we've Decided is, you know, the only storage
29:49that will support will support S3 compatible storage, but it's, it's
29:52ultimately our opinion was object storage is the right way to do storage for this.
29:58and then we have our, our wire protocol as well.
30:00WebSocket.
30:01Got it.
30:02That makes sense.
30:02Yeah.
30:03And I haven't managed yet to.
30:05Have, Kevin Jans here on the podcast, but he happens to also live in Berlin,
30:10and I've just seen him, for the, last local-first meetup that we've done here.
30:14So I think it's, well, about time that we hear from Kevin, about YJS, there's
30:20been, it's been such a rich ecosystem of different things around it, so I
30:26think we gotta make that happen as well.
30:28Yeah, you should.
30:29So I'm actually, I've been procrastinating editing a podcast that I did with Kevin.
30:33so we'll have that soon.
30:35There you go.
30:36we should put it in the show notes.
30:38So, YJS you've built, just, as you've seen that this is a, flavor of Sync
30:45server that can be hosted on, Jamsocket.
30:48So, is my understanding correct that, if I want to use YJS with Y Suite, I can just
30:55deploy that off the shelf on Jamsocket
30:59yeah, so you could deploy that.
31:01We have like a off the shelf offering that deploys it on Jamsocket.
31:04you can run it on your own servers as well.
31:07and it's one of the things we decided was like, regardless of how it's
31:10hosted, it should be the same API.
31:12So we have kind of the, what I call it, the document management API where
31:15you're, you know, create a document, give somebody an access token to that document.
31:19that is sort of just universal, no matter how it's deployed.
31:22Got it, so I think Yjs is one of the most mature options right now for
31:28people who want to build local-first apps, for people who are just, who've
31:32heard it a bunch of times, but maybe haven't yet come around to, fully.
31:37Implement their app using it.
31:39what are questions that people should ask themselves?
31:42Whether Yjs is a useful foundation for the app and in which scenarios
31:47would you say, actually, you probably want to build your own sync engine.
When to choose Yjs
31:51yeah, so I, I think the, one of the first dimensions to think about here is
31:54I see this sort of, there's two worlds.
31:56There's like database.
31:58sync engines and then there's document sync engines.
32:01So for database sync engine, I think of things like Linear, like things
32:04where you have some relational model data, you probably don't
32:08want the client to have all of it.
32:10You kind of have, the client storing some subset of a Database
32:14for each, for each account.
32:16Maybe you're sharing this data across multiple people in that account.
32:20database sync world where there's, Elastic SQL and, zero and, power sync
32:25and kind of a number of players there.
32:28instant DB and triplet and a number of others.
32:31on the document sync side, that's where you kind of have, you're sending
32:36all of the data down to the client.
32:38So you're dealing with kind of the unit of data that gets synced
32:41is in memory size on the browser.
32:43You're not dealing with like a terabyte of data here.
32:45you're not taking a subset of it.
32:46You're synchronizing the entire document.
32:48This would be kind of things like Figma or Google Docs, where
32:52there's a full local copy of.
32:54Some self standing piece of data.
32:57and generically in, Yjs, that's essentially like a
33:00JSON style or JSON shape data.
33:03So things like nested maps, things like nested lists, and
33:07text, and then JSON primitives.
33:10Is it fair to say that, so you've mentioning Figma, Google Docs, if I think
33:15about Figma and Google Docs, there is like a distinct boundary of a document.
33:20So I have a Google Docs document open.
33:23I have a Figma document open.
33:25is it wherever a product experience has sort of like for a given part of
33:32the experience is all centered around a document or tl draw comes to mind?
33:38is that a great fit for embracing the document model and anything that
33:44is more, rich in terms of, like a relational database where you can
33:49just freely join between things.
33:51That's where you would choose the other approach is that's a useful rule of thumb.
33:56Yeah.
33:57I think the words, that you use distinct boundary, I think that's really nails it.
34:00a, if there's kind of like a document with This is like self contained.
34:04It's distinct.
34:05you mentioned TL draw like, and actually, I mean, I think this gets
34:08to another point is that you can use both in the same application.
34:13So TL draw uses zero and their own document sync engine.
34:17Figma has built their own sync engine for both.
34:21and they're distinct sync engines.
34:23They can be used in tandem as well,
34:25right I mean, that gets us to a really interesting, topic more generally, which
34:29is combining multiple sync engines.
34:32And I think for people who've been dabbling in local-first, that might be
34:37more intuitive, but I think for, people who are just very new to, the local-first
34:42space, it's hard enough to wrap your head around, choosing the right sync engine.
34:47Now you're telling us, wait, you should choose multiple.
34:51Can you motivate a little bit more of like, how to think about that?
Choosing multiple Sync Engines
34:55so I think of it as like the app layer and the document layer.
34:57If you have a document based application, there's, you know, if you have a
35:00file viewer, for example, I think of that as app layer, you're not in
35:03a specific document at that moment,
35:05like in Figma where I'm on the home screen and I see my various projects.
35:09Yeah, exactly.
35:11and I think there's nothing that forces that part to be real time synced.
35:16In a lot of cases, I think a traditional Postgres database
35:20goes a long way for that.
35:21and then, but then once you're in the document, that's where I think you, you
35:25do kind of need a sync engine because, it's the type of thing that if you have
35:30two Google Docs open in two different tabs, you expect them to be in sync,
35:33even if you're just a single user.
35:35I think that actually motivates like 98 percent of the value of
35:38local-first is just somebody who has the same document open in two
35:41tabs and they've got 100 tabs open.
35:43I think that that's less of a given expectation these days for like a
35:48project view or something like that.
35:50I think that It's a nice surprise when that is in sync.
35:52And I think it is becoming the status quo, but I think that overall it's.
35:57less of an expectation that, Oh, you might have to refresh your Figma
36:00project, to sort of see the new assets that come up or that kind of thing.
36:06so yeah, but it is, I do think, and there's been a bit of Twitter debate
36:10about this lately, but like whether the same sync engine can handle both.
36:14I think that there are things that you are going to need transactions for, and if you
36:18need transactions, you're going to need a database with a single that is effectively
36:21a single bottleneck on updates.
36:23At the same time, if you have lots of documents, you don't want those documents
36:27to be bottlenecked in a single point.
36:29So I think unless there's a solution that offers both distributed and centralized
36:33with transactions, you kind of need both.
36:36Got it.
36:37So, if you're thinking more about the leaning into the document aspect of
36:42it, or even, when you say like, that something is bottleneck, let's say we
36:47also embrace the, database aspect of it.
36:50Maybe you have different.
36:52Workspaces, and, I think there's still like one aspect of like drawing
36:58boundaries around some body of data, where you say like, Hey, within
37:03that boundary, I care about certain constraints, maybe that there shouldn't
37:08be more than 10 documents ever.
37:11Or maybe you want to enforce some constraints around like
37:16users, access control, et cetera.
37:19can you share any sort of learnings or advice about how
37:23to approach this entire topic?
37:25Like, how do you decide this is a useful boundary about like how data should be
37:31modeled at and fragmented or petitioned.
Boundaries
37:40So I think in general, if it's not obvious what a document
37:43should be in an application, then it's probably the document model
37:47is probably not the right fit.
37:49I think things like Figma where, you know, you're, in a document at a time, like.
37:53You might have a different document in another tab, but
37:55you don't have two documents in the same tab concurrently open.
37:59it's taking up the whole screen.
38:01Like, I think that there's certain heuristics like that, that just
38:05tell you, like, this is definitely a document model application.
38:09Same with Google Drive or Google Docs.
38:11you kind of have one thing.
38:12Open at once,
38:13where would you put Linear?
38:15Since you could, for example, put each Linear issue into its own document.
38:21Why might that be a reasonable approach?
38:23where is this?
38:24Where might have not?
38:25I think I could see.
38:28That being reasonable, if there, if you really care about the tickets themselves
38:33being, you know, multiple people editing a ticket at one time and seeing the text.
38:37And, if you really wanted to make that kind of a first class experience.
38:41But in general, I think that, Linear just screams kind of database approach to me.
38:47although I do, I know they are, I believe using Yjs, for some of the issue.
38:51text now could be wrong, but I think they do use it or a CRDT.
38:55it might be a different CRDT, but I think they're using some sort
38:58of collaborative text editor.
39:00so given that you've seen quite a couple of different customers and products
39:05build their own sync engines, any sort of interesting, almost second order effects
39:11that you've seen there, unexpected things, new challenges that you didn't see in,
39:18in previous applications, things like.
39:21Database migrations or other things, which sort of challenges
Challenges in building Sync Engines
39:27Yeah, I think whenever you're dealing with data on S3, data migrations do
39:32become really interesting because you're not just sort of writing a
39:36database query and issuing an update.
39:39Usually some form of gradual lazy migration.
39:43So it's kind of like the application that's reading the data has to know
39:46how to transition from version one to two and two to three and then
39:50kind of apply those consecutively.
39:52And so that logic tends to linger around in the application for as long
39:57as you have old documents to support.
39:59and I think there's ways to do schema migrations or schema changes that
40:04don't require a migration as well.
40:06Like, I think that the, It was at Google and we, you know, there
40:09were certain rules about what you could do with protocol buffers.
40:12that would ensure that they were always backward and forward compatible.
40:16and so I think, you know, things like a required field always has to be required.
40:20And so.
40:21Deciding being delicate of when you call a field required.
40:25there's certain kind of things you can do at the schema design level and schema
40:28migration or schema change migration level that you can avoid kind of having
40:34to implement any sort of migration.
40:36It can kind of be more access time oriented.
40:39So I think doing that has been where I've seen.
40:43It will be successful with that, in terms of second order effects, I
40:46think kind of goes back to like, once you have the sync server, people are
40:50like, oh, this is now a place where I can trigger this notification or I
40:54can do this check or I can, you know, so I think we've sort of seen these,
41:00these backends kind of grow in scope.
41:02you know, we want that to be first class part of the application that
41:04can do whatever you want it to do.
41:07That makes a lot of sense.
41:08And yeah, I think this is, an area that, has already caused a lot of, headaches,
41:14schema migrations, data migrations in general, but now that we are rethinking
41:18the data architectures at large here, we also need to rethink that part and
41:24like you've mentioned, when you have all the data in a single Postgres database,
41:29then you can at least like apply like your old playbooks there, but now if all
41:33of your data is in an S3 bucket, laid out in whatever way, now you do need a
41:39different new approach to deal with that.
41:41And, That is one way to deal with it, to bake in the migration
41:46logic into your app logic.
41:48But, that is also, I think that also comes with its own downsides.
41:52This way you're like litter some of that code that was once very clear.
41:58and now you make it less clear because you need to account for.
42:02That historical evolution, a project that I want to shout out here is the
42:07project, Cambria by the folks at In I've actually studied this project myself quite
42:12intensively and I've rebuilt it, myself a few times once even on a type level
42:17just to, provide a nice type save API.
42:21Given that the original implementation rather lets you specify those
42:25sort of projection rules in YAML.
42:28But, I've heard some rumors that they're thinking of like rebooting
42:31that project at some point.
42:32So fingers crossed for that.
42:34And yeah, another approach that I'm investigating heavily myself,
42:39given I have my fair share of like.
42:42Database migration traumas, that I tried to remedy with, starting Prisma,
42:48but now I'm, trying a different approach with event sourcing.
42:52Where if you basically split up your documents your database
42:58into a dedicated write model and derive the read model from it.
43:02The core insight here is basically that if you split this up into two
43:07parts, the schema for your read model, that is typically the thing.
43:11That changes orders of magnitudes more often where you have different kind of
43:16queries that you want to do different sort of aggregations and where you want
43:21to maybe change the database layout to make certain queries faster and more
43:25efficient and then the write operations.
43:29Those are much more bound to the domain of when stuff actually happens.
43:34So, and that's changes way less over time.
43:37Like, maybe you want to capture, someone's preference on email,
43:42marketing emails on, on signup, but historically you can way easier say,
43:47like, actually we default to no.
43:49but a user signup event.
43:52Is always valid and way easier to upgrade.
43:56And then you can basically reapply all prior events into the new read
44:00model that you can change very easily.
44:03And you can even have like multiple read models all at once.
44:06So, that is what I'm exploring right now on the umbrella of Livestore.
44:12But, that also comes then requires that rigor to split it
44:16up into a read and write model.
44:18But yeah, curious whether you have thoughts on that.
44:20Yeah, that's really interesting.
44:21I think that event sourcing in general does sort of simplify migrations.
44:27If you're willing to kind of go back over the event source log and regenerate,
44:30because then as long as you represented all of the data that matters, then you
44:35can essentially just add fields as.
44:38As needed to
44:39another problem that emerges in that world is like, if your domain produces
44:44a lot of events, so let's say you build a TL draw and whenever you move
44:50a rectangle, that creates, you could model it in a way that when you let go
44:55of the rectangle that creates an event, but you could even model it in a way.
44:59Where, like whenever the browser registered a new move event, dragging
45:03it can cause 5, 000 events and that can lead to a very long history of events.
45:10So now you gotta keep that in mind as well.
45:13And, whereas in the traditional mixed read and write model approach, you
45:18would basically just overwrite the position and it would not necessarily
45:24cause the database to explode.
45:26because you have too much data.
45:27but yeah, it's all about trade offs that that is like, what
45:30data management is all about.
45:32maybe a slightly different aspect about data that, you've also written about,
45:37which is in regards to encrypting data.
45:41So, you've written a great blog post about that.
Data encryption
45:49Yeah, so this came out of when we were with Y-Sweet.
45:52We wanted to do We wanted to have store the data locally in the
45:56client, at least as an option.
45:58so we looked at the options that were available or, you know, local
46:01storage, indexed db, opfs, origin, private file system, realized that
46:06indexed db was really the kind of the right way to go for this right now.
46:10have high hopes on opfs, but they're still, I mean, they
46:14all kind of have flaws, but.
46:16Index DB is like the best people know the flaws the best, I guess,
46:20and how to work around them.
46:21So, looked at index DB.
46:23But the problem that we found with all of them is that all of them store
46:27the data in plain text, and that's not just a theoretical problem.
46:32There is at least a couple months ago.
46:34Now, there was some, you know, NPM and pie pie modules out there that
46:39would read some application data from these plain text sources.
46:43it's kind of a real problem that people have identified.
46:46And has been exploited.
46:48so we wanted to make sure that we provided an option that at least as,
46:51as best as possible would prevent that.
46:54so we said, okay, well, browsers have web crypto.
46:57We can encrypt all this.
46:58but then there's this problem of where do you store the key?
47:01because you could start on on the server, but then kind of defeats
47:04the purpose if you're offline, of then accessing that data.
47:07So realize that.
47:10don't really have a good way to store a key kind of credential.
47:16we've got like WebAuthn, but WebAuthn is a bit more secure, like, which is where
47:21you have pass keys and things like that.
47:23It's a bit more opinionated.
47:24It uses the operating systems key chain, but it, doesn't really expose that to
47:29you as any sort of low level API that you can store your own secrets in.
47:34What has started happening is that some browsers, particularly Chromium
47:37based browsers, Google Chrome, Edge, Rave, have built in something called
47:44App-Bound encryption, and they're just using this for cookies, but the idea
47:48is that the browser will store, cookies in, you know, on disk as they always
47:54have, but they'll be encrypted on disk, and then the symmetric key to that will
47:59be stored in the, Operating systems keychain and the operating system is set
48:05up to at least in theory, and there's been some vulnerabilities here, too.
48:09But, at least in theory, only give that private key back to
48:14the browser process itself not to another process that attempts to
48:18impersonate, the browser process.
48:20So what we landed on, which was pretty surprising to me, that this was kind
48:24of the best available path right now.
48:27But if you enable local storage, we encrypt it stored in index DB and
48:33then store the key in a cookie and.
48:35Kind of piggyback on that being App-Bound encrypted in at least
48:39in browsers to support it.
48:41That is very interesting.
48:42Yeah, I've been studying, cryptography, particularly in a browser context,
48:46also a bit more for various reasons.
48:49I am, trying to see what would it take to, do the entire, sync.
48:55messages for Livestore, what would it be, for them to be enter and encrypted,
49:01but the hard part is not the encryption, but the hard part is the end to end
49:07where, the various ends own their keys.
49:11And there's a, we should do an entire episode just about that.
49:14what's difficult about it, but, it can all be distilled down to
49:18the hard part about, anything cryptography related as key management.
49:23And you can either around the side of like being a little bit more loose with like
49:27how you manage keys, but that defies a lot of the, purposes and the benefits here.
49:32but then also the, browser makes that really, really tricky because it has very
49:38constrained APIs and historically it's always been rather a web document viewer
49:43than a fully fledged application platform and, we're getting the building blocks.
49:49I mean, you can, use the, web crypto API.
49:52I'm also using the Libsodium projects, compiled to WASM, which is very
49:57powerful and gives you a couple.
50:00of advanced, algorithms, et cetera, that you can use for, symmetric or
50:05asymmetric encryption, signing, et cetera.
50:08and pass keys, I think are also like, a super important foundation.
50:13But, they also get you just so far.
50:16And I think they don't really help you for the encryption as such,
50:20but rather for signing messages.
50:23So I think we're still lacking a few building blocks.
50:25So very excited to hear about this what, what it was again, App-Bound.
50:31App-Bound encryption, so ideally at some point, this goes even beyond cookies that,
50:37this can be applied for other storage mechanisms, but I like the approach
50:41to, basically encrypt it and then you reduce it to the key management problem
50:46and that you put into a cookie, which also, there's another question, which is
50:52what happens if that cookie goes away?
50:55did you figure out a, an answer for that?
50:58we don't.
50:58We just set it to a long expiration, but it's the thinking there was like,
51:02if the user is clearing their cookies on that tab or on that hosting, they
51:08probably want to destroy the data.
51:10And so are they, you know, they want to be logged out.
51:13so we actually saw it as the right thing to do to, bind it.
51:17The other nice thing about that is like, unlike indexed DB cookies can
51:21actually have an expiration date.
51:22So we could set an expiration of a week.
51:25we're still relying on the browser to enforce that, but if the browser enforces
51:28that, and then, you know, two weeks later, that person is fully hacked, including
51:33their operating system key chain, the browser, at least in theory, will have
51:36deleted that private key and then the data that's in IndexedDB will be gone.
51:40So that's actually, funny enough, additional functionality.
51:43It was just incidental to the, to using cookies for that.
51:46Right.
51:46I like this trick a lot and I got to look into it.
51:49One thing to point out still is, you've mentioned that this mechanism is only
51:54available in Chromium browsers anyway, but, cookies and IndexedDB, OPFS, et
52:00cetera, all of that is available in other browsers and namely Safari as well.
52:05One thing that, people find out the hard way about Safari is that it automatically
52:11deletes a user's data after seven days if they haven't visited that website.
52:16So if you're building a fully local-first web experience where
52:21someone, creates some precious data, in Safari and maybe doesn't sync it
52:26yet to somewhere else, go on vacation, come back and poof, the data is gone.
52:32So I think as app builders, we need to be aware of that and
52:36detect, Hey, is this Safari?
52:38And in Safari, make this part of the product experience show sort of like
52:42a message, like, Hey, be careful.
52:44Your data might go away.
52:46There are ways to remedy that.
52:48And, to, for example, if you make the Safari app, a, progressive web
52:53app by adding it to the home screen.
52:56That limitation goes away.
52:58but app builders need to be aware that they can make the app users aware.
53:04it's just something that, I think is important to, note.
53:08Yeah, I think that's an example of a number of cases where the
53:11browsers are just not optimized for local-first apps, unfortunately.
53:15you know, the, I think the ability to just store low level access to the operating
53:21systems key chain is another, where.
53:23Browsers have improved a ton in terms of what they expose of the APIs, but I
53:28think they're still lagging when it comes to that storage and encrypted storage.
53:32Yeah, totally.
53:34So, maybe slightly, moving to another browser related topic.
53:39you've been both through your work, Through your prior role, and
53:43also as part of Jamsocket, you've been dealing with quite a bit
WebAssembly
53:51guess I really wanted.
53:53I wanted to build the company around WebAssembly.
53:55I wanted WebAssembly to take off, particularly like server side,
53:59client side, that kind of having isomorphic client side server
54:03side code would be a big thing.
54:05And I've, I guess, just generally soured on WebAssembly a little bit.
54:10I think that it where I've seen it work really well is when it's
54:14in the application layer and you kind of have an application.
54:18there's a couple examples I like to go to that are like effectively the same model.
54:21the same kind of architecture, Figma.
54:24There's a company called Modify a few others that I'm, I'm blanking on, but the
54:28architecture is essentially a JavaScript UI, with a, webAssembly, WebGL, WebGPU
54:36kind of rendered canvas, behind it, so like Figma, you know, the core engine
54:41is, I believe, in C, talking to WebGL, with Modify, it's in Rust and WebGPU,
54:47but it's literally like, The application is layered that way that on screen,
54:52there is the canvas behind the UI.
54:55They're written in two different languages and they just talk to each other.
54:58so I think that is the most promising architecture that I see for WebAssembly,
55:02where I think it's been harder.
55:05To get right is building something like a library that is ultimately consumed
55:09by JavaScript developers, but written in WebAssembly, I think there's just so much
55:15friction still in the bundling that, I've kind of soured on that as an approach.
55:19Right.
55:20I mean, I agree in that regard that I wish there was already, we'd be further
55:25along with WebAssembly, but I think it's a bit of a chicken egg problem that
55:30we need more inspiring applications.
55:33That makes people feel like, wow, that is possible.
55:36I didn't recognize that this was the web.
55:39it feels so fast.
55:40And I think that is still true and, more true than, than ever
55:45that WebAssembly, I think.
55:46Can unlock whole new experiences.
55:49And there is a few Lighthouse examples like Figma that stand out here.
55:53Also a big shout out, to the folks building Makepad, which is
55:58a super ambitious project, which is, basically the same way as like.
56:03I'm probably going to do it, I don't do it justice by pitching it, but,
56:07I just want to speak to the ambition where it's basically like Unreal Engine
56:12is sort of like it's full engine.
56:14They, they're building their own platform and including, like a, a rendering layer
56:20and, sort of like as a few people think about, think that MakePad is an editor.
56:26No, MakePad has just as an example app.
56:30Build an editor in which they build make pad, which is just so phenomenal.
56:34So, and make pad is just such an incredibly fast app.
56:38So you should definitely check it out, go to make pad.
56:41dev and then press the option key, to see like how the entire code editor expands.
56:47So apps like that get me very excited about what's possible with
56:51WASM, but, they're fully, they're building everything in Rust.
56:54They're fully leaning into everything, there.
56:57And I think the either or, where you want to like, combined one step at a time.
57:02I think that's a. Tooling problem, partially it's also a trade off
57:07problem where if you move a lot of data back and forth between WASM and
57:12JavaScript, that doesn't come for free.
57:14So I think, you got to keep that in mind.
57:17I've seen a few, I think the, the RepliCache folks actually in the past have
57:22written a lot of their stuff in Rust and then moved to JavaScript because of that
57:27boundary crossing being, too expensive.
57:30But, I think not every use case suffers from that problem, but, I want to turn
57:35it around and, invite anyone who is excited about WebAssembly as, seeing
57:42that as an opportunity to make things significantly better, like working
57:46on projects like WasmBindGen or other things, I think the Deno folks are
57:52pushing heavily on that, so I'm seeing this glass half full and I think the
57:56glass is going to get full pretty soon.
57:58Yeah, I think to your point about like, the JavaScript WebAssembly boundary
58:03crossing, and I think that it comes down to just placing that boundary in the right
58:07place when it comes to, applications like the Figma model of sort of JavaScript
58:12front end with, renderer in WebAssembly, make pad is is a great example.
58:17I think of going like all the way in on WebAssembly.
58:21Another one's called Remix.
58:22and I think what's notable about both cases is that to do that
58:26well, they've had to basically be living in the GUI toolkit layer.
58:30Like, they've been writing their own code or adapting a
58:33lot of their own code for it.
58:34So I think that's, Not for the faint of heart.
58:36I think that people who have done it have built amazing software, but what comes
58:41up more often when I talk to people is like they there's a scarcity of rust
58:45developers and they want to optimize the rust developers to working on kind of the
58:50engine component and then be able to hire React developers and Svelte developers and
58:56kind of front end web developers to work on the GUI where it may not be Like, you
59:03know, think about Figma's UI components.
59:04Like they're not super performance sensitive in the way that the canvas is.
59:08Yeah, totally.
59:09I think it just takes, some bold thinkers and this is not something where you're
59:14gonna rebuild the world in two weeks, this is really something you gotta,
59:19put in the five, 10 years possibly.
59:22to really build something phenomenal, but I think the, rewards are massive and,
59:28I'm really looking forward to getting kind of alternatives to something like
59:33React that provide different trade offs and that allow you to build like really,
59:37really high performance applications and fundamentally React biases towards
59:41simplicity and biases towards that you can, Prevent, not so experienced
59:47engineers to, hurt themselves or others and drag the application down.
59:53But I think there's a different, trade off space as well, where you bias more
59:57towards performance and you need to know a little bit more what you're doing.
1:00:01And particularly now with AI being on the horizon, I think we can rethink
1:00:06a lot of trade offs significantly where engineering team sizes, maybe,
1:00:11get reduced as well, but that's the topic for another conversation.
1:00:16But, related though, in regards to AI, you've recently also launched a new
1:00:21project, that is certainly adjacent to AI.
1:00:25It's called ForeverVM.
ForeverVM
1:00:29Yeah.
1:00:30So pretty much from the beginning with Jamsocket, one of the ways we've seen
1:00:32people use it because we run these sandbox processes on demand is people
1:00:37have run LLM generated code in them.
1:00:39actually.
1:00:40Going back to the beginning, it wasn't even LLM generated.
1:00:42This was sort of pre chat GPT.
1:00:44it was things like Jupyter notebooks, but over time we see
1:00:47more and more LLM generated code.
1:00:50and it's, you know, it's good.
1:00:51I think we're, we're like competitive with other products for that, but we
1:00:54kind of realized, first of all, we're not really positioning the product that way.
1:00:58but also.
1:00:59We're not building the product necessarily to be like the best
1:01:02for that from first principles.
1:01:04Like if we were just say, like, I want an LLM to be able to execute code.
1:01:08What would that look like from first principles?
1:01:10And we kind of thought, well, we don't really care about the session.
1:01:13We don't really care about, you know, we want it to From the LLM's point of
1:01:17view, feel like it can always run code.
1:01:19It doesn't have to start a sandbox and stop a sandbox when it's done to cut
1:01:22down on costs and things like that.
1:01:24We, we kind of like cut out the rest of it, make that into the abstraction and
1:01:29build it, frankly, into something that we can position for those products so
1:01:32that we're not confusing people who are like, I thought you did sync engines.
1:01:36now you're telling me running AI code and it's like, architecturally, they
1:01:40can actually be fairly similar, but, we wanted to build a product around that.
1:01:43So We have forever VM, it is.
1:01:46Way to think about it.
1:01:47It's like an API that runs Python code in a unbounded session.
1:01:52so by that, I mean, if you, kind of make an API call and get a machine
1:01:56ID, maybe ABC 123, you can run instructions on that machine set a
1:02:02equals three or something like that.
1:02:03Two years from now, if you kept that machine around, you can query
1:02:07the value of a, you know, a plus five, and then you get back a value.
1:02:12and the way we're doing that behind the scenes is using, memory snapshotting
1:02:16of the underlying Python process.
1:02:19So we kind of from the ground up architected the whole system
1:02:23around this and it's kind of neat
1:02:24fascinating.
1:02:25Yeah.
1:02:26My mind is also going to other technologies like mentioned Temporal
1:02:30before, but there's also really fascinating project called Golem VM,
1:02:35which I think is also, Also employing some really interesting tricks, to use WASM
1:02:41and knowledge about, the WASM memory to make sort of checkpoints where you can
1:02:47restore and resume computation or retry.
1:02:51And yeah, I love that, Yeah, we, we get some bolder ideas out there.
1:02:56and particularly now when there is the cost of writing code has come
1:03:02down so much and, now it's also people write that code who know even
1:03:07less about whether it's good or not.
1:03:09So we need to put it into boxes that are somewhat blast safe.
1:03:15but also long, like durable in a way that doesn't break the bank.
1:03:19And I love how that is like an entirely different product, but yet leverages all
1:03:24the benefits and all the, foundations that he's built with Jamsocket, or with,
1:03:29I guess with Plain for that matter.
1:03:31That is very, very cool.
1:03:33Yeah, thanks.
1:03:34one of the things that's been really cool to see is that if we give an LLM
1:03:39the ability to write this code and get responses back very quickly, like kind
1:03:42of just treat it as a local-repl, that the AIs can kind of do more like they get
1:03:48that fast feedback loop and they can make mistakes and correct them almost faster
1:03:52than, and in some cases we've observed them doing this faster than a reasoning
1:03:56model could kind of just generate the right code in the first place.
Outro
1:04:00Nice.
1:04:02Any other things that you would like to share with the audience?
1:04:06if you want to find me online, I'm, paulgb on Twitter or X and paulbutler.
1:04:11org on BlueSky.
1:04:13jamsocket.
1:04:13com is the site, jamsockethq on Twitter.
1:04:16also on BlueSky is jamsocket.
1:04:18com.
1:04:19and, yeah, forevervm.
1:04:20com is the product we were just talking about.
1:04:22Perfect.
1:04:23We're going to put links to all of those things in the show notes.
1:04:27Paul, thank you so much for coming on the show today.
1:04:29I've learned a lot about so many different topics and yeah, really enjoyed it.
1:04:34Thank you.
1:04:35Thank you, Johannes.
1:04:36And really looking forward to seeing you at local-first in Berlin this year.
1:04:40Perfect.
1:04:40See you then.
1:04:41See you then.
1:04:42Thank you for listening to the localfirst.fm podcast.
1:04:45If you've enjoyed this episode and haven't done so already, please
1:04:48subscribe and leave a review.
1:04:50Please also share this episode with your friends and colleagues.
1:04:53Spreading the word about the podcast is a great way to support
1:04:56it and to help me keep it going.
1:04:58A special thanks again to Jazz for supporting this podcast.