A while ago, one of my projects reached the point where the initial exploratory prototyping had done its job. I’ve figured out an overall technological approach that will be its foundation—and the time came to begin work on its initial version.
Unlike most of the other projects I’ve done, this is explicitly an experimental research project. I have other projects for paying my bills, which take priority and are the main reason why progress on the project is so slow, but this one is entirely there for the new.
I’m a big fan of genre literature—stories that have boundaries and expectations that the author can play with and challenge, as long as the end result is recognisable as an entry in the genre. Software has its genres too, and this is an experiment in making a new spin on the notetaking or writing app genre.
I set myself a few guidelines:
- The UI would be based on proper research, both of the literature and actual user research. This drove most of my work and prototyping on the project last year. I documented much of this work on the Colophon Cards website and in three essays on my personal blog:
- The final app would first and foremost be a web app. I consider the various app stores to objectively be too much of a risk for anybody except for the biggest and most influential software companies. Whatever benefit they pose is outweighed by the downside, which can include massive delays, having updates blocked, a disconnected feedback loop, no direct communication with your customers, and decision-making that is often genuinely mean-spirited and capricious. The app stores are liabilities, not advantages.
- The app would be designed with the real world in mind. And our world is in fundamental decline, at a constant risk of collapse, is built on infrastructure that is decaying and not getting the repairs it needs, and is filled with countries on the verge of fascism. This means the app has to be built with privacy, redundancy, and network resilience in mind. This is tricky. Go too far, and your app becomes unusable. Do too little, and it becomes theatre: meaningless gestures revealed as useless as soon as you encounter a genuine problem.
- It would prioritise performance over other concerns. When privacy comes into conflict with the ability to use the web app on an old computer, access and performance should come first. I don’t think this will affect the end result as much as I thought, but it takes a few potential features off the table.
- It would prioritise ease of use over popular trends.
Figuring out an overall technical approach that would work within the boundaries set by these guidelines took a while. I looked into CRDTs, various syncing strategies such as JMAP or the Dropbox API, but nothing seemed to fit.
It wasn’t until I was forced to take six weeks off work (COVID, followed by a few weeks of brain fog so bad that I was unable to work) that a clear path forward came to me. I came up with an approach that I think is genuinely new: I combined aspects of distributed version control systems and CRDTs in a way that should work within the capabilities of modern web browsers. I have no idea whether it will be usable in practice, but I’ve been having a lot of fun implementing it.
As I said above, this is a research project. Its purpose is to try strange new things and see if they work.
The prototype
I wrote the initial, very rough prototype in Node.
The prototype proved that the overall idea is implementable even though I don’t yet know whether it’s practical.
This was a very frustrating experience. Making a performant, offline-first web app that nonetheless is lightweight on the front end, is necessarily going to be based on various kinds of Workers (Service, Web, and Shared). Node kind of sucks for writing isomorphic code that’s supposed to run both on the server and in the browser. The Node environment, even after recent efforts to add support for more browser-compatible features, is subtly incompatible with the browser environment in several ways. When you use Blob or streams, Typescript will complain because it’s trying to reconcile two incompatible environments.
Accommodating the two is full of tiny points of friction that drag the work. I had to switch most modules that imported isomorphic dependencies to plain JavaScript. This was fine, but it meant that Typescript didn’t benefit the project as much as it could.
For the first dev version that isn’t a proof-of-concept prototype, I wanted to try a new approach that didn’t have the same friction. I considered dropping Typescript entirely. After all, that was the source of most of the friction. I looked at Cloudflare, but they have a strong event horizon of proprietariness (for lack of a better word). Its proprietary services (like Durable Objects) exert a strong pull that inevitably leads you to build apps that only work with their systems. This would be fine if I trusted Cloudflare’s management, but I don’t, so it’s not.
I then had another look at Deno, tested it out by porting a bit of code, and it turned out to be perfect for this sort of project. There is little friction in developing isomorphic code, even with Typescript.
Deno
Working with Deno has, so far, a few benefits beyond “works a bit more like a browser”.
Because it has built-in tools for testing (with snapshots and coverage), linting, formatting, benchmarking, and generating documentation, there is much less variation from project to project than with Node. For any given Deno project, you can expect to be familiar with all of the tools it’s using. None of the “uvu? WTF is uvu?” kind of surprises you often get when checking out a Node project.
This, in turn, simplifies project and work environment setup. There’s a Deno plugin for Visual Studio Code, for example, that does a great job of integrating all of Deno’s tools with VS Code. There is much less complexity in getting up and running from scratch with Deno than with Node. No faffing about. When in doubt, it’s probably going to work the same as the browser.
With esbuild for deno and the http-fetch plugin, you get Deno-compatible bundling for the browser.
(I would forgo bundling for the front end if it weren’t for Firefox, which is the only major browser that hasn’t shipped support for modules in workers, and this project is out of necessity heavy on workers.)
A small experiment I put together a couple of days ago demonstrates a few of Deno’s advantages.
I had the idea that reliance on ternary operators was one of the things holding JS tagged templates back.
Maybe something like this:
html`<html>
<head>
<title></title>
<meta charset="UTF-8" />
</head>
<body>
<p>Test text</p>
${IF(secondary)}
<p>${trusted(secondary)}</p>
${ELSEIF(text)}
<p>${text}</p>
${ELSE()}
<p>Otherwise</p>
${END()}
</body>
</html>`;
Would be more readable than this:
html`<html>
<head>
<title></title>
<meta charset="UTF-8" />
</head>
<body>
<p>Test text</p>
${secondary
? html`<p>${trusted(secondary)}</p>`
: text
? html`<p>${text}</p>`
: html`<p>Otherwise</p>`}
</body>
</html>`;
This wasn’t going to work for DOM-based tagged template libraries like uhtml or lit-html but should be doable for generating HTML strings.
So, I put together a naive but straightforward benchmark using Deno’s benchmarking tool and compared a few pre-existing approaches. Important comparison points were existing string template libraries that didn’t use tagged templates and tools based on JSX that integrate more ergonomically with JavaScript control flow (albeit in a non-standard way):
The JSX-based approaches were 5-40x slower than any of the string template approaches, which all performed roughly the same.
Then I forked one of the tagged template modules and implemented the control logic, with tests, while keeping one eye on the benchmarking results to make sure I wasn’t making a big performance mistake somewhere.
Doing this in Node would have added friction to multiple steps in this experiment. It would have immediately become A Bit of A Project, just from having to choose benchmarking and testing tools.
In the end, I got a small utility that implemented the conditional rendering I’d hoped for while also:
- Should work equally well in Deno, a browser, CloudFlare, or even Node.
- Performs around the same as existing string template approaches.
- Auto-escapes text by default that hasn’t been marked as trusted.
You can find the end result here, extracted into its own repository.
All of this is a long way of saying that working with Deno is a joy. I recommend it highly. It isn’t a question of one thing or the proverbial ‘killer app’ but of a large number of sensible decisions where the overall experience is improved in thoughtful ways.
I’m definitely going to choose Deno for new projects in the future, whenever I can.