Post-pandemic update

2023-12-01 12:34:45 -0500 EST Updates

Oh wow, it's been a while since I last posted something here. The pandemic absolutely wrecked my productivity, and when I did get it back up to something workable, that productivity was mainly confined to work. Even at the start of the pandemic, I had two blog posts in the middle of being written and a few other things in the pre-work phase, but that slowed down considerably as time went on. And in the middle of that I promised I would post things that two and a half years later I still hadn't.

This awkward and probably disjoint post is going to do all that. Circumstances have come up that have caused me to need to post a number of things relatively quickly, but I still want to keep my promises first (to some extent at the very least), and provide if not an explanation then an excuse for the status of my open source projects.

The obituary post I promised myself (and a few other people) I'd make

Actually, that one is going to wait a little bit. I originally wanted it to be here, but a) it is tonally distinct from the rest of this post, b) it made this post go on too long, and c) its audience is significantly different (and arguably smaller) than that of the rest of this post. So I will still post it, but it will most likely be after the next urgent post.

A quick update on Crying Dragon: per-ROM checksums

My last post was the release of a prototype of an unreleased video game called Crying Dragon. Unbeknownst to me at the time I purchased it, it had already passed through someone else (who will remain anonymous), who also did a chip-by-chip dump of the game. My dump with a cartridge dumper matches theirs, but some projects (MAME in particular) would like to know what the checksums of those individual chips are. That previous owner provided me a text file with that information that I said I would release, and so I am releasing it now.

Download the text file (611 bytes).

I've since acquired a pre-made adapter for the odd EPROM chips that prototype used, so if (when?) I run across another one with those chips, I'll be able to dump them myself. (Thanks again, Devin!)

Are you still working on libui?

First, a bit of background information. When I created libui's predecessor, the Go-specific package ui, my goal was to use the programming language made by the old Bell Labs people I obsessively read everything of when I was in high school to build a program for making music to replace the old abortive Qt-based programs I had tried building before. At the time, I was regularly switching between a desktop iMac and a laptop dual-booting Windows and Linux, so portability was a priority. Things quickly got out of hand, and before long I had effectively rewritten the project multiple times, including the big rewrite to C that resulted in libui, and the music software project kept slipping away. Meanwhile, this was right around the time web-based desktop software tools like Electron started gaining serious traction, often to the frustration of desktop and laptop users who disliked the clunkiness of those web-based systems as they existed back then (and arguably still dislike their resource usage), and so were excited by a new library that explicitly was not that. It also did not help that I presented myself in a far more professional manner that led people to think the project was far more complete than it was.

Ultimately, the C version that I had built had serious structural issues that made fixing some particularly troublesome bugs difficult to fix, and so in 2019 I decided to commit to one last real rewrite to get things right. Since the implementation of much of the system was still usable as they were, I called this a "remodel" instead. At the same time, I decided to switch the library's test suite from a largely ad-hoc series of manual tests to an automated test suite, and to write documentation as I went. As you can probably guess, this has been a slog, and as the brain rot from 2020 progressed, progress and motivation slowed down considerably, to the point that I admittedly felt overwhelmed thinking about what to write next.

And that's where things currently stand. I still have a still-in-its-early-phases remodel git branch that contains enough work done to open a top-level window on Windows, macOS, and Unix systems. I started looking into doing a Haiku version again concurrently, but dropped that once it became clear that would just slow things down even more. However, the Haiku threading model does give me a new idea on how to implement the Windows backend cleanly, avoiding some of the issues that affect only that particular backend; I have yet to actually act on this idea.

The biggest personal hurdle to getting libui further along, more so than my productivity slowing down, has been the sense of perfectionism I've approached my projects with. I've been trying to build this library "right" all this time, both in terms of exposing the "right" API that only let people do the "right" things and making sure all the abstractions and implementation layers are "right" and clean and sensible, and the point of the remodel branch has been to do it "right" once and for all. And thinking about that has been overwhelming, especially when trying to figure out the design of each piece as I build it. I know I've had to accept leaky abstractions and imperfect structure at work in some other projects I've felt similarly passionate about, and I'm not quite sure if that's what breaking perfectionism looks like on a personal project like this with no authority-imposed deadline, but it is something that I will probably have to overcome if I commit to finishing again.

The march of time is another major issue that affects things going forward. One thing I've always tried to do with libui was make it compatible with old-but-not-too-old versions of the supported OSs, usually going back a couple of versions. I've always felt that perfectly good OS versions are being abandoned too quickly in the name of either security or pushing new product (or both), and I've felt that process has only accelerated (especially for the latter reason) as of late. The problem is that while I can make libui backward-compatible, a lot of the tools that libui depend on are slowly dropping their backward compatibility, and that will make building and using libui the way I want increasingly difficult going forward.

This is of particular concern on Windows, where I consider it unethical to require anything newer than Windows 7. While Windows 8 and 8.1 had a compromised UI that I didn't feel comfortable forcing on users, and even Windows 7 has some amount of diagnostic telemetry, Windows 10 and newer feature such an alarmingly high volume of telemetry to the point that I would consider it explicitly surveillance, and over time the amount of ads, user-hostile software choice decisions, "rate us 5 stars" dialog boxes on built-in tools, and replacement of built-in tools with paid subscription software — none of which has any business being in a buy-once-install-forever OS. In other words, Windows 7 support is mandatory, period. Unfortunately, now that Microsoft has discontinued their support for Windows 7, an increasing number of my dependencies have dropped support for Windows 7, with more surely to come. A major one is MSYS2, the current preferred means of distributing MinGW on Windows. MinGW is needed to continue to maintain the Go bindings to libui, since Go can't build C code with anything other than MinGW. For that, I might have to make it so that building libui itself can only be done on Linux (since I don't think MinGW itself is going to drop support for older versions of Windows ... yet), which will make building and testing libui for Windows increasingly inconvenient. (Of course, I do want libui to take advantage of Windows 8/8.1/10/11 tech if available, falling back to Windows 7 functionality if not available.)

A similar thing is happening on macOS, especially now that Apple is bumping the major OS version number on a yearly basis, web browser style. Perfectly usable versions of macOS are slowly being dropped by various software programs, even if they aren't using the new tech in the newer OSs. Furthermore, Apple has made it harder to build software for old OSs, or to make such software submittable to the App Store, and as such open source projects lacking the expertise or resources to keep supporting old versions are simply dropping official support, even if there is no technical reason to do this. And I understand this — continuous integration tools that libui uses for testing and producing release builds also have been dropping support for older OSs, and I'm likely going to have to set up my own CI system if I want to keep going as well (something that I do have the resources and expertise to do, but which would still be time-consuming).

In the end, the chorus of interested potential users wondering where I went has gotten loud enough that some of them have started forking the project to keep it "alive". I know I've let those users down, and I'm sorry about that. There are times when I wonder if libui would even have been the suitable GUI library some of them probably expect it to be, given that one of the things I was perfectionist about was not creating a library that could be misused, and (combined with the sorts of primitive UIs I was building in my test suite and example programs) that this would prove inflexible. But I do have to say: I can't make any promises, but I currently don't want to kill this either. I just need to get the motivation to keep going, and that isn't quite back yet. Perhaps after a few other things. And I wish the best of luck to the forks of libui; forking is a good part of the open-source ecosystem and I support it even under these circumstances.

I do have one other note about a pipe dream of mine with libui, but I'll bring that up in the next section. I also have similar comments to some of the ones above about reallymine, my other major project which was for hard drive backups, but which was even more ephemeral in the end — I used it for my own purposes and now haven't needed to use it myself anymore. Not sure what will happen to that. And if you're curious about some of the design and implementation hurdles that libui saw over its development, someone in the Rust community tried to create a similar project specifically for Rust and documented their findings in an excellent post; I can personally attest to a number of these.

So what now?

I actually did produce one major open-source contribution in 2022: emulation of the Yamaha DX100 synthesizer in MAME. Previously this was just a skeleton that could boot the firmware but nothing else and was written by someone else; now it's fully functional and usable. This was done as part of a larger investigation into the sound chip used by the keyboard, the Yamaha YM2164. This chip has some features that aren't used by other devices that use it; hese features are used by the DX100, so getting this emulation working helped to demystify things. I'll still need to play around with the chip directly to actually figure out what these extra features do; I know now they have to do with the breath controller. I don't know when that'll happen, but I do have all the hardware I need to do it.

One thing that accelerated during the pandemic was my spending. A few months before the pandemic I started digging into some music hardware for 1980s computers, and that turned into me joining the big, annoying, and unjust "everyone buys retro hardware all at once and drives the prices way way up" bandwagon. I've now accumulated a large number of machines and software to have fun with. This includes more of the prototypes like Crying Dragon that I hope to dump and release soon, and retro machines that i might explore interesting subjects about soon. In fact, during the start of the pandemic I was in the process of working on one of those, and that will most likely be the major project that gets a blog post next.

One set of machines I did acquire were older (mostly 90s) machines that ran Unix-derived and Unix-like operating systems, and I got them explicitly to try to port libui to those. Part of that major project I mentioned in the previous paragraph involves running some code on a system that has always been near and dear to me, and after a decade of not seeing what little code I had written for it running on real hardware, my first time running my own code on that hardware brought an indescribably joyous feeling. And for someone like me who grew up fantasizing about fake esoteric computers, the opportunity to run my own code on real esoteric computers was too good to pass up. But this goal flies directly against all the other goals of libui — it brings to the forefront the thing I haven't addressed up to this point, and that's whether this was really a project I'm making for myself or not. And that will come up again when I get to the point of trying to set up these machines, which I really want to do.

I had two blog posts partially or mostly written during the start of the pandemic, both directly related to libui: fixing a nasty bug in Solaris 11 that causes X11 to not work in VMware, and building VMware's shared folder access tool for Linux on OpenBSD (and possibly the other *BSDs). I might still finish these posts, but the passage of time will necessitate some rewrites, and I'm not even sure the Solaris 11 one is still feasible (it assumes Oracle still makes everything I need for it available). I do have a few backburner post ideas as well, including (and especially) one where I investigate a notorious incompatibility between an old video game and a popular peripheral for the system it runs on. I can't promise a schedule though.

One thing I need to do at some point in the near future is impose a schedule on myself so I actually get everything I want to do outside work done. I still have to write those music and fiction writings I've reserved sections on my website for ages ago, and I'm only getting older (something I admit to having anxiety about, possibly learned from wider society). I'll leave anything beyond that to myself though.

All that being said, this post is the first in a number of posts I'm going to release in somewhat rapid succession. I've recently paid a larger-than-I-would-prefer amount of money for something, and I don't intend to be nerd-sniped on that thing, so I'll be sharing that most likely tomorrow. I've been sitting on two similar things for a few months (if not years) and will also share those things shortly afterward (possibly separated by a week or two each). Two of these things are previously undumped pieces of software and one of them is a prototype of something that was released, so stay tuned.