Just read https://technicalwriting.dev/ml/embeddings/overview.html, and while it didn’t present me with a lot of new stuff, it had one idea that would be interesting to implement; providing embeddings through an API. I might take a stab at adding sqlite-lembed and sqlite-vec, and create embeddings locally for all content, and serve it up to anyone who wants it (mainly me, I suppose). Will also get me most of the way to some semantic search functionality and «related threads».

Idea for learning: feature flag service built on etcd, with a OpenFeature provider implementation and CEL for server-side runtime evaluation

I did avtually follow up on this, though not quite as described. I went for an implementation using BadgerDB instead, with clustering provided through the Raft consensus protocol. Whole project is here: https://github.com/frodejac/switch Incidentally, this was built when testing out Cursor for the first time, and I was pretty amazed at how quickly I could go from nothing to a functional MVP. Have to admit the amazement faded a bit after working with it for a while, as the tool quickly reaches some limits once the complexity starts to grow. And some routine tasks such as refactoring can quickly eat up hours of your time if left to the AI to figure out. The simple task of moving around existing code can get really hard when it’s done through «memorizing» the existing code and then reproducing it from memory instead of just copy-pasting things or using proper rafctoring tools.

I found that a much better approach for AI assisted refactoring is to have the AI suggest changes instead of running in agent mode. Get it to suggest new structure, which pieces can be extracted, etc., then have it create the scaffolding for the new files, and then lastly do all the copying and fixing manually. Much less hassle. And you don’t run the risk of suddenly getting a new feature you didn’t ask for.

Note to self: add some linkblog functionality. A simple form with a input field for the link and a free text form for short commentary.

I find a lot of interesting links to articles that would be great to keep in one place for later.

Found this one quite interesting (copy-pasted from https://news.ycombinator.com/item?id=42620446): > [...] all you need to know is velocity. The Y Combinator people call it doing things > that don’t scale. Here is how it works for absolutely anything: > > 1. Get the right tools in place. This is an intrinsic capability set you have to build. > People tend to fail here most frequently and hope some framework or copy/paste of a > library will just do it for them. Don’t be some worthless pretender. Know your shit from > experience so you can execute with confidence. > > 2. Build a solid foundation. This will require a lot of trial and error plus several > rounds of refactoring because you need some idea of the edge cases and where you the > pain points are. You will know it when you have it because it’s highly durable and > requires less of everything compared to the alternatives. A solid foundation isn’t a > thing you sell. It’s your baseline for doing everything else at low cost. > > 3. Create tests. These should be in writing but they don’t have to be. You need a list > of known successes and failures ready to apply at everything new. There are a lot of > whiners that are quick to cry about how something can’t be done. Fuck those guys and > instead try it to know exactly what more it takes to get done. > > 4. Finally, measure things. It is absolutely astonishing that most people cannot do this > at all. It looks amazing when you see it done well and this is ultimately what separates > the adults from the children. This is where velocity comes from because you will know > exactly how much faster you are compared to where you were. If you aren’t intimately > aware of your performance in numbers from a variety of perspectives you aren’t more > special than anyone else. > > People who accomplish hard things are capable of doing those because they didn’t get > stuck. They had the proper tools in place to manipulate their environment, redefine > execution (foundation), objectively determine what works without guessing, and then > know how much to tweak it moving forward. It's perhaps a bit rough, and written from a tech perspective, but I think it makes sense and the principles can be applied to most hard problems.

Some changes of note from the last few days: * UI tweaks * Made the content container a little wider on large displays * Switched to using Google fonts instead of self-hosting * More minimal UI * Bunch of fixes for smaller screens, e.g. text-overflow: ellipsis for long usernames ++ * Button styling * Added a link to the signup page from the login page * Tweaked username/email validation, made it more restrictive * Implemented basic flash messages on pages where it makes sense * User pages now display threads/posts authored by that user (with pagination!). Categories for user pages are broken, though, will have to fix that. It's a lot of fun working on this, even though nothing may come of if it the end. If nothing else, I will be able to reuse big parts of the codebase for another project I have in mind that has a much narrower scope and more targeted audience. More on that some other time!

Got user page categories working, though the category links on the posts still point to / so filtering has to be done manually.

Just did a major refactor of the codebase to get a better structure, and hopefully improve velocity for future features. Not a lot of interesting stuff to say about it. There's a decent change I'll refactor it some more at some point, as I'm still trying to figure out the most ergonomic way of creating request handlers in Go. The codebase has really low test coverage, so I hope I didn't break anything. #devlog

I'm thinking of categorizing threads through #tagging. Right now it's not possible to query threads by topic, and I don't want to go all full-text or semantic search (yet at least). Without reading up on any best practices for a system like this, I'm thinking of the following process: - user submits post to thread (also what happens on thread creation) - store thread and return to user, but also kick off a background job to process the text content and categorize the thread based on tags found in it's posts - add a (repeatable) c query parameter to /t that, if present, filters threads by catgeory - add a op query paramter to /t that defaults to any, but can also be set to all to only include threads that has all the selected categories To avoid a very complicated parser to extract tags, I'll restrict it to - only try to extract tags from the last line - the line must start with a '#' character - tags are on the format #[a-z]{3,}

I have to make some UI changes if I want to display tags on posts as links to a thread query that includes the given tag. I think it would be best to gather the tags at the top of the thread and display it under the thread ID and author heading. If I’m using # to denote tags, I’ll start using $ to denote thread and post IDs instead of # to avoid any confusion.

If I've done everything correct, it should be possible to categorize threads now using the syntax described above. I dropped changing the symbol used to denote thread and post IDs, but might change that later. #devlog

One UI thing I missed: don't include the thread-categories div if there are no categories to display.

Tweaked the UI a little. Removed the border between threads, and added a border-top on the categories div. That also removed the need to hide the categories div if it's empty. Also, my initial rough implemntation of categorization took the entire thread and processed all posts every time. Now it just takes the post that has been written and processes that. I think I can call this feature complete. Only thing I'm considering is if I should remove the text line containing the tags from the post content after processing. Makes for a cleaner look. Or, I could keep it in the database and remove it when it's rendered. Either will work.

If I decide to remove it, I should take care to handle posts consisting of only tags correctly.

Another day with some spare time, and another improvment incoming. I've set up Litestream on my VPS to do streaming replication of the SQLite database backing this site. So far I've set up ansible to install dependencies (sqlite and litestream itself, latter installed from a downloaded .deb), create the litestream config file from a template, and start the service. Service will also be reloaded on configuration changes. Right now I'm just replicating the database to another directory on the host to verify that everything is working. Next step is to set up replication to Backblaze B2 so I can have a proper backup setup.

Streaming replication to Backblaze works like a charm! Seeing as this is a very low-traffic site, I've set the sync interval on the s3 bucket to 1m. This should really limit any costs associated with this backup, even in the case of a massive traffic surge. For normal operation, I'm pretty sure I'm looking at free backup here. We'll see in a month, I guess.

One thing I actually should try is a recovery of the B2 backups. Not today, though.

Feeling inspired today, so I've got some more work done: * Rate limiting of signin attempts * Improved error handling * Now that I have a working rate limiter, I've enabled signups! No button anywhere that will take you there, though, so if you're reading this head to /signup and fill in the form, and I'll probably activate your account. A side note: I have a linux desktop, and a macbook, and I've previously only built and deployed micronotal from the linux machine. Running the ansible playbook from my mac, I hit a snag with mattn/sqlite3 and CGO. Luckily, there's a very straight-forward how-to in the README (https://github.com/mattn/go-sqlite3?tab=readme-ov-file#cross-compiling-from-macos): $ brew install FiloSottile/musl-cross/musl-cross $ CC=x86_64-linux-musl-gcc CXX=x86_64-linux-musl-g++ GOARCH=amd64 GOOS=linux CGO_ENABLED=1 go build -ldflags "-linkmode external -extldflags -static"

Made a couple of incremental improvements today! Yet to actually deploy the updates, but the short changelist is: * Fixed pagination buttons. Renamed from next/previous to older/newer, as the sematics are a bit clearer. Also, it's no longer possible to paginate into the void, as the the buttons are hidden if there are no more results to paginate to. * Switched from storing passwords using bcrypt to argon2id. Added some logic to autoupgrade users on login (since that's the only point in time where I have access to the password in plain text). Not exactly going to be a big migration, as I'm the only user, still nice to do it properly. * Slight improvement of the profile page. Now displays some stats like when you joined, and how many threads/posts you've written.

Deployed, and it looks to be working great!