I read a bit more on SQLite this morning, and came across this https://www.sqlite.org/np1queryprob.html. Being used to work with client/server databases and being accustomed to thinking that n+1 is the enemy, this was a little surprising. A quick refactor of the function responsible for getting threads, and it's now not only more readable, but threads and posts are even correctly limited when doing pagination.

One thing missing from micronotal is pagination. Let's see how quick I can get something working.

Turns out I'd actually already implemented pagination using thread UUIDs as cursors! So this should be quick. I'd managed to mix up > and < when comparing the UUIDs, so ?after=<uuid> gives you every thread _before_ the provided ID, but that's a quick fix.

No, I was actually right earlier. Got after and before semantics mixed up when combined with ordering on newest first.

Got a very basic implementation working now. It's far from flawless, as it allows you to navigate to empty pages, and pagination buttons are never hidden. But now it's actually possible to see old threads!

Got to do a bit more working on the styling as well.

Calling it good enough now. You can still paginate into the void, but the buttons look ok at least. Biggest issue remaining to solve with pagination is how the limiting in the query to fetch threads and posts is applied. Right now I’m joining thread and post tables and taking the limit on the entire result. This can cause the last thread in the result to not have all posts included. Not critical to fix, but it’s definitely on the todo list.

Fixed pagination by embracing n+1 https://micronotal.com/t/019289d4-e55b-7b19-b46a-215840607cf0

I don’t know exactly what it is about this microblogging style, but it’s got me started writing, and I’m enjoying it. I just finished and published a blogpost on my personal site https://frodejac.dev/blog/go-poor-mans-cron.html It’s not groundbreaking stuff, but it’s actually written by me (not a smidgen of gen AI involved)!

Speaking of throwaway projects; I’ve been working on setting up a little service that scrapes the access logs from Caddy and the block logs from UFW. For now I’m just pushing it into BoltDB, but I’m looking to use it to generate some stats on who’s accessing (and attempting to access) my VPS. Side note: is there really any difference between a honeypot and a VPS? There’s lots of small things to get right that I’ve stumbled a bit on: * Using journalctls cursor to retrieve the logs emitted since last time we stored something (as opposed to storing a timestamp which could lead to reading the same entries multiple times, or missing some entries) * Whatever is served up by the API should use a read-only transaction with Bolt * Only one process can access a Bolt database file at the time, so any cron-like job needs to run in the same process as the web server with the read-only connection. I’m using a separate goroutine with a time.Timer to check for new logs at a regular interval and write them to the DB. Works like a charm * Using journalctl -o json gives you access to lots of great stuff, plus it’s way easier to parse when you can just use json.Unmarshal

Note to self: A guide on how to set up a poor man’s cronjob with a goroutine and a ticker is a decent first blog post

I got a working POC for this running at https://honey.frodejac.dev. Next steps are to do geolocation on the IPs and group the request counts by country.

I’m really enjoying this whole having my own VPS-thing. I’ve now successfully moved my personal site frodejac.dev from fly.io to my VPS. It used to be a Go app that served some basic static pages, plus acted as a playground for small ideas I had. Now my personal site has been revamped, and is simply served as a static site by Caddy. No SSG, and no JS, just some handcrafted HTML+CSS. Fun thing about having a whole VPS to play with instead of just a free tier fly.io app is that I can now run all the little ideas as a separate systemd service and route traffic to them with subdomains instead of paths. Makes all the throw-away projects isolated and easy to take up and down.

Never used systemd to manage a service before. I'm liking it so far. It's really easy to set up a sandboxed environment using systemd-analyze <service>. It does a good job of listing the various security-related settings and how they're currently set, and calculating a rough 'exposure level' for the given service.

I'm always learning something new when using Go to build stuff. If you strive for minimal dependencies, it's fairly bare-bones, and you end up having to figure out and deal with a bunch of stuff that is typically hidden away in the bowels of web frameworks. Today it was browser caching of static files. The Go http.FileServer can use modification timestamps on the files to set cache headers. However, since I'm embedding the static files in the binary this info gets stripped away. This results in the fileserver not setting any cache headers at all, and the browser requests all static assets for every request. Not optimal. Most obvious symptom was a flash of unstyled content on every page load, as the fonts served up by my webserver was requested for every page load. To fix this, I implemented a small wrapper around http.FileServer that returns a http.HandlerFunc that sets Cache-Control, Expires, and Last-Modified headers. The timestamp used for Last-Modified is set at build time using the -ldflags option with go build. Found that to be a good combination of reasonable and easy, as static files cannot change unless the binary changes. Of courser, the binary might change more often than the static files, but I have a plan to solve that using etags. Just needed to get something basic and working out.

Latest project I've been working on is gathering up all odd bits and ends deployed on various PaaSes (Vercel, fly.io, Supabase, ++) and hosting them myself on a VPS. It's been a fun journey learning about setting up a VPS in a way that is reasonable secure to leave exposed to the Internet. One of the things I'm migrating is this site. And, since I'm trying to get rid of a bunch of my dependencies on various providers' free tiers I'm rewriting the site to get off Supabase. It's currently built with SvelteKit with a Supabase backend, but for ease of deployability I have staerted to port it to Go, backed by a SQLite database, and with simple html/template HTML templates that just renders the pages server-side and servers them to the user. I'm also dropping JS entirely (for now at least) and moving away from the SPA feel. This site is pretty basic anyways, and it feels overkill to use a complete fullstack JS framework for something like this. So far I've mostly ported the backend. There are still a few things missing from the backend, but nothing huge. Most of the frontend is also in place, but the styling is incomplete. As is common when rewriting, some parts are changed along the way. So far I've dropped OAuth, opting instead for managing my own users. I'm not going to have open sign-up, but will make it possible to request access (that will have to be manually reviewed by me). I've also dropped a couple of things related to user profile, etc., focusing instead on just getting the basics up and running. Next big step is doing a dump of the Supabase database and writing a migration script so I can get off Mr. Bones' wild ride

Okay, so this is pretty cool. This follow-up is written on an instance no longer served by Vercel and Supabase, but instead running a Go binary backed by a SQLite database on my own VPS! Today I've managed to do a bunch of stuff. First I finished up some styling issues with the rewrite. The old app used tailwind and postcss, so I had to rewrite that into plain CSS. There's still a few bits missing, but nothing on the public pages. Next, I added a bunch of structured logging using the slog package. Makes for easier debugging, and let's me see if there's any activity hitting the application. I've also got the access logs from Caddy, but I think it's nice to get a bit more info on how the app handles it. I also added signal handling of SIGINT and SIGTERM so that the http server waits to finish up any ongoing requests before shutting down. Should make deployments way more graceful. A big piece of the puzzle was migrating data off of Supabase. The old database was PostgreSQL, and used Supabase's row-level security for authorization. Luckily, I was the only one who ever wrote anything, so all posts belonged to me. That made the migration way easier, as I could just create a user in the new database, dump the threads and posts as JSON using row_to_json, and insert them all with the new user as the owner. I've also moved post and thread IDs from UUIDv4 to UUIDv7. Probably a premature optimization, but having them time-sortable on the primary key is a nice property in my opinion. That part was the most finicky, and required me to rip off some of the code from github.com/google/uuid and make it accept a time.Time and set the time-bits based on that (instead of current time). Finally, time came to get it deployed. I created an Ansible role for the app that includes setting up a app-specific user and group with no shell, creating all the necessary directories with the correct permissions, cloning the app repository onto the ansible host (not the remote) and building the binary for the correct OS and architecture. This binary is then pushed to the remote, along with a systemd unit file, and it makes sure to start/restart the service as necessary. The final playbook also includes the Caddy role to make sure that the Caddyfile is updated to let Caddy reverse-proxy requests to the app.

With all that done, the only thing remaining was switching the DNS and all should be good, right? Wrong. As I had configured Caddy before changing DNS, Caddy had started making requests to LetsEncrypt to get a SSL certificate for micronotal.com. But the ACME challenge failed each time, as micronotal.com was still pointing to Vercel. As this certificate dance is done on each reload, and I was doing a bit of trial-and-error-debugging, I quickly hit the LetsEncrypt rate limit on failed authorization requests. After changing the DNS and waiting an hour for the rate-limit window to open it was time to try again. And it failed, again. The error messages were a bit cryptic, so I went for the good ol' if at first it doesn't work, try again. And again. And a few more times. Maybe it works this time? No? Okay, one more time should do it. Unsurprisingly, it didn't magically resolve itself. Went a bit back and forth, but managed to find the cuplrit when I check the record for micronotal.com using dig, and then attempted to curl said IP and getting a response that definitely wasn't from a service of mine. Turns out I left out a '6' in the DNS record ¯\_(ツ)_/¯ Got that fixed, waited 5 mins for the TTL, waited another 5 minutes for the LE rate-limiter, and now we're here!

I think I've actually completed the migration now. I've fixed most if not all of the remaining cosmetic issues, and the site is super snappy which is really fun. I just deleted the old projects on Vercel and Supabase. Feels good!

I've been working on setting up Manjaro Linux (i3) lately, and it's overall been a good experience. Hitting some snags with customizing the terminal (mostly with powerline and nerd fonts), and all my tinkering has led to a bad state that seems hard to recover from. I'm going ahead and doing a clean install, but before that I want to note down the apps and packages I've installed that I actually want to reinstall. In no particular order: * bitwarden * bitwarden-cli * chromium * mpv * neovim * with lazy.nvim package manager * noto-fonts-emoji (if fonts aren't rendering properly in the browser) * py3status * docker * spotifyd * tmux * ufw Some of these might actually be there out of the box, so it's partially a list to remind me to configure them. Finally, a big fat note to self: remember to backup all config files before changing them! Should prevent the need for a clean install if anything breaks

To update, so far I've done: * backup i3 config, cp .i3/config .i3/config.bkup * set resolution, echo 'xrandr -s 5120x1440' >> .i3/config * update pacman mirrors, sudo pacman -f 30 * install brave browser, sudo pacman -S brave-browser I'm partially following these guides: * https://github.com/haraldwb/manjaro-i3wm-post-install-guide * https://jasoneckert.github.io/myblog/configuring-i3/

Changing default browser: * modify .i3/config, set bindsym $mod+F2 exec brave * modify .config/mimeapps.list (:%s/userapp-Pale Moon/brave-browser/g) * set $BROWSER in .profile, export BROWSER=/usr/bin/brave

On the issue of fonts in uxrvt: Seems like the default .Xresources config shipped with Manjaro sets the URxv.font without escaping spaces. The correct thing seems to be to escape them using \

Finally doing a sudo pacman -Syu to update installed packages

Fix for missing emojis: * pacman -S noto-fonts-emoji * add the following to /etc/fonts/conf.d/66-noto-emoji.conf: <?xml version="1.0"?> <!DOCTYPE fontconfig SYSTEM "fonts.dtd"> <fontconfig> <alias> <family>emoji</family> <prefer> <family>Noto Color Emoji</family> </prefer> </alias> </fontconfig>

TODO: get emojis working in the terminal

A few things done based on the aforementioned post-install guide: # Enable fstrim sudo systemctl status fstrim.timer sudo systemctl enable fstrim.timer # Set swappiness sudo vim /etc/sysctl.d/99-swappiness.conf vm.swappiness=10 # Enable ufw sudo ufw enable sudo systemctl status ufw sudo systemctl enable ufw

I think I found a working font and procedure for urxvt: sudo pacman -S ttf-0xproto-nerd edit ~/.Xresources: URxvt.font xfg:0xProto Nerd Font Mono:pixelsize=12 # no escaping xrdb -merge ~/.Xresources Close all running terminals to restart urxvt

AFter all this messing about with getting urxvt correctly rendering fonts, I think I'm going to swap to alacritty anyways (:

Install alacritty: $ sudo pacman -S alacritty Make it the default terminal: $ vim /usr/bin/terminal #!/bin/sh alacritty "$@" Create config file: $ mkdir -p ~/.config/alacritty/alacritty.toml Edit config file, set MesloLGS font: [font] size = 11 normal = { family = "MesloLGS NF", style = "Regular" } bold = { family = "MesloLGS NF", style = "Bold" } italic = { family = "MesloLGS NF", style = "Italic" } bold_italic = { family = "MesloLGS NF", style = "Italic" } Change default shell: $ chsh -s /usr/bin/zsh Configure poweline $ p10k configure

pacman -S mpv

I think I've been fairly successful in setting up neovim with the lazy.nvim package manager. I've got this config structure: $ tree .config/nvim .config/nvim ├── init.lua ├── lazy-lock.json └── lua ├── config │   └── lazy.lua └── plugins ├── cmp.lua ├── init.lua ├── lsp.lua └── treesitter.lua And the follwing plugins installed: cmp-buffer, cmp-cmdline, cmp-nvim-lsp, cmp-path, cmp_luasnip, LuaSnip, nvim-cmp, nvim-lspconfig, nvim-treesitter, nvim-ts-autotag, tokyonight.nvim

Today I went ahead and installed oh-my-zsh. Plugins enabled: git, asdf, dotenv

Embarking on a new adventure, setting up ansible to automate my VPS configuration. I want to manage my python versions with asdf, and I used the zsh asdf plugin to easily install asdf. Just clone the repo and enable the plugin. Next step is installing some dependencies that (I think?) are necessary for python, or at the very least recommended by asdf-python, though they link to pyenv docs for some reason, not digging in to that now: pacman -S --needed base-devel openssl zlib xz tk asdf plugin-add python asdf install python 3.12.6 pacman -S python-pipx pipx install poetry The rest will be put in version control, so I won't bother writing everything down in this thread

It's interesting to see this site still up. I had totally forgotten about it.

Not to mention that it's still working