2024-10-10 20:33:04
Speaking of throwaway projects; I’ve been working on setting up a little service that scrapes the access logs from Caddy and the block logs from UFW. For now I’m just pushing it into BoltDB, but I’m looking to use it to generate some stats on who’s accessing (and attempting to access) my VPS. Side note: is there really any difference between a honeypot and a VPS?
There’s lots of small things to get right that I’ve stumbled a bit on:
* Using journalctls cursor to retrieve the logs emitted since last time we stored something (as opposed to storing a timestamp which could lead to reading the same entries multiple times, or missing some entries)
* Whatever is served up by the API should use a read-only transaction with Bolt
* Only one process can access a Bolt database file at the time, so any cron-like job needs to run in the same process as the web server with the read-only connection. I’m using a separate goroutine with a time.Timer to check for new logs at a regular interval and write them to the DB. Works like a charm
* Using journalctl -o json gives you access to lots of great stuff, plus it’s way easier to parse when you can just use json.Unmarshal