Since re-launching my blog, I wanted to monitor traffic and logs more closely. Nothing groundbreaking, but it had been a while since I'd run Grafana, Prometheus, and Loki on my own hardware.
Turns out there's this handy all-in-one docker-compose setup that runs on Synology NAS. It fired up with minimal fuss, and soon I had metrics machinery humming in my basement—except the package didn't include Loki. A quick docs consultation got it running alongside the rest.
My blog is a static site hosted via AWS S3 and CloudFront. Both services dump logs into an S3 bucket, but I'd never bothered reading them before—and didn't want to start now. Instead, I loaded up Claude.ai and described my problem:
I want to get logs out of CloudFront. I have enabled new-style log delivery that stores gzipped JSON logs in an S3 bucket at s3://lmorchard-logs/blog.lmorchard.com/ with names like E5YXU82LZHZCM.2025-06-04-04.d024d283.gz
Can you help me write a script for my home Loki server to download only new log files and push them into Loki?
Claude stepped right up:
I'll help you create a script to process CloudFront logs and push them to Loki. Let me write a Python script that tracks processed files and handles the gzipped JSON format.
After some vibey iteration, we landed at this artifact:
It's quite verbose and could use some tightening up. But, I really don't care—it does the quick & dirty needful.
I wrote zero Python. I just henpecked Claude to add features until the script did what I needed. I wasn't even in an IDE, just the Claude.ai interface in a browser. An interesting thing to note is that Claude didn't have access to my AWS resources—I didn't even give it a sample of my logs. But, still, what I told it about JSON, S3, and CloudFront was enough for it to be off to the races.
Anyway, after a quick review and a satisfactory dry run, I dropped it into a cronjob to grab new logs every 5 minutes. Then I pestered Claude with Grafana dashboard questions I could have figured out myself. But why read docs when you can just ask? (Which I realize is ironic, since I wrote Too long? Read anyway. but I think I make an exception for LLMs.)
Total time from idea to working dashboard: about an hour.
Not revolutionary, but pretty satisfying for barely having to think about it.