Building a news radar and a job hunter in n8n

I've had n8n running on my home server for about a year, mostly doing small glue jobs nobody would ever notice. This month I finished two workflows that felt worth writing about, because they solve problems a lot of people in tech seem to have, and because building them taught me more about n8n's rough edges than the previous year of glue jobs did.

Both run on the same small box: two cores, under 6 GB of RAM, with n8n and Postgres in Docker next to about two dozen other containers. That limit came up again and again, and it explains a few of the choices below.

Keeping up without drowning

Things move fast enough at the moment that I'd started to worry about falling behind. A new model one week, a Next.js release the next, a React advisory, a tool that everyone is suddenly using. I was picking it all up from X, a couple of newsletters, Hacker News when I remembered, and a WhatsApp group. Lots of duplicates, lots of noise, and I'd still miss things that mattered.

Before building anything, I spent an evening going back through the previous month and noting where the links I'd actually found useful had come from. It was a shorter list than I expected: Hacker News, a few subreddits, dev.to, GitHub (new repos picking up stars quickly), Lobsters, and some Medium. The newsletters were mostly repeating those a day or two later, which was a bit deflating, but it made the design easy.

So AI Radar reads those sources every hour and sends me the best few links on Telegram. Hacker News comes in through the Algolia search API, filtered by points and age. Reddit comes in over RSS, because the JSON API returns 403 to my server's IP while RSS works fine (I don't know why, and I stopped caring once RSS worked). dev.to has a public API you can query by tag. GitHub search gives me recently created repos sorted by stars. Lobsters has a JSON feed, and Medium has tag feeds.

Everything ends up in one Code node called "Rank & Format". It turns each item into the same shape, {title, url, source, score, at}, gives it a score from points, comments, age and a boost for keywords I care about, and picks the top handful.

I set two rules before writing any of it. The first was that a link must never be sent twice. The same article turns up on HN, Reddit and Lobsters, sometimes days apart, and getting it three times is exactly the noise I was trying to get rid of. So URLs are normalised (tracking parameters, www. and trailing slashes stripped) and every sent URL is remembered for 30 days. The second was that React and Next.js always get a look in. AI news drowns everything else at the moment, so the first three slots in each message are kept for React and Next.js items when there are any worth sending, and AI and general full stack links fill the rest.

Where that "already sent" list lives turned out to matter a lot, which I'll come back to.

The digest now sits in a Telegram chat I look at a few times a day. Most hours it sends four or five links and I open one or two, which feels about right. If I was opening all of them the ranking would be too strict, and if I opened none it'd be noise again.

A job hunter that waits for my yes

The second workflow came from a more practical need. I wanted to watch for roles in the UK at companies that hold a licence to sponsor visas, which narrows things down a lot and makes ordinary job alerts mostly useless.

The government publishes the register of licensed sponsors as a CSV, and it's around 122,000 organisations. The names rarely match how companies appear on job boards ("Acme Technologies Ltd" on the register, "Acme" on the advert), so a plain lookup misses most of them. I load the register into Postgres weekly and match on a normalised name plus trigram similarity:

create extension if not exists pg_trgm;

select name, similarity(norm_name, norm_company($1)) as score
from jobhunt.uk_sponsors
where norm_name % norm_company($1)
order by score desc
limit 1;

norm_company lowercases the name and strips things like "ltd", "limited", "plc", "uk" and punctuation. With a threshold around 0.6 it's right far more often than it's wrong, and borderline matches get flagged for me to look at rather than trusted.

Three times a day the workflow pulls new roles from job board searches and throws away anything it has already seen, anything that's a contract, and anything from my current employer. What's left gets checked against the sponsor register. Then a cheap model reads the description and answers a few plain questions: is this really front end or full stack, what level is it, and is there a sentence saying they won't sponsor. Only the roles that pass go to a stronger model, which drafts a short cover letter. And then the draft comes to me on Telegram with Yes and No buttons, and nothing is emailed unless I press Yes.

The two-model split was a cost lesson. My first version sent every job to the big model and spent about a dollar on a handful of roles, which is fine once and silly three times a day. Now Claude Haiku does the screening for well under a cent a job, and Sonnet only writes letters for the few that pass. The letters are just as good, and it costs a fraction of what it did.

The approval step isn't going anywhere. An automation that emails people in my name without me reading it first is a risk to my reputation, and the time it would save is a few seconds a job. That's not a trade I want.

Where n8n bit me

Some of these cost me more time than I'd like to admit.

The Code node sandbox has no URL class. new URL(link) doesn't fail in any way you'd notice from the canvas, it just silently dropped every item that went down that path, and I spent the best part of an hour convinced a search was returning nothing. I swapped it for a small regex:

function parseUrl(u) {
  const m = /^(https?):\/\/([^/?#]+)([^?#]*)(\?[^#]*)?/i.exec(u || "");
  return m ? { host: m[2].toLowerCase(), path: m[3] || "/", query: m[4] || "" } : null;
}

The Merge node tops out at 10 inputs, and with 11 sources I had to merge in two groups and then merge the merges. Not hard, just odd to discover halfway through.

n8n import:workflow needs an id in the JSON, and without one you get a SQLite NOT NULL error that doesn't mention ids anywhere.

The one that actually hurt was static data. Re-importing a workflow wipes it, and that's where I'd put the "already sent" list for AI Radar. The first time I fixed a typo and re-imported, the next digest was full of links I'd already seen, which was the one thing I'd promised myself it would never do. Anything that has to survive a deploy now lives in Postgres, and static data is only a cache. On a related note, runs started with the CLI execute command don't save static data either, so testing from the terminal behaves differently from the scheduled run. If state matters, test with the real trigger.

And running CLI commands inside the live container clashes with the task runner on its broker port, so one-off commands need a different N8N_RUNNERS_BROKER_PORT.

Why not just write a service

I'm a developer, so the obvious question is why I didn't write a Go service with a cron job. For the core logic I do write code, it just lives in Code nodes. What n8n gives me is the dull surrounding stuff: schedules, retries, credentials, Telegram buttons, and an execution log I can open on my phone. That last one is the real reason. When a run fails at 8am I can see which node failed and what it was given, without opening a laptop or SSH-ing anywhere.

If one of these grows into something other people rely on, the logic will move into a proper service. That's more or less what happened with Doorstep, which also started life as a personal alert workflow.

First published on my blog. Cover photo: falco, CC0, via Wikimedia Commons (source).