JobsDB: An Autonomous Job-Search Platform

Abstract

JobsDB is a self-hosted job-search platform that runs the entire funnel — discovery, matching, analysis, résumé generation, and application — without a human in the loop until the moment a form needs a signature or a captcha needs solving. It crawls 14 job sources, scores every listing against 15 specialized résumé profiles, analyzes descriptions with an on-premise language model, resolves aggregator links back to the employer’s real application URL, scans that form headlessly to predict whether it can be filled, compiles a job-tailored résumé in XeLaTeX, and submits — directly by API to Greenhouse, Lever, and Ashby, or by driving a browser everywhere else. It exposes 70 API endpoints behind a React dashboard at jobs.rrecktek.com, runs in Docker with Prometheus, Grafana, and Consul, and stores everything in PostgreSQL. This is a walk through the whole machine.

The pipeline

Every listing takes the same journey. It is crawled, de-duplicated, scored against all fifteen profiles, read by the language model, its true application link is hunted down, its form is scanned, a résumé is tailored to it, and — if the odds are good — it is applied to. The stages divide into four phases: discovery, intelligence, link resolution and recon, and the apply itself.

The rest of this post follows that flow, stage by stage.

Discovery: crawling fourteen sources

A dedicated crawler container polls fourteen job boards — Indeed, Dice, Arbeitnow, RemoteOK, Jobicy, Jooble, an email intake, and others — on a schedule, with per-source search queries. Each new posting is fingerprinted by title and company and de-duplicated against everything already stored, so the same job surfacing on three boards becomes one row. A feed-health panel shows exactly where the inventory comes from and whether each source is still alive.

In the snapshot above the store holds 7,259 jobs: Indeed contributes the overwhelming majority, Dice the next slice, and a long tail of niche boards the remainder. The health table records each source’s total, its last-24-hour and last-7-day counts, when it last ran, and whether it is active — the operational truth of the crawl, not a guess.

Intelligence: scoring and reading every job

Discovery produces volume; the intelligence phase produces judgment. Two things happen to every job.

First, scoring. The system carries fifteen résumé profiles — Bioinformatics, DevOps, Security, Data Engineering, and so on — each with its own curated keyword list of about twenty terms. Every job is scored against all fifteen simultaneously: each keyword match contributes to a percentage, and the result is a fifteen-way match vector, not a single number. A job can be a 75% match for one profile and a 20% match for another, and the dashboard shows both.

Second, semantic analysis. An on-premise Mistral language model reads the description and extracts the structured facts a keyword never captures: the true work model (is “remote” actually remote, or hybrid-with-relocation?), the tax type (W-2, 1099, or corp-to-corp), the seniority level, travel requirements, and security-clearance needs. This is the difference between a job that says remote and a job that is remote.

The detail panel gathers all of it in one place: the match percentage, the compensation and deadline cards, the tech stack detected in the posting (here, Rust), the matched keywords that drove the score, and a résumé keyword gap — the terms present in the job description but missing from your profile. The action bar across the top is the job’s control surface: Review, Apply Primary, Apply +0 (API submit), Recon, Analyze, Gap, and Block to suppress a company entirely.

Link Hunter: finding the real application

Here is the problem that makes most job automation fail. A listing from Indeed or LinkedIn does not link to an application — it links to the aggregator, or bounces to a company careers landing page with a search box and no form. You cannot fill a search box.

Link Hunter resolves that. Given an aggregator URL, it works to recover the employer’s actual application link through three strategies: matching against the known URL patterns of two dozen applicant-tracking systems (Greenhouse, Lever, Ashby, Workday, iCIMS, Breezy, Workable, SmartRecruiters, Taleo, and more), looking up the company’s own domain and its /careers path, and searching for “[company] [title] careers apply.” An aggregator-detection guard makes sure it never resolves one aggregator to another.

This distinction is not academic. An early batch scan taught the lesson the hard way: of 2,060 jobs scanned, 1,984 came back empty — a 99% failure rate — because Link Hunter had resolved Indeed URLs to career landing pages like careers.humana.com, which are search interfaces, not forms. The fix was to recognize that only direct ATS URLs carry fillable forms, and to scan only those. jobs.lever.co/{company}/{id} has a form. careers.company.com usually does not.

Recon: scanning the form before touching it

Once the real application URL is known, JobsDB does not immediately try to apply. It runs recon first — a headless Playwright pass that opens the form, dismisses cookie banners and popups, and inventories the page without submitting anything. It detects every field, its type, its label, whether it is required, whether there is a captcha, and whether the ATS demands an account before you can apply.

From that inventory it computes a submission probability from observed penalties — a captcha costs 25 points, a required login costs 35, and unusual field counts subtract more — so a job’s odds are known before a single keystroke. And critically, recon pre-maps every detected field to the answer that belongs in it.

The screenshot shows a recon result in full: 54 fields, 30 required, account required, 20% success. Every row is a detected field mapped to its answer — email, first and last name, phone, street address, city, ZIP, work authorization, “need future sponsorship?”, education level, salary expectation, start date. Green means the field is confidently mapped; red flags a required field still needing attention. This is the map the auto-filler will follow, reviewed before anything is submitted.

The recon lessons compound into a discipline: recon first, always; apply only when three or more fields are fillable; if more than 30% of fields come back as fallback or unmatched, stop; and never close the browser before a human has solved the captcha. Auto-fill works reliably on Lever and Greenhouse, is hit-or-miss on the rest, and finds nothing on career landing pages — which is exactly why Link Hunter and recon run first.

Résumé intelligence: a document per job

JobsDB does not send one résumé. Its résumé database holds the raw material — 18 positions, 21 publications, 9 W3C memberships, 10 certifications, and 48 curated accomplishment bullets — and each of the fifteen profiles selects and orders that material differently, with its own summary and keyword targeting.

For a specific job it goes further and generates a job-tailored document: it runs the keyword-gap analysis against that posting, re-orders bullets by relevance to it, and can weave selected missing terms into the summary before compilation. The document is compiled to PDF with XeLaTeX (Linux Libertine typeface, narrow elegant style) through the shared pandoc container, then stamped with EXIF metadata — author, keywords, the profile name as subject — and named canonically, RonaldReck_YYYY-MM-DD_Profile.pdf, with epoch-stamped archival copies kept alongside.

Application: API where possible, browser where not

With a mapped form and a tailored résumé in hand, the apply itself takes one of two routes. For Greenhouse, Lever, and Ashby, JobsDB submits directly through the ATS API — no browser, no rendering, just a structured POST. For everything else it drives a real browser with Playwright: it fills fields using adaptive fuzzy matching against the recon map, applies ATS-specific selectors, dismisses cookie banners, and can even automate account creation on Workday, iCIMS, and Dice. A configurable daily cap (25 by default) with duplicate prevention keeps volume sane, and every attempt is logged to both a file and a PostgreSQL apply_attempts table.

The platform also researches the employer. A company-investigation panel runs SLM-powered research — size, funding, culture, tech stack — and lets you rate a company and mark your interest hot, warm, cold, or dead, so the same context is there the next time one of their jobs appears.

The dashboard that ties it together

Everything above is driven from a single React dashboard — around ninety distinct interactions across the surface. Job cards carry a match-score ring, salary intelligence, a tier badge, the recon probability, a source chip, and an age indicator. A row of toggle chips includes or excludes any of the fourteen sources with a click. A quick-filter bar cuts the set by remote/not-remote, has-rate, easy-apply, auto/semi/manual apply tier, W-2/1099/C2C, clearance, seniority, and travel — and every filter state persists to local storage across sessions. Profile scores render as color-coded marbles with configurable visibility thresholds, salary is normalized hourly and annually and rated, and description length becomes a demand indicator. The detail panel keyboard-navigates with the arrow keys.

Infrastructure

JobsDB is built to the same pattern as the rest of the fleet. Every service is a Docker container on a shared network; every agent exposes the standard /health, /status, /config, /job, and /batch endpoints, publishes Prometheus metrics on port 9090, and registers itself with Consul. Grafana carries pre-built dashboards for job counts, crawl health, and apply rates. PostgreSQL with pgvector stores jobs, profile scores, application tracking, and company research. Caddy serves the frontend with automatic HTTPS. The external dependencies are few and explicit: the Mistral SLM for analysis, a captcha-solving service, IMAP/SMTP for the email-verification steps of account creation, and the ATS APIs themselves.

Methods and provenance

The feature descriptions in this post reflect the JobsDB implementation as documented in its repository and the operational lessons recorded during its build; the source counts, field counts, and probabilities shown are from live screens of the running system. Link Hunter’s aggregator-to-ATS behavior, the recon field-mapping and probability model, and the ATS-specific form patterns are drawn from the platform’s own recorded lessons — including the corrections, such as the 99%-empty batch that taught the career-page-versus-ATS distinction. The system runs on-premise at jobs.rrecktek.com.