AK
← All projects

Custom Software

LinkedIn Job Scraper (PPS)

A Python scraper and dashboard that collects public LinkedIn job listings at scale, cleans and de-duplicates them, and exports structured data for recruitment and market research.

PythonPlaywrightVPSDashboardCSV export
Private — internal tool
Diagram — LinkedIn public job pages scraped, cleaned, de-duplicated and exported

Why it exists

A recruitment business needs to know who is hiring, where, and for what — every day. Manual LinkedIn searching cannot keep up, and paid data vendors are expensive.

What it does

  • Scrapes public LinkedIn job search pages with Playwright, in a fresh logged-out browser context each time.
  • Enriches each listing with the job detail page and company information.
  • Cleans and de-duplicates across runs so the same job is never counted twice.
  • Filters by title, location, and work type.
  • Exports to CSV and shows results in a small web dashboard.

Engineering notes

  • Strict operating rules are documented in the repo: never log in, never bypass a block, never make a live request without approval.
  • Every run writes a status report. A silent-truncation bug (runs that finished “clean” but skipped detail pages) was found through these reports and fixed with explicit accounting of attempted versus collected pages.
  • The scraper feeds two other tools: the ERS CRM and the Agency Signal analysis.