Methodology
Exactly how the numbers are made, and what they can't tell you.
Sources
Every day we read the public, no-login job-board APIs that companies use to publish their own openings: Greenhouse (5,502 boards), Lever (2,132) and Ashby (3,322). We don't scrape LinkedIn, Indeed or any site whose terms forbid it. Requests are rate-limited and follow each host's robots.txt. The list of company boards was seeded from two open directories, the LastRound AI ATS directory (CC BY 4.0) and the Job board directory (CC0). We then checked every board against the vendor's API.
What counts as a data & AI posting
Job titles are matched against fixed rules for 12 role families: data engineer, data architect, analytics engineer, data scientist, ML engineer, AI engineer, AI research scientist, data analyst, BI developer, database engineer, data governance, and data & AI leadership. Titles for sales, recruiting, data-center, data-entry, annotation and AI-tutoring jobs are excluded. Level (intern through director) comes from title words only, and titles without a level word are reported as “level not stated”.
Skills
A fixed taxonomy of 151 skills (platforms, formats, orchestration, BI, languages, cloud, ML, GenAI, governance). A posting counts toward a skill if its title or description matches that skill's patterns. A vendor's own postings don't count toward its own product, so Databricks' job ads don't inflate “Databricks”. Company boilerplate can add noise, so treat small differences with care.
Pay
We only use pay the employer published: structured pay ranges from the job-board API, or a strict range pattern in the posting text (for example “$150,000 – $200,000”). Medians use each range's midpoint for US postings in USD, annualized (hourly × 2,080). A median is shown only when at least 10 postings have pay. Pay-transparency laws mean US pay data skews toward states that require ranges.
Weekly index
Each ISO week is measured on its latest daily snapshot. Week-over-week change is this week's share minus last week's, in percentage points. New vs closed postings compare the two weekly snapshots. Both need at least two weeks of history and say “needs more history” until then. Closed postings vanish from job-board APIs, so the daily snapshot archive can't be rebuilt later. It is the core of this index.
What we publish, and what we don't
Only aggregates: counts, shares and medians. We never republish job descriptions, titles, links or per-company figures. The aggregates CSV is free to reuse with credit to Data Skills Index (CC BY 4.0).
Limitations
Coverage is companies that use Greenhouse, Lever or Ashby, which skews toward tech and venture-backed firms and away from enterprises on Workday. Postings for several locations can appear more than once. Classification is rule-based and imperfect. Corrections are welcome.