JobsEQ RTI Update: Better Sources and Years of Experience

We have recently finished an extensive update of our Real-Time Intelligence (RTI) job postings dataset, including both current and historical data.

Updating classification models

On a daily basis, we download and classify new job posts and incorporate them into RTI, to keep the dataset as timely as possible for our clients. These jobs coming in daily are run through set versions of our AI-powered classification models – so a job downloaded last month uses the same models as a job downloaded today in order to ensure consistency and comparability across the entire dataset. While these daily updates occur on set versions of our models, we are constantly working on updating and improving our classification models. When updates are ready to be released, rather than just updating the new jobs coming in, we reclassify the entire RTI dataset to again maintain consistency. In addition to updates to our existing models, such as our Standard Occupation Classification (SOC) code model, we also use these larger updates to introduce other significant changes to the dataset.

Improving quality of data sources

The biggest change in this update involved the website sources utilized. In addition to continuously adding new company sites, we have phased out some lower quality and troublesome aggregator sources and brought in new higher quality sources, most notably LinkedIn. This specific source shift was implemented in the historical data beginning with August 2021, with the overall impact being an increase in coverage and volume starting at that point. The chart below shows a comparison between the new RTI run and the previous data for total USA volume by month.

In the chart, the blue line shows this update while the orange line is the previous RTI. Prior to August 2021, the volume is virtually unchanged between the new data and the old. In August 2021 the volume temporarily increases by 45% due to the source changes but settles to around 29% higher through 2022 and 2023. One additional point worth noting is that while both the new and previous data have shown a decline in volume since peaks in May 2022, the decline is a little less severe in the new data, falling by 48% from May 2022 to March 2024, compared to 58% during the same period in the previous data.

Adding experience requirements to job posting data

New with this update is years of experience data. These data provide the minimum listed experience requirement in a job posting, broken out by number of years when specified. We also group ads into a Not Specified bucket, which indicates that an experience requirement was not found in the job description, as well as a No Experience Needed bucket, which contains jobs where it is explicitly stated within the job description that no experience is required. Below is an example of RTI showing experience requirements for Registered Nurses in Ohio from jobs newly posted in the last three months.