Skip to content

andrewlengly.com/sourcing-channels-casestudy :: where the best hires come from case study

# Which hiring channels bring in people who perform well and stay two years, and how a mid-size employer should split its recruiting budget.

designed page ......... /sourcing-channels-casestudy

THE SHORT VERSION

  A data study of which hiring channels bring in people who perform well and stay two years,
  tracking 311 hires by source. It runs survival models on retention and compares
  performance ratings by channel. Built with Python, pandas, lifelines and DuckDB, it
  recommends a two-tier referral bonus and a separate channel mix for production and office
  roles.

the question .......... Referrals, job boards, LinkedIn, job fairs and search ads all bring
                        in hires. Which ones bring in people who stay and do well?
what i did ............ Tracked 311 hires by source through retention, survival models and
                        performance ratings, then checked the pattern against BLS quit rates
                        and published hiring data.
where it stands ....... Done in November 2026. I recommend a two-tier referral bonus and a
                        separate channel mix for production and office roles.

kind .................. Case study · Data study
role .................. Analyst
year .................. 2026
tools ................. Python, pandas, lifelines, SciPy, DuckDB, Jupyter, BLS API

the question .......... Which hiring channels bring in people who stay two years, and how
                        should a recruiting budget split by role?

THE QUESTION

  # A recruiting budget is a bet on which channel brings in people who stay.

  Most teams measure channels by cost per hire. That number stops at the offer letter. A
  cheap hire who leaves in month eight costs the full search again.

  So I measured each channel by what happens after the hire. Who is still there at two
  years, and how do their managers rate them?

HOW I DID IT

  # One table of hires, followed from the offer to the latest review.

  # From raw HR records to a channel mix
  01. Clean
      311 hires, 2006 to 2018
      Tenure to the last review date
  02. Retention
      Kept at 12 and 24 months
      Wilson intervals, Fisher test
  03. Survival
      Kaplan-Meier curves
      Cox model with controls
      Controls: Role family and hire year, since earlier hires had longer to leave
  04. Cost
      Cost per hire still there at 24 months
      Editable cost table
  05. Benchmark
      BLS JOLTS quit rates
      Published referral research

kept at 24 months ..... still employed two years after the hire date, counted only for
                        people hired at least two years before the last review
cost per retained hire
                        cost per hire / share kept at 24 months
  Channel costs are assumptions. Referrals assume a $2,500 bonus plus about $1,000 of
  recruiter time. Every other channel uses SHRM’s published average of $4,700.

WHAT THE DATA SHOWS

  # Referral hires stayed longest and were rated below expectations least.

  # Still employed at 24 months, channels with 20+ hires
employee referral (31)
                        97%
linkedin (76) ......... 95%
careerbuilder (23) .... 91%
diversity job fair (29)
                        90%
indeed (84) ........... 87%
google search ads (49)
                        84%
  Hire counts in parentheses. Huebner HR Dataset, synthetic.

  # Hires rated Needs Improvement or put on a PIP
employee referral ..... 3%
linkedin .............. 8%
google search ads ..... 8%
indeed ................ 10%
company website ....... 15%
diversity job fair .... 17%
careerbuilder ......... 17%

  # Cost per hire still there at 24 months, assumed costs
employee referral ..... $3,617
linkedin .............. $4,961
careerbuilder ......... $5,148
diversity job fair .... $5,242
indeed ................ $5,408
google search ads ..... $5,617
  A referral stays the cheapest until it costs about $4,548 per hire. That leaves room for a
  bonus of about $3,500.

  The gaps all point the same way, and the sample is too small for any one of them to pass a
  significance test. Referral hires kept 97% at two years against 90% for everyone else
  (Fisher p = 0.33). The Cox model puts their risk of leaving 22% below Indeed hires, with
  an interval from 0.30 to 1.99.

  Published data at far larger scale shows the same pattern. On Ashby’s platform referrals
  were about 1% of applications and about 18% of hires, and hiring managers scored them
  highest.
source ................ Referral statistics, with Ashby and New York Fed figures

BY ROLE FAMILY

  # Referrals held up everywhere. The runner-up changed with the job.

  Production roles
    best .............. Referrals and CareerBuilder kept 100% at two years, LinkedIn kept
                        95%
    watch ............. Indeed kept 83%, and 8% of its production hires quit in year one

  Office roles
    best .............. The careers page kept all 9 of its hires. LinkedIn and referrals
                        each kept 93%
    watch ............. Google search ads kept 80% from a small base

  BLS puts manufacturing’s annual quit rate at 23% on average from 2021 to 2025. Every
  channel here beat that in year one, so this workforce is steadier than the national
  average.
source ................ BLS Job Openings and Labor Turnover Survey

MY RECOMMENDATION

  # Pay for referrals that stay, and split the mix by role.

pay the bonus in two parts
                        Half at 90 days and half at 12 months, so the bonus rewards hires
                        who stay.
two tiers ............. About $1,500 for production roles and up to $3,500 for office and
                        technical roles, under the break-even point.
production mix ........ Referrals and LinkedIn first, CareerBuilder as backup, and a smaller
                        share for Indeed.
office mix ............ Referrals and LinkedIn first. The careers page earns a test with
                        more volume.
search ads last ....... Lowest two-year retention (84%) and the fewest Exceeds ratings (4%).
track it quarterly .... Two-year retention and first-year rating by source, with the mix
                        revisited once a year.

LIMITS

  # What this data can’t tell you.

synthetic data ........ The Huebner dataset was built for teaching HR analytics. Its numbers
                        show the method, and the published research carries the conclusion.
one source per hire ... Someone referred by a friend who then applied on Indeed may be
                        logged under Indeed.
assumed costs ......... A real employer should swap in invoices and recruiter hours.

WHAT’S NEXT

  # What I’d run with a real company’s data.

a bonus test .......... Roll the two-part bonus out to half the departments for two quarters
                        and compare referral volume and 12-month retention.
source of influence ... Ask new hires where they first heard about the job, beside where
                        they applied, to catch referrals logged under job boards.
quality of hire score
                        Blend 12-month rating, retention and time to first independent work
                        into one score per channel.
referrer retention .... Check whether employees who refer someone also stay longer, a
                        pattern iCIMS reported.
a live dashboard ...... Rebuild the cleaned tables in Power BI so recruiting can watch
                        channel quality every month.

TL;DR

studied ............... Which hiring channels bring in people who perform well and stay two
                        years, tracked across 311 hires.
found ................. Referral hires stayed longest, with 97% still employed at 24 months
                        against 90% for everyone else.
  # by the numbers
  97% ................. Referral hires still employed at 24 months
  3% .................. Referral hires rated Needs Improvement or put on a PIP
  $3,617 .............. Referral cost per hire still there at 24 months
the catch ............. The sample is too small for any gap to pass a significance test. The
                        referral gap has p = 0.33.
next .................. Roll the two-part referral bonus out to half the departments for two
                        quarters and compare retention.

LINKS

open where the best hires come from
                        /studies/sourcing-channels.pdf
the analyst side ...... /machine/analyst

# end of /machine/sourcing-channels-casestudy. andrew.leng.ly@gmail.com · leave machine mode