In the first post in this series, I talked about the benefits of a consistent, reliable driver scoring system that lets you see drivers from the FMCSA’s perspective. In this post, we’ll look at the second problem a CSA driver scoring system needs to get right: Driver BASIC Percentiles.
When I discussed the first problem of driver scoring systems – Normalizing CSA Points – I explained that, CSA points must be normalized to be most useful. That is, you cannot simply add up CSA points and identify the driver with the most points as the one who should be your highest priority. We identified that the FMCSA explains how to create normalized CSA scores – and it is in the form of BASIC Percentiles. Unfortunately, using BASIC Percentiles effectively requires a clear understanding of what they are and how they can best be used. Ready? Here we go…
What are BASIC Percentiles?
BASIC Percentiles for drivers are scores of zero through one hundred for each of the seven CSA BASICs. The FMCSA has published the “rules” for calculating driver percentiles scores in its Safety Measurement System (SMS) PDF 4.2 MB. Carriers have one set of rules (Carrier SMS) and drivers have a different set (Driver SMS). Driver SMS is similar to Carrier SMS but with many significant differences. Here are a few:
- Driver SMS uses three years of history while Carrier SMS uses only two years
- Driver SMS excludes some violations where the driver is not responsible, while Carrier SMS includes all CSA violations.
- Driver SMS peer safety event grouping within a BASIC uses different criteria than Carrier SMS uses for safety event grouping.
For example, here are the seven BASIC scores for fictional driver Eric Johnson. Good things are starting to happen! We can meaningfully compare Eric’s seven BASICs against each other. We can meaningfully compare Eric against other drivers using his BASIC scores.
So, what can we tell about Eric from his BASIC scores? Quite a bit it turns out:
- Eric’s worst BASIC is the Crash Indicator BASIC. 98.7% of drivers nationally with a crash in the last 3 years have a better scores than he does.
- Eric’s best BASIC is the Drug/Alcohol BASIC in which this driver has had no violations – hence the score of insufficient Data.
- In all the other BASICs, Mr. Johnson has had violations or he would have had a score of zero or insufficient data. Many BASICs require at least 3 relevant inspections, so Eric has 3 or more inspections on his record for the past 3 years.
So, driver BASICs are appropriately normalized and can be meaningfully compared between drivers. Driver scoring problem solved, right? Not so fast, friend!
Challenges with using Driver BASIC Percentiles
Even with meaningfully comparable percentile scores, there are numerous challenges to using your drivers’ BASIC percentile scores as your driver primary prioritization tool:
- CHALLENGE: The FMCSA does not provide driver BASIC percentile scores.
True! As I mentioned previously: Although the FMCSA computes seven BASIC scores for every single CMV driver for which it has data, and although the FMCSA allows law enforcement to use these same scores during interventions, the FMCSA will not release these scores to motor carriers or to the drivers themselves! Do not fret, however, because Vigillo has recreated the FMCSA’s published methodology (PDF 4.2 MB)for driver BASIC percentiles and we have used our extensive database of approximately one million drivers to provide our customers with BASIC percentiles for every single one of their drivers. To my current knowledge, Vigillo is the only place on the planet a motor carrier or a driver can get their driver BASIC percentiles since the FMCSA will not reveal this information.
- CHALLENGE: How to use seven separate percentile scores to provide a single overall score for drivers? Good point! Even if you do have meaningfully comparable scores across the seven BASICs for every single one of your drivers, which drivers should get your focus first? The driver with the worst BASIC percentile score in Crash? or maybe in Fatigued Driving? The bottom line is that if you have seven scores per driver, you still haven’t gotten to the promised land. What you need is one single comprehensive score. So how do you turn seven Driver Percentile BASICs scores into one? Do you average them? Add them up? Some other wizardry?
- CHALLENGE: What about other important items outside the scope of percentile scores? Even if you can figure out how to turn seven scores into one, are you missing important information? Indeed. The FMCSA prioritizes drivers for review using BASIC percentiles – no question. But they also use other factors not included in BASIC percentile calculations – such as red flag violations and shadow violations (to be explored in coming posts on this topic)
Summing up – Driver BASIC percentile scores are very useful in a driver prioritization system, because they are scores that can be meaningfully compared across BASICs and across drivers. Vigillo calculates these extremely complicated percentiles using the FMCSA published rules and provides them to our scorecard customers and their drivers. Having BASIC percentiles for drivers, while hugely helpful, is not the end of the story because we are left with seven scores per driver when we need one…and driver BASIC percentiles are not fully comprehensive as there are important items that are out of the scope of the percentile calculations.
– – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – –
In the next post (available HERE), I’ll look at the next problem with scoring drivers – Red Flag Violations. You can also watch the video of a recent webinar of me discussing these same problems HERE.