The New CSA/IRT Scorecard

The New CSA/IRT Scorecard

The Next Generation of CSA

Introducing the new CSA/IRT Scorecard

The new CSA will measure a motor carrier’s

Safety Culture with a single score

CSA is undergoing major changes and almost everything we know about CSA—severity weights, time weights, BASIC measures and Safety Event Groups—will no longer factor into the new scoring methodology under the FMCSA’s planned 2019 release.

Our new IRT/CSA Scorecard provides you access to your new CSA Safety Culture Score—an entire year ahead of the FMCSA planned release date.

Establish Your Safety Score NOW!

[mrkto_frm id="1303"]

Be the first to know your CSA Safety Culture Score

The two-year look back is already in effect. We are offering you visibility into your new CSA Safety Culture Score to help you get ahead of how it measures your company’s safety culture—and show you what you can do now to prepare.

CSA Safety Culture Score

The new CSA FAST Act Score Model utilizes Item Response Theory (IRT) methodology and almost completely changes the building blocks of the current CSA scoring model.

It more accurately measures a company’s Safety Culture as your new score is based on inspections, violations and violation groups, and also factors in relevant variables such as drivers, inspections, VMTs, number of power units and more.

BASIC Comparison Scores

Get a side-by-side view of your current  CSA scores next to your new CSA/IRT  scores and start to understand the new  CSA scoring model.

Violation Groups

See your activity by violation group. The new  methodology identifies the same possible 945  CSA violations and assigns them to one of 66  violation groups. These groups are assigned  across the BASICs.

Industry Benchmark

How do you rank in the Exposure Risk Index?  See how you stack up against like motor  carriers in the industry and where you rank  amongst your peers for safety culture.

Violation Group Detail

Get the big picture with violation count, two  year timeline, severity indicator, and violations  falling off. See where you need to focus to  maintain or improve your CSA Safety Culture Score.

Violation Detail

Drill down to the specific list of individual  violations that make up your overall violation  count in any given group.

Driver Detail

The driver detail page will show you violation, date and location by driver.

Protect Your Business Now!

Click Here

New IRT CSA Methodology

New IRT CSA Methodology

How Safe Are You?

Are you ready for CSA Reform?

CSA is changing, and finally for the better. In 2008, Vigillo was first to run CSA scores on behalf of the industry. Since then, thousands of motor carriers have entrusted Vigillo as the industry experts to bring them powerful insights and valuable data analytics for CSA.

Since the FAST Act of 2015, we have been at the forefront of R&D of the new Item Response Theory (IRT) methodology for CSA.

We are proud to announce that following the roadmap put forth in the July 2017 report, Improving Motor Carrier Safety Measurment, by the National Academies – SambaSafety, powered by Vigillo, is able to provide customers with access to a preview of their new Safety Culture Score.

Almost everything you know about CSA, severity weights, time weights, BASIC Measures and Safety Event Groups are out the window.  SambaSafety, Powered by Vigillo is ready for it.  Make sure you are too.

Preview Scorecards available soon, and early adopter discounts won’t last long.  Give us a shout and we’ll get you set up with visibility into the next generation of CSA.  Act now, its still a two year look-back, so you are already accruing violations that the new system will compute into your new score.

Protect Your Business Now!

Click Here

FMCSA submits CSA overhaul plan to Congress

FMCSA submits CSA overhaul plan to Congress

Here we go!!!

As reported this morning by our friends at CCJ: FMCSA submits CSA overhaul plan to Congress…read more.

Almost three years since the FAST Act demanded CSA Reforms, looks like FMCSA is rolling it out.

Major changes pending to CSA methodology; Vigillo, a SambaSafety Company has been working in parallel to ensure we are ready this year to help customers interpret and act on the new scores and move Beyond Compliance with the right combination of data and Wisdom Deployed.

Are you ready for this?

Existing Methodology Example:

 

New methodology Example:

 

Happy Anniversary FMCSA – Meet The Parents

Happy February everybody, and Happy Anniversary!

Its been exactly 6 months (August 1 – Feb 1) since FMCSA launched their 24 month Crash Preventability Demonstration Program to see what impact it would have if FMCSA reviewed DOT Record-able crashes and made Preventable/Non-Preventable determinations for purposes of CSA crash scores. So happy 6-month (26 week) anniversary!

According to Match.com findings a 6-month relationship is important because this is when three key relationship milestones take place:

  • Revealing of one’s imperfections (24 weeks)
  • First argument (23 weeks)
  • Parental introductions take place (24 weeks)

So I ran some numbers FMCSA, its time to meet Momma.

Imperfection #1 – Permitted time period out of sync with CSA Scores

Lets start at the highest level.  Motor Carrier (CSMS) CSA scoring is based on the most immediate past 2 years of data including crashes.  In the two years prior to January 1, 2018, FMCSA recorded 307,732 crashes.  (MCMIS as of December snapshot).

All Crashes – Two Years

But FMCSA only allows crashes after June 1, 2017 to be submitted for reviews under its new program that started August 1, 2017.  So less than 25% of CSA crashes can be submitted at this time, it will only catch up with the 2 year CSA window, as the Demonstration Program comes to a close at the end of 24 months.

Crashes Qualifying for Review – 76,122 (24.7% of all Crashes)

Crash Locations – States (Oh look, Texas wins again!)

So as we sit here at the 6-month milestone, we can only judge the preventability project on the 1,149 submitted crashes out of 76,122 crashes that could possibly be submitted under the limitations put in place for this program, or 24.7% of all possible crashes. I never understood the reasoning behind limiting crash submissions to those since June, when FMCSA uses crashes back two years for scoring purposes. Seems fair to align those dates, but hey, we were early in the relationship and we didn’t want to have that first argument.

Imperfection #2 –  Transparency could be better

On August 1, as soon as the Demonstration Program launched, Vigillo started capturing all crashes submitted through DataQ’s.  FMCSA publishes the outcomes of crash reviews publicly, but cycles them through every 30 days, removing the previous month from public view.  There is no running total, so we created one.  We set up a daily capture and can now report that 1,149 crashes have been submitted for review by 588 different motor carriers since August 1.  That’s just 1.5% of the possible 76,122 that have been recorded in that same time period. This is starting to look an awful lot like the DataQ’s system we already dated and broke up with.  Lets get some reporting in there team, this is a demonstration program, lets see the results without having to do backflips. Motor Carriers, lets get busy, submit those crashes.

Imperfection #3 – Only certain predetermined categories can be submitted

FMCSA predetermined that only crashes in the following categories can be submitted for review:

  • When the commercial motor vehicle (CMV) was struck by a motorist driving under the influence (or related offense)
  • When the CMV was struck by a motorist driving the wrong direction
  • When the CMV was struck in the rear
  • When the CMV was struck while it was legally stopped or parked, including when the vehicle is unattended
  • When the CMV struck an individual committing, or attempting to commit, suicide by stepping or driving in front of the CMV
  • When the CMV sustained disabling damage after striking an animal in the roadway
  • When the crash was the result of an infrastructure failure, or falling trees, rocks, or debris
  • When the CMV was struck by cargo or equipment from another vehicle

And this is the breakdown now 6 months into the Program:

Actual Submissions by Crash category – 1,149 submitted

Actual Review Outcomes by Crash category – 596 Non-Preventable (52%)

Our first argument – Many clearly Non-Preventable Crashes are not being reviewed.

So to recap, Crashes going back two years should be permitted for submission, we need better and more transparent reporting from FMCSA, and with the huge investment in technology, including camera’s, there should not be predetermined categories of acceptable crashes.  It is impossible for anyone outside FMCSA to know how many crashes are being submitted and rejected out of hand because they are not in one of the pre-determined categories, and even less knowable, how many carriers don’t even try.

The link below is from a motor carrier who will remain anonymous, and it shows a crash that is clearly not preventable by the driver or motor carrier.  But it is a side collision, and FMCSA has predetermined that no side collision can be reviewed.  It is a Demonstration Program, lets demonstrate what is possible today with technology and video.  Until you do, I’m not sure how I feel about continuing our relationship, or having you meet my parents.

 

Steve

 

UPDATE: https://www.overdriveonline.com/fmcsa-clarifies-how-to-submit-crash-preventability-review-requests-for-pilot-program/?utm_source=daily_utm_medium=email_utm_content=02-07-2018_utm_campaign=Overdrive_ust_id=b5c8550d89cbb74bab233e6e8dc92bec

Motor Carrier Survey – for MC&E

Hello everyone, Our biggest annual event is upon us.  The American Trucking Assn’s Management Conference and Exhibition (MCE) starts tomorrow in Orlando, FL.  On Sunday morning, I will be participating in a panel discussion of the changes to CSA that have been suggested by the National Academies in the CSA Reform study they completed in June.

One of the recommendations made is to implement a new statistical model into CSA called Item Response Theory (IRT).  In my session Sunday morning at MC&E at 9:45 AM in room W305AB (shameless plug, please attend if you’re here), I will be using the results of a silly surey to explain fundamentally how IRT works.  I could use your help, I need some data.  Please click the survey link below and answer the 10 survey questions.  Make up a nickname, its totally anonymous.  I’ll use the aggregate responses, along with some specific right/wrong responses and show how IRT analyzes things like survey questions and map that to how IRT might do a better job of analyzing vioilation data in CSA.

www.pollev.com/vigillo

Appreciate your help, see you Sunday morning in Orlando.

 

Steve

CSA Reform Status – Vigillo Comments

Truck-Driving-with-graphs3

PDF Version Here

 IMPROVING MOTOR CARRIER SAFETY MEASUREMENT

Panel on the Review of the Compliance, Safety, and Accountability (CSA) Program of the Federal Motor Carrier Safety Administration

Initial Comments by Steve Bryan, President & Founder of Vigillo, A SambaSafety Company

June 28, 2017

Yesterday, June 27, 2017, 18 months after the FAST Act removed CSA Scores from public view, the National Academies of Science, Engineering and Medicine (NAS) published their preliminary findings outlining their recommendations to reform the Compliance, Safety and Accountability program we all know and love as CSA.

Their 132 page report starts with a nod to the FMCSA for the work they have done on CSA and for the funding for this study (amount undisclosed).  NAS starts by thanking Joseph DeLorenzo, director, FMCSA Office of Enforcement and Compliance, for his presentations, which were not just useful, but “enormously useful”, way to go Joe!

That was followed by acknowledgements to the other twenty-five presenters the first of which was yours truly (it was alphabetical). We are all thanked but our contributions are not characterized on a usefulness scale.  Item Response Theory (IRT) applied to the relative informational value of each of us would have been nice, but hey, more on that later.

NAS categorizes their report into the following four sections:

  1. Assessment of the Current Safety Measurement System
  2. A More Natural Statistical Model
  3. Data Improvements
  4. Transparency, Reproducibility, and Public Disclosure of Safety Rankings

In one paragraph, I’ll summarize. My take on this entire 132 page report is stated as follows.

“Good job FMCSA on what you have done with CSA so far, we (NAS) give you a solid 5 out of 10 for your efforts so far.  But there needs to be a sound statistical model applied to replace the ad hoc nature of how CSA is formulated today. More, and more reliable data needs to be gathered, and the data utilized needs to be more transparent and software developed to make such data more useful by all stakeholders.  Only then should CSA Scores be released to the public.”

(more…)

National Academies CSA Reform Report to be Released Tuesday

NAS Building

 

Happy Friday everybody,

It seems to be pretty rock solid now that the CSA Reform Report, 18 months in the making, will be released to the press on Monday, and the Public on Tuesday, June 27.

I’ll be reading the report as soon as I can get a hold of it, I have a team at Vigillo ready to dive into it, and we’ll be issuing comments by Wednesday, perhaps sooner.  Watch this blog for those comments.  Where possible, we’ll be running the numbers in addition to issuing opinions.

I posted a status report last December HERE and outlined the 8 items that the FAST Act cited as areas that need review.  Those 8 areas of reform are:

Reform Requirements

1. The accuracy with which the BASIC’s identify high risk carriers and predict or are correlated with future crash risk, crash severity, or other safety indicators for motor carriers. (Safety Event Groups)

2. The accuracy of safety data, including the use of crash data from crashes in which a motor carrier was free from fault. (Crash Accountability)

3. Whether BASIC percentiles for carriers of passengers should be calculated separately from motor carriers of freight. (Freight vs Passenger systems)

4. The differences in the rates at which violations are reported to the FMCSA by various enforcement authorities, including States, territories, and Federal inspectors (State Difference)

5. How members of the public use the SMS and what effect making the SMS information public has had on reducing crashes

6. Whether the SMS provides comparable precision and confidence, through SMS alerts and percentiles, for the relative crash risk of individual large and small motor carriers;  (Small Carriers largely unscored)

7. Whether alternatives to the SMS would identify high risk carriers more accurately

8. FMCSA has fully implemented or satisfactorily addressed the issues raised in the report titled ‘‘Modifying the Compliance, Safety, Accountability Program Would Improve the Ability to Identify High Risk Carriers’’ of the GAO dated February 2014.

As I conduct my review, these are the areas I’ll be looking for action.

Have a good weekend, I’ll talk to you all early next week

Steve

The Ties That Bind – Roadcheck 2017

You walk cool, but darlin’, can you walk the line…And face the ties that bind
Bruce Springsteen – The Ties That Bind (1980)

Roadcheck 2017 with its focus on Cargo Securement will run June 6-8, 2017.  Several weeks after that, we’ll have the data to report on the results, but I thought it might be interesting to give a little sneak peek into the current state of violation trends and into Cargo Securement violations specifically.

The following chart shows the historical trend of all Commercial Motor Vehicle Inspections and Violations since CSA (Compliance, Safety, and Accountability) launched, starting in January 2010.  The spikes that occur early each June (orange) represent the rather dramatic increase in Inspections that occur during the inspection blitz that CVSA calls Roadcheck.  Commercial Vehicle Enforcement puts in overtime for three intensive days of large truck inspections during the annual Roadcheck weekend. The accompanying spike in violations (blue) are of course the result of this heightened enforcement effort. Lesser known, but just as predictable is the downward spike in Inspections and Violations that occur in the middle of each December.  CVSA does not officially name this program, we call it Christmas.

CHART 1 Historical Inspection and Violation Trends by Week

Inspections-Violation Trends - all

(more…)

CSA Reform Progress Report – Scores Offline For One Year

CSA Reform Status

 Steve Bryan, Vigillo CEO/Founder gives comments to National Academies of Sciences in Washington DC last June

Steve at NAS

Background

On December 4th, 2015, President Obama signed the FAST Act (Highway Bill) that secured funding for infrastructure, removed CSA Scores from public view, and defined a reform process for the CSA Program that would lead to the return of the Scores once proper reforms were made and Congress authorized their re-publication to the public.

Reform Summary

The FAST Act required the FMCSA to retain the services of the National Academies of Sciences – Engineering – Medicine. (NAS) An 18 month process was outlined where the NAS would address the flaws of CSA and issue recommendations as to how they might be addressed.  I have previously summarized those flaws from the FAST Act as follows:

1. The accuracy with which the BASIC’s identify high risk carriers and predict or are correlated with future crash risk, crash severity, or other safety indicators for motor carriers. (Safety Event Groups)

2. The accuracy of safety data, including the use of crash data from crashes in which a motor carrier was free from fault. (Crash Accountability)

3. Whether BASIC percentiles for carriers of passengers should be calculated separately from motor carriers of freight. (Freight vs Passenger systems)

4. The differences in the rates at which violations are reported to the FMCSA by various enforcement authorities, including States, territories, and Federal inspectors (State Difference)

5. How members of the public use the SMS and what effect making the SMS information public has had on reducing crashes

6. Whether the SMS provides comparable precision and confidence, through SMS alerts and percentiles, for the relative crash risk of individual large and small motor carriers;  (Small Carriers largely unscored)

7. Whether alternatives to the SMS would identify high risk carriers more accurately

8. FMCSA has fully implemented or satisfactorily addressed the issues raised in the report titled ‘‘Modifying the Compliance, Safety, Accountability Program Would Improve the Ability to Identify High Risk Carriers’’ of the GAO dated February 2014.

(more…)

FMCSA UPDATE: WEDNESDAY 12/21/2016

Hi everyone,

We have not updated you about the FMCSA upgrades in a  little while but we’ve been back to Washington DC and met with the FMCSA since last time and have some news:

The FMCSA upgrade is still not complete but thankfully the FMCSA are now posting news about the upgrade status, you can see those below. At Vigillo what we are seeing is that some DOT’s still cannot log in to FMCSA Portal. The FMCSA are very aware of this and we contact them multiple times per day and have escalated this to the very highest levels.

SMS 

The SMS BASIC Percentile scores for this month were calculated with inspection, violation and crash data only up through November 16th, not up to the SMS Snapshot date of November 25th. The data for the latter half of November (Nov 17th – Nov 25th) will be included in next month’s SMS update. This may cause you some anomalies in your BASIC Percentile scores this month as only half the usual amount of data is being used. Still, the CSA Methodology is a relative scoring system so much of this should mostly balance out but we do see some significant Percentile changes for the month.

We are seeing that some States are still not reporting inspection, violation or crash data, or are not reporting it completely. We don’t yet know if this data will be backdated at a later time but we will keep you updated on this.

Here is what the FMCSA are reporting:

  • Display of the Safety Measurement System (SMS) Behavior Analysis and Safety Improvement Categories (BASICs) is currently disabled within the Portal. To view your SMS data, log into the CSA SMS Website.
  • Not all State inspections/crashes are uploaded at this time. FMCSA is working/troubleshooting with some individual States to complete the inspection/crash upload.
  • Individual Carrier Safety Profile generation is not working for some carriers. FMCSA is working to restore this functionality to all carriers

QC Mobile API

The QC Mobile API used to programmatically retrieve L&I information is back online although it does have some intermittent problems and was down for much of last weekend. It is working correctly at this moment. The FMCSA are working on this.

The FMCSA have reported to us that they expect to have all problems relating to the Compass Portal upgrade addressed by the time of the next SMS monthly update which is on the week of January 9th 2017.

We will keep you updated as we find out more and Happy Holidays from Vigillo.

FMCSA UPDATE: FRIDAY 12/16/2016

Today we have some good news about the L&I API:

QC Mobile L&I API

This API is the main and only way to bulk or programmatically retrieve L&I information from the FMCSA. Many DOT’s and software companies use this API to populate L&I information. Unfortunately this API has been unavailable since the FMCSA’s move to Amazon’s servers (called AWS) since November 16th 2016:

GOOD NEWS: This API is back working today 12/16

We don’t know how reliable it will be but here at Vigillo we have turned our access back on and L&I information is now flowing into our Carrier Select program once more.

We will keep you updated on this.

FMCSA Compass Portal Access

While many DOT’s now have access to their FMCSA Portal account we have significant reports of accounts still locked out, accounts are “stuck in a loop”.  We know from discussions with the FMCSA that they are highly aware of this and are working on it as a priority.

Vigillo’s Inspection Management System compares FMCSA inspections with company provided inspections. Normally these match up to a high degree. However, we are recently seeing higher rates of mismatch. This appears to indicate that the inspection, violations and crash data in FMCSA Portal may not be complete. We will continue to investigate this and keep you updated.

SMS Monthly Update

The SMS snapshot date is November 25, 2016 but there isn’t any inspection/violation or crash data later than November 16th currently in SMS. We have investigated this and preliminary results show that the FMCSA appear to have used only this data to compute this month’s BASIC Percentile scores. We should have a complete answer on this  very soon and will keep you updated.

As previously mentioned  the FMCSA has added 10 new Acute/Critical violations this month.

 

FMCSA UPDATE: WEDNESDAY 12/14/2016

The FMCSA’s Compass Portal migration to Amazon is still experiencing challenges today. Here is the latest update:

FMCSA Compass Portal Access

While Vigillo can now access daily data from portal for many customers there is still widespread Portal Account access issues. The FMCSA are aware of this (we just spoke with them this morning) and have escalated this issue. If you are having problems accessing your portal account then this should get addressed by the FMCSA hopefully soon (no need to call as they won’t submit a ticket since they already know about this).

Our Support team are working directly with customers on this issue. If you have any questions please call us at 503-688-5100 x102 or email us at support@vigillo.com

SMS Monthly Update

While the latest SMS snapshot date is November 25, 2016 there doesn’t appear to be any inspection/violation or crash date later than November 16th in SMS. This begs the question; what data was used by the FMCSA to compute this month’s CSA Percentile Scores? Is it all the data up to the snapshot of Nov 25th, or is it just half the regular data displayed up to Nov 16th. Here is what the FMCSA says:

“The CSA Safety Measurement System (SMS) Website has been updated with the November 25, 2016 snapshot. Please note: inspection and crash data is current as of November 16, 2016. The next snapshot will include inspection and crash data for November 17-25. Motor carriers, check your own safety assessments now.

We are working on identifying this issue.

Additionally, the two Inspections and Violations files available for download from SMS that contain the publicly available inspection and violation data used for the current month’s SMS calculations also truncates on November 16th – the data only goes up to 11/16/2016 (instead of the SMS Snapshot date of 11/25/2016). The rest of this data will be posted in next month’s SMS update. Here is what the FMCSA says about this:

Inspection and crash data is current as of November 16, 2016. The next snapshot will include inspection and crash data for
November 17-25. Motor carriers

 

Changes to CSA This Month.

A change this month is that the FMCSA has added 10 new Acute/Critical violations.

QC Mobile L&I API

This API is still down so no access to L&I information is available in bulk or programmatically, it is one DOT at a time on the L&I website at:

http://li-public.fmcsa.dot.gov/LIVIEW/pkg_carrquery.prc_carrlist

The FMCSA report this as a difficult problem so we have no ETA on when this will get resolved.

Upcoming FMCSA Changes

The FMCSA are going to remove DOT and PIN login to SMS. No more PINS! The only access to SMS in future will be through Compass Portal. This means all motor carriers will need a Portal account. The timeframe from the FMCSA on this is “in the coming months”, here is how the FMCSA phrases it:

“In the coming months, access by U.S. DOT Number and U.S. DOT Number PIN will be disabled on SMS and motor carriers will need to log in using the FMCSA Portal”

This is the latest information we have. The FMCSA are aware of all of these issues and are working to correct them.

We will keep you updated as we learn more

FMCSA UPDATE: FRIDAY 12/9/2016

Happy Friday!

More positive FMCSA news today in that we can successfully access most DOT’s inspections, violations and crashes from the FMCSA Compass Portal now on a daily basis and provide this information in Vigillo’s scorecards

The most recent Post Date is Thursday 12/8/2016 which is the most up the date information available which is good

We still have a few DOT’s with no access and we are working on those. If you have problems accessing the FMCSA Compass Portal, try Chat with them here (this is usually responsive):

  • https://ask.fmcsa.dot.gov/app/chat/chat_launch

If you are running Vigillo’s VDU, and you most likely are if you have the Vigillo Daily Update feature, there are a few customers who still need to upgrade to the new version released on 12/1/2016. We will be contacting you about this but please go ahead and update if you can (there should be pop-up that appears on your screen asking to update)

The BASIC Percentile scores on the Compass Portal home are still incorrect (they are shifted) so look to SMS for the correct scores:

  • https://ai.fmcsa.dot.gov/SMS

The QC Mobile API used to gather L&I information is still down.

FMCSA UPDATE: THURSDAY 12/8/2016

Positive news today in that we are having good success accessing FMCSA Compass Portal inspections, violations and crashes.

We are seeing new data now with the latest Post Date of Wednesday 12/7/2016 which is the most recent data.

We are noticing quite a lot of Clean Inspections and we are not yet sure these are all correct. In the past, during Compass Portal data outages, we have first seen inspections recovered as Clean inspections initially but then later become populated with their true violations so please keep an eye out for that.

We are still having problems with accessing some DOT accounts and we still have reports of customers with problems accessing Compass Portal although it appears to be improving

Contacting the FMCSA: If you are still having problems logging into Compass Portal the most immediate way to get help from the FMCSA is through their Chat system at:

  • https://ask.fmcsa.dot.gov/app/chat/chat_launch

Here are some other direct links to FMCSA Support:

  • https://ask.fmcsa.dot.gov/app/home
  • https://ask.fmcsa.dot.gov/app/ask
  • https://www.fmcsa.dot.gov/mission/field-offices

FMCSA Contact Center support number:

  • 1-800-832-5660

The remaining issues from yesterday such as Portal CSA Scores, L&I API etc. are not addressed yet although the FMCSA are actively working on these issues and we will keep you updated