New technologies such as artificial intelligence (AI) present a real opportunity for the government and DVSA to work smarter, to improve services and make roads safer.
The Minister for Implementation, Oliver Dowden, has just launched the Government’s Technology Innovation Strategy. I thought I would share with you what we're doing here at DVSA.
AI at DVSA
We’re leading the way and are already using AI to identify and target MOT garages and testers that are not testing vehicles properly - endangering people’s lives. We’ve already picked up a prestigious UK IT award and have been nominated for another.
How does it work?
Every year 40 million vehicles receive a MOT. We’ve developed an approach that analyses this vast amount of testing data. It uses AI technologies combined with day-to-day operations so we can develop a continually evolving risk rating algorithm.
A range of features were engineered to produce a MOT tester profile, however, we lacked a precise view of what constituted a high-risk tester profile. Therefore, an unsupervised clustering approach was used to identify outliers in the data using the Local Outlier Factor (LOF) model, which is a density-based anomaly detection algorithm. This model grouped testers with similar profile scores and identified the outliers as potentially high-risk testers.
The result is that the algorithm:
- focuses on individual MOT tester’s performance for the first time, which stays with testers as they move between garages
- considers changes to testing behaviour over time
- refreshes regularly, enabling accurate, up-to-date risk scoring
- incorporates an expanse of features, such as test volumes, frequency, duration and pass rates, as well as disciplinary history
The score from this is combined with the outcome of any previous enforcement action to produce the risk rating.
From this we're able to direct our enforcement officers’ attention to garages or MOT testers who may be either underperforming or committing fraud.
We’ll then investigate these garages and MOT testers further to see if there’s evidence of wrongdoing.
It gets results
Since we introduced this approach to risk rating 6 months ago, we’ve already seen really positive results. The new system has found more cases than we would have expected to see using the previous way of investigating. The approach has highlighted some garages that we have immediately been able to stop testing, instantly making our roads safer.
A good example was a garage owner in Devon that the system flagged up as having issues with testing. On further investigation by our examiners, we found out he’d carried out more than 300 fraudulent MOTs. This led to the garage owner being prosecuted, given a suspended prison sentence and banned from testing.
Building capability
The DVSA digital team worked with our partners Kainos to deliver an approach using machine learning. This helped build our capability, and, once they’ve finished working with us, will leave us with the skills and ability to develop this model. We’ll then be able to apply to other areas of our business and support other parts of the Civil Service.
Keeping learning
The current risk rating algorithm focuses on characteristics that might indicate fraudulent or deliberate testing. We're now using the AI approach to further develop this to help identify where errors might be being made in the MOT.
This is really important to us, as the vast majority of garages are trying to do a good job. It’s great that we can drive out the bad eggs, but we need to ensure we can also support those who can improve.
So, the team are currently working on expanding the risk ratings algorithm to include a ‘predictive vehicle failure model’. This will fine-tune our approach and help our officers to better understand the likelihood of a vehicle passing or failing its MOT. We can then use this to target testers who repeatedly record results contrary to the prediction.
We're confident that our innovative use of AI is the right approach. It allows us to be flexible, and to develop approaches to risk rating that keep getting better. We're more able to respond to what we learn and are better at responding to any changes in tester behaviours.
But above all, it's helping us catch fraudulent MOT testers and helping everyone stay safe on Britain’s road.