ANPR - Automatic Number Plate Recognition

Overview

The Problem

The incident management system had no way to identify vehicles automatically. When a security incident involved a car, there was no reliable way to capture who it belonged to, and parking areas had no automated way to check whether a vehicle was authorized to be there.

The goal was to build a recognition capability from scratch that could work across two very different camera sources (static CCTV and a moving robot) and deliver consistent results in real time, under real-world conditions.

I owned the entire journey: researching possible approaches, running experiments, evaluating models, and finally wrapping the AI engine in business logic and integrating it into the live system.

The Challenges

1. Unstable and low-quality footage

The robot's camera shakes while it moves, and many CCTV feeds are low resolution. A single frame often isn't clear enough to read a plate correctly, and in a security context a wrong reading can be worse than no reading at all.

My approach: I designed a reliability layer at the end of the pipeline that ensures the system delivers a trustworthy final reading, even when individual inputs are poor.

2. Many plate templates

Plates come in many different designs, layouts, and formats. A system that works perfectly on one template can fail completely on another, and new templates keep appearing over time.

My approach: I built a lightweight, low-latency post-processing layer that handles the variety of templates and is designed to scale, so supporting a new template is simple rather than a re-engineering effort.

3. Real-time performance on a shared edge device

The engine runs on an edge device alongside other heavy AI models, all competing for the same limited resources. It had to stay real-time without slowing down the rest of the system.

My approach: I optimized the models for the target hardware and carefully engineered the processing steps around them, keeping the full pipeline fast and resource-efficient.

4. No ready-made dataset

There was no existing dataset that matched our cameras, conditions, and plate types, and each evaluation stage needed reliable data of its own.

My approach: I collected the data by hand and built an AI-assisted annotation workflow, with human validation on top, to create dedicated, trustworthy datasets for every evaluation step.

Research & Process

I approached the project as a series of focused phases, each building on the last, so that every decision was based on evidence rather than assumptions.

1. Start with the core

I began with the most critical component of the pipeline and benchmarked several candidate approaches against each other, so the foundation of the system was chosen on measured results.

2. Build out the pipeline

With the core in place, I moved through the remaining components one by one, evaluating options at each stage to find the models best suited to our conditions and constraints.

3. Get the right data

Benchmarks only matter if the data reflects reality. I collected data from the same distribution the system would face in the field, matching the actual cameras, environments, and plate types.

4. Test on real cases and diagnose

I ran the pipeline on real scenarios and tracked exactly where performance dropped. For every failure, I identified the root cause: was it a limit of the model's capability, or of the logic around it? This distinction decided whether the fix belonged in the model or in the engineering.

5. Stress-test the edge cases

I kept testing across different scenarios and documented every edge case, turning each one into a known, handled situation rather than a surprise in production.

6. Optimize for the real world

Finally, I focused on latency, tuning the system to reach the best balance between accuracy and speed for real-time use on the edge device.

Results & Performance

A new capability, built from zero. The incident management system can now identify vehicles automatically, in real time, from both fixed CCTV cameras and mobile security robots.

Built where no ready solution existed. There was no off-the-shelf solution suited to the plate formats and conditions this system had to handle, so the full model stack and logic were designed and tuned specifically for them.

Better and faster with every iteration. The first version reached decent accuracy but was too slow for real-time use. Through better component selection and refined logic, the final version improved accuracy, especially on difficult edge cases, while significantly cutting latency on the edge device.

Real-time, multi-vehicle recognition. The engine handles several vehicles in the same scene simultaneously, in real time, while sharing the edge device with other heavy AI models.

Reliable under difficult conditions. The system delivers trustworthy readings even with shaking cameras and low-resolution footage.

Built to scale. New plate templates can be supported with minimal effort, without rebuilding the system.

Validated and production-ready. The engine passed multiple live demonstrations and is fully integrated with the incident system, ready for deployment.

Learnings & Next Steps

As my first production-level project, built entirely on my own, this engine taught me lessons that now shape how I approach every system I build.

Diagnose before fixing. When performance dropped, the most valuable step was finding the true source of the problem: the model's capability, or the logic around it. Getting that right meant every fix landed in the right place.

Accuracy is not enough. A model that performs well in isolation can still fail in production. Real value comes from balancing accuracy, latency, and limited resources on the device the system actually runs on.

Edge cases define production quality. The common cases work early. What turns a prototype into a reliable product is the patient work of finding, documenting, and handling the unusual ones.

Owning the full pipeline. Building everything from research to integration showed me how each decision affects the rest of the system, and how business logic matters as much as the models themselves.

What I'd do differently. I would invest more structure upfront in gathering requirements and in organizing the development and testing cycle. I'd also set the right goal from day one: not a perfect system, but a reliable one that keeps improving.

What's next. A system like this is never finished. There is always room to push accuracy higher, bring latency lower, and close the remaining edge cases, and the engine was built to support that kind of continuous improvement.