Computer Vision Model Testing Services
Independent evaluation for object detection, classification, segmentation, tracking, OCR, face systems and vision-language models across real deployment conditions.
What We Test
| System | Measurement |
|---|---|
| Object detection | Per-class precision, recall, localisation, FPPI and miss rate by size |
| Classification | Per-class accuracy, confusion and calibration |
| Segmentation | Per-class IoU, boundary accuracy and instance separation |
| Tracking & video | Identity switches, fragmentation and re-identification |
| OCR & document AI | Character and field accuracy by script, layout and capture |
| Face & person systems | Rates across cohorts, conditions and appearance factors |
| Vision-language models | Grounding, counting and hallucination |
| 3D & point cloud | Accuracy by range, density, weather and sensor |
Why mAP Is Not the Whole Story
An illustrative aggregate mAP of 0.91 can still contain an unusable night-rain, small-object or rare-class slice. All values are illustrative.
| Metric | Question answered | Interpretation |
|---|---|---|
| Per-class AP | Which classes work | Report instance counts |
| Precision / recall at operating threshold | Production trade-off | Avoid unused thresholds |
| FPPI | Operator burden | Absolute alerts matter |
| Miss rate by size / distance | Where detection falls off | Treat small objects separately |
| IoU distribution | Whether boxes are usable | A mean can hide a split |
| Per-slice breakdown | Condition and cohort failures | Where production failures live |
| Confidence calibration | Whether scores mean what logic assumes | Confident errors are costly |
| ID switches & fragmentation | Identity continuity | Tracking averages can hide failure |
Test Sets Built for Deployment
We control device, lighting, weather, geography, scale, occlusion, rarity and temporal drift. Image data collection can source missing evaluation imagery.
Distribution shift
- Capture shift
- Environmental shift
- Semantic drift
- Population shift
Face, Person and OCR Testing
Face and person systems are evaluated across cohort × condition cells. We use skin reflectance where it provides a more useful measurement than perceived skin tone. Formal fairness framing belongs in the AI Bias & Fairness Audit.
| Dimension | Coverage | Why it matters |
|---|---|---|
| Script | Latin and eight major Indian scripts | Conjuncts, diacritics and ligatures |
| Print quality | Digital to degraded or stamped-over | Copy loss has its own curve |
| Handwriting | Print, cursive and mixed | Field accuracy diverges |
| Layout | Tables, forms, rotation and skew | Structure errors differ |
| Capture | Scanner and phone imagery | Angle, glare, curl and shadow |
| Field extraction | Correct value in correct field | What workflows consume |
Vision-Language Model Evaluation
We test visual hallucination, grounding, spatial reasoning, counting, text-in-image reading, fine-grained distinction and uncertainty.
Ground Truth Is Part of the Test
Reference annotations use documented edge cases, adjudication and reported agreement. Image annotation creates training labels; testing measures fitness to ship.
How an Engagement Runs
- Scope and slice definition
- Test-set design and sourcing
- Reference annotation
- Evaluation
- Failure analysis
- Report and walkthrough
What You Receive
| Deliverable | Contents |
|---|---|
| Slice-level accuracy report | Metrics per class, condition, device, location and cohort |
| Failure atlas | Actual images grouped by cause and counted |
| Error taxonomy | Root causes ranked by frequency × cost |
| Operating-point analysis | Precision and recall at production thresholds |
| Versioned test set | Reusable imagery and reference annotations |
| Annotation agreement | Reference quality reported before findings |
| Class-definition findings | Ambiguities surfaced through review |
| Live walkthrough | Evidence reviewed with the vision team |
Where Vision Testing Goes Wrong
| Failure | Consequence | Control |
|---|---|---|
| Single mAP headline | A collapsing class disappears | Per-class and per-slice reporting |
| Training-like test set | Home-turf measurement | Deployment-led sourcing |
| Post-hoc slices | Flattering cuts | Pre-agreed slices |
| Noisy test annotation | Label error looks like model error | Reference-standard adjudication |
| Unused thresholds | Results do not describe production | Actual operating point |
| Small objects averaged in | Distance failures vanish | Size-band miss rate |
| Demographics alone | Condition effects stay hidden | Cohort × condition |
| Character accuracy only | Fields remain unusable | Workflow-level metrics |
| Fluent VLM descriptions | Hallucination scores well | Grounding and abstention |
| One-time evaluation | Drift goes unseen | Versioned re-testing |
What We Do Not Do
We do not tune the client's model, report one unexplained aggregate, treat unaudited labels as ground truth, or support unlawful biometric surveillance.
How to Engage
| Programme | Best fit | Window |
|---|---|---|
| Baseline vision evaluation | Full test-set and slice measurement | 6–7 weeks |
| Test-set construction only | Sourcing and reference annotation | 4–5 weeks |
| Release-cycle testing | Repeat against baseline | 1–2 weeks |
| Deployment-site audit | Focused drift investigation | 2–3 weeks |
Related Services
Image Annotation Video Annotation Document & OCR Annotation 3D Point Cloud & LiDAR Annotation AI Bias & Fairness Audit Data Collection
Computer Vision Model Testing FAQs
What is computer vision model testing?
Structured measurement of how a vision model performs on imagery that reflects its deployment environment — reported per class, per condition, per device and per cohort rather than as a single aggregate figure.
Why isn't mAP enough?
Because it is an average, and production vision failures concentrate rather than spread. We report per class and per slice, at the confidence threshold you actually deploy.
Our model scores well but fails in production. Why?
Usually because the test set resembles the training data rather than the deployment environment. We examine capture, environmental, semantic and population shift.
Can you build the test images, or do we have to supply them?
We can build them across regions, devices, lighting and conditions when deployment imagery is missing.
Do you test face recognition for bias?
We test cohort × condition cells and appearance factors, including skin reflectance where it provides a more useful measurement than perceived skin tone. Formal fairness and regulatory mapping moves to our bias and fairness audit service.
Do you test OCR in Indian scripts?
Yes — Devanagari, Bengali, Tamil, Telugu, Gujarati, Gurmukhi, Odia and Urdu, across print quality, handwriting, layout and capture method.
Can you evaluate vision-language models?
Yes. We cover visual hallucination, grounding, spatial reasoning, counting, reading text in images, fine-grained distinction and abstention.
How good does our ground truth need to be?
Better than training data: documented edge cases, adjudication and reported agreement are required for a defensible test.
How is this different from your annotation services?
Annotation produces labelled training data. Computer vision testing produces error rates and failure analysis used to decide whether a model is fit to ship.
How long does it take, and is the test set reusable?
A first programme typically runs 6–7 weeks, or 4–5 with suitable imagery. The versioned test set is delivered for reuse.