Active Learning for Computer Vision: Cut Labelling Effort

Active learning for computer vision is a training strategy in which the model chooses which images a human should label next, rather than labelling thousands up front. It directs scarce expert time at the few images where a label genuinely improves the model.

Active learning for computer vision is a training strategy in which the model itself chooses which images a human should label next, instead of engineers labelling thousands of images up front and hoping the model learns from them. For inspection teams in nuclear, energy and infrastructure, this matters because labelled data — not algorithms — is almost always the bottleneck. The images exist; what is scarce is the time of the chartered engineer or qualified inspector who can say, with authority, "that indication is a crack and that one is a weld toe." Active learning directs that scarce expert time at the handful of images where a label genuinely improves the model, and skips the thousands where it would teach the model nothing new.

This is the pillar guide to label-efficient computer vision for engineering teams. It explains what active learning is, why conventional data labelling fails for inspection problems, how the query loop works in practice, and what a realistic label-efficient workflow looks like from the first unlabelled image to a deployable model.

What is active learning?

Definition: Active learning is a machine-learning approach in which the model is trained iteratively and, at each iteration, selects the unlabelled examples it is most uncertain about (or that are most informative by some other measure) and asks a human to label only those. The newly labelled examples are added to the training set, the model is retrained, and the loop repeats until performance is good enough for the task.

The contrast is with what most teams do by default — sometimes called passive learning: collect a large set of images, pay for every one of them to be annotated, then train once. Passive learning treats every image as equally valuable. In real inspection datasets that assumption is badly wrong. Drone surveys, borescope runs and CCTV crawls produce enormous volumes of near-identical frames of healthy asset. A model learns almost nothing from the ten-thousandth image of intact concrete; it learns a great deal from the one frame showing an unusual corrosion morphology it has never seen. Active learning formalises that intuition and lets the model, not a random sample, decide where expert attention goes.

Why is labelling the bottleneck in engineering inspection?

Generic computer-vision projects can outsource image annotation cheaply, because labelling a cat or a car requires no special knowledge. Inspection imagery is different in three ways that compound each other.

First, the labelling itself requires engineering judgement. Distinguishing a rolling defect from a fatigue crack in a rail head, or crazing from structural cracking in concrete, is a competence question — bodies such as the British Institute of Non-Destructive Testing (BINDT) exist precisely because qualified visual inspection is a certified skill. You cannot crowd-source it, so every label costs senior engineering time that is already over-committed.

Second, access to the asset is expensive and rationed. Images from inside a reactor building, a live substation or a rail possession are gathered in narrow outage windows. The dataset you have is the dataset you get; you cannot simply "collect more data" the way a consumer-app team can. That makes it doubly important to extract maximum value from every image you already hold.

Third, the classes you care about are rare by design. A well-maintained asset should show mostly healthy surfaces, so defects may appear in a fraction of a percent of frames. Randomly sampling images to label mostly buys you redundant pictures of healthy asset, while the rare defect examples — the ones the model most needs — stay buried in the unlabelled pile.

Put together, the standard playbook of "label 10,000 images, then train" is not just slow for engineering teams; it is often economically impossible. Label-efficient methods are not an optimisation. They are the difference between a project happening and not happening.

How does the active-learning loop work?

A label-efficient workflow runs as a loop with four stages.

1. Start from a baseline, not from zero

Modern foundation models make it possible to get a working first-pass detector with no training labels at all. Open-vocabulary detection lets you describe what you are looking for in words — "corrosion", "spalling", "missing bolt" — and get bounding boxes or segmentation masks immediately, before any labelling has taken place. The baseline will not be perfect, but it does two valuable things: it gives stakeholders something concrete to react to on day one, and it gives the active-learning loop an informed starting point rather than a randomly initialised model. This zero-label starting point is the approach described in our guide to automated visual inspection.

2. Score every unlabelled image by informativeness

The model is run across the whole unlabelled pool and each image is scored by how much a label on it would be expected to help. The most common family of query strategies is uncertainty sampling: prioritise the images where the model's predictions are least confident — probabilities near the decision boundary, disagreement between augmented views, or high predictive entropy. Diversity-based strategies add a second criterion: of the uncertain images, pick a set that covers different conditions (lighting, viewpoint, asset type) rather than fifty near-duplicates of the same uncertain frame. Practical systems combine both.

3. A human labels only the selected images

The inspector is presented with a short, ranked queue — typically tens of images, not thousands — often with the model's provisional predictions pre-drawn so the task becomes "correct this outline" rather than "draw from scratch". Correcting a proposed mask is far faster than cold annotation, which is why label-efficient pipelines pair active learning with pre-labelling. The forms these annotations take, from bounding boxes to pixel-level segmentation, are covered in our field guide to image annotation.

4. Retrain and repeat

The new labels go into the training set, the model retrains, and the loop returns to step 2 with a sharper sense of what it still finds confusing. Each pass, the images the model asks about become genuinely harder edge cases — which is exactly where you want your expert's eyes. The loop stops when the model's performance on a held-out validation set meets the acceptance criteria you set at the start.

The mechanics of query strategies — entropy, margin sampling, ensemble disagreement — are treated in more depth in our explainer on active learning and uncertainty sampling.

How many labels does this actually save?

The honest answer is: it depends on how redundant your data is, and inspection data is usually very redundant. The academic literature on active learning has repeatedly shown that models can reach the accuracy of a fully supervised baseline with a fraction of the labels when the unlabelled pool contains many uninformative examples — the classic survey by Settles (2009, University of Wisconsin) collects results across domains showing large reductions in labelling burden from uncertainty-based querying. For inspection imagery, where healthy-asset frames dominate, the conditions for large savings are favourable: most of the labelling budget in a passive approach is spent re-confirming what the model already knows.

Two caveats keep this claim trustworthy. First, savings are problem-specific; a dataset that is already small and diverse benefits less than a large redundant one. Second, active learning changes which images get labelled, so your validation set must still be sampled independently — otherwise you are measuring performance on the model's own choices. Any supplier quoting a universal "90% fewer labels" figure without reference to your data should be pressed on both points. We will publish a worked example on a public weld-defect dataset later in this series showing a label-versus-accuracy curve end to end, so the shape of the trade-off is visible rather than asserted.

Uncertainty is a feature, not a by-product

There is a second, quieter benefit of building your workflow around model uncertainty: the same uncertainty estimates that drive the query strategy can drive deployment. In safety-critical sectors, no responsible engineer proposes removing humans from the inspection loop — and regulators would not accept it. The Office for Nuclear Regulation's assessment framework expects risks to be reduced as low as reasonably practicable (ONR), and any automated aid has to make the human review process better, not thinner.

An uncertainty-aware model supports that directly. At inference time, predictions above a confidence threshold you control can be accepted into reporting automatically, while low-confidence cases are routed to a qualified inspector for review. The result is not "AI replaces the inspector" but "the inspector's time concentrates on the frames that need judgement" — the same principle that governs the labelling loop, applied to operations. Calibration, thresholds and risk-based review deserve their own discussion, which this series covers in a dedicated post on uncertainty quantification in computer vision.

What does a label-efficient project look like in practice?

For a mechanical or civil engineering team, a realistic first project runs like this. Week one: upload existing inspection imagery — drone survey frames, outage photographs, CCTV stills — and stand up an open-vocabulary baseline by describing the defect classes in plain language. No labelling has happened yet, and you already have a reviewable first-pass result. Weeks two and three: run short active-learning cycles. Each cycle, the system queues a few dozen images; a competent inspector spends an hour or two correcting proposed labels; the model retrains. Week four: evaluate against an independently sampled validation set, set confidence thresholds for automatic acceptance versus human review, and decide whether performance justifies a pilot on live inspection data.

The through-line is that expert time is spent only where it changes the outcome: correcting the model's hardest cases and adjudicating its least confident predictions. Everything routine is absorbed by the model. Compare that with the conventional route — a months-long annotation contract before anyone sees a result — and the appeal to engineering teams with scarce labelled data, expensive expert labelling and safety-critical confidence requirements is clear. These are precisely the three problems VisionEngine is built around: an instant open-vocabulary baseline with zero training data, active learning that asks for the fewest and most informative labels, and uncertainty quantification that routes only low-confidence cases to a human.