smaple.tr
AI & Data

Computer Vision Applications: Where It Works and Where It Struggles

Mehmet Kurtipek
August 9, 2026
13 min read
AI & Data
Computer vision
Object detection
OCR
Edge AI

Computer vision applications are systems that turn images or video into a decision or a record: a scratched part pushed off a production line, a number plate logged at a gate, the total read off an invoice, a suspected bleed flagged for a radiologist, a weed sprayed while the crop next to it is left alone. The ones that work reliably have three things in common. The camera sees roughly the same scene every time, the question asked of each image is narrow, and a wrong answer is cheap to catch. When one of those is missing, expect a slower and more expensive project.

The projects we can describe first-hand are on the document side: an LLM workflow that reads oilfield field tickets, and extraction-quality checks for an AI bookkeeper that reads invoices and receipts. Everything else below comes from published trials, regulators and vendor documentation, linked where it is used.

What a vision model returns

Every application is built from a handful of outputs, and choosing one is the first design decision because it decides how you label data and what the model costs to run.

  • Classification gives one label per image: defective or fine, ripe or not.
  • Object detection draws and names a box around each object, so you can count and locate cars, people, parcels or weeds.
  • Segmentation marks the exact pixels of an object or region. You need it when area or shape matters: the outline of a tumour, the length of a crack, the free road in front of a car.
  • Keypoints and tracking find specific points (the joints of a body, the corners of a part) and follow objects from frame to frame, so a person walking through a door is counted once.
  • OCR and document understanding read text and layout and return fields: invoice number, date, line items.
  • Vision-language models (VLMs) answer free-form questions about an image ("is the safety guard closed?") and can return boxes and masks too.

A model that only says "defect or no defect" is cheaper to label for and to run than one that outlines every defect, so pick the smallest output that supports the decision.

Computer vision applications by industry

Manufacturing: inspection, counting and guidance

Quality inspection is the classic application. Cameras check that a label is present and straight, that a seal is complete, that a weld or solder joint looks right, that a roll of fabric has no holes or stains. A lot of this was solved decades ago with rule-based machine vision: fixed lighting, a fixed camera and measured thresholds on edges, colours and dimensions. Deep learning earns its place where defects vary too much for rules, such as scratches on textured or shiny surfaces, food and wood with natural variation, or patterned fabric.

Lighting is usually the first problem, because it decides what the camera can see at all: a scratch that only shows under low-angle light needs a low-angle light, and no model makes up for a shadow. The second is that real defects are rare, so you may have thousands of images of good parts and a few dozen of bad ones. The usual way round that is anomaly detection: train on good parts only, flag anything that looks different, and label the flagged images until there are enough to train a proper defect model.

Documents and back-office work

Reading documents is computer vision too, and it is often the first application a company tries: the documents already exist, and the manual work they replace is easy to measure. OCR turns pixels into characters; the hard part was always deciding which number is the invoice total when every supplier lays out the page differently. Templates broke whenever a supplier changed its layout, and LLMs and VLMs now read semi-structured pages well enough to replace most of them. One check before you build anything: if your invoices already arrive as structured e-invoices (XML), read the XML and skip the image entirely.

The field-ticket workflow we built with an agency partner for an oilfield services company uses that LLM approach. Tickets arrive as PDFs by Gmail. Every evening an n8n workflow picks up the day's tickets, an LLM extracts the customer, location, AFE/PO number, line items and hours as JSON, each PDF is filed with its JSON in a dated Google Drive folder, and the team gets a consolidated invoice report by email. We tested extraction on real tickets for a month before it went live in September 2025, and it has run every day since, at about 100 documents a day. Most of what broke in production was the LLM API hitting rate limits (HTTP 429). We added fallback routes to alternative models, so a ticket still gets processed when one model is rate-limited.

Extraction also needs a way to notice what it missed. For the AI bookkeeping assistant at Jenesys AI, which reads invoices and receipts and proposes the bookkeeping entry, our engineer built extraction-comparison checks that raise an alert when the category, VAT rate or tracking category comes back empty, so the gap no longer passes silently.

Healthcare and medical imaging

Medical imaging is where computer vision is most established as a regulated product. Of the 1,614 devices on the FDA's list of AI-enabled medical devices (content dated 4 September 2026), 1,230 have radiology as their lead review panel, by our count of the published table. The FDA says the list is not comprehensive, but the skew towards radiology is plain. Typical products outline organs for radiotherapy planning, flag suspected bleeds or fractures so those scans are read first, take measurements, or act as a second reader in screening programmes.

Two things set this area apart. The software is a medical device: in the US it needs FDA authorisation, and in the EU software that informs a diagnosis falls under the Medical Device Regulation. And adoption depends on workflow. A tool that scores well on a test set but adds clicks to a radiologist's reading routine, or doesn't talk to the hospital's image archive (PACS), tends to go unused.

Retail, warehouses and logistics

Shops use cameras to spot empty shelves, and self-checkouts use them to recognise loose fruit on the scale. Warehouses read labels and barcodes from any angle, measure parcels for shipping charges and photograph them at handover to settle damage claims, and picking robots rely on vision to grab items from bins.

The difficulty in retail is the long tail: thousands of products whose packaging changes every season, seen at odd angles on a crowded shelf. The systems that last usually narrow the problem (one category, one fixed camera above the scale) or keep a person in the loop for the cases the model is unsure about.

Traffic, cities and security

Automatic number-plate recognition (ANPR) is one of the most mature applications. Car parks, toll roads, gated sites and enforcement cameras have relied on it for years. Traffic counting and vehicle classification, pedestrian and cyclist counts, and driver-assistance features in cars are close behind. Driverless vehicles, most of which combine cameras with lidar and radar, remain the hardest case because unusual road situations never run out.

Face systems come in two kinds that the law treats very differently. Face verification (is this the person on the ID card?) is routine in bank onboarding and building access. Face identification (who is this, out of everyone?) is far more tightly regulated. In the EU, the AI Act has prohibited several practices since February 2025, including untargeted scraping of internet or CCTV images to build facial recognition databases, emotion recognition in workplaces and schools, and real-time remote biometric identification for law enforcement in public spaces, with narrow exceptions. Remote biometric identification, along with some other biometric uses that remain legal, is classed as high-risk, and those obligations apply from 2 December 2027, the date set by the AI Omnibus amendment that entered into force in July 2026. Under the GDPR, biometric data used to identify a person is already special-category data.

Agriculture

Drones and cameras mounted on sprayers detect weeds, disease and crop stress. The best-known product is John Deere's See & Spray, which sprays only where its cameras see a weed. An Iowa State University Extension trial on five soybean fields (415 acres) measured average herbicide savings of 76%. Individual fields ranged from 44% to 91%, and the authors tie the difference to weed pressure at spraying time: the cleaner the field, the bigger the saving.

Sports and media

Ball and player tracking feed broadcast graphics, officiating (line calls in tennis, semi-automated offside in football) and performance analysis, and media companies use vision to moderate content and search video archives. The officiating systems work from several fixed, calibrated cameras around the court or pitch.

Which applications are mature and which are still hard

Application

Maturity

What usually makes it hard

Number-plate recognition

Mature

Dirty or damaged plates, glare at night

Document extraction

Mature with LLMs and VLMs

Validating output, API limits and cost at volume

Inspection on a fixed line

Mature when lighting is controlled

Rare defects, product changes, lighting drift

Medical image triage and measurement

Mature but regulated

Authorisation, workflow and PACS integration

Shelf monitoring

Works in narrow scopes

Thousands of changing products, occlusion

Weed and crop detection

Proven on large farms

Field conditions, cost of the machinery

Driverless vehicles

Hardest

Endless edge cases, proving safety

For your own idea, four questions predict most of the outcome:

  1. Could a person make the call from the same image in a second or two? If an expert needs extra context, the model will need it too.
  2. Does the scene stay the same? Fixed camera, fixed lighting, fixed distance. Every variable you can pin down in hardware is one the model doesn't have to learn.
  3. What does each kind of mistake cost? A false reject wastes a good part; a missed defect reaches a customer. Agree on which one matters more before anyone quotes an accuracy figure, because that choice sets the threshold.
  4. Can you collect examples of the rare cases? If the event you want to catch happens twice a year, you can't train for it directly. Plan for anomaly detection or a human review step.

You may not need to train a model

A few years ago most of the applications above meant labelling thousands of images before anything worked. Four developments changed the starting point:

  1. General VLMs detect and segment from a text prompt. Google's Gemini API, for example, returns bounding boxes and segmentation masks as JSON when asked to find objects, so a prototype can work in an afternoon with no labelled data.
  2. Meta's SAM 3 (November 2025) segments every instance of a concept named in a short phrase ("forklift", "yellow helmet"), which makes it a good pre-labelling tool.
  3. Pretrained backbones such as Meta's DINOv3 (August 2025) make a small labelled set go further when you do fine-tune.
  4. Real-time detectors keep improving: Ultralytics released YOLO26 in January 2026, and Roboflow's transformer-based RF-DETR is a strong alternative.

Check licences before you choose. Ultralytics YOLO is AGPL-3.0, and Ultralytics' licence page says an Enterprise License is required to use it without open-sourcing your entire project, which covers closed-source products, SaaS and devices. RF-DETR's base models are Apache 2.0, though its largest variants use a different licence, and SAM 3 and DINOv3 come under Meta's own licences. Read the licence of the exact weights you ship.

Chat assistants such as ChatGPT, Gemini or Claude will happily describe a photo or read a scanned form, which is enough for many low-volume jobs. Per image, though, they are slower, costlier and less consistent than a small dedicated model. You still want your own trained model when you process thousands of images an hour and per-call prices add up, when the camera has no reliable connection, when the answer is needed in milliseconds, or when the same image must always get the same answer. A common path is to prototype with a VLM, use it (with human review) to pre-label your own images, and then train a small detector that runs cheaply next to the camera.

Our own lesson comes from the Jenesys work, where custom models we built were later overtaken by frontier models. If a general model is nearly good enough today, it will probably be better next year, so put your effort into the data, the validation checks and the review loop, which survive a model swap. And if a hosted model sits in the critical path, plan for the API failing: in the field-ticket workflow, rate limits were the main thing that broke.

How to scope a computer vision project

  1. Write down the decision ("reject the part", "send the invoice to review") and what a false alarm and a miss each cost.
  2. Collect images from the real camera in real conditions. Ultralytics' training guidance says images must be representative of the deployed environment: different times of day, lighting, seasons and cameras. One quiet afternoon of test shots doesn't cover that.
  3. Build a quick baseline with a VLM or a pretrained model. If it fails, find out why before buying hardware.
  4. If you train, label consistently. The same guide recommends at least 1,500 images and 10,000 labelled objects per class for best results, and every object in every image must be labelled. Treat those as a production target; a pilot can start much smaller, and the test in the next step tells you when you have enough. Our post on data labeling covers keeping labels consistent across people.
  5. Test on images from days the model never saw, and measure what the business cares about: missed defects and false rejects per thousand parts, or corrected fields per hundred documents. Benchmark scores like mAP help you compare models; they say little about whether a line is ready.
  6. Decide where it runs. Cloud inference is simplest. Running it on edge hardware next to the camera, such as an NVIDIA Jetson module, makes sense when bandwidth, latency or privacy rule out sending video away.
  7. Monitor after launch. A new product variant, a replaced lamp, a dirty lens or a new supplier template all shift the input. Log low-confidence cases, review a sample every week and retrain on what you find; our MLOps post covers the tooling.

If you have a document or image problem like these and want a second opinion on whether it needs a custom model at all, our data and AI team can take a look.

Related Articles

August 11, 2026

MLOps Guide: What to Set Up First, and What Can Wait

MLOps gets a model into production and keeps it useful there. Here is the order we'd set it up in, with current MLflow and DVC examples, and what changes for LLM features.

Read More: MLOps Guide: What to Set Up First, and What Can Wait
August 4, 2026

AI Agent Development: When You Need One and How to Build It

An AI agent is a language model working in a loop with tools. When you need one, the loop in Python, choosing a framework, keeping it safe, and what each run costs.

Read More: AI Agent Development: When You Need One and How to Build It
June 28, 2026

How to Build an MCP Server: Python Code, Testing, Hosts and Auth

A working Python MCP server for the 2026-07-28 spec, how to test it and connect it to Claude or ChatGPT, and the tool design and auth rules that matter once it goes remote.

Read More: How to Build an MCP Server: Python Code, Testing, Hosts and Auth