Vehicle Damage Detection with YOLOv8-seg: A Multimodal Web + Desktop + Mobile System

The Problem

Insurance companies and car dealerships face a recurring challenge: assessing vehicle damage quickly and reliably. Manual assessments are slow and subjective. This is where computer vision comes in.

My goal was to build a system that detects which parts of a vehicle are damaged at the segmentation level — and serves results across three clients: web, desktop, and mobile.

Why Segmentation Over Detection?

A standard bounding box tells you "there's damage in this area." Segmentation tells you the exact pixels. That's crucial when you need to isolate a specific panel like the front bumper or left door. YOLOv8-seg delivers both speed and pixel-level masks.

Architecture

Client (Web / Desktop / Mobile)
         ↓
    FastAPI Backend
         ↓
  YOLOv8-seg Model (Python)
         ↓
  Damage Region + Cost Estimate

Backend: FastAPI

FastAPI's async support and automatic Swagger docs made it the obvious choice. Images arrive via POST to /predict, the model runs inference, and results return as JSON with segment masks and confidence scores.

Web Client: Next.js 15

React Server Components for fast loading, TailwindCSS for the dark theme. Drag-and-drop image upload, with analysis results rendered as an overlay on the original image.

Desktop: Tauri 2

Far lighter than Electron. Rust backend, WebView frontend. The same Next.js codebase wrapped as a native app. Tauri's native API handles file system access for local image processing.

Mobile: React Native (Expo)

Camera integration for live photo capture. expo-camera sends images directly to the backend. Results displayed in a mobile-optimized interface with swipeable damage reports.

Challenges

1. Model size: YOLOv8s-seg is 22MB. Server-side inference was the right call for mobile clients.

2. CORS: Three different clients meant careful CORS configuration in FastAPI.

3. Tauri hot reload: Running Tauri + Next.js dev servers together required a custom startup script.

Results

Live demo on Vercel. YOLOv8-seg model achieved mAP50: 0.89 on the test set. Next step: automated cost estimation from detected damage regions.

---

View the project: GitHub | Live Demo