SPROUT

How SPROUT is built

Four parts: a vision model trained on a GPU, a search engine over 237 extension guides, a question engine that keeps every number in code, and a small web stack that serves it all.

Right disease in the top three, from the photo alone
96%
Photo to diagnosis, median
0.4 s
Tap to next question, median
6.5 ms
Extension guides searched
237

The system

A static site, one Python server, and an AI model that only writes text.

Browserstatic site,Cloudflare Workers Flask server Image model, ONNX on CPU Guide search Odds engine and stop rule Background text workers Spend caps: visitor, IP, day AI model APIwrites questions,advice, replies Offline trainingPyTorch on a CUDA GPU photo, answers, chat odds, questions, replies guide passages JSON, checked by code model file + calibration
The browser keeps nothing but the id of the current check. The server owns the odds, so a refresh or a second tab never changes the result.

The vision model

ConvNeXt V2 Tiny, fine-tuned on 19,260 photos: 12,918 from the lab and 6,342 taken in fields and gardens.

Photos12,918 lab6,342 field5 public datasets Clean splitsnear-duplicatesgrouped by hash,never split apart Fine-tunePyTorch + timmCUDA, RTX 4070 GPUmixed precision Calibratetemperature scaling93% bar to answer,stricter for healthy ExportONNX, 29 classesONNX Runtimeon a plain CPU MLflow logs every run, DVC versions the data, and every number on this site traces to a saved run record.
Trained on lab photos alone, the model scored 0.24 macro-F1 on real field photos. Training on lab and field photos together took it to 0.81, ahead of a DINOv2 baseline at 0.72.

Every question, piece of advice and chat reply starts from passages in 237 university extension guides.

237 guidesPDF and web pagesfrom universityextension services Chunk and tagpassages taggedwith crop anddisease Two indexesBM25 keywordsBGE-smallembeddings Filtered searchonly this plantand these suspects,results merged RerankMiniLM cross-encoder picks thetop passages
The reranker is what made the difference: the right passage lands in the top five for 83% of test questions, against 72% for embedding search alone.

The question engine

The AI model writes the words. Code does the counting, so the same answers always give the same result.

Background, while you read Question writer: 6 per plant group Advice writer: top two suspects Photo odds29 conditions Two plants?ask the plant first Pick a questionmost informative You answeryes, no, not sure Update oddsx3 or x0.35 93% sure, or 5 questions?yes: diagnosis with the advice ready no: next questionin 6.5 ms
Worked example: early blight at 54%, a yes multiplies it by 3 to 162 against 46 for everything else, so 162 ÷ 208 = 78%. Questions are scored by how much they would narrow the odds; the best one is asked first.

Built with

Python end to end, from data checks to the server; plain HTML, CSS and JavaScript for the site.

Training

  • PyTorch and timm, ConvNeXt V2 Tiny pretrained on ImageNet-22k
  • AdamW, batch size 32, 224 px inputs, fresh lab and field mix every epoch
  • CUDA mixed precision on an RTX 4070 laptop GPU

Data

  • Five public datasets: PlantVillage, PlantDoc, PlantSeg, PlantWild, Plant Pathology 2021
  • Perceptual hashing and DINOv2 similarity to keep near-duplicates out of the test set
  • DVC for versioned datasets, pandas manifests for every split

Evaluation

  • Macro-F1, top-3 accuracy, bootstrap confidence intervals, McNemar’s test
  • Temperature scaling and expected calibration error
  • MLflow for every run; each published number traced to a run record

Serving

  • ONNX export, served by ONNX Runtime on CPU
  • Flask API with a background thread pool and per-visitor spend caps
  • Photos read with Pillow, turned upright, never stored

Search and text

  • pypdf extraction, passages tagged by crop and disease
  • BM25 plus BGE-small embeddings via fastembed, MiniLM cross-encoder reranker
  • An AI model API for questions, advice and chat, checked by code

Quality

  • About 950 automated tests with pytest
  • Ruff linting and GitHub Actions on every push
  • This site: hand-written HTML and CSS, inline SVG diagrams, no framework
Take the walkthrough Live demo