I sent my 4090 six receipts a second. The queue kept growing.

In the first minute, receipts waited an average of 1.56 seconds before a client worker started their request. By minute five, that wait had grown to 10.39 seconds. The queue filled and started refusing work.

I was sending six receipts a second to Nemotron Parse on my RTX 4090. The model takes a receipt image and returns its text. Sending five receipts a second kept the queue short for the same five minutes. One extra receipt per second was enough to expose the difference.

The lower rate still had failures. Twenty-six requests timed out. Every one was for the same receipt.

The test measured what happened when receipts kept arriving at a fixed rate. The client scheduled another receipt whether the earlier ones had finished or not. At six per second, a new request was due roughly every 167 milliseconds. A slower response could not quietly slow down the test's arrival rate.

I used 56 receipts from CORD-v2, rows 24–79 of its test split. One RTX 4090 ran Nemotron through vLLM, with the model and serving settings fixed across the trials. Images were prepared and encoded before timing began, so these measurements exclude document preparation.

The chart below measures delay before a client worker starts, including scheduling delay. A request can then wait again inside vLLM. Runner and settings.

For the capacity tests, the client cycled through the same 56 receipts after warming each image. These were repeated, cached inputs. Separate short trials sent each new image once after resetting the parser's cache. Those trials let me inspect responses to fresh inputs, but were too short to establish sustained capacity for new documents.

The initial sweep tested two, four, six, eight and twelve incoming receipts per second. Each rate ran three times, with a minute of arrivals followed by time for outstanding requests to finish or expire.

At six per second, those short trials already showed waiting and timeouts, but none rejected an arrival because the client queue was full. I then ran five-minute checks at five and six per second.

Average delay from scheduled arrival until a client worker starts, including scheduling delay. Full-queue rejections are excluded from the average and counted separately. One five-minute trial per rate, using warmed, repeated receipts. At six per second, the full queue rejected 13 arrivals in minute four and two in minute five. The five-per-second run still had 26 timeouts. Source: per-minute audit.

A normal completion here means nonempty output that ended with the model's stop signal. It says nothing about whether the receipt was transcribed correctly.

Six per second completed more work overall, but it also accumulated waiting work. By the fourth minute, the bounded queue had started refusing arrivals. The one-minute trials had understated that risk.

Five per second kept queueing low over the observed five minutes. That puts the observed queue-stability boundary between the two tested rates for this workload and configuration. Each longer condition ran once. Five per second avoided a growing backlog, but its timeouts prevent calling it a reliable service rate.

All 26 timeouts in the lower-rate run belonged to receipt 71. Separate fresh-input trials returned unrelated repeated characters and exhausted the output allowance for that same image.

Two different tests of the same image, not a reconstruction of the timed-out responses. The failure review records source and output hashes. Visual checks were performed by Codex, without independent human adjudication.

Image: NAVER CLOVA and Park et al., CORD (2019), CC BY 4.0. Open the original receipt image. The image is unchanged, including the source blurring.

In each of the first two fresh-input trials, this one receipt accounted for about 49% of all returned output tokens. That is a count of generated output, not a measurement of GPU time or evidence that this receipt caused the queue to fill at six per second.

The same document failed repeatedly even at the rate that kept queueing low. The longer run exposed two distinct problems: a backlog at six receipts per second, and recurring timeouts at five.

This post records the v0.2.0 scaling experiment. Its evidence, configuration and reproduction links are pinned to that experiment's commit. The current documentation follows main and can change as later experiments are added.

All 9,228 measured requests have a saved outcome, including rejections, timeouts and outputs that reached their length limit. Those requests reused 56 documents; they are not 9,228 independent examples of receipt quality.

The full results include the short saturation curve, resource counters and limitations. The reproduction guide explains how to rerun the experiment or rescore saved responses without running the model again. The earlier 24-receipt experiment examines the separate question of whether normally completed responses preserve the printed amounts.

Companion experiment: inference-lab v0.2.0 — the sustained receipt-load experiment, with saved request outcomes, queue measurements and offline scoring.