What I Learned Stress-Testing a Local Translator

While prototyping local translation for this portfolio, I prepared a deliberately long and repetitive text to keep the model busy for longer. I wanted to learn whether the entire flow—reading Markdown, translating through Ollama, rebuilding the document, and writing the result—could finish without running out of memory or being interrupted.

The probe served that exploratory purpose, but calling it a “GPU stress test” promised far more than the evidence could support. This is the analysis I should have published in the first place.

What the prototype ran

The translator used an 8B Llama 3 model served by Ollama. For technical articles, it traversed Markdown nodes, protected images with placeholders, and sent each fragment to the local endpoint. The historical request included these options:

Javascript

{
  model: "llama3",
  options: {
    num_ctx: 4096,
    num_gpu: 99,
    temperature: 0.1,
    top_p: 0.3
  },
  stream: false
}

num_gpu: 99 requested that Ollama offload as many layers as possible to the GPU, but the setting alone did not prove where each layer ran. Supporting that claim would have required recording Ollama’s response and observing CPU, GPU, RAM, and VRAM throughout every run.

What the probe actually established

It produced a small but useful body of evidence:

the pipeline could process a document longer than the usual portfolio notes;
the ten-minute timeout did not fire during that run;
image placeholders survived the round trip;
the English output was written without the process terminating unexpectedly.

This tested the resilience of the translation pipeline, not GPU performance. I did not record the exact model version, quantization, input and output token counts, warm-up time, memory use, temperature, power, or multiple comparable runs. Without those measurements, publishing performance figures or attributing the result to a particular piece of hardware would be misleading.

The most important failure was not performance

The test corpus contained artificial repetition and broad claims about artificial intelligence. The translator completed the job, yet produced unnatural constructions and preserved hardware conclusions unsupported by telemetry. A technically complete output could still be editorially unacceptable.

That distinction changed the success criterion. The question was no longer “Can it process the whole file?” but “Does it reduce the total amount of work without changing the meaning?” In this case, reviewing and rebuilding the output cost more than preparing a deliberate English adaptation.

How I would design a reproducible benchmark today

Dimension	Minimum evidence
Environment	CPU, GPU, RAM, operating system, and Ollama and driver versions.
Model	Name, digest, quantization, context size, and effective parameters.
Workload	Versioned corpus, token count, prompt, and file hash.
Timing	Separate warm-up, total duration, and tokens per second.
Resources	CPU/GPU load, RAM/VRAM use, temperature, and layer-offload errors.
Repetition	Several runs under the same conditions and a measure of dispersion.
Quality	Reviewed sample, error taxonomy, and comparison with a reference.

I would also report three measurements separately: model inference, Markdown transformation, and human review. Optimizing only the first can make the total workflow slower when the output introduces omissions or semantic changes.

Decision

The probe revealed a design limit; it did not demonstrate the capability of a GPU. I retired automatic translation and adopted an editorial workflow in which each English version is reviewed and approved independently. The architecture and reasoning behind that decision are documented in the Ollama translator retrospective.

The experiment remains worth documenting because of that outcome: a professional benchmark is not merely a large file that finishes. It is a bounded question, a recorded environment, and evidence another person can interpret or reproduce.