Reading a technology claim: sources, measurements, and uncertainty

Announcements, measurements, and interpretations answer different questions

A developer's announcement says what they released. A measurement reports what happened with particular inputs and devices. Commentary explains the meaning its author draws from those results. All three may appear in one article; labelling the entire article evidence hides these distinctions.

The QuickConvert file converter and every timing in this guide are invented teaching examples. We did not benchmark a real product. The aim is a reusable reading method for a claim encountered in a news story or an AI answer.

Turn the headline into a testable sentence

Suppose the headline says “QuickConvert is now twice as fast.” It omits the version, file, and operation used for comparison. A testable version is: “On the same device, converting the same 10 MB CSV file to JSON took 400 ms instead of 800 ms in this example.” Say whether loading the file and downloading the result are excluded.

When an AI answer supplies a link, opening it is only the beginning. Does the source actually contain this operation, version, and comparison? Follow secondary citations back to the original measurement. If the underlying document cannot be found, record “original measurement unavailable” rather than “verified.”

• Claim: What behavior changed?
• Scope: Which version, device, and input?
• Evidence: Where are the measurements or reproducible method?

Twice as fast versus a 50% reduction in elapsed time

Reducing 800 ms to 400 ms gives (800 − 400) / 800 × 100 = 50% less time. The old-to-new time ratio is 800 / 400 = 2. For the same task, halving duration could ideally fit two tasks into the former interval. It does not establish that the whole application runs twice as fast everywhere.

Now assume unchanged file selection, reading, and output preparation take 1,200 ms. End-to-end time drops from 2,000 ms to 1,600 ms: (2,000 − 1,600) / 2,000 × 100 = 20%. Improving one stage does not improve the entire experience by the same percentage. Whenever a story quotes a percentage, first identify its denominator.

Do not substitute the best run for the whole distribution

Our fictional old timings are 780, 800, 810, 790, 820 ms; the new timings are 390, 400, 410, 395, 900 ms. Sorting them gives medians of 800 and 400 ms. Hiding the new series' 900 ms run conceals a delay a user might encounter. Five trials also cannot establish a reliable probability estimate for a broad population of users.

For a real browser experiment, record the same file, software version, and timing boundaries, and separate warm-up runs. MDN explains that performance.now() measures elapsed time using a monotonic clock. That addresses system-clock changes; it does not remove device load, cache effects, or biased experiment design.

Keep a short evidence record

Start with four lines: the claim; the source and access date; the finding the source actually supports; and an unresolved question. Here the finding could be “the median conversion time for the same hypothetical task halved.” An open question is “what happens to memory use and slow runs with larger files?” Do not invent an answer missing from the evidence.

An announcement date may differ from a measurement date. A page's last-update date is not necessarily the event date either. A recent article can retell an old test. For a current decision, check the software version covered by the document. Five sites quoting one another do not amount to five independent experiments.

Share the result together with its limits

A proportionate summary is: “In this teaching scenario, median conversion time fell from 800 ms to 400 ms; including assumed fixed preparation time gives a 20% total improvement. This is not a real product benchmark.” The number, scope, and uncertainty travel together.

For a community discussion, bring the original source, the sentence you do not understand, and your calculation. Use a small synthetic example instead of private files or personal data. If better evidence changes the conclusion, explain exactly which part changed. A percentage calculator helps with arithmetic; it cannot decide whether a source is honest or a comparison fair.