Data literacy · Before testing a pattern

Does Benford’s law fit your data? Check before calling a pattern suspicious

Every amount in a spreadsheet begins with 2. That looks unusual beside a Benford chart, where 1 is the most frequent leading digit. But if the spreadsheet contains eighty payments of a fixed 25-unit fee, the pattern has a straightforward explanation. A perfectly accurate file can fail the wrong comparison.

Before using Benford’s law, establish what generated the numbers and what selected them into your file. A mismatch can justify further questions when the comparison is appropriate. It cannot identify fraud by itself, and a close fit cannot certify honest records.

The reference pattern, in one calculation

For data that follow the first-digit Benford distribution, the probability of leading digit d is log10(1 + 1/d), for digits 1 through 9. This gives about 30.1% for 1, 17.6% for 2 and 4.6% for 9. The Bureau of Labor Statistics research paper by Swanson and colleagues (2003) describes the formula in the context of expenditure-survey screening.

These are reference proportions, not quotas that every genuine spreadsheet must meet. “Leading digit” means the first nonzero significant digit: 0.025 begins with 2 for this purpose. A zero has no such digit. Keep missing values distinct from zeros, and state how you treat refunds or other negative amounts before counting.

Three files that should stop the shortcut

Constructed teaching examples: the following files are invented to expose problems with choosing a comparison. They are not an audit, a customer case or observed results from a real organisation.

For the first file, multiplying 80 by the Benford proportion for 2 gives about 14.1, while the actual count is 80. That large difference is entirely explained by the fixed fee in this exercise. Computing an impressive test statistic would not repair the unsupported assumption that these payments should follow Benford.

The second file has the wrong kind of variable. Identifiers organise records; they do not measure expenditure. Preserve them as labels when importing data, as described in the CSV identifier guide. For the third file, ask for the unfiltered scope and the export settings before interpreting the digit frequencies.

Describe the mechanism before choosing a test

Write down what one row represents. A row might be an invoice, an invoice line, a daily total or an updated copy of an earlier entry. Counting all four together can give repeated transactions disproportionate influence. Check the file’s definitions and retain a record of any exclusions.

Then identify constraints: fixed prices, minimum publication thresholds, maximum reimbursement limits, rounded values and filters applied before export. Ask whether the values are measured quantities or assigned codes. A broad spread across scales can support consideration of a Benford comparison, but it is not sufficient proof that the distribution should fit.

Eckhartt and Ruxton’s 2023 study discusses data suitability, test choice and false signals. It also stresses that a statistically significant departure need not come from manipulation. An independently obtained, genuinely comparable dataset can help evaluate the expectation; it should not be chosen because its chart happens to look convenient.

Do not mix currencies, change units or drop inconvenient rows merely to make the curve look better. Define a coherent scope first, preserve the original file and document transformations. If a published table omits transactions below a threshold, that absence is part of the evidence about the dataset—not a detail to hide in the cleaning step.

What to write when the model is unsuitable

For the fixed-fee example, a useful finding is: “The first digits reflect a 25-unit tariff. A first-digit Benford comparison is not justified for this column. We have not assessed whether the recorded payments occurred.” That statement resolves the statistical question while leaving the substantive one open.

The next check should address the claim being investigated. For authorised payment records, it might be reconciliation with source documents or checking duplicate entries and totals. A digit chart cannot tell you whether a missing invoice exists, whether a service was delivered or who entered a record.

If a suitable dataset does depart from the expected pattern, record the test, sample size, preprocessing and size of the departure. Investigate data errors and legitimate process changes alongside possible manipulation. Repeatedly testing different subsets until something looks significant changes how the result must be interpreted; keep a record of the comparisons attempted.

A short decision memo before you count

Complete these six lines: research question; meaning of one row and the selected column; collection and export rules; reason to expect this digit distribution; treatment of zeros, negatives and duplicates; evidence needed after an anomaly. If you cannot explain the fourth line, pause the Benford test and investigate the data’s origin.

Attach that memo to your research report. For the three practice files, the appropriate output is a documented reason not to use the shortcut. That is a useful analytical result, even though it produces no fraud score.