We tested 8 free PDF redaction tools. Five left the redacted data inside the file.

We redacted the same test document in eight free redaction tools, downloaded the results, and checked each one with an independent PDF parser. Two tools left every marked value readable on the page. Five left values inside the document's metadata. Two others produced a “clean” file by destroying the text layer entirely.

Disclosure: the last row of the table below is our own tool, PlainFile. It fails one of the three checks, and that failure is published here with the rest.

Why this test exists

Most advice about PDF redaction ends with a warning that drawing a black rectangle does not remove anything. That warning is correct, and it is also where most people stop — so the practical question goes unanswered: if I use a real redaction tool, does the content actually go away? We could not find a public test that answered it with reproducible evidence, so we built one. The black-box failure mode is well documented; what happens with real tools is not.

How we measured

Three checks per file, each independent:

1. Text layer — what any PDF parser, copy-paste or search sees (extracted with MuPDF).
2. Metadata — the document's Info dictionary: author, title, subject, keywords, producer.
3. Raw bytes — a literal scan of the file, which also catches XMP streams.

Matching is deliberately two-stage: exact first, then normalised (spaces, hyphens and case ignored), so a tool that re-flows text cannot look cleaner than it is. The test document has three pages and nine synthetic values, none of them real: a fictional name, an SSN from a non-issued range, an invented account number, an IBAN that fails its check digits, an example.com address, a phone number in the reserved fiction range, a date of birth, a case reference, and a value drawn in white on white — present in the text layer, invisible on screen. Four of those values were also planted in the document's metadata.

We scored the eight visible values on page one, the ones a person actually marks while redacting. Values the tester could not see, or did not cover, are reported separately rather than counted as tool failures.

Results

“Text left” is how much text is extractable from the output at all. The source document has 447 characters; zero means the text layer is gone and the page was rasterised into an image.

What the numbers actually say

1. Metadata is the most common failure, and nobody warns you about it

Five of the eight outputs still contained values we had marked for removal — not on the page, but in the file's Info dictionary: AvePDF, DumPDF, iLovePDF, Sejda and Smallpdf. For AvePDF, iLovePDF and Smallpdf the page itself was properly cleared, which makes the metadata the only remaining leak and the purest form of this finding: the screenshot looks perfect, and exiftool still prints the name, the account number and the case reference.

2. Two tools removed nothing at all

Sejda and DumPDF returned a file with all 447 characters intact and all eight marked values still extractable — the page was covered, not cleared. Worth noting what DumPDF says about itself on its own homepage: “No Upload, 100% Secure” and “Permanently Blackout Text Locally.”

3. The invisible text layer survived in seven of nine outputs — including ours

The test document contains a value drawn in white on white. It cannot be seen, so a person marking regions by hand will never select it, and no tool in this test flagged its existence. The only thing that removes text you cannot see is a search-based redaction: if you know the value, search for it instead of drawing a box over it. Our own tool caught the marked values and left this one, which is exactly the kind of gap this test was built to expose. The full checklist covers what a thorough job involves.

4. “Clean” means two very different things

HuePDF and PDF2Go produced zero leaks because they destroyed the text layer completely: 0 of 447 characters remain, and the output files are 52× and 24× the size of the source. PDF24 did the same partially (151 characters left, 17× larger). That is a legitimate approach — it is the strongest guarantee you can get, and it is why some tools offer a flatten or maximum-security mode — but it silently costs you search, copy-paste, screen readers and any further editing. AvePDF, iLovePDF, Smallpdf and PlainFile instead removed only what was marked, leaving 290–302 characters of real text behind. Both outcomes are defensible; neither is labelled.

5. One tool undersold itself

PDF2Go's own page promises you can “redact a PDF by drawing black boxes.” Its output was clean. We are reporting that as readily as the failures: a test in which every competitor fails reads as marketing, and would deserve the scepticism it got.

If you are redacting something today

• Check the metadata, not just the page. Open the result and look at what the file says about itself. If the tool has no metadata step, assume it is still there.
• Search, do not draw. Search-based redaction is the only method that reaches text you cannot see, including invisible layers and text behind images.
• Verify with a second tool. Extract the text from your own output. “It looked black” is not evidence.
• Know what rasterising costs. Flattening to images is a real security upgrade for a one-off document, and a real downgrade for anything you need to search, index or edit later.
• Assume a black rectangle removes nothing. It is the failure mode that keeps appearing in public document leaks, and our fixture reproduces it on demand.

Limitations

One tester edited all documents by hand, on one date, with one test document. Page three of the fixture was not covered consistently across tools, so its results are excluded from the comparison above. Four of the twelve shortlisted tools are not in the table: Adobe's free online redaction URL returned 404 when we checked, Xodo and HonestPDF were not completed, and PrivacyScanPDF could not open the test document and exported a JSON report instead of a PDF, so we are not scoring it either way. Whether a tool uploads your file, requires an account, or adds a watermark was not recorded in this run and is not claimed here. Tools change; these results describe one moment.

Reproduce it

The fixture generator and the checker are part of this site's AGPL-3.0 repository, so you can point the same test at your own tool of choice:

git clone https://github.com/plainfile/plainfile && cd plainfile/app
npm install
npm run audit:fixture        # builds fixtures/audit/audit-fixture.pdf
# redact that file in any tool, download the result, then:
npm run audit:redaction -- ~/Downloads/<tool>-redacted.pdf

The fixture is generated from source, contains no real personal data, and every value in it is either invalid or drawn from a range reserved for documentation and fiction. The checker does not care whose output it is looking at.

Working on a document right now? Redact it in your browser — the file never leaves your device — and read the privacy page to verify that claim yourself in the Network tab.