Email Verifier Benchmark: 8,000 Hard Addresses Tested

Every email verifier looks accurate on an easy list. Point five tools at a set of ordinary addresses on ordinary mail servers and they will agree with each other almost every time.

The disagreements start on the hard cases: catch-all domains that accept mail at any address whether or not the mailbox exists, and domains behind security gateways that greylist incoming connections. Those are exactly the addresses where a verifier is supposed to earn its money, and where headline accuracy figures become much less useful.

So we built a benchmark out of nothing but hard cases: 8,000 addresses, chosen specifically because a standard SMTP check cannot resolve them. Then, for every address where the tools disagreed, we ran a delivery test and recorded whether the destination accepted it or hard-bounced.

The full dataset is published below, one row per address, so you can check the arithmetic yourself.

What We Tested

Five verifiers: Anymail Finder, ZeroBounce, BounceBan, MillionVerifier and NeverBounce.

Before any of the numbers, the thing you should weigh them against: the sample is drawn from our own results, so it is not a neutral sample of the internet. All 8,000 addresses are ones Anymail Finder had already ruled on, 4,000 it called valid and 4,000 it called invalid. It is a set of cases where we already had a position, which we then invited four competitors to challenge. Everything below should be read in that light, and the limitations section lists what the design cannot show.

The 8,000 addresses were split into four groups of 2,000, by the mail infrastructure sitting in front of the domain:

How We Validated Disagreements

Whenever another verifier's call contradicted ours, we escalated that address to Mailprobe, our internal delivery-testing tool. Mailprobe establishes whether the destination accepts the message or returns a hard bounce. The outcome is binary and observable - accepted or hard-bounced - rather than one more verifier's opinion. We report it as acceptance rather than inbox delivery, because acceptance is what is actually measured.

That produced an observed outcome for 663 contested addresses. Every disagreement, with every tool, was resolved against that same outcome. None were left open.

Two things follow from that design, and they cut in opposite directions:

• Because we only escalated disagreements, we do not have an observed outcome on all 8,000. Cases where all five tools agreed and all five were wrong are invisible to this test.
• Because every tool was scored on the same 663 rows, the comparison between tools on those rows is like-for-like. Nobody got an easier set.

Result 1: Email Verifier Coverage on Catch-All Addresses

Before accuracy, coverage. A verifier that returns "catch-all" or "unknown" has not answered the question you paid it to answer - it has handed the decision back to you.

ZeroBounce's 188 are not all abstentions: 79 are "do_not_mail", 73 "unknown", 28 "catch-all" and 8 "abuse". Two of those are specific classifications rather than an unresolved result, which is why the column counts anything that is not a valid/invalid call rather than calling them all catch-alls. It makes no difference to the scoring, since none of them tells you whether the mailbox exists.

NeverBounce declined to call 95.6% of the list. On a benchmark made entirely of catch-all and greylisted addresses that is a defensible engineering position, but it means NeverBounce cannot be scored as a full-coverage verifier here, and we have not tried to.

Coverage also holds steady or collapses depending on what sits in front of the domain, and MillionVerifier is the only tool where it swings: it answers 78.6% of Mimecast addresses and 2.0% of Proofpoint ones. For a list concentrated on Proofpoint-protected enterprises, 2.0% coverage makes it of very limited practical use. The other four are flat across all four environments, at whatever level they operate.

Result 2: Email Verifier Accuracy on 663 Contested Addresses

This is the part that matters. 663 addresses, observed acceptance outcomes for all of them, every tool scored on the identical rows.

We score every tool against all 663 cases, counting anything that is not a valid/invalid call as a failure to answer. Every row adds up to 663, so the four outcome columns account for the whole test with nothing left over.

Those 663 addresses sit on only 245 domains, and the ten largest account for 53% of them. Per-address scoring therefore lets a single large employer swing the result. So we scored it a second way, giving every domain one equal vote regardless of how many addresses it contributed:

The order is identical under both. We also re-scored the set with a hard cap on addresses per domain, at 25, 10, 5, 3 and finally 1, resampling 200 times at each level: the ranking came out the same in every one of those 1,000 draws. We quote the per-address figure throughout this article because it is the lower one for us.

MillionVerifier is the striking one: it abstained on 194 of the 663 hard cases and still got 344 of the remaining 469 wrong. Low coverage bought it nothing.

Anymail Finder vs Other Email Verifiers

The same data, sliced as a direct duel. When we and another tool disagreed, which verdict matched the observed outcome?

Result 3: False Positives vs False Negatives in Email Verification

A false valid, also known as a false positive, is an address classified as valid that hard-bounced. A false invalid, or false negative, is an address classified as invalid that was actually accepted. A verifier can fail in either direction, and they cost you very different things.

Saying valid when the address bounces is the expensive error. It puts a bad address into your sending list, and bounces are what damage your sender reputation.

Saying invalid when the address would in fact have been accepted is the quiet error. It costs you a reachable prospect, which may well be the more expensive loss for your business, but unlike a false valid it adds nothing to your sending infrastructure and you never find out it happened.

Split that way, the tools separate cleanly:

Of the 663 contested addresses, 172 hard-bounced and 491 were accepted, so those are the two denominators. The total column ranks the tools on both mistakes added together, which deliberately weights a bounce and a lost lead as equally bad. They are not equally bad, as the rest of this section argues, so treat the total as a rough ordering rather than a verdict.

The second column counts every reachable address the tool would have cost you, whether it called the address invalid or simply declined to call it, because an unresolved address gets dropped exactly like a rejected one.

This scores abstention differently from the scoreboard above, which asked whether the tool answered at all. Here, abstaining on an address that ultimately bounced counts as a safe outcome, so MillionVerifier and NeverBounce record 179 and 187 safe outcomes against their 125 and 20 actual correct verdicts. For scale, a tool that abstained on all 663 cases would still avoid all 172 false valids, or 25.9%, without successfully identifying a single valid mailbox. NeverBounce's 187 is barely above that floor.

That is what makes NeverBounce's first column misleading on its own. It produced almost no false valids because it barely classified anything as valid, and in exchange it would have discarded 473 of the 491 reachable addresses in this set.

Read both columns together and each tool's chosen trade-off is visible. BounceBan and Anymail Finder sit at the same end, few false valids and the fewest reachable addresses lost. On this contested set ZeroBounce produces 3.6 times as many false valids as we do and still drops almost twice as many reachable addresses. MillionVerifier is worse again on each count.

Across the 4,000 addresses Anymail Finder classified as valid, we identified 17 that bounced during disagreement testing, or 0.4% of the full valid sample. Because addresses every tool agreed on were never independently tested, that is an observed error floor rather than an overall false-valid rate: the true figure can only be higher, not lower. Those 17 are spread across 15 different domains, so they are scattered misses rather than one misread employer.

ZeroBounce's 61 false valids are the opposite shape: they come from 15 domains too, but 24 land on a single one.

ZeroBounce's problem is specific and severe: 57 of its 61 false valids were Mimecast addresses. Mimecast's gateway accepts almost anything at the perimeter and rejects later, so a verifier that trusts the perimeter response marks dead mailboxes as valid. If you are verifying lists heavy in Mimecast-protected companies, that is the number to worry about.

Where Anymail Finder Loses

Our worst result in this study is 107 addresses we called invalid that were actually accepted. They are not spread evenly:

Those 107 misses are not spread across 107 companies. They come from 12 domains, and four of them account for 99:

That reframes the problem. This is not a general Microsoft 365 weakness, it is a handful of large corporate tenants whose configuration defeats our checks, repeated across every contact we hold at those companies.

The weakness is mostly shared. ZeroBounce and BounceBan rejected nearly all of the same accepted Microsoft 365 addresses. MillionVerifier was the exception: it classified 99 of the 100 as valid, and NeverBounce returned catch-all or unknown throughout. Taking the 100 addresses we wrongly rejected on Microsoft 365, here is how many each tool got right:

So the honest statement of our weakness is narrower than "Microsoft 365 is hard": there is a class of large enterprise tenant that accepts mail for mailboxes we reject, and ZeroBounce and BounceBan read those tenants the same way we do. MillionVerifier does not, and on these addresses it was right and we were wrong.

That comes with a caveat worth stating, because it is the same trade-off running in the other direction. MillionVerifier's approach on Microsoft 365 is to call most things valid: across the 257 contested addresses in that group it returned valid on 173, of which 70 hard-bounced. It resolves these tenants correctly, and it pays for that with the highest false-valid count in the study. We would rather close this gap without buying it at that price, and it is the top item on our verification roadmap.

It is worth saying that this failure direction is the one we chose. Given a genuinely ambiguous mailbox, we would rather drop a usable address than classify an address as valid when it will bounce, because a bounce lands in your sender reputation and a dropped lead does not. Whether that is the right trade for you depends on what a missed prospect is worth against a damaged domain. That preference is why our false valids sit at 17 while our false invalids sit at 107.

What This Means for Your List

This benchmark measures how five verifiers performed on deliberately hard addresses. To compare them more broadly on pricing, features and use cases, see our roundup of the best email verification tools.

• On a clean, mainstream list, the tool you pick barely matters. These gaps only open up on catch-all and greylisted domains. If that is not most of your list, buy on price.
• If you sell to enterprises, it matters a lot. Large companies are exactly the ones sitting behind Mimecast and Proofpoint, and those are the domains where coverage and accuracy both fall apart.
• Check the environment, not just the headline accuracy figure. No tool in this test was uniformly good. In this benchmark ZeroBounce's largest concentration of false valids was on Mimecast, and MillionVerifier's coverage held on Mimecast but collapsed on Proofpoint. Our own concentrated weakness was false invalids on Microsoft 365.
• Treat an unresolved catch-all result as a non-answer. MillionVerifier and NeverBounce gave no valid/invalid verdict on roughly 60% and 96% of this list. Whatever you were going to do with that address, you still have to decide it yourself.

Limitations

This is our own study of our own product, so here is what it cannot tell you:

• The sample is anchored on our results. All 8,000 addresses come from Anymail Finder's output, split 4,000 valid and 4,000 invalid. A sample built from a competitor's output would look different.
• These are not overall accuracy rates. The 81.3% figure is the share of 663 deliberately contested hard cases resolved correctly. It is not what any of these tools would score on a normal list, where every one of them would be far higher.
• Errors everyone shared are invisible. We only ran delivery tests where tools disagreed. If all five were wrong together, this study did not catch it, and every tool's error count here is a floor rather than a total.
• The per-environment figures are thin. The 663 contested cases are not spread evenly: Microsoft 365 has 257 and Mimecast 246, but Google Workspace has 115 and Proofpoint only 45, with just 3 bounces among them. The advice to check your own environment stands, but the Proofpoint and Google numbers carry much less weight than the headline ones.
• The sample is concentrated. 8,000 addresses sit on 2,229 domains, and the 663 contested cases sit on only 245, with the largest ten making up 53% of them. We report per-domain accuracy alongside per-address for this reason, and the ranking holds under both. But any single number here is measuring a few hundred companies, not a few thousand.
• Verifiers change. Results were collected in August 2026. At least one Mimecast address we called invalid already resolves as valid in Anymail Finder today.

Download the Data

The complete dataset is public: email-verifier-benchmark-data.csv, 8,000 rows, one per address.

Each row gives the domain, the mail infrastructure group, all five verifier results, and the observed outcome where we have one. Local parts have been removed and only the domain is published, so no individual's email address appears in the file.

Frequently Asked Questions

Test It on Your Own List

We chose this sample, we ran the tests, and we published where we lose. You should still not take it as proof for your list.

The addresses that matter are your hardest ones: the catch-all domains and the enterprises behind Mimecast and Proofpoint. Run a few hundred of them through us and through whatever you use today, then compare the verdicts on the addresses that matter most to you. That is the only benchmark that settles anything.

Anymail Finder includes 100 free credits on signup and only charges for verified results: verify 100 emails free.