Data waterfalls get expensive, fast.
Most try to maximize “thoroughness” at first (find/validate EVERY email humanly possible), but costs grow quickly and it becomes clear that some tradeoffs need to be made between quality and cost.
The question is — “how can you make these tradeoffs if your email verifiers are black box?” Meaning, if you don’t know the overlapping capabilities of each one, how do you know what you’re losing by giving one up, given that some may be better for certain email providers, company sizes, industries, etc?
We decided to study this question in a number of experiments.
In this article, we’ll share the results of the first one, which tried to understand how much various providers overlapped in providing the EXACT same functionality on the backend, by tracing the technologies and 3rd party verification providers that they used.
Summary Of Findings
- MillionVerifier is the biggest winner
- You, the consumer are the biggest loser
Want to know why? Keep reading…
Experiment Design
There are two ways that we can try to understand the value each provider could bring to the table in our waterfall.
First, we can look at their inputs — which infrastructure, systems, techniques and third party providers they use to deliver the result. Second, we can look at divergence in their outputs — this is what we did previously, for example in our prior benchmark test here.
Given that this study is about measuring duplication of cost and not efficacy, we limited our scope in this test to only point #1
Hypothesis
We assumed there would be a huge amount of duplicated effort/waste between providers in most waterfalls today — especially the ones running on platforms like Clay.
Why?
We knew most “hot” companies in the email verification space were founded by ‘growth hacker’ founders who didn’t have technical chops or deep knowledge of the tech behind email. We also knew these guys were experts in building their own waterfalls, and experts in finding arbitrage opportunities.
So… we assumed that many of these companies were building their own waterfalls, buying credits in bulk from the ‘big players’ like MillionVerifier and making money by reselling them at retail rates.
Methodology
We created an email stack and used a number of email verification services to “verify” emails on that target server.
In order to try to get them to make the verification as elaborate as possible, we tried a number of variations in the setup including:
- Testing real/fake emails
- Turning catch-all protection on the email server in question
- Having overlapping records with Google/MSFT to try to trigger different ‘catchall verification’ techniques.
For each vendor, we captured all of the logs on the server like this:
Using these logs, we were able to find common patterns (domains, ip addresses, etc) across vendors to triangulate which services were reselling which vendors.
To help with this triangulation effort, we started out with vendors that we thought would be wholesalers (since these were the most likely to be using a single proprietary service), and then moved onto vendors we thought were resellers.
When we couldn’t identify the exact vendor that the resellers were using, we labeled them MysteryVendorA|B|C|etc just so we could at least see that there was an overlapping/duplicated effort between services, even if we didn’t know who the underlying provider was.
Findings
Resellers (Waterfall APIs)
| Verification Tool | Vendors Used |
| Prospeo | Zerobounce |
| Leadmagic | MillionVerifier, MysteryVendorA |
| Findymail | MillionVerifier, Debounce |
| IcyPeas | — No SMTP verification 😅 — which also led to both fewer and incorrect results in the sample. |
| MailVerify | MillionVerifier, Gmail SMTP |
| Snov.io | MillionVerifier & Bouncer |
| Enrow | Bouncer |
NOTE: the data above does not say these are the ONLY checks that these vendors are using — many have their own ‘catchall verification’ algorithms, and it is outside of the scope of this study to assess them. This purely looks at their use of 3rd party vendors which often represents the highest portion of their variable costs. And these costs are inherently passed onto their consumers.
Proprietary Tech Vendors
The following companies all appeared to use their own proprietary verification technology — we provide the infrastructure and a relative rating below (the rating matters in that lower quality rated infrastructure is more likely to yield inaccurate results in the verification process). Why?
Because higher quality infrastructure carries about half the weighting that most email servers use to block SMTP check requests, so if you are using a really high quality vendor, you probably don’t need to fall back to another.
| Verification Tool | Vendors Used | Primary Infrastructure Provider | Server Location |
| MillionVerifier | Proprietary | Custom ISP (B+) | Hungary (B) |
| Bouncer | Proprietary | Dataspace (B) | Poland (B) |
| EmailListVerify | Proprietary | OVHCloud (C) | Canada (A) |
| Emailable | Proprietary | AWS (A) | US (A) |
| Debounce | Proprietary | Hetzner Online (B) | Germany (A) |
| Zerobounce | Proprietary | GVM Sistem (B) | Europe (B) |
| Neverbounce | Proprietary | Custom ISP (B+) | US (A) |
| Hunter.io | Proprietary | OVHCloud (C) | UK (A) |
| Bounceban | Proprietary | OVHCloud (C) | UK (A) |
| Trykitt.ai | Proprietary & MillionVerifier | AWS (A) | US (A) |
Conclusions
We believe there is substantial evidence here that duplication of effort/cost is a real problem in waterfall enrichment workflows, especially when using the “hot” or “new” vendors in the Clay ecosystem.
We recommend that:
- Consumers should demand these vendors be more transparent in what 3rd party services are bundled and give them the option to disable them for a discounted rate, since waterfall creators will want more control. Trykitt.ai is the only email verification vendor that provides this level of control and transparency.
- Consumers should take some time to use this information to build more intelligence into their waterfall workflows to reduce their duplication of effort and cost. Where possible, they may want to pick just one catchall verification vendor that’s good enough and a single backup wholesale vendor like MillionVerifier (the market clearly is voting this as the best value wholesale provider around!)
- MillionVerifier be given a medal 🥇 — they are the clear winners in this game, since they’re simultaneously monetizing each waterfall request up to 5X in the background while only having to do the lookup once!
eroltoker
Author