A website owner who sees an unfamiliar crawler in the server logs needs to know who operates it and why it is requesting files. Cloudflare has now approved FindFilesBot through its verified-bot submission process.
Our submission in Cloudflare BotBase now shows Approved status. For FindFiles.net, a search engine for publicly accessible files, this is a useful step toward making our automated requests easier to recognize and assess.
What Cloudflare Approved
The approved bot is FindFilesBot, operated by FindFiles.net UG (limited liability). Its stated purpose is crawling for FindFiles.net, a public file search engine. The submission records Direct access and reference content use, with an IP list as the identity attestation method.
Cloudflare's verified-bot documentation describes verification in terms of identifiable operators and non-abusive behavior, including respect for crawling directives and reasonable request rates. Its Direct label describes a bot operated by a single operator. That fits the purpose of our submission: identifying requests made by our own crawler.
For administrators, the value is practical. A recognizable crawler identity provides context when deciding how to handle automated traffic to a site.
Why This Matters for File Discovery
A useful file search result depends on more than a filename. A report may have moved, a dataset may have been withdrawn, or a download link may no longer work. Checking publicly accessible file URLs helps a file search service understand whether a resource is still reachable.
FindFiles.net's crawler supports that work. Our crawler documentation describes public file discovery and availability checks, along with separate agents for tasks such as retrieving site icons and fetching selected files for automated checks.
Consider a publisher making PDF manuals, CSV datasets, or CAD drawings available on its website. If that publisher wants those resources discoverable through FindFiles.net, being able to identify our requests helps it make an informed access decision. Cloudflare's approval supports that identification. The effect on access still depends on the publisher's settings.
FindFiles.net complements general web search by focusing on the file itself. Making the crawler's identity clearer supports that work where our service interacts with a publisher's website.
The Verification Behind the Approval
A crawler name in a request is useful for reading logs, but another client can send the same text. Verification needs a way to connect the claimed identity to the source of the request.
Cloudflare's IP validation documentation explains how an operator-provided IP list can be used alongside a user-agent match pattern. This is the method recorded in our approved submission. It ties recognition to declared network addresses as well as the crawler's name.
We submitted FindFilesBot on 11 October 2026. During review, Cloudflare requested changes concerning DNS-based identity verification, including the verifiability of the published IPv6 identity and the reverse-DNS suffix. The approved submission uses IP-list attestation. Its last-updated date is 13 November 2026.
This review underlines an operational responsibility: the identity information we publish and submit must stay aligned with the infrastructure sending our requests.
Five Published User-Agents
The approved submission includes five user-agent patterns. They distinguish the general crawler from more specific tasks, helping administrators understand the purpose of a request.
| User-Agent Pattern | Purpose |
|---|---|
FindFilesBot |
General public file discovery and availability checks. |
FindFiles.net-LinkChecker |
Checks whether public file URLs remain reachable. |
FindFiles.net-FaviconFetcher |
Retrieves site icons for source attribution in search results. |
FindFiles.net-VirusScan |
Fetches selected files for malware scanning. |
FindFiles.net-ImageNSFWClassifier |
Fetches images for automated adult-content and safety classification. |
These are match patterns from the submission. Full request headers may also contain a version number and a link to our documentation. The separate names explain why some requests check availability while others retrieve content for a specific processing task.
Website Owners Remain in Control
To check a request claiming to come from FindFiles.net, compare its actual client IP with our published crawler addresses and inspect its user-agent. The following steps work independently of whether you choose to allow the crawler.
1. Find the Request in Your Access Log
For an Nginx access log, this command finds entries containing one of our published agent names. Adjust the path for your server; a common Apache location is /var/log/apache2/access.log. Your log format must include the user-agent.
grep -E 'FindFilesBot|FindFiles[.]net-(LinkChecker|FaviconFetcher|VirusScan|ImageNSFWClassifier)' \
/var/log/nginx/access.log
Read the client IP and user-agent from each candidate entry. Check that the agent name appears in the user-agent field: a match elsewhere in the line, such as a requested URL, does not identify the crawler. A user-agent can also be copied by another client, so continue with the address check.
Behind Cloudflare, your origin log may show a Cloudflare proxy address. Use the original client IP restored through your server's trusted-proxy configuration. Cloudflare documents how to restore visitor IPs using headers such as CF-Connecting-IP. Trust those headers only on connections from your configured trusted proxies; a header supplied directly by an untrusted client is not proof of origin.
2. Compare the Client IP With Our Current IP List
Fetch the public crawler IP list over HTTPS:
curl --fail --silent --show-error https://findfiles.net/bot-ips.txt
As of 15 November 2026, the response contains:
65.21.31.180
2a01:4f9:3080:2b61::2
The request's client IP must match an address in the current list. Together with a published user-agent, that identifies the request as coming from our declared crawler infrastructure. Compare IPv6 addresses numerically if their formatting differs: expanded and compressed forms can represent the same address.
If the download fails, the check is incomplete. If the address is absent, you cannot confirm the request through our IP list. Fetch the current list when investigating traffic rather than relying on the snapshot in this article.
3. Combine Cloudflare Verification With the Source Address
In a Cloudflare WAF custom rule's expression editor, the following condition matches verified-bot requests from the addresses listed above:
cf.client.bot
and ip.src in {65.21.31.180 2a01:4f9:3080:2b61::2}
Cloudflare's cf.client.bot field identifies verified bots generally. Combining it with our source addresses narrows the match to our published infrastructure. Use the user-agent table above to identify the task, and update the addresses in any saved expression if our published list changes.
This expression defines a match condition; the rule's action remains your choice. Keep your site's crawling preferences and access restrictions in place. If a request appears inconsistent with our documentation, use the contact link on the bot page to reach us with its timestamp, client IP, and user-agent.
Where to Find Our Crawler Details
Cloudflare's approval is a welcome milestone for FindFiles.net. It supports a practical relationship between a file search service and the publishers whose public resources it helps people discover.
If you administer a website and see our crawler in your logs, visit the FindFiles.net bot documentation for our published identity, agent descriptions, and contact information.