CiteFlowBot
CiteFlowBot is the web crawler operated by CiteFlow, an AI content operations platform.
What we do
CiteFlowBot fetches publicly accessible pages of websites to analyse structural readiness for SEO, AEO (Answer Engine Optimisation), and LLMO (Large Language Model Optimisation). Our analysis runs on behalf of CiteFlow customers and the CiteFlow team's own product development.
How to identify us
User-Agent: CiteFlowBot/1.0 (+https://citeflow.co.uk/bot)
How to block us
Add the following to your robots.txt:
User-agent: CiteFlowBot Disallow: /
We respect robots.txt rules at all times.
How to rate-limit us
Specify a Crawl-delay directive in your robots.txt:
User-agent: CiteFlowBot Crawl-delay: 5
We respect Crawl-delay if it is longer than our configured rate.
What we store
We extract and store metadata signals from each page (title, meta description, headings, structured data, internal links). We do not store the full HTML or copy your content. Page signals are retained for analysis history and discarded according to our retention policy.
Contact
For questions or issues: contact@citeflow.co.uk