Details you only notice from far away
onthepluto.com compares how news outlets in different countries cover the same events. To do that it reads the public RSS and Atom feeds of about 400 outlets. This page says who we are and how the fetcher behaves.
Only public RSS/Atom feeds, and for a few outlets without a feed their public list of latest headlines. It stores the headline, a short summary, the time and the link. It never downloads article pages, images or video, never logs in, and never submits forms. Readers are sent to the original article through the link.
Each feed about every 15 minutes. Never two requests to the same site at once. A refusal (403, 429…) is asked once more under our feed-reader name (below) and then left until the next round.
Checked before every feed (cached for a day). A group for «onthepluto» applies, otherwise «*». A feed that robots.txt disallows is not read. A missing robots.txt allows everything; an unreachable one pauses reading for an hour.
Add «User-agent: onthepluto» with «Disallow: /» (or the feed’s path) to your robots.txt, or write to us — we will remove the feed.
User-Agent (always contains «onthepluto»; never a browser’s name):
Mozilla/5.0 (compatible; onthepluto/1.0; +https://onthepluto.com/bot)onthepluto-reader/1.0 (+https://onthepluto.com/bot)Requests come only from 2.58.66.195 (IPv4 only; the list as JSON: /bot/ips.json).
[email protected] — we answer and act on every request.