onthepluto.com

onthepluto feed fetcher

onthepluto.com compares how news outlets in different countries cover the same events. To do that it reads the public RSS and Atom feeds of about 400 outlets. This page says who we are and how the fetcher behaves.

What it reads

Only public RSS/Atom feeds, and for a few outlets without a feed their public list of latest headlines. It stores the headline, a short summary, the time and the link. It never downloads article pages, images or video, never logs in, and never submits forms. Readers are sent to the original article through the link.

How often

Each feed about every 15 minutes. Never two requests to the same site at once. A refusal (403, 429…) is asked once more under our feed-reader name (below) and then left until the next round.

robots.txt

Checked before every feed (cached for a day). A group for «onthepluto» applies, otherwise «*». A feed that robots.txt disallows is not read. A missing robots.txt allows everything; an unreachable one pauses reading for an hour.

How to stop it

Add «User-agent: onthepluto» with «Disallow: /» (or the feed’s path) to your robots.txt, or write to us — we will remove the feed.

How to recognise it

User-Agent (always contains «onthepluto»; never a browser’s name):

  • Mozilla/5.0 (compatible; onthepluto/1.0; +https://onthepluto.com/bot)
  • onthepluto-reader/1.0 (+https://onthepluto.com/bot)

Requests come only from 2.58.66.195 (IPv4 only; the list as JSON: /bot/ips.json).

Contact

[email protected] — we answer and act on every request.

Facts and findings of the day — on Telegram One post every morning: the main things from every section