VestSightBot

If you found this page through your server logs, this is what our crawler does and how to stop it.

Who it is

VestSightBot collects public information about exits in Indian companies for VestSight. Every request it makes carries this user agent:

VestSightBot/0.2 (+https://<this site>/about/crawler)

It never changes or hides that identity, and it does not rotate addresses or use proxies to avoid limits.

Public listing pages, RSS feeds and sitemaps of business news publications, and the article pages they point to. It keeps only the headline and the short excerpt a publication makes public.
Public offer documents from the Securities and Exchange Board of India, and filing listings.

It never signs in, never submits forms, never fetches content behind a paywall, and never visits LinkedIn or any other personal profile page.

It reads and obeys robots.txt before fetching from a site, including any Crawl-delay.
It waits between four and nine seconds between requests (longer if your Crawl-delay asks for it), and backs off when a server signals it is busy.
It does not decode obfuscated links or work around any access control.

Add this to your robots.txt:

User-agent: VestSightBot
Disallow: /

The crawler will stop at its next visit. To block only part of your site, disallow those paths instead.

Questions or problems: darshv.mishra@gmail.com.