User-agent: * Disallow: /members/ User-agent: AhrefsBot Disallow: / User-agent: BLEXBot Disallow: / User-agent: Yandex Disallow: / User-agent: Data2LifeBot Disallow: / User-agent: proximic Disallow: / User-agent: Mail.RU_Bot Disallow: / User-agent: GrapeshotCrawler Disallow: / User-agent: Genieo Disallow: / User-agent: ZumBot Disallow: / User-agent: MaxPointCrawler Disallow: / User-agent: iZSearch.com Disallow: / User-agent: spbot Disallow: / User-agent: ADmantX Disallow: / User-agent: xintellibot Disallow: / User-agent: Sogou web spider Disallow: / User-agent: WeSEE:Ads/PictureBot Disallow: / User-agent: WeSEE:Ads/PageBot Disallow: / User-agent: MegaIndex.ru Disallow: / User-agent: omgilibot Disallow: / User-agent: SemrushBot Disallow: / User-agent: SemrushBot-SA Disallow: / User-agent: SemrushBot-BA Disallow: / User-agent: SemrushBot-SI Disallow: / User-agent: SemrushBot-SWA Disallow: / User-agent: SemrushBot-CT Disallow: / User-agent: SemrushBot-BM Disallow: / User-Agent: trendictionbot Disallow: / User-agent: MJ12bot Disallow: / User-agent: dotbot Disallow: / User-agent: Clickagy Intelligence Bot v2 Disallow: / User-agent: Bytespider Disallow: / User-agent: MaxPointCrawler Disallow: / User-agent: adscanner Disallow: / User-agent: um-IC/1.0 Disallow: / User-agent: Mozilla/5.0 (compatible) SemanticScholarBot (+https://www.semanticscholar.org/crawler) Disallow: / User-agent: istellabot Disallow: / User-agent: ZoominfoBot Disallow: / User-agent: startmebot Disallow: / User-agent: oBot Disallow: / User-agent: linkdex.com/v2.0 Disallow: / User-agent: linkdex.com/v2.1 Disallow: / User-agent: linkdex.com/v2.2 Disallow: / User-agent: FeedlyBot Disallow: / User-agent: MojeekBot Disallow: / User-agent: Cocolyzebot Disallow: / User-agent: SeznamBot Disallow: / User-agent: MBCrawler Disallow: / User-agent: SEOkicks Disallow: / User-agent: The Knowledge AI Disallow: / User-agent: Scrapy Disallow: /