Today, I'm here to explain you how bad-bots can destroy your website. Usually, badbots are used to extract important data from the website. These bad bots can be harmed for a website, if it is not properly protected. How to protect your website from these bots visits? You can use user agent to stop visiting some unusual bots to your website by using a robots.txt file. I have extracted some bad bots from my log file, and would like to publish here. You can use under mentioned bots in your robots.txt file to prevent such harmed visits to your website.
List of Bad Bots
User-agent: Black Hole
User-agent: Titan
User-agent: WebStripper
User-agent: NetMechanic
User-agent: CherryPicker
User-agent: EmailCollector
User-agent: EmailSiphon
User-agent: WebBandit
User-agent: EmailWolf
User-agent: ExtractorPro
User-agent: CopyRightCheck
User-agent: Crescent
User-agent: Wget
User-agent: SiteSnagger
User-agent: ProWebWalker
User-agent: CheeseBot
User-agent: Teleport
User-agent: TeleportPro
User-agent: MIIxpc
User-agent: Telesoft
User-agent: Website Quester
User-agent: WebZip
User-agent: moget/2.1
User-agent: WebZip/4.0
User-agent: WebSauger
User-agent: WebCopier
User-agent: NetAnts
User-agent: Mister PiX
User-agent: WebAuto
User-agent: TheNomad
User-agent: WWW-Collector-E
User-agent: RMA
User-agent: libWeb/clsHTTP
User-agent: asterias
User-agent: httplib
User-agent: turingos
User-agent: spanner
User-agent: InfoNaviRobot
User-agent: Harvest/1.5
User-agent: Bullseye/1.0
User-agent: Mozilla/4.0 (compatible; BullsEye; Windows 95)
User-agent: Crescent Internet ToolPak HTTP OLE Control v.1.0
User-agent: CherryPickerSE/1.0
User-agent: CherryPicker /1.0
User-agent: WebBandit/3.50
User-agent: NICErsPRO
User-agent: Microsoft URL Control - 5.01.4511
User-agent: DittoSpyder
User-agent: Foobot
User-agent: WebmasterWorldForumBot
User-agent: SpankBot
User-agent: BotALot
User-agent: lwp-trivial/1.34
User-agent: lwp-trivial
User-agent: Wget/1.6
User-agent: BunnySlippers
User-agent: Microsoft URL Control - 6.00.8169
User-agent: URLy Warning
User-agent: Wget/1.5.3
User-agent: LinkWalker
User-agent: cosmos
User-agent: moget
User-agent: hloader
User-agent: humanlinks
User-agent: LinkextractorPro
User-agent: Offline Explorer
User-agent: Mata Hari
User-agent: LexiBot
User-agent: Web Image Collector
User-agent: The Intraformant
User-agent: True_Robot/1.0
User-agent: True_Robot
User-agent: BlowFish/1.0
User-agent: JennyBot
User-agent: MIIxpc/4.2
User-agent: BuiltBotTough
User-agent: ProPowerBot/2.14
User-agent: BackDoorBot/1.0
User-agent: toCrawl/UrlDispatcher
User-agent: WebEnhancer
User-agent: TightTwatBot
User-agent: suzuran
User-agent: VCI WebViewer VCI WebViewer Win32
User-agent: VCI
User-agent: Szukacz/1.4
User-agent: QueryN Metasearch
User-agent: Openfind data gathere
User-agent: Openfind
User-agent: Xenu's Link Sleuth 1.1c
User-agent: Xenu's
User-agent: Zeus
User-agent: RepoMonkey Bait & Tackle/v1.01
User-agent: RepoMonkey
User-agent: Zeus 32297 Webster Pro V2.9 Win32
User-agent: Webster Pro
User-agent: EroCrawler
User-agent: LinkScan/8.1a Unix
User-agent: Keyword Density/0.9
User-agent: Kenjin Spider
User-agent: Cegbfeieh
Post new comment