A Flash Developer Resource Site

Results 1 to 4 of 4

Thread: destroy a spam crawler

  1. 02-18-2008, 11:48 PM #1
    cloudzero
    cloudzero is offline
    Junior Member
    Join Date
    Mar 2006
    Posts
    18

    destroy a spam crawler

    hi,

    so this Microsoft URL Control - 6.00.8862 keeps hitting my site
    i tried denying the IPs, but that still spams the crap out of my log files.

    does anyone have a method of maliciously dealing some damage back to this parasite of a bot?
    Reply With Quote Reply With Quote
  2. 02-19-2008, 01:35 PM #2
    Musicman
    Musicman is offline
    Registered User
    Join Date
    Feb 2001
    Posts
    13,041
    Hi,

    the URL Control is not necessarily bad - but maybe someone was scripting a web crawler and did not understand how to do. If it is on a fixed ip, denying should also get rid of log file entries. If it is targeting a script, and you dont want to deny, you could just delay

    Musicman
    Reply With Quote Reply With Quote
  3. 02-28-2008, 05:03 PM #3
    cloudzero
    cloudzero is offline
    Junior Member
    Join Date
    Mar 2006
    Posts
    18
    its a spam crawler advertising a link as a fake referrer, because the link is also included in the user-agent. i have my own written logging scripts that i use to detect 403-forbidden requests, and these crawlers are filling it up
    Reply With Quote Reply With Quote
  4. 02-29-2008, 01:29 AM #4
    cloudzero
    cloudzero is offline
    Junior Member
    Join Date
    Mar 2006
    Posts
    18
    no one knows how?
    Reply With Quote Reply With Quote
« Previous Thread | Next Thread »

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

Forum Rules



Click Here to Expand Forum to Full Width

HTML5 Development Center