A Flash Developer Resource Site

Results 1 to 4 of 4

Thread: destroy a spam crawler

  1. #1
    Junior Member
    Join Date
    Mar 2006
    Posts
    18

    destroy a spam crawler

    hi,

    so this Microsoft URL Control - 6.00.8862 keeps hitting my site
    i tried denying the IPs, but that still spams the crap out of my log files.

    does anyone have a method of maliciously dealing some damage back to this parasite of a bot?

  2. #2
    Registered User
    Join Date
    Feb 2001
    Posts
    13,044
    Hi,

    the URL Control is not necessarily bad - but maybe someone was scripting a web crawler and did not understand how to do. If it is on a fixed ip, denying should also get rid of log file entries. If it is targeting a script, and you dont want to deny, you could just delay

    Musicman

  3. #3
    Junior Member
    Join Date
    Mar 2006
    Posts
    18
    its a spam crawler advertising a link as a fake referrer, because the link is also included in the user-agent. i have my own written logging scripts that i use to detect 403-forbidden requests, and these crawlers are filling it up

  4. #4
    Junior Member
    Join Date
    Mar 2006
    Posts
    18
    no one knows how?

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  




Click Here to Expand Forum to Full Width

HTML5 Development Center