Twiceler is broken. Today it went closed loop and made many queries per second agains front page of my site. Had to block IP. Even tried to call this company and mailed. NOTHING.
This Bot is identified as a search engine, when and where can or will the results be available for visitors? Is there anymore information on the type of search engine. I assumme it will be available to the general public?
They don’t obey robots. They don’t stop from hammering your site. They hit your site a zillion thousand times a day. They claim to be a search engine but where’s the page to search for something? BS. My site just went offline due to excess bandwidth due to this thing. I sent them an e-mail but I am honestly not expecting much.
Twiceler has to be the most retarded bot ever.
Instead of just blocking them (like the comment above), do a 301 redirect to their own site. Hopefully the bot is broken enough so that it will visit their own site instead.
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} ^Mozilla/5(.*)Twiceler
RewriteRule . http://www.cuill.com/your_bot_sucks [R=301]
well this week i’ve been visited by that crap and i want to kick it out of our site…
but, i’m really new, and don’t know where to find this .htaccess file you mentioned before.
.htaccess files can only be used with Apache web servers. In most cases, the .htaccess file is placed in the top directory of the web site. Search for .htaccess in Google and you will find a lot of information.
I use this on my sites, works pretty well: if (!(strpos(strtolower($_SERVER["HTTP_USER_AGENT"]), 'twiceler') === false)) {
sleep(25);
echo 'Twiceler is bad software. Use Google instead. Support proper search engines!';
die();
}
Twiceler’s such engine just launched at cuil.com and are being touted as the new Google. What a joke, what until all these buried posts about how the the twiceler bot has been abusing webmasters over the last year start to surface.
Interestingly, Twiceler does not seem to know how to follow HTTP “301 Moved Permanently” redirects.
My site currently gets very few hits from it (about 40 per week), but all of those hits are directed to a retired hostname which automatically supplies known bots with 301 redirects to equivalent URLs on its new hostname (other clients get a “300 Multiple Choices” so they read the page, followed by a 5 second Refresh if they don’t click the link). Googlebot and Yahoo! Slurp understand them properly (they immediately rerequest it from the new URL and update their index), but when Twiceler receives a 301 response, it simply gives up and goes to the next page in its list without ever following the redirect.
I don’t know if it’s smart enough to follow temporary redirects (302 and 303), since I don’t use them anywhere on my site (except for login forms and such, which bots aren’t going to follow).
August 16th, 2007 at 10:02 pm
Twiceler is broken. Today it went closed loop and made many queries per second agains front page of my site. Had to block IP. Even tried to call this company and mailed. NOTHING.
August 21st, 2007 at 7:35 am
This Bot is identified as a search engine, when and where can or will the results be available for visitors? Is there anymore information on the type of search engine. I assumme it will be available to the general public?
December 25th, 2007 at 5:08 pm
They don’t obey robots. They don’t stop from hammering your site. They hit your site a zillion thousand times a day. They claim to be a search engine but where’s the page to search for something? BS. My site just went offline due to excess bandwidth due to this thing. I sent them an e-mail but I am honestly not expecting much.
December 26th, 2007 at 5:24 pm
If you don’t like this bot, block it with these lines at the top of your .htaccess file:
Options +FollowSymlinksRewriteEngine on
RewriteCond %{HTTP_USER_AGENT} twiceler
RewriteRule ^.*$ - [F]
December 28th, 2007 at 11:25 am
Twiceler has to be the most retarded bot ever.
Instead of just blocking them (like the comment above), do a 301 redirect to their own site. Hopefully the bot is broken enough so that it will visit their own site instead.
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} ^Mozilla/5(.*)Twiceler
RewriteRule . http://www.cuill.com/your_bot_sucks [R=301]
February 18th, 2008 at 7:19 pm
Their crawler is pure garbage. It’s crawling one of my sites with all kinds of garbage query strings and made up URL’s.
I’d say it’s more likely that it’s surfing for security holes.
They are obviously a bunch of flakes as they are just internet noise.
March 28th, 2008 at 11:50 pm
well this week i’ve been visited by that crap and i want to kick it out of our site…
but, i’m really new, and don’t know where to find this .htaccess file you mentioned before.
March 29th, 2008 at 2:05 pm
.htaccess files can only be used with Apache web servers. In most cases, the .htaccess file is placed in the top directory of the web site. Search for .htaccess in Google and you will find a lot of information.
July 29th, 2008 at 8:19 am
I use this on my sites, works pretty well:
if (!(strpos(strtolower($_SERVER["HTTP_USER_AGENT"]), 'twiceler') === false)) {
sleep(25);
echo 'Twiceler is bad software. Use Google instead. Support proper search engines!';
die();
}
July 30th, 2008 at 7:22 am
Twiceler’s such engine just launched at cuil.com and are being touted as the new Google. What a joke, what until all these buried posts about how the the twiceler bot has been abusing webmasters over the last year start to surface.
August 28th, 2008 at 8:29 pm
Interestingly, Twiceler does not seem to know how to follow HTTP “301 Moved Permanently” redirects.
My site currently gets very few hits from it (about 40 per week), but all of those hits are directed to a retired hostname which automatically supplies known bots with 301 redirects to equivalent URLs on its new hostname (other clients get a “300 Multiple Choices” so they read the page, followed by a 5 second Refresh if they don’t click the link). Googlebot and Yahoo! Slurp understand them properly (they immediately rerequest it from the new URL and update their index), but when Twiceler receives a 301 response, it simply gives up and goes to the next page in its list without ever following the redirect.
I don’t know if it’s smart enough to follow temporary redirects (302 and 303), since I don’t use them anywhere on my site (except for login forms and such, which bots aren’t going to follow).
September 18th, 2008 at 5:26 pm
Is been two days Twiceler crawling my site constantly.. I decided to block this Bot. Now i know how to block this Bot.I hope i can stop it.