BitNinja – The Road to Success with the new Browser Integrity Check Feature

One of the major challenges as a web hosting provider is keeping all our servers and all customers’ websites secured. We always prioritize security, making sure to implement all possible active and passive measures in order to block all attacks and malicious intents in their tracks. BitNinja is one security solution that we have started using on the FastComet platform a few months ago. Thus far, we are witnessing great net results, especially in terms of preventing malicious traffic. With the aid from the integrated set of modules system, we have managed to prevent several potential exploits, unwanted bots, malicious objects, but what’s more important – we saved a large amount of server resources that can now be used meaningfully and with a legitimate activity by all our clients.

IP Reputation Like Never Before

BitNinja inspects each connection to our servers, seeking out obvious malicious activity, and then verifying the kind of bots that are accessing the server – if they are “good” or “bad” bots, in addition to logging all of the suspicious incidents in a central database. This analysis is performed in the most intelligent way possible, with many different checks and balances that help ensure there are virtually no false positives. The analysis detects all IP addresses that behave in a suspicious way (e.g. – if a visitor is accessing one and the same web page in small intervals, tries to fill in a password that’s incorrect multiple times, etc.). BitNinja would mark such kind of behavior as potentially dangerous for your site, which results in adding the IP address to a “greylist” that identifies your IP as potentially harmful. All suspicious activity logs are being reported and the attack types – categorized. The BitNinja greylist is a very precise list of the dangerous IP-s of the Internet containing more than 2.1 million IPs. This list is constantly fine-tuned automatically, and every BitNinja server automatically contributes information to this list.

It is the “greylisting” concept that is exactly what makes BitNinja exceptionally effective. Traditionally, a simple firewall would only ban or allow IP addresses when trying to reach the server. The problem here is that in case there is even the slightest chance of a false positive, and the visitor was a person, they will not be able to access the site anymore. What BitNinja does differently is that if an IP address is listed on their greylist, they won’t block each connection from that same address.

The CAPTCHA Challenge Problems

Up until now, greylisted IP addresses were given a ‘CAPTCHA’ page that allowed them to insta-unblock their IPs.

This however opened space for some very common issues. Even though the only action you had to perform was to click on “I am not a robot” icon and then have your IP address delisted, some of the website visitors found this too much of a challenge. They didn’t really know why they had to fill out a CAPTCHA despite the obvious fact that it was just one click. Some of the users were afraid of filling CAPTCHAs. Others did not see the CAPTCHA page at all. There were also those who proceeded through the CAPTCHA page but got the message: ERR_CONNECTION_TIMED_OUT. Whatever the reason was, there was an increasing amount of complaints that were filed to us by our customers as a BitNinja partner.

This made us take the decision to disable the IP Filtering module as the lesser of two evils to avoid any further confusion with the annoying CAPTCHA issues. It was obvious that a new security solution was needed where the visitors with suspicious incident cases in their past don’t have to type in anything. What’s more, they don’t need to click anywhere as well.

Why Having a High Threat IP Reputation Matters

Nowadays, malicious traffic is a considerable problem which probably affects each website that’s online. The time for a malicious login attempt into a brand new site after it goes live is maybe just a few weeks. The traffic is most likely to come from automated botnets, programmed to crawl the web for sites and locate all login panels with the use of preset conditions. The attacks aren’t targeted and carried out manually. Nevertheless, the activity remains a serious problem. Bots use a large amount of server resources with all their login attempts. Because of the potential number of sent requests, a brute force attack could actually work as a DoS attack, which would lead to many sites going down because of the high utilization of CPU/Memory. Even without big volumes of bot activity resulting in a service denial, it can still make your hosting more expensive by making you exceed the resources of your account. That’s because the account needs to handle not just your legit visitors’ traffic, but also unwanted bot traffic. The BitNinja IP Reputation system prevents this via filtering all requests and maintaining a 4-level reputation ranking.

Goodbye CAPTCHA! Hello Browser Integrity Check!

It is commonly understood that one of the best ways to develop a successful product is to create something that solves real problems. With customers in mind, the ninjastic team of BitNinja has decided to make an even more convenient method for validating browsers and traffic. This is how they came up with the idea to create a Browser Integrity Check (BIC) instead of only the reCAPTCHA. The BIC does background checking of the browser, followed by the automatic delisting of the IP address.

BitNinja decided to create an even more convenient way to validate browsers and valid traffic. So they came up with the idea to build in Browser Integrity Check (BIC) instead of using the reCAPTCHA alone. It does some background checking of the browser and after that, it automatically delists the IP address. Say Goodbye to the old CAPTCHA problems. BIC is a much easier and convenient way for validating normal visitors and malicious attackers.

We wasted no precious time and we activated the new module right away. For you, as a FastComet customer, this means more stable hosting experience, reduced resource usage by your website and peace of mind knowing that no search engine bots are being blocked by the system.

How Does the Browser Integrity Check Work?

Since the BIC is connected to the IP Reputation module, it will appear to those who are on the BitNinja greylist and connected to BitNinja-protected servers through HTTP or HTTPS (in case of HTTP Protection module is enabled). After the validation, the IP would be removed from the BitNinja greylist and requests to the BitNinja protected servers would not be blocked.

What BIC actually does is looking for common HTTP headers that are most commonly abused by spammers and then deny access to your page. It would also challenge the visitors that do not have a user agent or those that have a non-standard user-agent (also commonly used by abuse bots, crawlers or visitors).

Why is the BIC better than the CAPTCHA/reCAPTCHA?

The website visitors won’t have to fill or type anything, they just have to wait for 5 seconds while we ascertain by measuring some feedback information from the browser while we run some simple JavaScipt calculations in the background.

Will There Be any Exceptions?

It is important to note that BitNinja will keep the old method as well in some cases when there are too many incidents from an IP address, or for some reason this new method fails. This way you can avoid all the hassles with the CAPTCHA and hopefully, your website visitors will no longer complain about the captcha and they will rather get a feeling that we do care about security and we do it in a very convenient way.

We want to thank all customers, who have been among the first affected by the CAPTCHA issue for the patience and for helping us and BitNinja team become better in dealing with it. Of course, should you experience any issues with the incoming or outgoing connections, do not hesitate to contact our team of Technical Support experts. They would be glad to look into this.