As a regular browser of the Internet, you must have surely come across 404 errors which tell you that the page you were looking for could not be found. Sometimes these are quite funny too, like the one by mint.com. On your own site too, you may see the occasional 404 error. These usually occur as your site evolves and old links stop working. However, this is only applicable if site owners or administrators have left the site in a messy state.
Hackers on the prowl are always looking for vulnerabilities on your site that can be exploited. Some of these vulnerabilities can be found by just loading the frontpage of your site and scanning the content. Some other vulnerabilities might not be directly visible, and hence hackers will use automated scripts to shoot in the dark using some commonly known methods. There is an off-chance that they will succeed, but most of the attempts will actually fail. Each of these failed attempts will lead to a 404 error. By blocking out users when a certain number of 404 errors are generated, you can effectively protect your site against these attacks.
There are many security plugins that support 404 detection, iThemes Security being one of them. Apart from keeping the bad guys at bay, it also helps you uncover broken links on your site that result in 404 errors. Here is a snapshot of the 404 settings for the iThemes Security plugin. You can set the threshold for the number of 404 errors within a time frame that should lead to a lockout.
404 based blocking brings its own challenges too. There may be pages that you know are missing for which you wouldn’t like record 404 errors. For instance, you may have concatenated a bunch of stylesheets in a single file or moved a set of images to a different sub-folder for better organization. Unless you remove the old links in these cases, your visitors are likely to see 404 errors on accessing them, eventually leading to a lockout. In order to avoid this, you can add the list of such pages or file types to a white list as shown in the screenshot above.
Search engines spider the whole site by going through every available link to the site. Due to changes on the site over time, the search engines too will trigger 404 errors. In such cases, if they get blocked out, then it could lead to major impact on the traffic. Your SEO rankings also get affected. This could be a major setback for a site owner. Given the pitfalls, we do not recommend using this way of securing your site.