Monday, November 30, 2009

Building a Secure Squid Web Proxy, Part III



We've been building a secure Squid Web Proxy the past few months, and we'll continue to do so for a couple more. Last time [May 2009], we got Squid installed, running and restricted to serve only local clients (based on their IP addresses). This month, we delve deeper into Squid's Access Control List (ACL) capabilities and other built-in security features.

ACL Review
As you may recall from my last column, all we had to do to get Squid running on a standard Ubuntu 8.04 system was add two lines to the file /etc/squid/squid.conf:

acl mick_network src 10.0.2.0/24
http_access allow mick_network

We inserted those two lines, which allow outbound proxy connections from clients whose IP addresses fall within the network 10.0.2.0/24 (that is, addresses 10.0.2.1 through 10.0.2.254), right above Squid's default “deny all” ACL, which looks like this:

http_access deny all

You can correctly infer from this that, by default, Squid denies proxy connections from all clients. This is a refreshing change in default server application configurations during the past few years. Whereas in the past, many applications had default configurations that would “just work”, which is a very user-friendly but also excessively open stance, nowadays few network applications will do much of anything without some administrative intervention. This is only sensible. Connecting things to the Internet that you don't even know how to configure is the way of pain.

Getting back to our example ACL, the acl statement itself is fairly self-explanatory: acl tells Squid we're defining an ACL; mick_network is its name; src indicates it matches the client's source IP address or network address; and 10.0.2.0/24 is the network address in CIDR notation that will match this ACL.

This is the simplest type of ACL and still one of the most useful. In February 2002, if the New York Times had had a simple source-IP/network ACL correctly configured on its Internet-facing corporate Web proxies, the rogue hacker Adrian Lamos couldn't have gained access quite so easily to its editorial-page contributor database or its Lexus-Nexus portal.

ACLs in More Depth
Besides clients' (source) IP addresses, Squid also can match a great deal of other proxy transaction characteristics. Note that some of these deal with arcane HTTP headers and parameters, many of which are minimally useful for most Squid users anyhow.

Table 1. Complete List of ACL Types Supported in Squid 2.6


ACL TypeDescription
srcClient (transaction source) IP address or network address.
dstServer (transaction destination) IP address or network address.
myipLocal IP address on which Squid is listening for connections.
arpClient's Ethernet (MAC) address (matches local LAN clients only).
srcdomainClient's domain name as determined by reverse DNS lookup.
dstdomainDomain portion of URL requested by client.
srcdom_regexRegular expression matching client's domain name.
dstdom_regexRegular expression matching domain in requested URL.
timePeriod of time in which transaction falls.
url_regexRegular expression matching entire requested URL (not just domain).
urlpath_regexRegular expression matching path portion of requested URL.
urlloginRegular expression matching requested URL's “login” field.
portRequested site's (destination) TCP port.
myportLocal TCP port on which Squid is listening for connections.
protoApplication-layer protocol of request (HTTP, HTTPS, FTP, WHOIS or GOPHER).
methodRequest's HTTP method (GET, POST or CONNECT).
browserMatches the client's browser, per HTTP “User-Agent” header.
referer_regexRegular expression matching the unreliable HTTP “Referer” header (that is, the supposed URL of some page on which the user clicked a link to the requested site).
identMatches specified user name(s) of user(s) running client browser, per an “ident” lookup. Note that ident replies, which often can be spoofed, should not be used in lieu of proper authentication.
ident_regexRegular expression defining which client user names to match per ident lookup.
src_asMatches client IP addresses associated with the specified Autonomous System (AS) number, usually an ISP or other large IP registrant.
dst_asMatches destination-server IP addresses associated with the specified AS number.
proxy_authMatches the specified user name, list of user names or the wild card REQUIRED (which signifies any valid user name).
proxy_auth_regexRegular expression defining which user names to match.
snmp_communityFor SNMP-enabled Squid proxies, matches client-provided SNMP community string.
maxconnMatches when client's IP address has established more than the specified number of HTTP connections.
max_user_ipMatches the number of IP addresses from which a single user attempts to log in.
req_mime_typeMatches a regular expression describing the MIME type of the client's request (not the server's response).
req_headerMatches a regular expression applied to all known request headers (browser, referer and mime-type) in the client's request.
rep_mime_typeMatches a regular expression describing the MIME type of the server's response.
rep_headerMatches a regular expression applied to all known request headers (browser, referer and mime-type) in the server's response.
externalPerforms an external ACL lookup by querying the specified helper class defined in the external_acl_type tag.
urlgroupMatches a urlgroup name, as defined in redirector setups.
user_certMatches specified attribute (DN, C, O, CN, L or ST) and values against client's SSL certificate.
ca_certMatches specified attribute (DN, C, O, CN, L or ST) and values against client certificate's issuing Certificate Authority certificate.
ext_userMatches specified user name(s) against that returned by an external ACL/authentication helper (configured elsewhere in squid.conf).
ext_user_regexMatches a regular expression describing user names to be matched against that returned by an external ACL/authentication helper.

I've presented the full range of possible ACL types to give you a taste for how rich Squid's ACL functionality is. Needless to say, however, I can't cover usage scenarios for (or even adequately explain) all of these. ViServe's “Squid 2.6 Configuration Manual” (see Resources) gives complete syntax and usage examples for all.

Many, if not most, Squid installations don't go much beyond a few src ACLs, along with perhaps a few simple dstdomain blacklist entries thrown in for good measure. Many of the other most useful ACL types, such as myip, time, port, proto, method, dst_mime_type and rep_mime_type, should be reasonably self-explanatory (or at least easy enough to understand from the examples shown in squid.conf's comments).

One category of less-intuitive ACL types is particularly powerful and useful: the ones that enable Squid to authenticate client users via external authentication authorities. Before we tackle authentication, however, we should give a little more attention to ACL operators, the tags that perform some action (most commonly, to allow or deny a request) based on a matched ACL.

By far, the most important ACL operator is http_access, which specifies whether Squid should allow the transaction matching the specified ACL to proceed. Going back to the example ACL/operator pair from the beginning of this section, after we defined the ACL mick_network as all transactions involving client/source IP addresses within 10.0.2.0/24, we operated on it with this line:

http_access allow mick_network
This is simple enough to understand: “allow HTTP requests matching the ACL named mick_network.”

The most common use of ACLs is to specify a list of ACLs and http_access statements, ending (as we've seen) with a “drop by default” line, like this:

http_access deny all
This has the effect of creating a “whitelist”— a list of types of transactions that are allowed, with all others being denied.

Squid recognizes a number of additional ACL operators besides http_allow, including no_cache, ident_lookup_access, always_direct, never_direct and snmp_access. Because most of these concern cache performance, HTTP redirects and communications with other Squid servers rather than security per se, I'll leave it to you to explore those (or not) as your particular needs dictate. The Squid User's Guide referenced in the Resources section is a good source of information about Squid's various ACL operators.

Squid Authentication
As I mentioned previously, one of Squid's most handy capabilities is its ability to authenticate proxy users by means of a variety of external helper mechanisms. One of the simplest and probably most commonly used helper applications is ncsa_auth, a simple user name/password scheme that uses a flat file consisting of rows of user name/password hash pairs. The HOWTO by Vivek Gite and, to a lesser extent, the Squid User's Guide, explain how to set this up (see Resources).

Briefly, you'll add something like this to /etc/squid/squid.conf:

auth_param basic program /usr/lib/squid/ncsa_auth /etc/squid/squidpasswd
auth_param basic children 5
auth_param basic realm Squid proxy-caching web server at Wiremonkeys.org
auth_param basic credentialsttl 2 hours
auth_param basic casesensitive off
And, in the ACL section:

acl ncsa_auth_users proxy_auth REQUIRED
http_access allow ncsa_auth_users
The block of auth_param tags specifies settings for a “basic” authentication mechanism:

program is the helper executable ncsa_auth, using the file /etc/squid/squidpassd as the user name/password hash list (created previously).

children, the number of concurrent authentication processes, is five.

realm, part of the string that greets users, is “Squid proxy-caching Web server at Wiremonkeys.org”.

credentialsttl, the time after authentication that a successfully authenticated client may go before being re-authenticated, is two hours.

casesensitive, which determines whether user names are case-sensitive, is off.

In the ACL section, we defined an ACL called ncsa_auth_users that says the proxy_auth mechanism (as defined in the auth_param section) should be used to authenticate specified users. Actually in this case, instead of a list of user names to authenticate, we've got the wild card REQUIRED, which expands to “all valid users”. The net effect of this ACL and its subsequent http_access statement is that only successfully authenticated users may use the proxy.

The main advantages of the NCSA mechanism are its simplicity and its reasonable amount of security (only password hashes are transmitted, not passwords proper). Its disadvantage is scalability, because it requires you to maintain a dedicated user name/password list. Besides the administrative overhead in this, it adds yet another user name/password pair your users are expected to remember and protect, which is always an exercise with diminishing returns (the greater the number of credentials users have, the less likely they'll avoid risky behaviors like writing them down, choosing easy-to-guess passwords and so forth).

Therefore, you're much better off using existing user credentials on an external LDAP server (via the ldap_auth helper) on an NT Domain or Active Directory server (via the msnt_auth helper) or the local Pluggable Authentication Modules (PAM) facility (via the pam_auth helper). See Resources for tutorials on how to set up Squid with these three helpers.

Note that Squid's helper programs are located conventionally under /usr/lib/squid. Checking this directory is a quick way to see which helpers are installed on your system, although some Linux distributions may use a different location.

Other Squid Defenses
Access Control Lists really are Squid's first line of defense—that is, Squid's primary mechanism for protecting your network, your users and the Squid server itself. There are a couple other things worth mentioning, however.

First, there's the matter of system privileges. Squid must run as root, at least while starting up, so that, among other things, it can bind to privileged TCP ports such as 80 or 443 (although by default it uses the nonprivileged port 3128). Like other mainstream server applications, however, Squid's child processes—the ones with which the outside world actually interacts—are run with lower privileges. This helps minimize the damage a compromised or hijacked Squid process can do.

By default, Squid uses the user proxy and group proxy for nonprivileged operations. If you want to change these values for effective UID and GID, they're controlled by squid.conf's cache_effective_user and cache_effective_group tags, respectively.

Squid usually keeps its parent process running as root, in case it needs to perform some privileged action after startup. Also, by default, Squid does not run in a chroot jail. To make Squid run chrooted, which also will cause it to kill the privileged parent process after startup (that is, also will cause it to run completely unprivileged after startup), you can set squid.conf's chroot tag to the path of a previously created Squid chroot jail.

If you're new to this concept, chrooting something (changing its root) confines it to a subset of your filesystem, with the effect that if the service is somehow hacked (for example, via some sort of buffer overflow), the attacker's processes and activities will be confined to an unprivileged “padded cell” environment. It's a useful hedge against losing the patch rat race.

Chrooting and running with nonroot privileges go hand in hand. If a process runs as root, it can trivially break out of the chroot jail. Conversely, if a nonprivileged process nonetheless has access to other (even nonprivileged) parts of your filesystem, it still may be abused in unintended and unwanted ways.

Somewhat to my surprise, there doesn't seem to be any how-to for creating a Squid chroot jail on the Internet. The world could really use one—maybe I'll tackle this myself at some point. In the meantime, see Resources for some mailing-list posts that may help. Suffice it to say for now that as with any other chroot jail, Squid's must contain not only its own working directories, but also copies of system files like /etc/nsswitch.conf and shared libraries it uses.

Common Squid practice is to forego the chroot experience and to settle for running Squid partially unprivileged per its default settings. If, however, you want to run a truly hardened Squid server, it's probably worth the effort to figure out how to build and use a Squid chroot jail.

Conclusion
Setting ACLs, running Squid with nonroot privileges most or all of the time and running Squid in a chroot jail constitute the bulk of Squid's built-in security features. But, these are not the only things you can do to use Squid to enhance your network and end-user systems' security.

Next time, I'll show you how to use add-on tools such as SquidGuard to increase Squid's intelligence in how it evaluates clients' requests and servers' replies. I'll also address (if not next time then in a subsequent column) some of the finer points of proxying TLS/SSL-encrypted sessions. Until then, be safe!

Resources

Wessels, Duane: Squid: The Definitive Guide. Sebastopol, CA: O'Reilly Media, 2004. Includes some tips on creating and using a Squid chroot jail.

The Squid home page, where you can obtain the latest source code and binaries for Squid: www.squid-cache.org.

The Ubuntu Server Guide's Squid chapter: https://help.ubuntu.com/8.10/serverguide/C/squid.html.

The Squid User's Guide: www.deckle.co.za/squid-users-guide/Main_Page.

ViSolve's Squid 2.6 Configuration Manual and Comprehensive squid.conf Reference: www.visolve.com/squid/squid26/contents.php.

“The Homeless Hacker v. The New York Times”, Jennifer Kahn's article in Wired about Adrian Lamos: www.wired.com/wired/archive/12.04/hacker_pr.html.

Chris Wichura's slideshow “The Squid Caching Proxy”: www.uniforum.chi.il.us/slides/squid/UniForum-Squid.ppt.

Vivek Gite's tutorial “Howto: Squid proxy authentication using ncsa_auth helper”: www.cyberciti.biz/tips/linux-unix-squid-proxy-server-authentication.html.

Vivek Gite's Tutorial “Configure squid for LDAP authentication using squid_ldap_auth helper”: www.cyberciti.biz/tips/howto-configure-squid-ldap-authentication.html.

David Bolton's “Howto: Squid + msnt_auth + Active Directory”: www.davidbolton.com/?p=32.

Paul Matthews' HOWTO “Squid with PAM Authentication and Squish Download Manager”: www.opensourcehowto.org/how-to/squid/squid-with-pam-authentication--squish-download-manager.html.

Thread from the squid-users mailing list, on what should go into a Squid chroot jail: www.squid-cache.org/mail-archive/squid-users/200609/0782.html.

Thread from the squid-users mailing list, about some of the finer points of running Squid in a chroot jail: www.squid-cache.org/mail-archive/squid-users/200811/0411.html.

Mick Bauer (darth.elmo@wiremonkeys.org) is Network Security Architect for one of the US's largest banks. He is the author of the O'Reilly book Linux Server Security, 2nd edition (formerly called Building Secure Servers With Linux), an occasional presenter at information security conferences and composer of the “Network Engineering Polka”.

Taken From: Linux Journal Contents #183, July 2009

Building a Secure Squid Web Proxy, Part II

Get a Squid caching proxy up and running, securely.

Last month, I began a series of articles on Squid Web proxy security by introducing the theory, benefits and architecture of Web proxies. This month, we dive right in to basic Squid installation, configuration and testing, and begin hardening our Squid proxy.

What We're Doing (Review)
As you'll recall from last month, a Web proxy provides a control point for restricting which external Web sites your users can reach. It allows you to permit Web access without allowing non-Web traffic (or even publishing a default route to the Internet), and it provides a convenient place to perform content filtering and transaction logging.

As you also may recall, unlike a firewall, a Web proxy doesn't need to be a physical choke point through which all traffic must pass for a physical path to the outside. Instead, you can use firewall rules or router ACLs that allow only Web traffic, as a means of ensuring your users will use the proxy. Accordingly, your Web proxy can be set up like any other server, with a single network interface.

This is the case with the Web server I show you how to build in this and subsequent columns. This month, we focus on Squid itself; we'll cover add-ons like SquidGuard in future columns.

Obtaining and Installing Squid
So, where do you get Squid software? Naturally, the Squid Web site (see Resources) is the definitive source. But, because Squid has been the gold standard for Linux Web proxies for so many years, chances are it's a fully supported package in your Linux distribution of choice. If so, that's how I recommend getting it; it's easier to keep it patched that way, and you'll have greater assurance of stability and compatibility with the other things on your system.

On Ubuntu and other Debian variants (not to mention Debian itself), you need the packages squid and squid-common. On Red Hat and its variants, you need the package squid. And, on SUSE and OpenSUSE systems, you need squid.

At the time of this writing, all three of these families of distributions (Debian, Red Hat and SUSE) are maintaining separate packages for Squid version 3; the packages cited above are for version 2. This is because although the Squid development team recently declared Squid 3.0 to be a stable release (in November 2008), at the time of these three distributions' most recent production releases, Squid 3.0 still was considered to be a beta code branch, with 2.6 or 2.7 as the preferred production versions.

On the one hand, by the time you read this, Squid 3.0 (maybe even 3.1, which is in beta right now) may be mainstreamed into your Linux distribution of choice. On the other hand, maybe not. So for now, I'm going to use examples from Squid 2.6.18, the version on my Ubuntu system. They still should be perfectly valid for later versions—generally, later versions have additional options and features, not replaced options. I can cover Squid 3.0 in a future column.

I leave it to you to use the package manager of choice to install Squid packages on your RPM-based system, but on Debian-based systems, the most direct way is usually with the command:

bash-$ sudo apt-get install squid
(apt-get automatically will determine that it also needs squid-common and will install that too.)

By the way, you do not need to install Apache or any other Web server package on your Squid server, unless, of course, you're also going to use it as a Web server or want to use some Web-based administration tool or another. Squid itself does not need any external Web server software or libraries in order to proxy and cache Web connections.

Configuring Squid: Basic Functionality
Creating a basic, working configuration for Squid isn't much harder than installing it. Like so much else in Linux, it's a matter of making small changes to a single text file, in this case, squid.conf. In all three distribution families I mentioned, its full path is /etc/squid/squid.conf.

To get started, first open a command window, and back up the default squid.conf file (non-Ubuntu users can su to root and omit the sudo from these examples):

bash-$ cd /etc/squid
bash-$ sudo cp squid.conf squid.conf.default

Next, open squid.conf with your text editor of choice. You actually may prefer a graphical editor, such as gedit, but I've always used vi for its simplicity and ubiquity—if it's UNIX-like, it's got vi.

(Note to the emacs-loving alpha geeks among you: yes, emacs is more powerful; it's written in LISP; God kills a kitten every time someone installs Gvim; you win! But, I still like vi.)

Believe it or not, all you need to do to get Squid running is add two lines to the ACL (Access Control List) section of this file: an object definition that describes your local network and an ACL allowing members of this object to use your proxy. For my network, these lines look like this:

acl mick_network src 10.0.2.0/24
http_access allow mick_network

The first line is the object definition. The acl signifies that I'm about to define an ACL object. mick_network is the name I've chosen for this object. src means that it represents the IP address or range of addresses of hosts initiating TCP transactions with my proxy (that is, proxy clients). Finally, 10.0.2.0/24 is my LAN's network address in CIDR notation, which in this case translates to “the range of IP addresses from 10.0.2.1 through 10.0.2.254”.

The second line declares an actual ACL: allow transactions involving the object mick_network—that is, transactions initiated by hosts having IP addresses in the range 10.0.2.1 through 10.0.2.254.

If more than one network address comprises your local network, you can specify them as a space-delimited list at the end of the acl statement, for example:

acl mick_network src 10.0.2.0/24 192.168.100.0/24

Because ACLs are parsed in the order in which they appear (going from top to bottom) in squid.conf, do not simply add these acl and http_access lines to the very end of squid.conf, which will put them after the default “http_access deny all” statement that ends the ACL portion of the default squid.conf file. On my Ubuntu system, this statement is on line 641, so I inserted my custom acl and http_access lines right above that.

In case you haven't guessed, all is a wild-card ACL object that means “all sources, all ports, all destinations” and so forth. Any transaction that is evaluated against any http_access statement containing any will match it, and in this case, will be dropped, unless, of course, it matches a preceding http_access line.

Now that you've created an object and ACL for your local network, you should save squid.conf and then restart Squid by typing this command (see earlier note about su root shells vs. sudo):

bash-$ sudo /etc/init.d/squid restart

In fact, if you're editing squid.conf from a sudo vi squid.conf session, you don't even need to leave your editing session; just do a :w to save your work, then type :! /etc/init.d/squid restart to restart Squid from within vi.

To test whether things are working, you need to configure a machine other than the proxy itself to use your proxy. (Squid comes configured by default to allow transactions from 127.0.0.1, the local loopback address, to be proxied.)

Figure 1 shows the dialog for setting up Firefox to use our example proxy.

Figure 1. Setting Up Firefox to Use Proxies

In Figure 1, we've selected Manual proxy configuration and entered in an HTTP Proxy address (which can be either a hostname or IP address) of 10.0.2.2 and Port number 3128, which is Squid's default listening port for client connections. We've also selected the box to Use this proxy server for all protocols, resulting in the same values being copied automatically to the subsequent settings for other types of proxies.

We've left No Proxy for: at its default value of localhost, 127.0.0.1. The reason for not proxying connections to Web pages hosted locally on the client system is probably obvious, but you can additionally list URLs or IP addresses elsewhere on your local network that there's no need to use the proxy to reach.

At this point, you may be wondering, what does the connection between a client and a Web proxy look like? Is there some special protocol, or maybe a subset of HTTP commands or flags?

In fact, proxy connections are simpler than you may think. Normally, when you click on a hyperlink or enter a URL, your browser resolves the URL you typed or clicked on, using its own local DNS capabilities. It then takes the IP address and sends an HTTP/HTTPS request to that IP address, with the original (non-resolved) URL in the body of the request.

A proxied connection is the same without any DNS resolution. Your browser simply sends its HTTP/HTTPS request to the proxy server without trying to resolve the URL. The body of that request is identical to the one it would otherwise send directly to the Web server you're trying to reach.

Instead of configuring your Web browser's proxy settings directly, if you use the GNOME desktop on your client test system, you can set global proxy settings that can, in turn, be used by Firefox and other Internet applications. Note, however, that the proxy settings you set in GNOME will be applied only to applications that are, in turn, configured to use system settings—for example, by selecting the option Use system proxy settings shown in Figure 1. Other applications will continue to use either their own proxy settings or no proxy at all.

GNOME's Network Proxy Preferences applet, which should appear in your System→Preferences menu, is shown in Figure 2.

Figure 2. Setting Global Proxy Options in GNOME

It may seem like I'm spending a lot of ink explaining client-side configuration just for testing purposes, given that this is an article about building Squid servers. But, of course, the way you set up a proxy client for testing is the same as for one in production, so I would have had to explain this sooner or later anyhow.

In fact, future installments in this series may go further in covering client configuration topics. Autoproxy.pac files, for example (which is what Figure 1's Automatic proxy configuration URL setting is for), can be very handy in managing very complex or very highly scaled proxy environments.

Once you've configured your test client system to use your Squid proxy, you can attempt to navigate to some Web page to see if everything works. It's a good idea to tail Squid's access log simultaneously. To do so, enter this command on your Squid system:

bash-$ sudo tail -f /var/log/squid/access.log

If browsing works but nothing zings by in this log-tailing session, your client-side configuration is incorrect—it isn't actually using the proxy. If browsing doesn't work, you may see some useful server-side message in the log-tailing session. Squid usually returns fairly useful messages directly to client browsers as well.

If things don't work, your browser session is simply timing out and nothing is showing up in access.log, try using the ping command from your client to your proxy and vice versa. If pinging doesn't work, the problem is at the network level and has nothing to do with Squid.

Squid's Performance Benefits

The Paranoid Penguin is a security column, so naturally, security is our primary focus in dealing with Squid (or it will be, once I've walked you through the basics of getting it up and running). But, you should be aware that Squid is not a security application per se. Squid's main purpose in life is to cache commonly accessed Web and FTP content locally, thereby both reducing Internet bandwidth usage and speeding up end users' download times.

The negative side of this is that Squid doesn't have as rich of a security feature set built in to it as commercial security-oriented Web proxies, such as BlueCoat and Sidewinder. In fact, Squid (years ago) used to ship with a default configuration that allowed completely open access.

The good side is that Squid can be configured, especially along with add-ons like SquidGuard, to provide some of the most important Web proxy security features. And, even if those features are your main reason for deploying Squid, you'll still enjoy the performance benefits of having commonly accessed Web content cached locally by Squid.

Seldom, in the security business, do we enhance end users' experience when we add security controls.

Conclusion
With any luck, at this point, chances are that everything works! Your Squid proxy software is installed, configured to accept only client connections from itself and from hosts on your local network, and it's hard at work proxying your users' connections and caching commonly accessed content. Not a bad day's work!

Not difficult, was it? Like most server applications, Squid's default configuration file is designed to maximize your chances for success, while minimizing the odds of your shiny-new Squid server being hacked. But, also like other server applications, there's certainly more that you can and should do to secure your Squid proxy than the default settings will do for you.

That will be our starting point next month. Among other things, we'll delve much deeper into Squid's Access Control List features to further harden Squid. Until then, be safe!

Resources

The Squid home page, where you can obtain the latest source code and binaries for Squid: www.squid-cache.org

The Ubuntu Server Guide's Squid Chapter: https://help.ubuntu.com/8.10/serverguide/C/squid.html

The Squid User's Guide: www.deckle.co.za/squid-users-guide/Main_Page

Mick Bauer (darth.elmo@wiremonkeys.org) is Network Security Architect for one of the US's largest banks. He is the author of the O'Reilly book Linux Server Security, 2nd edition (formerly called Building Secure Servers With Linux), an occasional presenter at information security conferences and composer of the “Network Engineering Polka”.

Taken From: Linux Journal Contents #181, May 2009

Building a Secure Squid Web Proxy, Part I

Consider the venerable Web proxy—back when the Internet was new to most of us, setting up a Web proxy was a convenient way to grant users of an otherwise non-Internet-connected network access to the World Wide Web. The proxy also provided a convenient point to log outbound Web requests, to maintain whitelists of allowed sites or blacklists of forbidden sites and to enforce an extra layer of authentication in cases where some, but not all, of your users had Internet privileges.

Nowadays, of course, Internet access is ubiquitous. The eclipsing of proprietary LAN protocols by TCP/IP, combined with the technique of Network Address Translation (NAT), has made it easy to grant direct access from “internal” corporate and organizational networks to Internet sites. So the whole idea of a Web proxy is sort of obsolete, right?

Actually, no.

After last month's editorial, we return to technical matters—specifically, to the venerable but assuredly not obsolete Web proxy. This month, I describe, in depth, the security benefits of proxying your outbound Web traffic, and some architectural and design considerations involved with doing so. In subsequent columns, I'll show you how to build a secure Web proxy using Squid, the most popular open-source Web proxy package, plus a couple of adjunct programs that add key security functionality to Squid.

What Exactly Is a Web Proxy?
The last time I discussed proxies in this space was in my December 2002 article “Configuring and Using an FTP Proxy”. (Where does the time go?) A quick definition, therefore, is in order.

The concept of a Web proxy is simple. Rather than allowing client systems to interact directly with Web servers, a Web proxy impersonates the server to the client, while simultaneously opening a second connection to the Web server on the client's behalf and impersonating the client to that server. This is illustrated in Figure 1.

Figure 1. How Web Proxies Work

Because Web proxies have been so common for so long, all major Web browsers can be configured to communicate directly through Web proxies in a “proxy-aware” fashion. Alternatively, many Web proxies support “transparent” operation, in which Web clients are unaware of the proxy's presence, but their traffic is diverted to the proxy via firewall rules or router policies.

Why Proxy?
Just because nowadays it's easy to interconnect TCP/IP networks directly doesn't mean you always should. If a nasty worm infects systems on your internal network, do you want to deal with the ramifications of the infection spreading outward, for example, to some critical business partner with whom your users communicate over the Internet?

In many organizations, network engineers take it for granted that all connected systems will use a “default route” that provides a path out to the Internet. In other organizations, however, it's considered much less risky to direct all Web traffic out through a controlled Web proxy to which routes are internally published and to use no default route whatsoever at the LAN level.

This has the effect of allowing users to reach the Internet via the Web proxy—that is, to surf the Web—but not to use the Internet for non-Web applications, such as IRC, on-line gaming and so forth. It follows that what end users can't do, neither can whatever malware that manages to infect their systems.

Obviously, this technique works only if you've got other types of gateways for the non-Web traffic you need to route outward, or if the only outbound Internet traffic you need to deal with is Web traffic. My point is, a Web proxy can be a very useful tool in controlling outbound Internet traffic.

What if your organization is in a regulated industry, in which it's sometimes necessary to track some users' Web access? You can do that on your firewall, of course, but generally speaking, it's a bad idea to make a firewall log more than you have to for forensics purposes. This is because logging is I/O-intensive, and too much of it can impact negatively the firewall's ability to fulfill its primary function, assessing and dealing with network transactions. (Accordingly, it's common practice mainly to log “deny/reject” actions on firewalls and not to log “allowed” traffic except when troubleshooting.)

A Web proxy, therefore, provides a better place to capture and record logs of Web activity than on firewalls or network devices.

Another important security function of Web proxies is blacklisting. This is an unpleasant topic—if I didn't believe in personal choice and freedom, I wouldn't have been writing about open-source software since 2000—but the fact is that many organizations have legitimate, often critical, reasons for restricting their users' Web access.

A blacklist is a list of forbidden URLs and name domains. A good blacklist allows you to choose from different categories of URLs to block, such as social networking, sports, pornography, known spyware-propagators and so on. Note that not all blacklist categories necessarily involve restricting personal freedom per se; some blacklists provide categories of “known evil” sites that, regardless of whatever content they're actually advertising, are known to try to infect users with spyware or adware, or otherwise attack unsuspecting visitors.

And, I think a lot of Web site visitors do tend to be unsuspecting. The classic malware vector is the e-mail attachment—an image or executable binary that you trick the recipient into double-clicking on. But, what if you could execute code on users' systems without having to trick them into doing anything but visit a Web page?

In the post-Web 2.0 world, Web pages nearly always contain some sort of executable code (Java, JavaScript, ActiveX, .NET, PHP and so on), and even if your victim is running the best antivirus software with the latest signatures, it won't examine any of that code, let alone identify evil behavior in it. So, sure enough, the “hostile Web site” has become the cutting edge in malware propagation and identity theft.

Phishing Web sites typically depend on DNS redirection (usually through cache poisoning), which involves redirecting a legitimate URL to an attacker's IP address rather than that site's real IP, so they're difficult to protect against in URL or domain blacklists. (At any rate, none of the free blacklists I've looked at include a phishing category.) Spyware, however, is a common blacklist category, and a good blacklist contains thousands of sites known to propagate client-side code you almost certainly don't want executed on your users' systems.

Obviously, no URL blacklist ever can cover more than a tiny fraction of the actual number of hostile Web sites active at any given moment. The real solution to the problem of hostile Web sites is some combination of client/endpoint security controls, better Web browser and operating system design, and in advancing the antivirus software industry beyond its reliance on virus signatures (hashes of known evil files), which it's been stuck on for decades.

Nevertheless, at this very early stage in our awareness of and ability to mitigate this type of risk, blacklists add some measure of protection where presently there's very little else. So, regardless of whether you need to restrict user activity per se (blocking access to porn and so forth), a blacklist with a well-maintained spyware category may be all the justification you need to add blacklisting capabilities to your Web proxy. SquidGuard can be used to add blacklists to the Squid Web proxy.

Just How Intelligent Is a Web Proxy?

You should be aware of two important limitations in Web proxies. First, Web proxies generally aren't very smart about detecting evil Web content. Pretty much anything in the payloads of RFC-compliant HTTP and HTTP packets will be copied verbatim from client-proxy transactions to proxy-server transactions, and vice versa.

Blacklists can somewhat reduce the chance of your users visiting evil sites in the first place, and content filters can check for inappropriate content and perhaps for viruses. But, hostile-Web-content attacks, such as invisible iframes that tell an attacker's evil Web application which sites you've visited, typically will not be detected or blocked by Squid or other mainstream Web proxies.

Note that enforcing RFC compliance is nothing to sneeze at. It constitutes a type of input validation that could mitigate the risk of certain types of buffer-overflow (and other unexpected server response) attacks. But nonetheless, it's true that many, many types of server-side evil can be perpetrated well within the bounds of RFC-compliant HTTP messages.

Second, encrypted HTTPS (SSL or TLS) sessions aren't truly proxied. They're tunneled through the Web proxy. The contents of HTTPS sessions are, in practical terms, completely opaque to the Web proxy.

If you're serious about blocking access to sites that are inappropriate for your users, blacklisting is an admittedly primitive approach. Therefore, in addition to blacklists, it makes sense to do some sort of content filtering as well—that is, automated inspection of actual Web content (in practice, mainly text) to determine its nature and manage it accordingly. DansGuardian is an open-source Web content filter that even has antivirus capabilities.

What if you need to limit use of your Web proxy, but for some reason, can't use a simple source-IP-address-based Access Control List (ACL)? One way to do this is by having your Web proxy authenticate users. Squid supports authentication via a number of methods, including LDAP, SMB and PAM. However, I'm probably not going to cover Web proxy authentication here any time soon—802.1x is a better way to authenticate users and devices at the network level.

Route-limiting, logging, blacklisting and authenticating are all security functions of Web proxies. I'd be remiss, however, not to mention the main reason many organizations deploy Web proxies, even though it isn't directly security-related—performance. By caching commonly accessed files and Web sites, a Web proxy can reduce an organization's Internet bandwidth usage significantly, while simultaneously speeding up end-users' sessions.

Fast and effective caching is, in fact, the primary design goal for Squid, which is why some of the features I've discussed here require add-on utilities for Squid (for example, blacklisting requires SquidGuard).

Web Proxy Architecture
Suppose you find all of this very convincing and want to use a Web proxy to enforce blacklists and conserve Internet bandwidth. Where in your network topology should the proxy go?

Unlike a firewall, a Web proxy doesn't need to be, nor should it be, placed “in-line” as a choke point between your LAN and your Internet's uplink, although it is a good idea to place it in a DMZ network. If you have no default route, you can force all Web traffic to exit via the proxy by a combination of firewall rules, router ACLs and end-user Web browser configuration settings. Consider the network shown in Figure 2.

Figure 2. Web Proxy Architecture

In Figure 2, Firewall 1 allows all outbound traffic to reach TCP port 3128 on the proxy in the DMZ. It does not allow any outbound traffic directly from the LAN to the Internet. It passes only packets explicitly addressed to the proxy. Firewall 2 allows all outbound traffic on TCP 80 and 443 from the proxy (and only from the proxy) to the entire Internet.

Because the proxy is connected to a switch or router in the DMZ, if some emergency occurs in which the proxy malfunctions but outbound Web traffic must still be passed, a simple firewall rule change can accommodate this. The proxy is only a logical control point, not a physical one.

Note also that this architecture could work with transparent proxying as well, if Firewall 1 is configured to redirect all outbound Web transactions to the Web proxy, and Firewall 2 is configured to redirect all inbound replies to Web transactions to the proxy.

You may be wondering, why does the Web proxy need to reside in a DMZ? Technically, it doesn't. You could put it on your LAN and have essentially identical rules on Firewalls 1 and 2 that allow outbound Web transactions only if they originate from the proxy.

But, what if some server-side attacker somehow manages to get at your Web proxy via some sort of “reverse-channel” attack that, for example, uses an unusually large HTTP response to execute a buffer-overflow attack against Squid? If the Web proxy is in a DMZ, the attacker will be able to attack systems on your LAN only through additional reverse-channel attacks that somehow exploit user-initiated outbound connections, because Firewall 1 allows no DMZ-originated, inbound transactions. It allows only LAN-originated, outbound transactions.

In contrast, if the Web proxy resides on your LAN, the attacker needs to get lucky with a reverse-channel attack only once and can scan for and execute more conventional attacks against your internal systems. For this reason, I think Web proxies are ideally situated in DMZ networks, although I acknowledge that the probability of a well-configured, well-patched Squid server being compromised via firewall-restricted Web transactions is probably low.

Yet to Come in This Series
I've explained (at a high level) how Web proxies work, described some of their security benefits and shown how they might fit into one's perimeter network architecture. What, exactly, will we be doing in subsequent articles?

First, we'll obtain and install Squid and create a basic configuration file. Next, we'll “harden” Squid so that only our intended users can proxy connections through it.

Once all that is working, we'll add SquidGuard for blacklisting, and DansGuardian for content filtering. I'll at least give pointers on using other add-on tools for Squid administration, log analysis and other useful functions.

Next month, therefore, we'll roll up our sleeves and plunge right in to the guts of Squid configuration and administration. Until then, be safe!

Resources

“Configuring and Using an FTP Proxy” by Mick Bauer, LJ, December 2002: www.linuxjournal.com/article/6333

The Squid home page, where you can obtain the latest source code and binaries for Squid: www.squid-cache.org

The Squid User's Guide: www.deckle.co.za/squid-users-guide/Main_Page

The SquidGuard home page—SquidGuard allows you to enforce blacklists with Squid: www.squidguard.org

The DansGuardian home page, a free content-filtering engine that can be used in conjunction with Squid: dansguardian.org

Mick Bauer (darth.elmo@wiremonkeys.org) is Network Security Architect for one of the US's largest banks. He is the author of the O'Reilly book Linux Server Security, 2nd edition (formerly called Building Secure Servers With Linux), an occasional presenter at information security conferences and composer of the “Network Engineering Polka”.

Monday, November 23, 2009

Sync Your Phone With Thunderbird - Funambol

Cooking with Linux - Linux, Thunderbird and the BlackBerry—a Love Story

July 1st, 2009 by Marcel Gagné in

Keeping various devices in sync with our Linux systems can be the source of nightmares for many. After all, asking for an open-source solution that can keep millions of smartphones, cell phones, e-mail clients, contact databases and calendars on the same planet, never mind the same page, seems akin to asking for the moon—to which Chez Marcel would like to ask, “Would you

While we wait for his return, let me tell you about François' dilemma. He has multiple portable devices, including a BlackBerry, an Android phone and a Motorola RAZR, all of which he wants to synchronize with Evolution on his Linux notebook. On the store workstation, he uses Thunderbird instead, and at home, something else. Getting those contact lists, calendars and so on synchronized is easier than it sounds, and it all can be done with Linux and open-source software.

All this is possible, and easy, with a great little package from a company called Funambol. The software itself also is called Funambol, and it is freely distributed and open source. Essentially, it's a program that lets you perform over-the-air (also known as OTA) synchronization of your contacts, calendars and so on, using your cell phone or smartphone, desktop contact management software (Evolution, Thunderbird, Outlook and so forth) and other hardware. Part of the magic behind all of it is SyncML (Synchronization Markup Language), which also is known as Open Mobile Alliance Data Synchronization (OMA DS). SyncML is an open standard for synchronizing information, such as calendars and contacts, that is platform-independent. Several mobile phone manufacturers, such as Motorola, Nokia and Sony Ericsson, already include SyncML in their devices. SyncML also supports e-mail, which is handy for those needing (or just plain wanting) an alternative to proprietary products, like the BlackBerry.

Funambol consists of a server component and a client for your device or application. Start by getting your copy of Funambol server from funambol.org, and save it somewhere on your system. The package file, with a .bin extension, needs to be made executable before you execute it:

chmod +x funambol-7.1.bin ./funambol-7.1.bin 

The whole thing takes only a few seconds. The steps that follow are extremely simple. Type yes at the “agree to the above terms” prompt (it's the GPL version 3). You'll be prompted for an installation directory which, by default, is /opt. It's best to accept the default unless you have a very good reason to do otherwise. The resulting folder will be /opt/Funambol. Once the product has been extracted, you'll be asked whether you want to start the server. Type yesand continue on. To make sure things are working properly, point your browser to http://localhost:8080/funambol/ds, and you should get status information back from the Funambol data synchronization server (Figure 1).

Figure 1. A Quick Test to Make Sure the Server Is Up and Running

Of course, if you aren't running this test directly on the server, you'll want to change localhost to the hostname or IP address of the server.

Funambol also comes with a simple Web app to test the contact as well as calendar creation and update before you turn it over to your mobile device. Point your browser to http://localhost:8080/funambol to bring up the demo page. You won't be able to do a great deal at this point, other than read the terms and conditions and test a very limited Web client. That demonstration will allow you to log in as guest with a password of guest and create contacts (Figure 2) or a calendar entry. Once you have done so, update a record or two, and make sure the changes are being saved.

Figure 2. The Web client demo lets you create calendars and contacts, making it a better test.

Now that you know it works, you still can't do a great deal with Funambol in this form. In order to do more interesting things, you need to do a little system configuration. On the server side, there is a graphical administration tool. You can start it from the command line like this:

cd /opt/Funambol admin/bin/funamboladmin 

A couple seconds later, you'll see the Funambol administration tool appear (Figure 3). To use the administration tool, you first need to log in. If you don't see the login window up front, click File on the menu bar, and select Login. By default, the admin password, sa, already is set (you always can change it later), but for now, simply click Login.

Figure 3. Funambol Administration Tool and Login Screen

The Funambol administration tool is divided into three panes: a navigator pane fills the top left half, an admin tool pane is at the top right, and a status pane is located along the bottom (Figure 4). Take a look at the navigator window, and you will see your system's domain name at the top. To expand the system tree, click the switch icon next to the domain name. You'll then see Server Settings (which expands into its own subtree), Users, Devices, Principals and Modules. That last one also expands into several other branches. To see how this all works and how you can configure and change things, let's deal with that admin password right now.

Figure 4. On the left, you can see the Funambol administration tool's system navigator with several expanded properties.

Double-click on Users and look at the admin tool window (Figure 5). The Search Users tool appears. You can search by user name, first name, last name and e-mail address. Enter admin in the search box beside Username, and click the Search button (notice that you can search by a part of the name as well as position of the text by clicking the drop-down box beside the label). Only one admin name should show up, so it naturally will be highlighted. If you did this by searching for part of a name, and you had multiple names, you would, of course, need to select the correct name.

Figure 5. Using the Tool to Change the Admin Password

Click the Edit button, change the password, and then save your changes. That takes care of controlling access to the tool. Your next step is to define access to the system. As it stands, your Funambol implementation allows connections only from localhost and then only to a limited set of users. You need to change that. Double-click on Server Settings in the navigator window. Now, look to the left and locate the Server URI field in the settings window (Figure 6).

Figure 6. As a final first step, you need to configure the URI to the Funambol service on your server.

Enter the hostname (or the IP address) of your server, then click Save. You should see a confirmation message in the status window below. It should look something like this:

http://yourdomain.com:8080/funambol/ds 

Believe it or not, that's pretty much it on the server end. Now, let's take a break, have François refill everyone's glass, and then let's see what we need to do on the BlackBerry end of things.

The first step is to install the BlackBerry client, which you can find at https://www.forge.funambol.org/download/downloads-bb.html. You will see an e-mail client in addition to the sync client, but, for the sake of this article, let's just concentrate on the sync client. Make sure you get the right client for your particular BlackBerry OS version.

Once installed, you will see the Funambol BlackBerry sync icon in your list of applications on the BlackBerry screen (Figure 7).

Figure 7. The Funambol Client Icon as It Appears on My BlackBerry

Click the icon, and you should see a status screen showing Contacts, Calendar, Tasks and Notes, all with Not Synchronized below the labels. To perform a sync, you need to configure the client. Press the menu key on your BlackBerry, and select Settings (Figure 8).

Figure 8. Press the menu key to configure the client settings.

When the Funambol client configuration screen appears (Figure 9), enter the URI for your machine's Funambol server. This is the same address that you entered when you configured the server. You also must enter your user name and password—that's your Linux server user name and password. A little farther down that screen, there are check boxes beside labels to Sync Contacts, Sync Calendar, Sync Tasks and Sync Notes. These are all checked by default, but you may decide you don't want to sync all those resources, so change it here if you like. You also can configure a scheduled sync and have the client update your information every 30 minutes (the default) or whatever period makes sense to you. That feature is not turned on unless you specify otherwise.

Figure 9. Funambol BlackBerry Client's Configuration Screen

When you're done, save your settings (on my BlackBerry, I just press the trackball or the back arrow). You'll find yourself back at the status screen, and now you're ready to synchronize for the first time. Press the menu key, and select Sync All from the menu. The Funambol client will connect with your server and start transferring the information on your BlackBerry. Underneath the labels for Contacts (and Calendar and so on), the client will show how many records are being transferred. Once complete, the status screen lists the last successful sync for each resource (Figure 10).

Figure 10. During synchronization, the status screen shows you the number of records transferred. Once complete, you can see the latest sync at a glance.

This is all wonderful, because the Funambol server effectively is keeping an over-the-air backup of your data—handy if you ever need to reload it. But, what if you use another client on your Linux desktop for e-mail, contacts and appointments, such as Evolution or Thunderbird? Funambol provides download clients for these and others as well. Figure 11 shows a screenshot of a pretty desolate-looking address book in Thunderbird.

Figure 11. My Thunderbird Address Book, without Any Contacts

The plugin you need for Thunderbird is available from the Funambol community download page. Download it, and save it to a local directory. Once that's done, click Tools on the Thunderbird menu bar and select Add-ons. When the Add-ons window appears, click the Install button, and navigate to the folder where you stored the file, then click on it and install it. Once finished, Thunderbird needs to restart to load the new extension. After Thunderbird restarts, you must configure the Funambol client to connect to your server. Click Tools from the menu bar, and select Funambol plugin. When the Funambol PIM Plugin window appears, click the Options button, and you'll see a screen that, although shinier than the one on the BlackBerry, is similar as it asks for the same information, namely the server URL, user name and password (Figure 12). Enter the information, then click Close.

Figure 12. Configuring the Funambol Thunderbird Plugin

That's it. To synchronize Thunderbird with the contacts from my BlackBerry, all I do is click the Synchronize button and wait while my contacts are transferred (Figure 13). How long this takes depends, of course, on how much information is being synchronized and how fast your connection is.

Figure 13. The Thunderbird Sync Plugin Happily Doing What It Is Built to Do

In this way, I can keep my desktop client in sync with my BlackBerry and the server itself. As an added bonus, I get over-the-air backup with my own server without having to shell out the dollars for a BES server. Funambol, Linux and my BlackBerry—it's a match made in open-source heaven.

With the help of Funambol, a great open-source application, you (and François), can keep all that personal information in sync without having to resort to entering the information manually or paying huge sums of money for a special server running proprietary code. Well, mes amis, the time is finally upon us. That old clock on the wall says closing time has arrived yet again. François will be happy to refill your glasses a final time while we say our goodbyes to one another. Please, mes amis, raise your glasses, and let us all drink to one another's health. A votre santé! Bon appétit!

Marcel Gagné is an award-winning writer living in Waterloo, Ontario. He is the author of the Moving to Linux series of books from Addison-Wesley. Marcel is also a pilot, a past Top-40 disc jockey, writes science fiction and fantasy, and folds a mean Origami T-Rex. He can be reached via e-mail atmarcel@marcelgagne.com. You can discover lots of other things (including great Wine links) from his Web sites at marcelgagne.com andcookingwithlinux.com.


Taken From: http://www.linuxjournal.com/article/10486

Creating VPNs - IPSEC and OpenVPN (SSL/TLS)

Creating VPNs with IPsec and SSL/TLS

VPN (Virtual Private Network) is a technology that provides secure communication through an insecure and untrusted network (like the Internet). Usually, it achieves this by authentication, encryption, compression and tunneling. Tunneling is a technique that encapsulates the packet header and data of one protocol inside the payload field of another protocol. This way, an encapsulated packet can traverse through networks it otherwise would not be capable of traversing.


Figure 1. A Basic VPN Tunnel

Currently, the two most common techniques for creating VPNs are IPsec and SSL/TLS. In this article, I describe the features and characteristics of these two techniques and present two short examples of how to create IPsec and SSL/TLS tunnels in Linux and verify that the tunnels started correctly. I also provide a short comparison of these two techniques.

IPsec and Openswan

IPsec (IP security) provides encryption, authentication and compression at the network level. IPsec is actually a suite of protocols, developed by the IETF (Internet Engineering Task Force), which have existed for a long time. The first IPsec protocols were defined in 1995 (RFCs 1825–1829). Later, in 1998, these RFCs were depreciated by RFCs 2401–2412. IPsec implementation in the 2.6 Linux kernel was written by Dave Miller and Alexey Kuznetsov. It handles both IPv4 and IPv6. IPsec operates at layer 3, the network layer, in the OSI seven-layer networking model. IPsec is mandatory in IPv6 and optional in IPv4. To implement IPsec, two new protocols were added: Authentication Header (AH) and Encapsulating Security Payload (ESP). Handshaking and exchanging session keys are done with the Internet Key Exchange (IKE) protocol.

The AH protocol (RFC 2404) has protocol number 51, and it authenticates both the header and payload. The AH protocol does not use encryption, so it is almost never used.

ESP has protocol number 50. It enables us to add a security policy to the packet and encrypt it, though encryption is not mandatory. Encryption is done by the kernel, using the kernel CryptoAPI. When two machines are connected using the ESP protocol, a unique number identifies this connection; this number is called SPI (Security Parameter Index). Each packet that flows between these machines has a Sequence Number (SN), starting with 0. This SN is increased by one for each sent packet. Each packet also has a checksum, which is called the ICV (integrity check value) of the packet. This checksum is calculated using a secret key, which is known only to these two machines.

IPsec has two modes: transport mode and tunnel mode. When creating a VPN, we use tunnel mode. This means each IP packet is fully encapsulated in a newly created IPsec packet. The payload of this newly created IPsec packet is the original IP packet.

Figure 2. An IPsec Tunnel ESP Packet

Figure 2 shows that a new IP header was added at the right, as a result of working with a tunnel, and that an ESP header also was added.

There is a problem when the endpoints (which are sometimes called peers) of the tunnel are behind a NAT (Network Address Translation) device. Using NAT is a method of connecting multiple machines that have an “internal address”, which are not accessible directly to the outside world. These machines access the outside world through a machine that does have an Internet address; the NAT is performed on this machine—usually a gateway.

When the endpoints of the tunnel are behind a NAT, the NAT modifies the contents of the IP packet. As a result, this packet will be rejected by the peer because the signature is wrong. Thus, the IETF issued some RFCs that try to find a solution for this problem. This solution commonly is known as NAT-T or NAT Traversal. NAT-T works by encapsulating IPsec packets in UDP packets, so that these packets will be able to pass through NAT routers without being dropped. RFC 3948, UDP Encapsulation of IPsec ESP Packets, deals with NAT-T (see Resources).

Openswan is an open-source project that provides an implementation of user tools for Linux IPsec. You can create a VPN using Openswan tools (shown in the short example below). The Openswan Project was started in 2003 by former FreeS/WAN developers. FreeS/WAN is the predecessor of Openswan. S/WAN stands for Secure Wide Area Network, which is actually a trademark of RSA. Openswan runs on many different platforms, including x86, x86_64, ia64, MIPS and ARM. It supports kernels 2.0, 2.2, 2.4 and 2.6.

Two IPsec kernel stacks are currently available: KLIPS and NETKEY. The Linux kernel NETKEY code is a rewrite from scratch of the KAME IPsec code. The KAME Project was a group effort of six companies in Japan to provide a free IPv6 and IPsec (for both IPv4 and IPv6) protocol stack implementation for variants of the BSD UNIX computer operating system.

KLIPS is not a part of the Linux kernel. When using KLIPS, you must apply a patch to the kernel to support NAT-T. When using NETKEY, NAT-T support is already inside the kernel, and there is no need to patch the kernel.

When you apply firewall (iptables) rules, KLIPS is the easier case, because with KLIPS, you can identify IPsec traffic, as this traffic goes through ipsecX interfaces. You apply iptables rules to these interfaces in the same way you apply rules to other network interfaces (such as eth0).

When using NETKEY, applying firewall (iptables) rules is much more complex, as the traffic does not flow through ipsecX interfaces; one solution can be marking the packets in the Linux kernel with iptables (with a setmark iptables rule). This mark is a member of the kernel socket buffer structure (struct sk_buff, from the Linux kernel networking code); decryption of the packet does not modify that mark.

Openswan supports Opportunistic Encryption (OE), which enables the creation of IPsec-based VPNs by advertising and fetching public keys from a DNS server.



OpenVPN

OpenVPN is an open-source project founded by James Yonan. It provides a VPN solution based on SSL/TLS. Transport Layer Security (TLS) and its predecessor, Secure Sockets Layer (SSL), are cryptographic protocols that provide secure communications data transfer on the Internet. SSL has been in existence since the early '90s.

The OpenVPN networking model is based on TUN/TAP virtual devices; TUN/TAP is part of the Linux kernel. The first TUN driver in Linux was developed by Maxim Krasnyansky.

OpenVPN installation and configuration is simpler in comparison with IPsec. OpenVPN supports RSA authentication, Diffie-Hellman key agreement, HMAC-SHA1 integrity checks and more. When running in server mode, it supports multiple clients (up tp 128) to connect to a VPN server over the same port. You can set up your own Certificate Authority (CA) and generate certificates and keys for an OpenVPN server and multiple clients.

OpenVPN operates in user-space mode; this makes it easy to port OpenVPN to other operating systems.


Example: Setting Up a VPN Tunnel with IPsec and Openswan

First, download and install the ipsec-tools package and the Openswan package (most distros have these packages).

The VPN tunnel has two participants on its ends, called left and right, and which participant is considered left or right is arbitrary. You have to configure various parameters for these two ends in /etc/ipsec.conf (see man 5 ipsec.conf). The /etc/ipsec.conf file is divided into sections. The conn section contains a connection specification, defining a network connection to be made using IPsec.

An example of a conn section in /etc/ipsec.conf, which defines a tunnel between two nodes on the same LAN, with the left one as 192.168.0.89 and the right one as 192.168.0.92, is as follows:

...
conn linux-to-linux
#
# Simply use raw RSA keys
# After starting openswan, run:
# ipsec showhostkey --left (or --right)
# and fill in the connection similarly
# to the example below.
left=192.168.0.89
leftrsasigkey=0sAQPP...
# The remote user.
#
right=192.168.0.92
rightrsasigkey=0sAQON...
type=tunnel
auto=start
...
You can generate the leftrsasigkey and rightrsasigkey on both participants by running:

ipsec rsasigkey --verbose 2048 > rsa.key
Then, copy and paste the contents of rsa.key into /etc/ipsec.secrets.

In some cases, IPsec clients are roaming clients (with a random IP address). This happens typically when the client is a laptop used from remote locations (such clients are called Roadwarriors). In this case, use the following in ipsec.conf:

right=%any
instead of:

right=ipAddress
The %any keyword is used to specify an unknown IP address.

The type parameter of the connection in this example is tunnel (which is the default). Other types can be transport, signifying host-to-host transport mode; passthrough, signifying that no IPsec processing should be done at all; drop, signifying that packets should be discarded; and reject, signifying that packets should be discarded and a diagnostic ICMP should be returned.

The auto parameter of the connection tells which operation should be done automatically at IPsec startup. For example, auto=start tells it to load and initiate the connection; whereas auto=ignore (which is the default) signifies no automatic startup operation. Other values for the auto parameter can be add, manual or route.

After configuring /etc/ipsec.conf, start the service with:

service ipsec start
You can perform a series of checks to get info about IPsec on your machine by typing ipsec verify. And, output of ipsec verify might look like this:

Checking your system to see if IPsec has installed and started correctly:
Version check and ipsec on-path [OK]
Linux Openswan U2.4.7/K2.6.21-rc7 (netkey)
Checking for IPsec support in kernel [OK]
NETKEY detected, testing for disabled ICMP send_redirects [OK]
NETKEY detected, testing for disabled ICMP accept_redirects [OK]
Checking for RSA private key (/etc/ipsec.d/hostkey.secrets) [OK]
Checking that pluto is running [OK]
Checking for 'ip' command [OK]
Checking for 'iptables' command [OK]
Opportunistic Encryption Support [DISABLED]
You can get information about the tunnel you created by running:

ipsec auto --status
You also can view various low-level IPSec messages in the kernel syslog.

You can test and verify that the packets flowing between the two participants are indeed esp frames by opening an FTP connection (for example), between the two participants and running:

tcpdump -f esp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
You should see something like this:

IP 192.168.0.92 > 192.168.0.89: ESP(spi=0xd514eed9,seq=0x7)
IP 192.168.0.89 > 192.168.0.92: ESP(spi=0x3a1563b9,seq=0x6)
IP 192.168.0.89 > 192.168.0.92: ESP(spi=0x3a1563b9,seq=0x7)
IP 192.168.0.92 > 192.168.0.89: ESP(spi=0xd514eed9,seq=0x8)
Note that the spi (Security Parameter Index) header is the same for all packets; this is an identifier of the connection.

If you need to support NAT traversal, add nat_traversal=yes in ipsec.conf; nat_traversal=no is the default.

The Linux IPsec stack can work with pluto from Openswan, racoon from the KAME Project (which is included in ipsec-tools) or isakmpd from OpenBSD.

Example: Setting Up a VPN Tunnel with OpenVPN

First, download and install the OpenVPN package (most distros have this package).

Then, create a shared key by doing the following:

openvpn --genkey --secret static.key

You can create this key on the server side or the client side, but you should copy this key to the other side in a secured channel (like SSH, for example). This key is exchanged between client and server when the tunnel is created.

This type of shared key is the simplest key; you also can use CA-based keys. The CA can be on a different machine from the OpenVPN server. The OpenVPN HOWTO provides more details on this (see Resources).

Then, create a server configuration file named server.conf:

dev tun
ifconfig 10.0.0.1 10.0.0.2
secret static.key
comp-lzo
On the client side, create the following configuration file named client.conf:

remote serverIpAddressOrHostName
dev tun
ifconfig 10.0.0.2 10.0.0.1
secret static.key
comp-lzo
Note that the order of IP addresses has changed in the client.conf configuration file.

The comp-lzo directive enables compression on the VPN link.

You can set the mtu of the tunnel by adding the tun-mtu directive. When using Ethernet bridging, you should use dev tap instead of dev tun.

The default port for the tunnel is UDP port 1194 (you can verify this by typing netstat -nl | grep 1194 after starting the tunnel).

Before you start the VPN, make sure that the TUN interface (or TAP interface, in case you use Ethernet bridging) is not firewalled.

Start the vpn on the server by running openvpn server.conf and running openvpn client.conf on the client.

You will get an output like this on the client:

OpenVPN 2.1_rc2 x86_64-redhat-linux-gnu [SSL] [LZO2] [EPOLL] built on
Mar 3 2007
IMPORTANT: OpenVPN's default port number is now 1194, based on an official
port number assignment by IANA. OpenVPN 2.0-beta16 and earlier used 5000
as
the default port.
LZO compression initialized
TUN/TAP device tun0 opened
/sbin/ip link set dev tun0 up mtu 1500
/sbin/ip addr add dev tun0 local 10.0.0.2 peer 10.0.0.1
UDPv4 link local (bound): [undef]:1194
UDPv4 link remote: 192.168.0.89:1194
Peer Connection Initiated with 192.168.0.89:1194
Initialization Sequence Completed
You can verify that the tunnel is up by pinging the server from the client (ping 10.0.0.1 from the client).

The TUN interface emulates a PPP (Point-to-Point) network device and the TAP emulates an Ethernet device. A user-space program can open a TUN device and can read or write to it. You can apply iptables rules to a TUN/TAP virtual device in the same way you would do it to an Ethernet device (such as eth0).

IPsec and OpenVPN—a Short Comparison
IPsec is considered the standard for VPN; many vendors (including Cisco, Nortel, CheckPoint and many more) manufacture devices with built-in IPsec functionalities, which enable them to connect to other IPsec clients.

However, we should be a bit cautious here: different manufacturers may implement IPsec in a noncompatible manner on their devices, which can pose a problem.

OpenVPN is not supported currently by most vendors.

IPsec is much more complex than OpenVPN and involves kernel code; this makes porting IPsec to other operating systems a much heavier task. It is much easier to port OpenVPN to other operating systems than IPsec, because OpenVPN runs entirely in user space and is not involved with kernel code.

Both IPsec and OpenVPN use HMAC (Hash Message Authentication Code) to authenticate packets.

OpenVPN is based on using the OpenSSL library; it can run over UDP (which is the default and preferred protocol) or TCP. As opposed to IPsec, which runs in kernel, it runs in user space, so it is heavier than IPsec in terms of performance.

Configuring and applying firewall (iptables) rules in OpenVPN is usually easier than configuring such rules with Openswan in an IPsec-based tunnel.

Acknowledgement
Thanks to Mr Ken Bantoft for his comments.

Resources

OpenVPN: openvpn.net

OpenVPN 2.0 HOWTO: openvpn.net/howto.html

RFC 3948, UDP Encapsulation of IPsec ESP Packets: tools.ietf.org/html/rfc3948

Openswan: www.openswan.org

The KAME Project: www.kame.net

Rami Rosen is a computer science graduate of Technion, the Israel Institute of Technology, located in Haifa. He works as a Linux and Open Solaris kernel programmer for a networking startup, and he can be reached at ramirose@gmail.com. In his spare time, he likes running, solving cryptic puzzles and helping everyone he knows move to this wonderful operating system, Linux.