As the state of the art in operating systems (OS) continues to advance, an unnerving trend has emerged: vulnerabilities in tightly integrated operating systems. How do you address this? With an effective combination of educated staff, proper procedures, and technology.

Rather than being a collection of separate utilities and daemons, the modern OS is moving toward a highly integrated system with numerous dependencies. As a result, the core of the OS is more easily exposed to a broader range of vulnerabilities. While Linux still largely a collection of separate components, Microsoft Windows is at the forefront of this design principle and, in fact, is moving to an even more tightly integrated system. The risks can become significant. Whenever a vulnerability is found in one of the core components of a tightly integrated OS, interdependent components are vulnerable as a result. Developing an appropriate approach to protecting systems with tightly integrated OSs is the key to maintaining a secure and safe network environment.

The rationale for a tightly integrated operating system is sound - reduced development costs and effort, a reduction in portability issues, and fewer components to break. The flip side is unprecedented exposure to vulnerabilities. In the past, when a single system component had a vulnerability the impact was isolated to that single component. However, due to the dependencies introduced by extensive integration, that one component may now impact multiple applications. It is this chain of dependencies that presents enormous risk.

A Practical Approach to Isolating the Exposures

A number of approaches exist for isolating - or at least reducing - your exposure in cases such as these. For the purposes of this article, the assumption is that it is impossible to catch every security flaw during development and that organizations will need to take measures to protect themselves until patches or upgrades are available that solve the security flaw.

The simplest approach to dealing with exploits aimed at integrated OSs is to turn off any services not required or restrict access to those services via network firewalls or network intrusion prevention systems (IPSs). Turning off a service entirely is rarely a practical option for Web servers or file servers. In the specific case of a Web server, doing so would certainly solve the problem, but then you wouldn't have a Web server!

A layered approach consisting of the following primary components is the most practical solution:

1. Education of your network and system administrators
2. A baseline of the current state of your network
3. Proper configuration of the host operating system, including current patches and service packs
4. Proper configuration of the network service being hosted
5. A generic network firewall to allow only specific traffic in and out
6. An IPS to cover the bases left open by the network firewall
7. An on-board firewall for each device (IPtables in Linux, TCP Filters in Windows)
8. In the case of a Linux system, a chrooted environment for each available network service, and optionally physical separation from the internal network

Having an educated and security-conscious staff is the most important of these options. The ability to recognize, understand, and correct a potential security exposure or configuration error is much more valuable than any technology solution. Your staff is truly the first line of defense.

Knowledge is power! Knowing your current exposures and configuration issues should be on your short list, regardless of how far into this process you may go. Rectifying the issues found should be the immediate next step - directly followed by another baseline to once again ascertain any new issues. Automated vulnerability management tools can help make this process straightforward and manageable.

Current shipping distributions of Linux as well as current shipping versions of Windows still contain many services that are not useful or appropriate for a device that will host publicly accessible network services. You should identify and disable these services before the device is ever connected to any network. Linux is able to fully function with far fewer resources than Windows, and you should take advantage of this. If the first step (i.e., a well-educated staff) was successful, your administrators will be able to identify which services to safely disable.

The network service itself, for example a Web server, should also be properly configured. No prepackaged examples or documents should be present anywhere within the document root, nor should any of this data be accessible by anyone over the network. For example, many exploits exist that rely upon these stock examples being installed in a default installation of the Microsoft IIS Web server.

Every network that is to be interconnected with any other network should have a firewall at the gateway. The firewall should be configured to only allow specific traffic both into and out of the network. Nearly every firewall controls inbound traffic, but few are configured to also control outbound traffic. For example, should an internal system ever be infected with a worm (as has happened both with Linux and Windows), the outbound controls will hopefully limit the impact and propagation of the worm.

An intrusion prevention system (IPS) is a great tool to fill in the cracks that a firewall leaves open. As most firewalls do not normally perform any type of content inspection (or very limited if they do), the allowed traffic is by no means assured to be free of malicious content or exploits. This is where an IPS really shines - the ability to inspect all traffic for attacks. Most IPS products also allow the traffic to be blocked, hence the prevention in intrusion prevention system. The value of an IPS is often discounted or misunderstood, yet for those in the know, an IPS represents a 24/7 partner that never stops preventing the malicious traffic from entering your network.

An on-board firewall is a critical component that will shield your organization from the inevitable configuration error. By restricting which types of network traffic may be passed into and out of each endpoint, you greatly reduce your exposure. Windows and Linux have this capability. Most Linux distributions use this out of the box; however, Windows must be configured after the fact to leverage this capability, although Service Pack 2 for Windows XP will change that.

Chrooted environments are an extremely effective means to isolate processes on a Linux system. Linux has native support for chrooted environments and most distributions ship with tools out of the box that will allow you to do this for nearly any network service (or any process for that matter!). Unfortunately, Windows has no good way to implement a chrooted environment. A somewhat feasible option for Windows includes running VMware, but the resources required are often too much, making this impractical. The primary benefit of a chrooted environment is the logical separation: if a process or application is exploited, the damage is limited to the chrooted environment, significantly reducing the impact to the rest of the system. How-to's exist for popular Linux network services and a quick search on Google will find those.

Conclusion

The rate of exploit attempts and network worms is rising and will continue to rise. The attack vectors are continually increasing in their sophistication, and attacks are becoming much more difficult to prevent or even contain. Both Linux and Windows can be made insecure in a network environment - and both can also be made secure enough to be safe. Regardless of your chosen platform, the most important tool available to you is an effective combination of your staff, proper procedures, and technology.