An organisation without an incident handling team (IHT) can be compared to a city without a fire brigade. The result can be widespread damage, with normal citizens finding themselves having to tackle fire fighting tasks that they were never trained to perform.
This is the reason why larger companies recruit dedicated incident handlers. These teams may be as small as just two people, but will still be able to handle all critical incidents, and have a positive stabilising effect on the organisation as a whole.
Unfortunately, it is becoming increasingly difficult to find experienced incident handling people on the market today, and those that are available are very experienced and very expensive. So, are more cost-effective routes available for organisations seeking to assemble their own efficient incident handling teams?
One solution would be to hire cheap trainees, but provide them with high-quality equipment with which to handle incidents. The problem with this approach, even with training, is that it could take a very long time before the rookies would have enough experience of handling incidents to become effective.
Training to use tools
I once heard a story about a very enthusiastic security manager who had figured out his own way to train the IHT. He had no experienced incident handling people on his staff at all, but he did have a good set of basic security protection tools, such as firewalls, VPNs and IPS devices.
One day he used a simple directory traversal attack against his own company's test web server. He launched the attack from the internet, and then waited to see if his staff would detect his actions. They didn't of course, because they were not in the habit of looking at IPS alerts and logs.
So the security manager told his incident management staff what he had done, and asked them to uncover evidence of his attacks. They eventually found the evidence in the IPS alerts and logs. So they created an incident case, collected all the information they could find, and finally printed a report for the security manager.
In this way, the incident management staff learned to use their equipment and were able to store the data for future use. They also learned not just to watch production systems, but also to secure test systems, which have often been used as stepping-stones to attack live production systems.
However, the security manager was not satisfied yet. He knew his staff could now detect intrusions from the internet, but wondered what would happen if an intrusion came from the DMZ area. He had noticed that the web server in the DMZ had a direct connection through the firewall to the internal domain name server. He also discovered that the firewall was not configured correctly, and allowed the web server to access any internal server using port 53 (the domain name port). He used all of this information and found himself a remote connection from the DMZ to one of the internal workstations.
The security manager went back to his incident management team and asked if it had discovered anything abnormal. This time his team reported that it had seen some weird activity from the web server in the DMZ, which had tried to resolve names from several internal servers. The security manager complimented his team for finding this information, but instructed it to look deeper into the traffic of that web server.
Finally, his team noticed from traffic captures that the web server was not sending name resolution queries to internal servers at all, but running remote desktop management. So they restricted the web server's access to the internal name server only. This meant the team would get an alert if the web server tried to send anything other than name server queries through port 53.
The security manager then went one step further and made a final attack from the internal network. This time the attack did not go through any firewalls, but it did go through an IPS that was protecting the internal web server. In just a couple of minutes he received a phone call from his now wary incident management team informing him that IP-address 1.2.3.4 had tried to attack the internal web server. The security manager opened the incident management window of the security system, and was able to follow in real time exactly how his team tracked the attack to his laptop. Now he was satisfied that his team knew how to use its security and audit systems effectively. These skills would also help his organisation claim compliance to different regulatory demands like the Payment Card Industry standard (PCI) or Sarbanes-Oxley.
This short example illustrates that it does not take a great deal of effort to train an incident management team, if it has good tools at its disposal.
Klaus Majewski is product specialist manager at Stonesoft, www.stonesoft.com
