/root/IDEA/doc/index.html

The purpose of this manual is to document the installation and configuration of the IDEA program. Since IDEA relies on Snort to obtain the real-time alerts we will have to install snort. IDEA works on a java platform, which allows you to run it on multiple platforms. IDEA currently also has the ability to pull data from various databases to include MySQL and PostgreSQL (will include Oracle and Microsoft SQL server soon). This manual will only show you how to set up IDEA on a Linux machine (more to come later).

2. ACKNOWLEDGEMENTS

Special Thanks Martin Roesch and Chris Green for developing Snort. Your time, efforts and coding skills have made snort one of the most versatile Intrusion Detection System. Thanks to sjscott007 for writing the snort manual. I have used your installation manual as a backbone to this one.

Ian Duffy for taking matters into his own hands and writing the best analysis tool for intrusions.

3. THE CONCEPT OF IDEA

IDEA provides a real-time analysis tool for your networks. IDEA can run in two separate modes: Standalone listener mode and Client-Server mode. In standalone listener mode, the console listens for incoming alerts from Snort sensors on TCP port 1051. Any xml alert that is received from the snort sensors will show up as a new line on the console alerts table in real time. However, when you shut down the IDEA console, all these alerts are gone from the screen. IDEA can however, query various databases to retrieve alerts. In Client-Server mode, the IDEA server listens on port 1051 and acts as a proxy for real-time alerts to IDEA clients. This server will listen for new alerts from Snort sensors and buffer a predefined number of alerts. Any client console that is connected to the server will receive new alerts in real time. Also, when a client registers with the server, it can optionally retrieve old alerts from the server's cache. This way, when you shut down the console, the alerts are still retained on the IDEA server for later review. In addition to buffering the alerts for clients to access, the IDEA server also acts as a chat server so that users using the IDEA-console in client-server mode can collaborate with each other on network events. This collaboration capability is only available in client-server mode. Finally, IDEA now comes with a Web Servlet that can display you alerts stored in the IDEA Server.

4. REQUIRED SOFTWARE

Below is a list of required software to obtain the full functionality of IDEA

The Versions show you what has been tested so far. If you have other configurations, please let me know so I can include them here.

Software	Version	Download Site	Files
RedHat	7.3 or 8.0	www.redhat.com	Either 7.3 or 8.0 distro
Snort	2.0.2	www.snort.org www.superhac.com	snort-2.0.2.tar.gz snortrules-stable.tar.gz snortd
MySQL	3.23	www.mysql.com	MySQL-3.23.X-X.i386.rpm MySQL-client-3.23.X-X.i386.rpm MySQL-shared-3.23.X-X.i386.rpm MySQL-devel-3.23.X-X.i386.rpm
IDEA	1.2.0	sourceforge.net	IDEA-1.2.0-bin.tar.gz
Java SDK	1.4.1.01	java.sun.com	j2sdk-1_4_1_01-linux-i586.bin
Apache	2.0.40	www.apache.org	httpd-2.0.40-i686-pc-linux-gnu-rh73.tar.gz
Tomcat	3.2 and 4.1.18	jakarta.apache.org/tomcat/index.html	tomcat4-4.1.18-full.1jpp.noarch.rpm

Also included here is a table to let you know what software you required based on the functionality that you would like to achieve with IDEA.

	IDEA-Console	IDEA-Server	IDEA-Servlet
Snort	Required	Required	Required
MySQL	Not Required, but nice	Not Required, but nice	Not Required
JAVA SDK	Not Required	Not Required	Required
Apache	Not Required	Not Required	Required
Tomcat	Not Required	Not Required	Required

5. INSTALL AND CONFIGURE MYSQL

5.1. Install MySQL

First, you need to install these various packages

# rpm -ivh MySQL-3.23.X-X.i386.rpm

# rpm -ivh MySQL-client-3.23.X-X.i386.rpm

# rpm -ivh MySQL-devel-3.23.X-X.i386.rpm

# rpm -ivh MySQL-shared-3.23.X-X.i386.rpm

5.2. Configure MySQL

First we need to setup the root password and create the database snort to store the snort alerts

# mysql -u root

mysql> set password for 'root'@'localhost'=password('yourpassword');

mysql> create database snort;

mysql> exit

start mysql on runlevel 2,3,4 and 5

# chkconfig --level 2345 mysql on

# mysql -u root -p

mysql> connect snort

Make the table using the create_mysql script

mysql> source create_mysql

mysql> grant CREATE, INSERT, SELECT, DELETE, UPDATE on snort.* to snort;

mysql> grant CREATE, INSERT, SELECT, DELETE, UPDATE on snort.* to snort@localhost;

mysql> connect mysql

Set the password for the account

mysql> set password for 'snort'@'localhost'=password('yourpassword');

mysql> set password for 'snort'@'%'=password('yourpassword');

mysql> flush privileges;

mysql> exit

Now that MySQL is installed, we need to

6. INSTALL AND CONFIGURE SNORT.

6.1 Install Snort

The first thing that you will need to do in order to install Snort and make it work with IDEA is to apply the spo_xml patch that was so kindly provided by Vigilant Minds. You can download the patch at the snort.org website here.

# cp snort-2.0.*.tar.gz /usr/src/redhat/SOURCES

# cd /usr/src/redhat/SOURCES

# tar -zxvf snort-2.0.*.tar.gz

# cd snort-2.0.2

# cp ../snort_xml_2.0.2/snort_xml_2.0.2.diff .

# patch –p1 < snort_xml_2.0.2.diff

This will apply the necessary code changes to your source tree so that you can get the XML output plugin to work with Snort 2.0.2 (since it was for some reason removed in Snort versions 2.0 and higher). At this point you are ready to build and install snort. From the snort base directory, type the following commands:

At this point, you are ready to build snort. To do so, you will need to configure it according to your needs. For this document, we will install snort with MySQL support. To do this, we need to configure and compile snort with MySQL support

# ./configure --with-mysql=”PATH TO YOUR MYSQL INSTALLATION”

# make

# make install

Now that snort is compiled and installed, we need to download and install the latest rules.

# mkdir /etc/snort

# cp snortrules.tar.gz /etc/snort

# cd /etc/snort

# tar -zxvf snortrules.tar.gz

6.2 Configure Snort

We need to make a directory in /var/log/ for snort

# mkdir /var/log/snort

Now we must edit the snort.conf file

Change the variable RULE_PATH to the absolute path (/etc/snort/rules)

In Section 3, you must configure the following:

Change the line

#output database: log, mysql, user=root password=test dbname=db host=localhost

output database: log, mysql, user=snort password=xxxx dbname=snort host=000.000.000.000 encoding=hex detail=full

where xxxx is the database password and 000.000.000.000 is the IP address for the MySQL database.

change the line

include classification.config

include $RULE_PATH/classification.config

and change the line:

include reference.config

include $RULE_PATH/reference.config

In the xml section, add the following line:

output xml: log, protocol=tcp host=IDEAhost port=1051 encoding=hex detail=full

This allows the alert to be logged by the IDEA database. Replace IDEAhost with the hostname of the system on which you are using IDEA.

That's it, save and close snort.conf.

Now, we create setup the snort daemon to start at bootup.

# cp snortd /etc/rc.d/init.d

# cd /etc/rc.d/init.d

# chmod 755 snortd

# chkconfig --level 2345 snortd on

Now we can start snort and check that it is working:

# service snortd start

# service snortd status

You should see a line like this:

snort (pid xxxx) is running...

Note: If you are running several sensors, we recommend using snortcenter. Snortcenter will allow you to manage multiple snort sensors. I will not cover the installation of Snortcenter here, but you can obtain the documentation from snortcenter.com. There is currently one flaw with snortcenter which does not pass the name of the sensor. I will post a fix here as soon as I figure it out.

Note: Also, be aware that when you first start running snort, you may get a lot of positives...so many that you could create your own Denial of Service attack. Make sure that you monitor the alerts coming from snort and modify your rules appropriately.

Now that Snort and MySQL are running successfully, we get into the...

7. INSTALLATION OF IDEA.

7.1. Install Java

To Install IDEA, you must download and install the Java Runtime Environment. Since the JRE is included in the SDK file which is required to run the Tomcat server, I recommend downloading the sdk file. However, if you do not plan on using the IDEA-Servlet and want to save room on your hard drive, you may download the jre file and follow the same steps. It will work for the IDEA-Console and the IDEA-Server.

For Linux 8.0, download j2sdk-1_4_1_01-linux-i586.bin

Install the sdk.

# cp j2sdk-1_4_1_01-linux-i586.bin /usr/local

# cd /usr/local

# ./j2sdk-1_4_1_01-linux-i586.bin

Read the license agreement and type "yes" at the bottom

Now you have to be able to run "java" in its new location so you have to add this to the path.

For the time being, you need to modify your PATH:

PATH=$PATH:/usr/local/j2sdk1.4.1_01/bin

To make it permanent, add the following line to /etc/profile

pathmunge /usr/local/j2sdk1.4.1_01/bin

7.2. Install IDEA for standalone and server mode

First, you need to choose where to uncompress the IDEA tar file. This is up to your personal preference. You may choose /usr/local/

To uncompess it, type:

# tar -zxvf IDEA1.2.0.tar.gz

now you need to be in that directory

# cd IDEA1.2.0

To use IDEA, we need to create two tables in the snort database for incident management. Ian was kind enough to write a script to make the creation of these tables painless...

# cat INC_TABLES_MYSQL.txt | mysql dbname -u username -p

replace dbname with the name of the database containing your snort alerts and username with a user that has access to the snort dbase (typically root or snort)

7.3. Run IDEA is standalone mode

Go to the directory where you uncompressed IDEA and...

# java -jar IDEA-Console.jar

By default, IDEA starts in standalone listener mode.

You first must add the sensors that will be allowed to send xml alerts to the IDEA console. Follow these steps to do so.

Click on FILE

Click on APPLICATION SETTINGS

Click on the SENSORS Tab

Click ADD

Fill in the Blanks to add your sensor, and don't forget to click "Accept alerts from this sensor", otherwise your console will ignore alerts received from that sensor.

To enable the console to retrieve alerts from MySQL, click on RTA settings, and make sure to enter the IP address of the MySQL server that you are running.

You can also add a map and physically located your sensors on the map. Ian has included 2 maps with the standard package that you can use. You can also create your own or use any other map you would like. The maps are created from standard JPEG or GIF files; to create a map, perform the following steps:

Click on VISUALIZATION

Click on 2D SENSORMAP

You will be prompted whether you want to create a new map from a file, or to use an existing map. Existing maps are maps that you have saved previously (they will contain geolocation data for your sensors). If you have never used the SensorMap before, you will want to create a new map from a file. A file selection dialog opens; choose the JPEG or GIF image which you would like to use as your map background.

The rest of the software is very user-friendly and if you came this far should have no problem figuring it out.

7.4. Run IDEA in server mode

In order to run the server, you must have 3 files in the same directory on the system on which you want to run the server:

1. IDEA-Server.jar -- the command line executable for the IDEA server

2. crimson.jar -- xml parser from Apache's Jakarta project

3. ServerUsers.cfg -- a config file that specifies who can connect to the server and from which hosts

7.4.1. Starting the CORBA ORB listener

Before you can start the IDEA server, you must have a CORBA ORB listening on TCP port 1050. In order to do this, you can use the CORBA ORB that is supplied by Sun Microsystems with its Java Runtime Environments. This ORB is found in the "bin" directory of your java installation, it's executable name is "tnameserv". Note that you can specify any port that is available; I simply chose to use 1050 for the TCP port, but you can use whichever port you feel is appropriate. To start the orb listening on TCP:1050, execute the following command:

# tnameserv -ORBInitialPort 1050 &

You will see some output telling you that the Transient Name Server has set the port for initial object references to 1050. This is how the console locates and defines remote objects on the server.

7.4.2. Setting up the ServerUsers.cfg file

Once the ORB has been started, you must set up the ServerUsers.cfg file. To do this, you must create a colon delimited ASCII text file (similar to the /etc/password file), with a list of users and hosts. General syntax for this file is as follows:

username:password:GECOS:permissions

username is the name of the user which will be granted access to the IDEA server

password is the users plaintext password (I know, I know, this will be fixed in the future)

GECOS is a descriptive text field describing this user

permissions is a comma delimited permissions list. Permissions are represented by a single character, and can be any of the following characters:

m – this grants the user permission to send messages to other users via the IDEA-Server’s messaging subsystem

c – this grants the user permission to connect to the IDEA-Server. This is a pre-requisite to all other permissions. Without c, your user won’t be able to do much!

r – this grants the user permission to retrieve alerts from the IDEA-Server. Without r, the user can only receive new alerts from the IDEA-Server.

d – this grants the user permission to delete alerts from the IDEA-Server. Use with caution – you’ve been warned.

That’s about it.

7.5. Starting the IDEA-Server

Once tnameserv is running and the access permissions have been set up, you can execute the server. Do this by typing

# java -jar IDEA-Server.jar 10 200 20 &

There are 3 parameters that you must specify to the IDEA server:

1. The maximum number of clients that can connect to the server (in this case, only 10 clients are allowed to connect to the server at any one time)

2. The maximum number of alerts to keep in cache (in this case, 200 alerts maximum will be cached -- after 200 alerts are received, the oldest alerts are discarded)

3. The keepalive interval -- this is the interval at which the server checks all registered clients to make sure that they are still "alive". If they are not, then they get "deregistered" from the server. This prevents server deadlock from hung or non-responsive clients and inadvertent denial of service.

These 3 parameters must be specified, in order, as the first 3 parameters to the console.

7.5.1. Optional Parameters

Optional parameters to the server are:

-f <ip address>: This switch specifies a forwarding host. When the server is run with this switch, any alerts that it receives will also be forwarded to another server (listening on TCP:1051). This can be useful in creating Distributed IDS hierarchies. Use caution to avoid having 2 servers forward to each other as this will cause a “forwarding loop”. Code is not yet implemented to avoid this scenario.

-o: This switch specifies that when forwarding alerts, the IDEA server should overwrite the name of the sensor in the alert with the name of the IDEA server. This can be useful in a DIDS hierarchy for determining where an alert is coming from, or for consolidation of a bunch of related sensors (and their output) to appear to be from one sensor.

-l <name>: When the -o switch is used, the -l switch can be used to specify a different name for the IDEA server. Otherwise, the DNS hostname of the IDEA server will be used.

-L <filename>: This specifies a filename to which to log server messages. If specified, all output messages from the server will be written to this log file. The default is to simply write the messages to the screen (standard out).

7.6. Configuration of the console to connect to the server

Once you have the server up and running, it is a simple matter to make the console work with the IDEA server. Use the following steps to connect to the server:

1. Start up the console as described previously in this document

2. Select File->Application Settings

3. On the top of the Application Settings window, you should see two checkboxes, one that says "Connect to RTA Server" and one that says "Use standalone listener". Click on the box that says "Use RTA Server".

4. In the boxes below, type in the IP address of the system on which the server is running, and type in 1050 for the port number (or whatever port number you decided to start the CORBA ORB tnameserv running on).

5. If you would like to retrieve alerts from the IDEA server's cache when the console starts up, the click on "Retrieve Cached Alerts on Startup", and specify the maximum number of alerts that you would like to retrieve from the servers buffer. Note that if you specify a number that is greater than the number of alerts that the server is caching, you will only receive the number of alerts that the server has in its cache.

6. Click on the "OK" button

7. Go to File->IDEA Server->Connect. If you have not entered your username and password, do so when prompted, and click “OK”. You should see a notification that you have successfully connected to the IDEA Server.

7.7 Install and setup the IDEA Web Servlet

7.7.1. Install Apache

I will eventually walk you through the installation of apache, however, if install it during your initial redhat installation, there will be nothing to do here.

7.7.2. Install Tomcat

Installing Tomcat is rather painless. Hopefully you have already downloaded the latest version of tomcat from http://jakarta.apache.org/tomcat/ From this point type:

# rpm -ivh tomcat4-4.1.18-full.1jpp.noarch.rpm

and you're done. Once that is done, pick a folder where you would like to untar the file. I picked /usr/local

# cp tomcat-4.1.18-LE-jdk14.tar.gz /usr/local

# cd /usr/local

# tar -zxvf tomcat-4.1.18-LE-jdk14.tar.gz

Now that you've uncompressed the file, go to the newly created directory

# cd /usr/local/jakarta-tomcat-4.1.18-LE-jdk14

You can navigate around the folders in there. Of importance are conf and bin

In conf, you will find a tomcat-users.xml file.

You will need to edit this file and add 2 roles: admin and manager

Once you've added these roles, you will have to add one user (or two) with these roles. As I've said before, tomcat requires the sdk package for java. This is why I recommended downloading the sdk package instead of the jre pack. Tomcat will require the JAVA_HOME variable to be set. For now, type:

# JAVA_HOME=/usr/local/j2sdk1.4.1_01 (assuming this is where you installed the java sdk package.

# export JAVA_HOME

To make this variable permanent, I recommend you edit your /etc/profile directory and type the same commands as above.

Once that's done, go to the bin directory and type

# ./startup.sh

This will startup the tomcat server. To shut down the tomcat server, type ./shutdown.sh

8. TESTED CONFIGURATIONS

IDEA has been successfully tested with RedHat 7.3 and 8.0, Snort 2.0.2 with xmlpatch_2.0.2, and IDEA 1.2.0. Some software that has worked with IDEA:

Apache 2.0.40

Tomcat versions 3.2 & 4

MySQL 3.23.X

9. LICENSE INFORMATION

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.

10. CONCLUSION

We hope that you find IDEA useful for your needs. IDEA is currently in its development stage and new features are added constantly. Check sourceforge.net often to obtain the latest version of IDEA and this manual. This manual can also be obtained at http://www.bedalis.net/IDEA/ .Questions can be addressed to bedalis@users.sourceforge.net. If you are interesting in becoming part of this project, you can contact me at the same e-mail address.