Syslog Collector Manager

Introduction

The goal of the Collector architecture designed by LUTEUS is to provide a scalable solution for handling huge quantities of management information and to help the administrator to classify, analyze and process them.

The product has been designed based on the following issues:

The main issue that system and network managers have to face is the collection and processing of huge quantities of management information.
They need to have detailed information of what is happening in their system and network devices by turning on in-depth logging and alarm facilities.
They want to receive the critical information immediately but also want to retrieve non-critical ones at a later time.
They want to have a centralized system that can collect information files without using too much network bandwidth.
They want to manage their infrastructure from a centralized manager.
They want to have access security and control of the management solution.

The concept of Collector in the LoriotPro management system architecture is based on two components:

The Collector agent software running on Microsoft Windows workstations. These agents are dedicated to the collection of the supervision information sent by the System and Network devices located in a predefined area.
The Collector Manager software running on top of our LoriotPro supervision software. This manager has to manage the agents avoiding any human intervention directly on the agent.

Syslog messages are collected by agents called, in our terminology, “Syslog Collector Agents.” Agents are designed to collect a large throughput of Syslog messages and to process them according to advanced filtering rules. Filtered messages can then be displayed on a viewer, the agent taking on the role of a simple Syslog server. Messages can be stored locally in files or forwarded to the central management system. Critical messages can be sent to the centralized management system either as LoriotPro proprietary-formatted event messages or as Syslog-formatted messages. Agents can be cascaded to build a hierarchical architecture of Syslog message relays.

Agents can be used as a standalone solution and act as a Syslog server or Syslog relay. Our LoriotPro Network Management System (NMS) and the Syslog Manager are not necessary in this case. Filtering rules can be defined from the Agent GUI and applied. Actions taken on conditions defined in the filtering rules can be displayed in a viewer, stored in files or forwarded to another Syslog server.

The Syslog Collector Manager is responsible for the management of the agents from a centralized position. Filtering rules are defined on the manager and pushed to the agent. The manager is also able to retrieve a filter rule previously loaded onto an agent. Filtering rules are stored in local text-only files. The manager is also able to upload Syslog files previously stored on the agent.

The Syslog files can be compressed on the fly during uploading, sparing precious bandwidth of WAN links or on-demand links. The manager works on top of our LoriotPro NMS as a Plug-In Service. As we have stated previously, the messages sent by the Syslog Collector Agents can be in the LoriotPro event format. The LoriotPro Event Manager receives them and processes them. They are first displayed in the Event Log window and if necessary, they trigger actions based on predefined conditions. Actions can send messages, start programs, play sounds, etc.

Launch Syslog Manager Plugin from LoriotPro

To start the Syslog Manager plugin select the Service Tab of LoriotPro and click the right button to open the contextual menu.

In the list choose the Syslog Manager service plug-in.

Using the Manager

Declare Syslog Collector agents

Use the Edit Agent List button.

edit agent list syslog
Edit Agent List button

The CollectorSyslogManagerLicence.ini is in text format and can be modified using Notepad.

syslog license
Edit Agents with the Notepad utility

For each agent, append a line containing the following information:

1) Agent name

2) Agent IP address

3) License key for this agent (the same key is set on the agent side)

4) The TCP listening port for this agent

5) The password for this agent (the same password is set on the agent side)

C If the password includes a space, the password should be specified between quotes.

Example: “agent italien” 182.2.3.4 101101 5003 « admin secret »

Fields	Parameters
Agent name	“agent italien”
Agent IP address	182.2.3.4
License key for this agent	101101
The TCP listening port of the agent	5003
The password for this agent	“admin secret”

Warning: The manager will not work if two agents own the same license number.

During the evaluation period, you can change the license number set by default in the CollectorSyslogAgentLicence.ini file located in the bin/collector/Syslog directory of the agent and set the same number on the Manager side.

Example:

Agent 1

IP Adresse : 192.168.1.1

Port TCP : 5003

Password : admin

CollectorSyslogAgent.ini

[ALARM]

syslogd_port 514

max_log_view_lines 50

collector_mode 0

hide_log_view 0

loriotpro_ip_add 193.1.1.1

loriotpro_event_send 16001

Loriotpro_event_port 5001

collector_tcp_manager_server_ip 193.1.1.1

collector_tcp_manager_server_port 5002

collector_tcp_agent_server_port 5003

collector_tcp_agent_server_timeout 5000

collector_tcp_server_password "admin"

CollectorSyslogAgentLicence.ini

30 days Evaluation

10001

AAAA-AAAA-AAAA-AAAAA

Agent 2

IP Adresse : 194.169.1.2

Port TCP : 5003

Password : admin

CollectorSyslogAgent.ini

[ALARM]

syslogd_port 514

max_log_view_lines 50

collector_mode 0

hide_log_view 0

loriotpro_ip_add 193.1.1.1

loriotpro_event_send 16001

Loriotpro_event_port 5001

collector_tcp_manager_server_ip 193.1.1.1

collector_tcp_manager_server_port 5002

collector_tcp_agent_server_port 5003

collector_tcp_agent_server_timeout 5000

collector_tcp_server_password "admin"

CollectorSyslogAgentLicence.ini Agent 2

30 days Evaluation

10002

AAAA-AAAA-AAAA-AAAAA

Manager

IP Adresse : 193.1.1.1

PORT TCP : 5002

CollectorSyslogManagerLicence.ini

# For each agent, Add a line with, Agent Name, Agent IP address, Agent License_ID, Agent password

# You will find the License_ID for this agent on line two of the CollectorSyslogAgentLicence.ini file,

# located on your agent. Each agent should have a unique license

# SyslogConnectorAgent_name SyslogConnectorAgent_ip_addr license_id server_port password

LocalAgent 127.0.0.1 1000 5003 admin

Agent1 192.168.1.1 10001 5003 admin

Agent2 194.169.1.2 10002 5003 admin

When done, if you use the combo box of the Manager you should see the three agents.

syslog agent
Combo box

Select one agent from the list and click the Get Filter button. If everything is configured properly the filter list of the agent appears in the Manager’s filter list editor.

The message « Configuration File Receive OK » should appears :

syslog conf
Result of a Get Filters operation on the agent “local agent.”

If the agent does not answer your request:

Ø Verify your configuration parameters.

Ø Do trace route or a ping to the agent to check that it is not a connectivity issue.

Ø Do a telnet ipadd_agent :TCP_Port, if the connection is established and stop the manager to check your agent configuration (password and License number).

If a firewall is located between you and the agent add the following rules to it.

Source	Port	Destination	Port	Protocol	Action
Agent	>1023	Manager	5002	TCP	Permit
Agent	>1023	Manager	5001	UDP	Permit
Agent	>1023	Manager	514	UDP	Permit
Manager	>1023	Agent	5003	TCP	Permit

Agent remote control

Control

Explanation

Selection and setting of the current agent.

Uploads the filter list of the selected agent in the Filter List Editor window.

Sends to the selected agent the filter list currently displayed in the manager editor and applies it to the agent filter process.

Note:

Filters are immediately applied to the agent but are not saved in the agent default filter file.

The agent answers with an acknowledge message: “Agent Filters Send OK (delete tmp file).”

destroy filter syslog

Note:

If you ask for the agent’s current status, it should notify you that the current applied filter list is not saved.

syslog status

syslog config

Sends the Save command to the agent. The agent saves its current filter list into the default filter list file.

The agent status is returned .

syslog configuration

Allows you to read the list of the current log files stored on the agent and to download if needed the selected files to your LoriotPro system. Select the file to download and click the Get Selected File button.

syslog files

The list includes csv and gz file formats. The gz file format is archived, compressed csv files. The GZip remote file before download checkbox allows you to force the agent to compress the file before download. The compression ratio s approximately 15.

Warning: The interface allows you to download one file at once.

The dialog box displays asking you to specify the local directory where the log file has to be saved.

syslog

The Download process progression bar is displayed. You can cancel the transfer using the Cancel button.

syslog request

Note:

During the transfer, the LoriotPro software is totally operational for other tasks.

Once the transfer is done, the manager offers to display the Download file.

syslog confirm

Note:

If you use the compression option, the manager software waits 60 seconds before starting the download. If this time is not enough for the agent to compress the required file, the download is cancelled. However, the agent still works on the compression of the file. The next time you open the list of remote log files you will see the new file in GZ format.

syslog get status

The agent can be managed remotely with a set of commands in this combo box.

Control	Function
Hide	Hides the agent GUI on the remote PC.
List	Retrieves and displays the list of log files.
Save	Saves the agent’s current applied filter list in the default filter list file on the agent.
Show	Unhides the GUI on the remote agent.
Start	Restarts the Syslog server daemon of the agent.
Status	Retrieves statistics of the agent.
Stop	Stops the Syslog server daemon of the agent.

Manager settings

The Manager parameters are located in the CollectorSyslogManager.ini file in the /bin directory of LoriotPro. These parameters are similar to those used on the agent.

CollectorSyslogManager.ini

[ALARM]

Loriot_event_port 5001

loriot_ip_add 127.0.0.1

loriot_event_send 16001

collector_mode 1

collector_tcp_manager_server_port 5002

collector_tcp_server_password "admin"

Parameters are loaded when the Syslog Manager plug-in starts and cannot be modified dynamically. However, it is possible to modify the manager port from the manager GUI.

syslog tcp port

If you change the value you should click the Reset Manager Server button to apply this setting.

Warning: If you change this parameter all agents must be reconfigured. Agents should be stopped, the CollectorSyslogAgent.ini has to be modified and agents restarted.

Managing filter list files

It is possible to save your filter list of each agent on the Manager using the Open Filters or Save Filters buttons.

Example:

You select a local filter list file, located on your local hard disk, edit it and push it to the agent.

Local filter list file selection

A window informs you that the current filter list present in the Manager Editor will be cleared.

The new filter list is loaded in the editor window.

New filter list loaded in the editor window

The next step is to select the agent destination as shown in the screen capture below.

Agent Selection

Then apply Agent selection the filter list by clicking the Send/Apply button.

New filter list is applied

Acknowledgment by the agent of receipt of filters

If you look at the Filter Management status bar you see that the filter list is applied.

The agent has received the new filter list

The Status option is another way of checking that the filter file has been received and applied.

The status option

Display of agent statistics

The Save button forces the agent to save the filter list in its default file.

Perform a save of the current agent filter list

The agent confirms the save operation.

The active filters are saved

Manager Filter List Editor

The Manager Filter List Editor has the same capabilities as the Agent Filter List Editor explained in the previous sections. Refer to the section “Agent Filter List Editor” for operations.

The Manager has an extended feature that allows it to use a text file called connectorsyslog-msg.txt located in the LoriotPro /bin Directory.

This file contains predefined character strings allowing you to search for message strings easily. By default this file contains the set of messages sent by Cisco Pix Firewalls.

The syntax of this file is:

Reference : comments

The colon “:” is used to separate the string from the comment.

To use it , simply click in the string field of the editor. A dialog box appears with available strings.

You can manually edit the proposed examples.

The result is set in the filter rule.