Basics of syslog-ng
Synopsis
Syslog-ng is a versatile and powerful logging tool that allows system administrators to collect, filter, and forward log messages from various sources to multiple destinations. It is an enhanced version of the traditional syslog protocol, providing additional features such as support for structured logging, message parsing, and integration with various external tools.
What is syslog-ng?
Syslog-ng is an open-source log management solution that extends the capabilities of the standard syslog protocol. It enables the collection, processing, and routing of log messages from a wide range of sources, offering advanced filtering, parsing, and formatting options. syslog-ng supports various protocols and data formats, making it a flexible choice for managing logs in complex IT environments.
Instructions: Installing and Configuring syslog-ng
Step 1 : Install syslog-ng on Ubuntu
sudo apt update
sudo apt install syslog-ng
Step 2: Basic Configuration
Once installed, the primary configuration file for syslog-ng is located at
/etc/syslog-ng/syslog-ng.conf. Here’s a basic configuration:
2.1 Check the version of syslog-ng, and then open the configuration file. You’ll need to know the version number to ensure your configuration is correct:
syslog-ng --version
sudo nano /etc/syslog-ng/syslog-ng.conf
2.2 Add the following lines to the the config file.
At the top of the config file the version should match your version of syslog-ng. In this case since we’re using version 3.35.1, we’ll change the top section to:
@version: 3.35
Toward the end of the config file you’ll add the following lines.
# Enable syslog-ng to receive logs from remote hosts
source s_network {
udp(port(514));
tcp(port(514));
};
# Define a destination for logs from remote hosts
destination d_remote {
file("/var/log/syslog-ng/remote/${HOST}.log");
};
# Define a log path for remote logs
log {
source(s_network);
destination(d_remote);
};
2.3 Save and close the file. Using nano, Ctrl + x
2.3 Add a new directory, change the ownership, and permissions.
sudo mkdir -p /var/log/syslog-ng/remote
sudo chown -R syslog:syslog /var/log/syslog-ng
sudo chmod -R 755 /var/log/syslog-ng
Step 3: Starting and Enabling syslog-ng:
sudo systemctl restart syslog-ng
sudo systemctl enable syslog-ng
Step 4: Verify the Configuration
To ensure that syslog-ng is running, you can check the status:
sudo systemctl status syslog-ng
Step 5: Test the changes to ensure you’re receiving logs.
Use the following command to create a log from the local host.
logger "Test from localhost"
Use the following commands to check for that log from the local host.
cd /var/log/
sudo grep -i "Test from localhost" syslog
Use the following command to create a log from a remote host
logger -n server_ip -P 514 "Test message from remote host"
Use the following command to check for that log from the remote host.
cd /var/log/syslog-ng/remote/
sudo grep -i "Test message from remote host" remote_ip.log
Use Case: Centralized Log Management
One of the primary use cases for syslog-ng is centralized log management. This involves collecting log messages from multiple servers and network devices and forwarding them to a central log server for processing and storage. This centralized approach simplifies monitoring, enhances security, and aids in compliance with regulatory requirements.
Example for Centralized Log Management
1. On the Client machine
Edit the syslog-ng configuration file to forward logs to the central server:
Note: Follow step 1
sudo nano /etc/syslog-ng/syslog-ng.conf
Note: Follow steps 2.2 to ensure your config file has the correct version. Also, add the lines below to the same modified section in the config file shown in 2.2.
# Define the destination for remote logging
destination d_remote {
tcp("server_ip" port(514));
};
# Log path to send all logs to the remote server
log {
source(s_src); # Ensure this matches your existing source
destination(d_remote);
};
sudo systemctl restart syslog-ng
sudo systemctl enable syslog-ng
2. On the Central log server:
Edit the syslog-ng configuration file to receive and store logs from clients:
Note: Same steps from 2.2
sudo nano /etc/syslog-ng/syslog-ng.conf
# Enable syslog-ng to receive logs from remote hosts
source s_network {
udp(port(514));
tcp(port(514));
};
# Define a destination for logs from remote hosts
destination d_remote {
file("/var/log/syslog-ng/remote/${HOST}.log");
};
# Define a log path for remote logs
log {
source(s_network);
destination(d_remote);
};
sudo systemctl restart syslog-ng
sudo systemctl enable syslog-ng
Conclusion
syslog-ng is a robust and flexible tool for log management, offering advanced features for collecting, filtering, and forwarding log messages. By implementing syslog-ng, organizations can enhance their log management capabilities, achieve better visibility into their IT infrastructure, and ensure compliance with security and regulatory requirements. Whether for centralized log management, real-time monitoring, or integration with other IT systems, syslog-ng proves to be an invaluable asset for system administrators.
Visual Walkthrough
Check out this video for a walkthrough Basics of syslog-ng
References
For more advanced configurations and use cases, refer to the official syslog-ng documentation.