Centralized Log Management with Rsyslog: A Guide to Remote Log Collection

Why Centralized Logging Matters

In modern IT environments, managing logs across multiple servers is critical for security auditing, troubleshooting, and compliance. Rsyslog (the “rocket-fast” system log processor) enables efficient remote log collection by centralizing logs from multiple sources into a single repository.

Key Rsyslog Concepts

1. Client (Sender): Servers/applications generating logs (e.g., web servers, databases).
2. Server (Receiver): Central log aggregator storing all logs.
3. Protocols:
   • UDP (Port 514): Fast but unreliable (no delivery guarantees).
   • TCP (Port 514): Reliable delivery with connection overhead.
   • RELP (Port 20514): Lossless protocol for critical environments.

Step-by-Step Configuration

1. Server Setup (Log Receiver)

Edit /etc/rsyslog.conf:

# Enable TCP/UDP listening
module(load="imudp")   # UDP module
input(type="imudp" port="514")

module(load="imtcp")   # TCP module
input(type="imtcp" port="514")

# Define template for storing logs by client hostname
template(
  name="RemoteLogs" 
  type="string" 
  string="/var/log/remote/%HOSTNAME%/%PROGRAMNAME%.log"
)

# Apply template to received logs
if $fromhost-ip != '127.0.0.1' then {
  action(type="omfile" dynaFile="RemoteLogs")
}

Restart Rsyslog:

sudo systemctl restart rsyslog

2. Client Setup (Log Sender)

Edit /etc/rsyslog.conf:

# Send all logs to central server via TCP (replace 192.168.1.100)
action(type="omfwd" 
       protocol="tcp" 
       target="192.168.1.100" 
       port="514"
       queue.type="linkedList" 
       queue.filename="forwarding_q"
       action.resumeRetryCount="-1"
)

Restart Rsyslog:

sudo systemctl restart rsyslog

Securing Log Transmission (TLS)

For sensitive environments, encrypt logs with TLS:

1. Generate Certificates (server/client):

   sudo openssl req -x509 -newkey rsa:2048 -keyout /etc/rsyslog/key.pem -out /etc/rsyslog/cert.pem -days 365

2. Server Config:

   module(load="imtcp" StreamDriver.AuthMode="x509/name" StreamDriver.Mode="1")
   input(type="imtcp" port="6514")

3. Client Config:

   action(type="omfwd" protocol="tcp" target="logs.example.com" port="6514"
         StreamDriver="gtls" StreamDriverMode="1" StreamDriverAuthMode="x509/name")

Verification & Troubleshooting

1. Check Server Logs:

   tail -f /var/log/remote/client_hostname/messages.log

2. Test Connectivity:

   # On client
   logger "Test message" && echo "Check server for 'Test message'"

3. Common Issues:
   • Firewall Blocking: Ensure TCP/UDP ports are open.
   • Permissions: Verify /var/log/remote exists and is writable.
   • SELinux: Temporarily disable with setenforce 0 to test.

Advanced Options

• Filtering: Route logs by severity/facility:

  
  if $syslogseverity  **Pro Tip**: Monitor `/var/log/syslog` on both client/server for Rsyslog errors during setup!  

Resources:

• Rsyslog Documentation
• TLS Configuration Guide