You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 26 Next »

Topics Discussed

You can add a remote log collector to the following remote devices (log sources):

Create a remote log source (Apache HTTP Server for Ubuntu 14)


To obtain Log Relay and to configure your account for remote log collection, you must have the following AMP permissions added to your account: 

  • Write Virtual Machine
  • Delete Log Management
  • Read Log Endpoints
  • Read Log Relays
  • Write Log Relays
  • Delete Log Relays

You can use this document to add a remote log collector to an Apache HTTP Server remote device (log source). 


Pre-deployment considerations

For remote log collection, you must have Log Relay added to your account.  


Create A Remote Log Source

  1. In the Armor Management Portal (AMP), in the left-side navigation, click Security.
  2. Click External Sources.
  3. Click Log Relay Source.
  4. Click the plus ( + ) sign. 
    • If you do not have any log sources already created, then click Add a New Log Source
  5. Complete the missing fields:
    1. In Endpoint, select the available Armor Endpoint.
    2. In Log Source Type, select Apache HTTP Server
    3. In Hostname, enter the system hostname that matches the system for syslog collection. For example, in Mar 10 08:52:55 node-77 systemd: < redacted >, the hostname would be node-77.

      1. The hostname is case-sensitive and must match the same letter casing as the logs that are sent into this log source.
    4. In Protocol, select TLS Syslog
  6. Click Save Log Source.


Implement Server

Before you begin, ensure that the following pages are installed:

  • rsyslog-gnutls
    • apt-get install -y rsyslog-gnutls
  1. Create a directory to hold Armor pem file: 
    • mkdir -pv /etc/rsyslog.d/keys/ca.d
  2. Change to the newly created directory: 
    • cd /etc/rsyslog.d/keys/ca.d
  3. Download Armor PEM files: 
  4. Create a file called /etc/rsyslog.d/54-nginx.conf with the template below. 
    • Replace access-log with FULL path of access log.
    • Repeat for each access log needed.
    • If more than one access log file is defined, then ensure that InputFileStateFile name is unique per log file ###

      #########################
      $ModLoad imfile
      
      
      # access log
      $InputFileName access-log
      $InputFileTag httpd:
      $InputFileStateFile stat-nginx-access
      $InputFileSeverity info
      $InputFilePollInterval 1
      $InputRunFileMonitor
      #########################
  5. Create a file called /etc/rsyslog.d/55-armor.conf with the template below. 
    • Replace target-name:port with the name of the configured endpoint and port.

      #########################
      #RsyslogGnuTLS
      
      global(
        # certificate files
        defaultNetstreamDriverCAFile="/etc/rsyslog.d/keys/ca.d/logs.armor.com.pem"
      )
      
      template(
        name="RFC3164Template"
        type="string"
        string="<%PRI%>%TIMESTAMP:::date-rfc3164% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n"
      )
      
      # make gtls driver the default
      $DefaultNetstreamDriver gtls
      
      # do not validate peer
      # if set to anon then $ActionSendStreamDriverPermittedPeer must not be set
      #$ActionSendStreamDriverAuthMode anon
      
      # run driver in TLS-only mode
      $ActionSendStreamDriverMode 1
      $ActionSendStreamDriverAuthMode x509/name
      $ActionSendStreamDriverPermittedPeer *.logs.armor.com
      
      ### Send auth or authpriv messages to Armor
      # https://www.ibm.com/support/knowledgecenter/en/SS42VS_DSM/t_DSM_guide_Linux_OS_syslog.html
      ## if ( $syslogfacility-text == "auth" or $syslogfacility-text == "authpriv" ) then {
      ##   & stop
      ## }
      ###
      
      ### Send httpd messages to Armor
      if ( $programname startswith "httpd" ) then {
        $ActionQueueType LinkedList
      
        # unique name prefix for spool files
        $ActionQueueFileName q_sendHttpdToArmor
      
        # infinite retries if host is down
        $ActionResumeRetryCount -1
      
        # 1gb disk queue
        $ActionQueueMaxDiskSpace 1g
      
        # save messages to disk on shutdown
        $ActionQueueSaveOnShutdown on
      
        # queue.workerThreads may not be raised above 1
        # Specifies the maximum number of worker threads that can be run parallel.
        $ActionQueueWorkerThreads 1
      
        # queue.dequeueSlowDown limited to 100 messages per second
        # Regulates how long dequeueing should be delayed. This value must be specified in microseconds (1000000us is 1sec). It can be used to slow down rsyslog so it won't send
        # things to fast. For example if this parameter is set to 10000 on a UDP send action, the action won't be able to put out more than 100 messages per second.
        $ActionQueueDequeueSlowDown 10000
      
        # queue.discardSeverity default 8
        # As soon as the threshold of the parameter queue.discardMark is reached incoming aswell as queued messages with a priority equal or lower than specified will be erased.
        # With the default no messages will be erased. You have to specify a numeric severity value for this parameter.
        $ActionQueueDiscardSeverity 6
      
      
        # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
        *.* @@target-name:port;RFC3164Template
      
        # ### end of the forwarding rule ###
        & stop
      }
      #########################
  6. Ensure that rsyslog configuration has no syntax errors:

    • rsyslogd -N1

  7. Restart rsyslog: 

    • service rsyslog restart

  8. Verify.


Create a remote log source (Apache HTTP Server For CentOS 6)


To obtain Log Relay and to configure your account for remote log collection, you must have the following AMP permissions added to your account: 

  • Write Virtual Machine
  • Delete Log Management
  • Read Log Endpoints
  • Read Log Relays
  • Write Log Relays
  • Delete Log Relays

You can use this document to add a remote log collector to an Apache HTTP Server remote device (log source). 


Pre-deployment considerations

For remote log collection, you must have Log Relay added to your account.  


Create a remote log source

  1. In the Armor Management Portal (AMP), in the left-side navigation, click Security.
  2. Click Log & Data Management.
  3. Click External Sources.
  4. Click the plus ( + ) sign. 
    • If you do not have any log sources already created, then click Add a New Log Source
  5. Complete the missing fields:
    1. In Endpoint, select the available Armor Endpoint.
    2. In Log Source Type, select Apache HTTP Server
    3. In Hostname, enter the system hostname that matches the system for syslog collection. For example, in Mar 10 08:52:55 node-77 systemd: < redacted >, the hostname would be node-77.

      1. The hostname is case-sensitive and must match the same letter casing as the logs that are sent into this log source.
    4. In Protocol, select TLS Syslog
  6. Click Save Log Source.


Implement Server

Before you begin, ensure that the following pages are installed:

  • rsyslog-gnutls
    • yum install -y rsyslog-gnutls
  1. Create a directory to hold Armor pem file: 
    • mkdir -pv /etc/rsyslog.d/keys/ca.d
  2. Change to the newly created directory: 
    • cd /etc/rsyslog.d/keys/ca.d
  3. Download Armor PEM files: 
  4. Place the following two lines in main http.conf or vhost config (will be needed for each vhost): 
    • LogFormat "%h %A %l %u %t \"%r\" %>s %p %b" ArmorCustom
    • CustomLog "|/usr/bin/logger -t httpd -p local6.info" ArmorCustom
  5. Create a file called /etc/rsyslog.d/55-armor.conf with the following template:
    • Replace target-name:port with the name of the configured endpoint and port.

      # make gtls driver the default
      $DefaultNetstreamDriver gtls
      # $ActionSendStreamDriverAuthMode anon
      $ActionSendStreamDriverMode 1 # run driver in TLS-only mode
      $ActionSendStreamDriverAuthMode x509/name
      $ActionSendStreamDriverPermittedPeer *.logs.armor.com
      
      # certificate files
      $DefaultNetstreamDriverCAFile /etc/rsyslog.d/keys/ca.d/logs.armor.com.pem
      
      # $ActionSendStreamDriverAuthMode x509/name
      # $ActionSendStreamDriverPermittedPeer *.logs.armor.com
      
      
      # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
      local6.*  @@<target-name:port>
      
      
      # ### begin forwarding rule ###
      # The statement between the begin ... end define a SINGLE forwarding
      # rule. They belong together, do NOT split them. If you create multiple
      # forwarding rules, duplicate the whole block!
      # Remote Logging (we use TCP for reliable delivery)
      #
      # An on-disk queue is created for this action. If the remote host is
      # down, messages are spooled to disk and sent when it is up again.
      # $ActionQueueFileName fwdRule1 # unique name prefix for spool files
      $ActionQueueMaxDiskSpace 1g   # 1gb space limit (use as much as possible)
      $ActionQueueSaveOnShutdown on # save messages to disk on shutdown
      $ActionQueueWorkerThreads 1
      $ActionQueueDequeueSlowDown 10000
      $ActionQueueDiscardSeverity 6
      $ActionQueueType LinkedList   # run asynchronously
      $ActionResumeRetryCount -1    # infinite retries if host is down
      # ### end of the forwarding rule ###
  6. Ensure that rsyslog configuration has no syntax errors:

    • rsyslogd -N1

  7. Ensure that httpd configuration has no syntax errors:

    • httpd -t

  8. Reload httpd:

    • service httpd reload

  9. Restart rsyslog: 

    • service rsyslog restart

  10. Verify.

Troubleshooting 

If enforced, RHEL/CentOS - SELinux will block rsyslog from using a remote port. To troubleshoot:

  1. Install SELinux troubleshooting tool:

    • yum install -y setroubleshoot-server

  2. Check to see if rsyslog is being blocked from using remote port

    • sealert -a /var/log/audit/audit.log

  3. Follow recommendations to allow: Example:

    • grep 72733A6D61696E20513A526567 /var/log/audit/audit.log | audit2allow -M mypol
      semodule -i mypol.pp



Was this helpful?
Your Rating: Results: 1 Star2 Star3 Star4 Star5 Star 0 rates