Skip to main content

Telemetry and collection

Everything the CyberOptix CTEM Platform can search, detect on, or alert about arrives through collection. This page covers pointing your own log sources at it.

Deploying the receiver itself is a separate job, covered in Log agents and collectors. Do that first: the configuration below sends to a collector that has to already be listening.

Throughout, replace <collector-host> with your collector's hostname and 6514 with whatever you set as its listen_port.

Linux servers, via rsyslog​

Install rsyslog with TLS support. Without rsyslog-gnutls the gtls stream driver is missing and rsyslog fails at startup rather than falling back to plaintext:

sudo apt install rsyslog rsyslog-gnutls -y

Create /etc/rsyslog.d/10-forward-tls.conf:

template(name="CEFFormat"
    type="string"
    string="CEF:0|-|%programname%|-|%syslogfacility-text%|%syslogtag%|%syslogseverity-text%|%rawmsg%"
)

*.* action(
    type="omfwd"
    StreamDriver="gtls"
    StreamDriverMode="1"
    StreamDriverAuthMode="x509/name"
    StreamDriverPermittedPeers="<collector-host>"
    template="CEFFormat"
    queue.type="linkedList"
    target="<collector-host>"
    port="6514"
    protocol="tcp"
)
sudo systemctl restart rsyslog && systemctl status --no-pager rsyslog
note
Why StreamDriverPermittedPeers matters

It pins the certificate name the collector must present. Without it, x509/name validation has nothing to compare against and the sender will accept any certificate - which defeats the point of encrypting the transport. The value has to match the collector's certificate, not just its DNS name.

The queue.type="linkedList" line is what makes the sender resilient: if the collector is briefly unreachable, rsyslog buffers in memory rather than discarding messages.

Apache​

Apache does not speak syslog directly, so it logs through logger into a facility that rsyslog then forwards.

In apache2.conf or the relevant virtual host:

LogFormat "CEF:0|Apache Software Foundation|Apache|2.4|HTTPAccess|Access Log|5|client_ip=%a src=%h dst=%v request=\"%r\" query_string=\"%q\" method=%m response=%>s referer=\"%{Referer}i\" user_agent=\"%{User-agent}i\"" cef

ErrorLogFormat "CEF:0|Apache Software Foundation|Apache|2.4|%E|Error Log|5|msg=\"%M\" src=%a log_level=%l request_id=%{UNIQUE_ID}e"

CustomLog "|/usr/bin/logger -t apache-cef -p local6.info" cef
ErrorLog "|/usr/bin/logger -t apache-error -p local6.err"

Then forward that facility, in /etc/rsyslog.d/20-apache-tls.conf:

if ($syslogfacility-text == 'local6') then {
    action(
        type="omfwd"
        StreamDriver="gtls"
        StreamDriverMode="1"
        StreamDriverAuthMode="x509/name"
        StreamDriverPermittedPeers="<collector-host>"
        queue.type="linkedList"
        target="<collector-host>"
        port="6514"
        protocol="tcp"
    )
}
sudo systemctl restart rsyslog apache2

NGINX​

NGINX can emit syslog itself, but sends it to the local rsyslog rather than to the collector, so one TLS configuration serves every source on the host.

In the http block of nginx.conf:

log_format cef 'CEF:0|F5|NGINX|1.0|HTTPAccess|Access Log|5|'
               'src=$remote_addr dst=$server_name request="$request" '
               'query_string=$query_string method=$request_method '
               'response=$status referer=$http_referer user_agent=$http_user_agent';

access_log syslog:server=127.0.0.1:514,facility=local6,tag=nginx,severity=info cef;
error_log syslog:server=127.0.0.1:514,facility=local6,tag=nginx_error,severity=error;

This needs the same local6 forwarding rule as Apache. If you have already added it, nothing further is required.

sudo systemctl restart rsyslog nginx
One facility, two sources

Apache and NGINX above both use local6. That is fine - the tag distinguishes them - but it means a single forwarding rule carries both, and dropping that rule silently stops both. If you want them separable at the firewall or in rsyslog, give one of them a different facility.

Network appliances​

Anything that can send syslog over TCP can send to the collector. Two things to check on the device:

  • TCP, not UDP. The collector listens on TCP. A device configured for UDP syslog will report success and deliver nothing.
  • TLS support. Many appliances only speak plaintext syslog. If yours does, either accept that traffic is unencrypted on your own network, or put a local rsyslog relay in front of it that terminates plaintext locally and forwards over TLS.

Confirming it arrived​

On the collector:

sudo journalctl -u syslog-collector.service -f

Then in the platform, search for events from the source host under Investigations. Events are searchable as soon as they are ingested - detection is a separate, opt-in stage and does not have to be enabled for collection to work.

Next​

Last updated on