diff --git a/docs/gettingstarted.md b/docs/gettingstarted.md index a43e906..a7fff9d 100644 --- a/docs/gettingstarted.md +++ b/docs/gettingstarted.md @@ -76,7 +76,7 @@ Splunk type. | [Docker CE + systemd single node](gettingstarted/docker-systemd-general.md) | First choice for Debian, Ubuntu, and CentOS distributions with limited existing docker experience | | [Docker CE + Swarm single node](gettingstarted/docker-swarm-general.md) | Option for Debian, Ubuntu, and CentOS desiring swarm orchestration | | [Docker CE + Swarm single node RHEL 7.7](gettingstarted/docker-swarm-rhel7.md) | Option for RedHat 7.7 desiring swarm orchestration | - +| [Bring your own Envionment](gettingstarted/byoe-rhel7.md) | Option for RedHat 7.7 (centos 7) with SC4S configuration without containers | # Scale out diff --git a/docs/gettingstarted/byoe-rhel7.md b/docs/gettingstarted/byoe-rhel7.md new file mode 100644 index 0000000..cc973d6 --- /dev/null +++ b/docs/gettingstarted/byoe-rhel7.md @@ -0,0 +1,145 @@ +#Warning + +The "Bring Your Own Environment" instructions that follow allow administrators to utilize the SC4S syslog-ng +config files directly on the host OS running on a hardware server or virtual machine. Administrators must provide an +appropriate host OS as well as an up-to-date syslog-ng installation either built from source (not documented) or +installed from community-built RPMs. Modification of the base configuration will be required for most customer +environments due to enterprise infrastructure variations. + +* NOTE: Installing or modifying system configurations can have unexpected consequences, and rudimentary linux system +administratrion and syslog-ng configuration experience is assumed. + +Read this [explanation](https://www.syslog-ng.com/community/b/blog/posts/installing-latest-syslog-ng-on-rhel-and-other-rpm-distributions) +on the reason syslog-ng builds are so dated in most RHEL/Debian distributions. + + +* Install CentOS or RHEL 7.7 +* Enable EPEL + * Centos 7 + + ```bash + sudo yum install epel-release + ``` + + * RHEL 7 + + ```bash + cd /tmp + wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm + sudo yum install ./epel-release-latest-*.noarch.rpm -y + ``` + +* Enable the optional repo for RHEL 7 only + + ```bash + sudo subscription-manager repos --enable rhel-7-server-optional-rpms + ``` +* Enable the "stable" unoffical repo for syslog-ng + + ```bash + cd /etc/yum.repos.d/ + sudo wget https://copr.fedorainfracloud.org/coprs/czanik/syslog-ng-stable/repo/epel-7/czanik-syslog-ng-stable-epel-7.repo + sudo yum install syslog-ng syslog-ng-http syslog-ng-python + ``` + +* Optional step: Disable the OOB syslog-ng unit file, as the syslog-ng process configured here will run as the `sc4s` +service. rsyslog will continue to be the system logger, and can be left enabled _only_ if it is configured to not +listen on the same ports as sc4s. + +```bash +systemctl stop syslog-ng +systemctl disable syslog-ng +``` +* Download the latest bare_metal.tar from [releases](https://github.com/splunk/splunk-connect-for-syslog/releases) on github and untar the package + +```bash +cd /tmp +sudo wget https://github.com/splunk/splunk-connect-for-syslog/releases/download/0.12.1/baremetal.tar +tar -xf baremetal.tar +sudo mkdir -p /opt/syslog-ng/etc +sudo mkdir -p /opt/syslog-ng/var +sudo cp -R etc/* /opt/syslog-ng/etc/ +``` + +* Install and verify gomplate verify the output is 3.5.0 or newer + +```bash +sudo curl -o /usr/local/bin/gomplate -sSL https://github.com/hairyhenderson/gomplate/releases/download/v3.5.0/gomplate_linux-amd64 +sudo chmod 755 /usr/local/bin/gomplate +gomplate --help +``` + +* create the sc4s unit file drop in ``/etc/systemd/system/sc4s.service`` and add the following content + +```ini +[Unit] +Description=SC4S Syslog Daemon +Documentation=man:syslog-ng(8) +Wants=network.target network-online.target +After=network.target network-online.target + +[Service] +Type=notify +ExecStartPre=/opt/sc4s/bin/preconfig.sh +ExecStart=/usr/sbin/syslog-ng -F $SYSLOGNG_OPTS -p /var/run/syslogd.pid +ExecReload=/bin/kill -HUP $MAINPID +EnvironmentFile=-/etc/default/syslog-ng +EnvironmentFile=-/etc/sysconfig/syslog-ng +EnvironmentFile=/opt/sc4s/default/env_file +StandardOutput=journal +StandardError=journal +Restart=on-failure + +[Install] +WantedBy=multi-user.target +``` + +* create the file ``/opt/sc4s/bin/preconfig.sh`` and add the following content + +```bash +#!/usr/bin/env bash +source scl_source enable rh-python36 + +cd /opt/syslog-ng +for d in $(find /opt/syslog-ng/etc -type d) +do + echo Templating conf for $d + gomplate \ + --input-dir=$d \ + --template t=etc/go_templates/ \ + --exclude=*.conf --exclude=*.csv --exclude=*.t --exclude=.*\ + --output-map="$d/{{ .in | strings.ReplaceAll \".conf.tmpl\" \".conf\" }}" +done + +mkdir -p /opt/syslog-ng/etc/conf.d/local/context/ +mkdir -p /opt/syslog-ng/etc/conf.d/local/config/ +cp --verbose -n /opt/syslog-ng/etc/context_templates/* /opt/syslog-ng/etc/conf.d/local/context/ +cp --verbose -R -n /opt/syslog-ng/etc/local_config/* /opt/syslog-ng/etc/conf.d/local/config/ +mkdir -p /opt/syslog-ng/var/data/disk-buffer/ +``` + +* set execute permissions on the file +``` +sudo chmod 755 /opt/sc4s/bin/preconfig.sh +``` + +* Create the file ``/opt/sc4s/default/env_file`` and add the following environment variables: + +```dotenv +SYSLOGNG_OPTS=-f /opt/syslog-ng/etc/syslog-ng.conf +SPLUNK_HEC_URL=https://splunk.smg.aws:8088/services/collector/event +SPLUNK_HEC_TOKEN=a778f63a-5dff-4e3c-a72c-a03183659e94 +SC4S_DEST_SPLUNK_HEC_WORKERS=6 +SPLUNK_CONNECT_METHOD=hec +SPLUNK_DEFAULT_INDEX=main +SPLUNK_METRICS_INDEX=em_metrics +#Uncomment the following line if using untrusted SSL certificates +#SC4S_DEST_SPLUNK_HEC_TLS_VERIFY=no +``` + +* Reload systemctl and restart syslog-ng + +```bash +sudo systemctl daemon-reload +sudo systemctl start sc4s +```