-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
10 changed files
with
233 additions
and
73 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| # Use the demo | ||
|
|
||
| The Splunk Connect for syslog demo uses docker and docker compose | ||
| to configure a instance of Splunk along with syslog-ng and a test | ||
| harness to simulate a mix of events. Ensure git, docker and docker-compose | ||
| are pre-installed and working prior to continuing. | ||
|
|
||
|
|
||
| - Clone the repository and cd into directory | ||
|
|
||
| ```bash | ||
| git clone git@github.com:splunk/splunk-connect-for-syslog.git | ||
| cd splunk-connect-for-syslog | ||
| ``` | ||
|
|
||
| - Create a working .env file * Note for demo purposes this file does not need to be modified | ||
|
|
||
| ```bash | ||
| cp .env.template .env | ||
| ``` | ||
|
|
||
| - Update the splunkbase username and password in .env this allows the splunk container to install required add-ons for the demo | ||
|
|
||
| - Start the demo environment | ||
|
|
||
| ```bash | ||
| ./demo-with-compose.sh | ||
| ``` | ||
|
|
||
| - Login to splunk by browsing to http://127.0.0.1:8000 user name admin password "Changed@11" | ||
|
|
||
| - Search the main index to see indexed events | ||
|
|
||
| ```spl | ||
| index = * | ||
| ``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,145 @@ | ||
| #Warning | ||
|
|
||
| The "Bring Your Own Environment" instructions that follow allow administrators to utilize the SC4S syslog-ng | ||
| config files directly on the host OS running on a hardware server or virtual machine. Administrators must provide an | ||
| appropriate host OS as well as an up-to-date syslog-ng installation either built from source (not documented) or | ||
| installed from community-built RPMs. Modification of the base configuration will be required for most customer | ||
| environments due to enterprise infrastructure variations. | ||
|
|
||
| * NOTE: Installing or modifying system configurations can have unexpected consequences, and rudimentary linux system | ||
| administratrion and syslog-ng configuration experience is assumed. | ||
|
|
||
| Read this [explanation](https://www.syslog-ng.com/community/b/blog/posts/installing-latest-syslog-ng-on-rhel-and-other-rpm-distributions) | ||
| on the reason syslog-ng builds are so dated in most RHEL/Debian distributions. | ||
|
|
||
|
|
||
| * Install CentOS or RHEL 7.7 | ||
| * Enable EPEL | ||
| * Centos 7 | ||
|
|
||
| ```bash | ||
| sudo yum install epel-release | ||
| ``` | ||
|
|
||
| * RHEL 7 | ||
|
|
||
| ```bash | ||
| cd /tmp | ||
| wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm | ||
| sudo yum install ./epel-release-latest-*.noarch.rpm -y | ||
| ``` | ||
|
|
||
| * Enable the optional repo for RHEL 7 only | ||
|
|
||
| ```bash | ||
| sudo subscription-manager repos --enable rhel-7-server-optional-rpms | ||
| ``` | ||
| * Enable the "stable" unoffical repo for syslog-ng | ||
|
|
||
| ```bash | ||
| cd /etc/yum.repos.d/ | ||
| sudo wget https://copr.fedorainfracloud.org/coprs/czanik/syslog-ng-stable/repo/epel-7/czanik-syslog-ng-stable-epel-7.repo | ||
| sudo yum install syslog-ng syslog-ng-http syslog-ng-python | ||
| ``` | ||
|
|
||
| * Optional step: Disable the OOB syslog-ng unit file, as the syslog-ng process configured here will run as the `sc4s` | ||
| service. rsyslog will continue to be the system logger, and can be left enabled _only_ if it is configured to not | ||
| listen on the same ports as sc4s. | ||
|
|
||
| ```bash | ||
| systemctl stop syslog-ng | ||
| systemctl disable syslog-ng | ||
| ``` | ||
| * Download the latest bare_metal.tar from [releases](https://github.com/splunk/splunk-connect-for-syslog/releases) on github and untar the package | ||
|
|
||
| ```bash | ||
| cd /tmp | ||
| sudo wget https://github.com/splunk/splunk-connect-for-syslog/releases/download/0.12.1/baremetal.tar | ||
| tar -xf baremetal.tar | ||
| sudo mkdir -p /opt/syslog-ng/etc | ||
| sudo mkdir -p /opt/syslog-ng/var | ||
| sudo cp -R etc/* /opt/syslog-ng/etc/ | ||
| ``` | ||
|
|
||
| * Install and verify gomplate verify the output is 3.5.0 or newer | ||
|
|
||
| ```bash | ||
| sudo curl -o /usr/local/bin/gomplate -sSL https://github.com/hairyhenderson/gomplate/releases/download/v3.5.0/gomplate_linux-amd64 | ||
| sudo chmod 755 /usr/local/bin/gomplate | ||
| gomplate --help | ||
| ``` | ||
|
|
||
| * create the sc4s unit file drop in ``/etc/systemd/system/sc4s.service`` and add the following content | ||
|
|
||
| ```ini | ||
| [Unit] | ||
| Description=SC4S Syslog Daemon | ||
| Documentation=man:syslog-ng(8) | ||
| Wants=network.target network-online.target | ||
| After=network.target network-online.target | ||
| [Service] | ||
| Type=notify | ||
| ExecStartPre=/opt/sc4s/bin/preconfig.sh | ||
| ExecStart=/usr/sbin/syslog-ng -F $SYSLOGNG_OPTS -p /var/run/syslogd.pid | ||
| ExecReload=/bin/kill -HUP $MAINPID | ||
| EnvironmentFile=-/etc/default/syslog-ng | ||
| EnvironmentFile=-/etc/sysconfig/syslog-ng | ||
| EnvironmentFile=/opt/sc4s/default/env_file | ||
| StandardOutput=journal | ||
| StandardError=journal | ||
| Restart=on-failure | ||
| [Install] | ||
| WantedBy=multi-user.target | ||
| ``` | ||
|
|
||
| * create the file ``/opt/sc4s/bin/preconfig.sh`` and add the following content | ||
|
|
||
| ```bash | ||
| #!/usr/bin/env bash | ||
| source scl_source enable rh-python36 | ||
| cd /opt/syslog-ng | ||
| for d in $(find /opt/syslog-ng/etc -type d) | ||
| do | ||
| echo Templating conf for $d | ||
| gomplate \ | ||
| --input-dir=$d \ | ||
| --template t=etc/go_templates/ \ | ||
| --exclude=*.conf --exclude=*.csv --exclude=*.t --exclude=.*\ | ||
| --output-map="$d/{{ .in | strings.ReplaceAll \".conf.tmpl\" \".conf\" }}" | ||
| done | ||
| mkdir -p /opt/syslog-ng/etc/conf.d/local/context/ | ||
| mkdir -p /opt/syslog-ng/etc/conf.d/local/config/ | ||
| cp --verbose -n /opt/syslog-ng/etc/context_templates/* /opt/syslog-ng/etc/conf.d/local/context/ | ||
| cp --verbose -R -n /opt/syslog-ng/etc/local_config/* /opt/syslog-ng/etc/conf.d/local/config/ | ||
| mkdir -p /opt/syslog-ng/var/data/disk-buffer/ | ||
| ``` | ||
|
|
||
| * set execute permissions on the file | ||
| ``` | ||
| sudo chmod 755 /opt/sc4s/bin/preconfig.sh | ||
| ``` | ||
|
|
||
| * Create the file ``/opt/sc4s/default/env_file`` and add the following environment variables: | ||
|
|
||
| ```dotenv | ||
| SYSLOGNG_OPTS=-f /opt/syslog-ng/etc/syslog-ng.conf | ||
| SPLUNK_HEC_URL=https://splunk.smg.aws:8088/services/collector/event | ||
| SPLUNK_HEC_TOKEN=a778f63a-5dff-4e3c-a72c-a03183659e94 | ||
| SC4S_DEST_SPLUNK_HEC_WORKERS=6 | ||
| SPLUNK_CONNECT_METHOD=hec | ||
| SPLUNK_DEFAULT_INDEX=main | ||
| SPLUNK_METRICS_INDEX=em_metrics | ||
| #Uncomment the following line if using untrusted SSL certificates | ||
| #SC4S_DEST_SPLUNK_HEC_TLS_VERIFY=no | ||
| ``` | ||
|
|
||
| * Reload systemctl and restart syslog-ng | ||
|
|
||
| ```bash | ||
| sudo systemctl daemon-reload | ||
| sudo systemctl start sc4s | ||
| ``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.