diff --git a/.circleci/config.yml b/.circleci/config.yml index d13a074..0d3076a 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -70,7 +70,7 @@ jobs: - run: name: BYOE Config command: | - tar rvf /tmp/workspace/baremetal.tar -C package etc + tar rvf /tmp/workspace/baremetal.tar -C package/etc . tar rvf /tmp/workspace/baremetal.tar -C package/sbin entrypoint.sh - store_test_results: diff --git a/docs/gettingstarted/byoe-rhel7.md b/docs/gettingstarted/byoe-rhel7.md index e91f5b1..f3d5645 100644 --- a/docs/gettingstarted/byoe-rhel7.md +++ b/docs/gettingstarted/byoe-rhel7.md @@ -21,7 +21,16 @@ for the reason why syslog-ng builds are so dated in almost all RHEL/Debian distr # BYOE Installation Instructions -* Install CentOS or RHEL 7.7 +These installation instructions assume a recent RHEL or CentOS-based release. Minor adjustments may have to be made for +Debian/Ubuntu. In addition, almost _all_ pre-compiled binaries for syslog-ng assume installation in `etc/syslog-ng`; these instructions +will reflect that. + +The following installation instructions are summarized from a +[blog](https://www.syslog-ng.com/community/b/blog/posts/introducing-the-syslog-ng-stable-rpm-repositories) +maintained by a developer at One Identity (formerly Balabit), who is the owner of the syslog-ng Open Source project. +It is always adivisable to review the blog for the latest changes to the repo(s), as changes here are quite dynamic. + +* Install CentOS or RHEL 7.7/8.0 * Enable EPEL (Centos 7) @@ -55,15 +64,14 @@ sudo systemctl stop syslog-ng sudo systemctl disable syslog-ng ``` -* Download the latest bare_metal.tar from [releases](https://github.com/splunk/splunk-connect-for-syslog/releases) on github and untar the package +* Download the latest bare_metal.tar from [releases](https://github.com/splunk/splunk-connect-for-syslog/releases) on github and untar the package in `/etc/syslog-ng` + +* NOTE: The `wget` process below will unpack a tarball with the sc4s version of the syslog-ng config files in the standard +`/etc/syslog-ng` location, and _will_ overwrite existing content. Ensure that any previous configurations of syslog-ng are saved +if needed prior to executing the download step. ```bash -cd /tmp -sudo wget https://github.com/splunk/splunk-connect-for-syslog/releases/download/0.12.1/baremetal.tar -tar -xf baremetal.tar -sudo mkdir -p /opt/syslog-ng/etc -sudo mkdir -p /opt/syslog-ng/var -sudo cp -R etc/* /opt/syslog-ng/etc/ +sudo wget -c https://github.com/splunk/splunk-connect-for-syslog/releases/download/latest/baremetal.tar -O - | sudo tar -x -C /etc/syslog-ng ``` * Install gomplate and confirm that the version is 3.5.0 or newer @@ -78,7 +86,7 @@ gomplate --version ```scl enable rh-python36 bash``` -* create the sc4s unit file drop in ``/etc/systemd/system/sc4s.service`` and add the following content +* create the sc4s unit file ``/etc/systemd/system/sc4s.service`` and add the following content ```ini [Unit] @@ -103,13 +111,20 @@ Restart=on-failure WantedBy=multi-user.target ``` -* create the file ``/opt/sc4s/bin/preconfig.sh``. This file should be made executable according to your file permission standards. Add the following content: +* NOTE: The `wget` process above will download a file called `entrypoint.sh` and place it in `/etc/syslog-ng`. This is the +preconfiguration file that is used for the container version of sc4s, and forms the foundation of the BYOE version of the file you will +create below. Do _not_ use it verbatim as there are differences between them (most notably the install location). However, it does include +the "latest and greatest" updates from the container, and should be used (with appropriate modifications) as the basis of the contents of +`preconfig.sh` below. + +* create the file ``/opt/sc4s/bin/preconfig.sh``. This file should be made executable according to your file permission standards. +Add the following content (but be sure to check the note above to ensure the latest updates are included): ```bash #!/usr/bin/env bash source scl_source enable rh-python36 -cd /opt/syslog-ng +cd /etc/syslog-ng #The following is no longer needed but retained as a comment just in case we run into command line length issues #for d in $(find /opt/syslog-ng/etc -type d) #do @@ -120,15 +135,14 @@ cd /opt/syslog-ng # --exclude=*.conf --exclude=*.csv --exclude=*.t --exclude=.*\ # --output-map="$d/{{ .in | strings.ReplaceAll \".conf.tmpl\" \".conf\" }}" #done -gomplate $(find . -name *.tmpl | sed -E 's/^(\/.*\/)*(.*)\..*$/--file=\2.tmpl --out=\2/') --template t=etc/go_templates/ - -mkdir -p /opt/syslog-ng/etc/conf.d/local/context/ -mkdir -p /opt/syslog-ng/etc/conf.d/local/config/ -cp --verbose -n /opt/syslog-ng/etc/context_templates/* /opt/syslog-ng/etc/conf.d/local/context/ -cp --verbose -R -n /opt/syslog-ng/etc/local_config/* /opt/syslog-ng/etc/conf.d/local/config/ -mkdir -p /opt/syslog-ng/var/data/disk-buffer/ -mkdir -p /opt/syslog-ng/var/archive/ -mkdir -p /opt/syslog-ng/tls/ + +gomplate $(find . -name *.tmpl | sed -E 's/^(\/.*\/)*(.*)\..*$/--file=\2.tmpl --out=\2/') --template t=go_templates/ + +mkdir -p /etc/syslog-ng/conf.d/local/context/ +mkdir -p /etc/syslog-ng/conf.d/local/config/ +cp /etc/syslog-ng/context_templates/* /etc/syslog-ng/conf.d/local/context/ +for file in /etc/syslog-ng/conf.d/local/context/*.example ; do cp -v -n $file ${file%.example}; done +cp -v -R /etc/syslog-ng/local_config/* /etc/syslog-ng/conf.d/local/config/ ``` * (Optional) Execute the preconfiguration shell script created above. You may also optionally execute it as part of the unit @@ -142,7 +156,7 @@ sudo bash /opt/sc4s/bin/preconfig.sh * Create the file ``/opt/sc4s/env_file`` and add the following environment variables: ```dotenv -SYSLOGNG_OPTS=-f /opt/syslog-ng/etc/syslog-ng.conf +SYSLOGNG_OPTS=-f /etc/syslog-ng/syslog-ng.conf SPLUNK_HEC_URL=https://splunk.smg.aws:8088 SPLUNK_HEC_TOKEN=a778f63a-5dff-4e3c-a72c-a03183659e94 SC4S_DEST_SPLUNK_HEC_WORKERS=6 diff --git a/docs/gettingstarted/docker-swarm-general.md b/docs/gettingstarted/docker-swarm-general.md index a26a1c1..9e1a85f 100644 --- a/docs/gettingstarted/docker-swarm-general.md +++ b/docs/gettingstarted/docker-swarm-general.md @@ -55,33 +55,34 @@ services: - /opt/sc4s/env_file volumes: - /opt/sc4s/local:/opt/syslog-ng/etc/conf.d/local:z - - /opt/sc4s/disk-buffer:/opt/syslog-ng/var/data/disk-buffer:z + - splunk-sc4s-var:/opt/syslog-ng/var # Uncomment the following line if local disk archiving is desired # - /opt/sc4s/archive:/opt/syslog-ng/var/archive:z # Uncomment the following line if custom TLS certs are provided # - /opt/sc4s/tls:/opt/syslog-ng/tls:z ``` +* Execute the following command to create a local volume that will contain the disk buffer files in the event of a communication +failure to the upstream destination(s). This will also be used to keep track of the state of syslog-ng between restarts, and in +particular the state of the disk buffer. This is a required step. +``` +sudo docker volume create splunk-sc4s-var +``` + * Create the subdirectory ``/opt/sc4s/local``. This will be used as a mount point for local overrides and configurations. - * The empty ``local`` directory created above will populate with templates at the first invocation -of SC4S for local configurations and overrides. Changes made to these files will be preserved on subsequent -restarts (i.e. a "no-clobber" copy is performed for any missing files). _Do not_ change the directory structure of + * The empty ``local`` directory created above will populate with defaults and examples at the first invocation +of SC4S for local configurations and context overrides. _Do not_ change the directory structure of the files that are laid down; change (or add) only individual files if desired. SC4S depends on the directory layout -to read the local configurations properly. - - * You can back up the contents of this directory elsewhere and return the directory to an empty state -when a new version of SC4S is released to pick up any new changes provided by Splunk. Upon a restart, -the direcory will populate as it did when you first installed SC4S. Your previous changes can then -be merged back in and will take effect after another restart. +to read the local configurations properly. See the notes below for which files will be preserved on restarts. -* Create the subdirectory ``/opt/sc4s/disk-buffer``. This will be used as a mount point for local disk buffering -of events in the event of network failure to the Splunk infrastructure. + * In the `local/config` directory, there are example log path files (`lp-example.*`) and a filter (`example.conf`) in the +appropriate subdirectories. These should _not_ be used directly, but copied as examples for your own log path development. +They _will_ get overwritten at each SC4S start. - * This directory will populate with the disk buffer files upon SC4S startup. If SC4S restarts for any reason, a new - set of files will be created in addition to the original ones. _The original ones will not be removed_. - If you are sure, after stopping SC4S, that all data has been sent, these files can be removed. They will be created - again upon restart. + * In the `local/context` directory, if you change the "non-example" version of a file (e.g. `splunk_index.csv`) the changes +will be preserved on a restart. However, the "example" files _themselves_ (e.g. `splunk_index.csv.example`) will be updated +regularly, and should be used as a template to merge new/changed functionality into existing context files. * Create the subdirectory ``/opt/sc4s/archive``. This will be used as a mount point for local storage of syslog events (if the optional mount is uncommented above). The events will be written in the syslog-ng EWMM format. See the "configuration" @@ -183,7 +184,7 @@ services: - /opt/sc4s/env_file volumes: - /opt/sc4s/local:/opt/syslog-ng/etc/conf.d/local:z - - /opt/sc4s/disk-buffer:/opt/syslog-ng/var/data/disk-buffer:z + - splunk-sc4s-var:/opt/syslog-ng/var # Uncomment the following line if local disk archiving is desired # - /opt/sc4s/archive:/opt/syslog-ng/var/archive:z # Uncomment the following line if custom TLS certs are provided diff --git a/docs/gettingstarted/docker-swarm-rhel7.md b/docs/gettingstarted/docker-swarm-rhel7.md index 251c9fa..c276f64 100644 --- a/docs/gettingstarted/docker-swarm-rhel7.md +++ b/docs/gettingstarted/docker-swarm-rhel7.md @@ -63,34 +63,35 @@ services: - /opt/sc4s/env_file volumes: - /opt/sc4s/local:/opt/syslog-ng/etc/conf.d/local:z - - /opt/sc4s/disk-buffer:/opt/syslog-ng/var/data/disk-buffer:z + - splunk-sc4s-var:/opt/syslog-ng/var # Uncomment the following line if local disk archiving is desired # - /opt/sc4s/archive:/opt/syslog-ng/var/archive:z # Uncomment the following line if custom TLS certs are provided # - /opt/sc4s/tls:/opt/syslog-ng/tls:z ``` +* Execute the following command to create a local volume that will contain the disk buffer files in the event of a communication +failure to the upstream destination(s). This will also be used to keep track of the state of syslog-ng between restarts, and in +particular the state of the disk buffer. This is a required step. +``` +sudo docker volume create splunk-sc4s-var +``` + * Create the subdirectory ``/opt/sc4s/local``. This will be used as a mount point for local overrides and configurations. - * The empty ``local`` directory created above will populate with templates at the first invocation -of SC4S for local configurations and overrides. Changes made to these files will be preserved on subsequent -restarts (i.e. a "no-clobber" copy is performed for any missing files). _Do not_ change the directory structure of + * The empty ``local`` directory created above will populate with defaults and examples at the first invocation +of SC4S for local configurations and context overrides. _Do not_ change the directory structure of the files that are laid down; change (or add) only individual files if desired. SC4S depends on the directory layout -to read the local configurations properly. +to read the local configurations properly. See the notes below for which files will be preserved on restarts. - * You can back up the contents of this directory elsewhere and return the directory to an empty state -when a new version of SC4S is released to pick up any new changes provided by Splunk. Upon a restart, -the direcory will populate as it did when you first installed SC4S. Your previous changes can then -be merged back in and will take effect after another restart. - -* Create the subdirectory ``/opt/sc4s/disk-buffer``. This will be used as a mount point for local disk buffering -of events in the event of network failure to the Splunk infrastructure. - - * This directory will populate with the disk buffer files upon SC4S startup. If SC4S restarts for any reason, a new -set of files will be created in addition to the original ones. _The original ones will not be removed_. -If you are sure, after stopping SC4S, that all data has been sent, these files can be removed. They will be created -again upon restart + * In the `local/config` directory, there are example log path files (`lp-example.*`) and a filter (`example.conf`) in the +appropriate subdirectories. These should _not_ be used directly, but copied as examples for your own log path development. +They _will_ get overwritten at each SC4S start. + * In the `local/context` directory, if you change the "non-example" version of a file (e.g. `splunk_index.csv`) the changes +will be preserved on a restart. However, the "example" files _themselves_ (e.g. `splunk_index.csv.example`) will be updated +regularly, and should be used as a template to merge new/changed functionality into existing context files. + * Create the subdirectory ``/opt/sc4s/archive``. This will be used as a mount point for local storage of syslog events (if the optional mount is uncommented above). The events will be written in the syslog-ng EWMM format. See the "configuration" document for details on the directory structure the archive uses. @@ -191,7 +192,7 @@ services: - /opt/sc4s/env_file volumes: - /opt/sc4s/local:/opt/syslog-ng/etc/conf.d/local:z - - /opt/sc4s/disk-buffer:/opt/syslog-ng/var/data/disk-buffer:z + - splunk-sc4s-var:/opt/syslog-ng/var # Uncomment the following line if local disk archiving is desired # - /opt/sc4s/archive:/opt/syslog-ng/var/archive:z # Uncomment the following line if custom TLS certs are provided diff --git a/docs/gettingstarted/docker-systemd-general.md b/docs/gettingstarted/docker-systemd-general.md index 55a4788..2f25337 100644 --- a/docs/gettingstarted/docker-systemd-general.md +++ b/docs/gettingstarted/docker-systemd-general.md @@ -28,21 +28,24 @@ net.ipv4.ip_forward=1 ```ini [Unit] Description=SC4S Container -Wants=network.target network-online.target -After=network.target network-online.target +Wants=NetworkManager.service network-online.target +After=NetworkManager.service network-online.target + +[Install] +WantedBy=multi-user.target [Service] Environment="SC4S_IMAGE=splunk/scs:latest" +# Required mount point for syslog-ng persist data (including disk buffer) +Environment="SC4S_PERSIST_VOLUME=-v splunk-sc4s-var:/opt/syslog-ng/var" + # Optional mount point for local overrides and configurations; see notes in docs Environment="SC4S_LOCAL_CONFIG_MOUNT=-v /opt/sc4s/local:/opt/syslog-ng/etc/conf.d/local:z" # Optional mount point for local disk archive (EWMM output) files # Environment="SC4S_LOCAL_ARCHIVE_MOUNT=-v /opt/sc4s/archive:/opt/syslog-ng/var/archive:z" -# Mount point for local disk buffer (required) -Environment="SC4S_LOCAL_DISK_BUFFER_MOUNT=-v /opt/sc4s/disk-buffer:/opt/syslog-ng/var/data/disk-buffer:z" - # Uncomment the following line if custom TLS certs are provided # Environment="SC4S_TLS_DIR=-v /opt/sc4s/tls:/opt/syslog-ng/tls:z" @@ -57,35 +60,36 @@ ExecStartPre=/usr/bin/docker run \ --rm $SC4S_IMAGE -s ExecStart=/usr/bin/docker run -p 514:514 -p 514:514/udp -p 6514:6514 \ --env-file=/opt/sc4s/env_file \ + "$SC4S_PERSIST_VOLUME" \ "$SC4S_LOCAL_CONFIG_MOUNT" \ - "$SC4S_LOCAL_DISK_BUFFER_MOUNT" \ "$SC4S_LOCAL_ARCHIVE_MOUNT" \ "$SC4S_TLS_DIR" \ --name SC4S \ --rm $SC4S_IMAGE ``` +* Execute the following command to create a local volume that will contain the disk buffer files in the event of a communication +failure to the upstream destination(s). This will also be used to keep track of the state of syslog-ng between restarts, and in +particular the state of the disk buffer. This is a required step. +``` +sudo docker volume create splunk-sc4s-var +``` + * Create the subdirectory ``/opt/sc4s/local``. This will be used as a mount point for local overrides and configurations. - * The empty ``local`` directory created above will populate with templates at the first invocation -of SC4S for local configurations and overrides. Changes made to these files will be preserved on subsequent -restarts (i.e. a "no-clobber" copy is performed for any missing files). _Do not_ change the directory structure of + * The empty ``local`` directory created above will populate with defaults and examples at the first invocation +of SC4S for local configurations and context overrides. _Do not_ change the directory structure of the files that are laid down; change (or add) only individual files if desired. SC4S depends on the directory layout -to read the local configurations properly. - - * You can back up the contents of this directory elsewhere and return the directory to an empty state -when a new version of SC4S is released to pick up any new changes provided by Splunk. Upon a restart, -the direcory will populate as it did when you first installed SC4S. Your previous changes can then -be merged back in and will take effect after another restart. +to read the local configurations properly. See the notes below for which files will be preserved on restarts. -* Create the subdirectory ``/opt/sc4s/disk-buffer``. This will be used as a mount point for local disk buffering -of events in the event of network failure to the Splunk infrastructure. - - * This directory will populate with the disk buffer files upon SC4S startup. If SC4S restarts for any reason, a new -set of files will be created in addition to the original ones. _The original ones will not be removed_. -If you are sure, after stopping SC4S, that all data has been sent, these files can be removed. They will be created -again upon restart + * In the `local/config` directory, there are example log path files (`lp-example.*`) and a filter (`example.conf`) in the +appropriate subdirectories. These should _not_ be used directly, but copied as examples for your own log path development. +They _will_ get overwritten at each SC4S start. + * In the `local/context` directory, if you change the "non-example" version of a file (e.g. `splunk_index.csv`) the changes +will be preserved on a restart. However, the "example" files _themselves_ (e.g. `splunk_index.csv.example`) will be updated +regularly, and should be used as a template to merge new/changed functionality into existing context files. + * Create the subdirectory ``/opt/sc4s/archive``. This will be used as a mount point for local storage of syslog events (if the optional mount is uncommented above). The events will be written in the syslog-ng EWMM format. See the "configuration" document for details on the directory structure the archive uses. @@ -149,26 +153,30 @@ documentation to identify the specific environment variables that are mapped to ```ini [Unit] Description=SC4S Container -After=network.service -Requires=network.service +Wants=NetworkManager.service network-online.target +After=NetworkManager.service network-online.target + +[Install] +WantedBy=multi-user.target [Service] Environment="SC4S_IMAGE=splunk/scs:latest" +# Required mount point for syslog-ng persist data (including disk buffer) +Environment="SC4S_PERSIST_VOLUME=-v splunk-sc4s-var:/opt/syslog-ng/var" + # Optional mount point for local overrides and configurations; see notes in docs Environment="SC4S_LOCAL_CONFIG_MOUNT=-v /opt/sc4s/local:/opt/syslog-ng/etc/conf.d/local:z" # Optional mount point for local disk archive (EWMM output) files # Environment="SC4S_LOCAL_ARCHIVE_MOUNT=-v /opt/sc4s/archive:/opt/syslog-ng/var/archive:z" -# Mount point for local disk buffer (required) -Environment="SC4S_LOCAL_DISK_BUFFER_MOUNT=-v /opt/sc4s/disk-buffer:/opt/syslog-ng/var/data/disk-buffer:z" - # Uncomment the following line if custom TLS certs are provided # Environment="SC4S_TLS_DIR=-v /opt/sc4s/tls:/opt/syslog-ng/tls" TimeoutStartSec=0 Restart=always + ExecStartPre=/usr/bin/docker pull $SC4S_IMAGE ExecStartPre=/usr/bin/docker run \ --env-file=/opt/sc4s/env_file \ @@ -177,8 +185,8 @@ ExecStartPre=/usr/bin/docker run \ --rm $SC4S_IMAGE -s ExecStart=/usr/bin/docker run -p 514:514 -p 514:514/udp -p 6514:6514 -p 5000-5020:5000-5020 -p 5000-5020:5000-5020/udp \ --env-file=/opt/sc4s/env_file \ + "$SC4S_PERSIST_VOLUME" \ "$SC4S_LOCAL_CONFIG_MOUNT" \ - "$SC4S_LOCAL_DISK_BUFFER_MOUNT" \ "$SC4S_LOCAL_ARCHIVE_MOUNT" \ "$SC4S_TLS_DIR" \ --name SC4S \ @@ -283,4 +291,5 @@ Oct 1 05:29:55 77cd4776af41 syslog-ng[1]: Syslog connection closed; fd='49', cl ``` If you see http server errors such as 4xx or 5xx responses from the http (HEC) endpoint, one or more of the items above are likely set incorrectly. If validating/fixing the configuration fails to correct the problem, proceed to the "Troubleshooting" section for more -information. \ No newline at end of file +information. + diff --git a/docs/gettingstarted/podman-systemd-general.md b/docs/gettingstarted/podman-systemd-general.md index 7b49762..cc4714e 100644 --- a/docs/gettingstarted/podman-systemd-general.md +++ b/docs/gettingstarted/podman-systemd-general.md @@ -10,21 +10,24 @@ Refer to [Installation](https://podman.io/getting-started/installation) ```ini [Unit] Description=SC4S Container -Wants=network.target network-online.target -After=network.target network-online.target +Wants=NetworkManager.service network-online.target +After=NetworkManager.service network-online.target + +[Install] +WantedBy=multi-user.target [Service] Environment="SC4S_IMAGE=splunk/scs:latest" +# Required mount point for syslog-ng persist data (including disk buffer) +Environment="SC4S_PERSIST_VOLUME=-v splunk-sc4s-var:/opt/syslog-ng/var" + # Optional mount point for local overrides and configurations; see notes in docs Environment="SC4S_LOCAL_CONFIG_MOUNT=-v /opt/sc4s/local:/opt/syslog-ng/etc/conf.d/local:z" # Optional mount point for local disk archive (EWMM output) files # Environment="SC4S_LOCAL_ARCHIVE_MOUNT=-v /opt/sc4s/archive:/opt/syslog-ng/var/archive:z" -# Mount point for local disk buffer (required) -Environment="SC4S_LOCAL_DISK_BUFFER_MOUNT=-v /opt/sc4s/disk-buffer:/opt/syslog-ng/var/data/disk-buffer:z" - # Uncomment the following line if custom TLS certs are provided # Environment="SC4S_TLS_DIR=-v /opt/sc4s/tls:/opt/syslog-ng/tls:z" @@ -39,35 +42,36 @@ ExecStartPre=/usr/bin/podman run \ --rm $SC4S_IMAGE -s ExecStart=/usr/bin/podman run -p 514:514 -p 514:514/udp -p 6514:6514 \ --env-file=/opt/sc4s/env_file \ + "$SC4S_PERSIST_VOLUME" \ "$SC4S_LOCAL_CONFIG_MOUNT" \ - "$SC4S_LOCAL_DISK_BUFFER_MOUNT" \ "$SC4S_LOCAL_ARCHIVE_MOUNT" \ "$SC4S_TLS_DIR" \ --name SC4S \ --rm $SC4S_IMAGE ``` +* Execute the following command to create a local volume that will contain the disk buffer files in the event of a communication +failure to the upstream destination(s). This will also be used to keep track of the state of syslog-ng between restarts, and in +particular the state of the disk buffer. This is a required step. +``` +sudo docker volume create splunk-sc4s-var +``` + * Create the subdirectory ``/opt/sc4s/local``. This will be used as a mount point for local overrides and configurations. - * The empty ``local`` directory created above will populate with templates at the first invocation -of SC4S for local configurations and overrides. Changes made to these files will be preserved on subsequent -restarts (i.e. a "no-clobber" copy is performed for any missing files). _Do not_ change the directory structure of + * The empty ``local`` directory created above will populate with defaults and examples at the first invocation +of SC4S for local configurations and context overrides. _Do not_ change the directory structure of the files that are laid down; change (or add) only individual files if desired. SC4S depends on the directory layout -to read the local configurations properly. +to read the local configurations properly. See the notes below for which files will be preserved on restarts. - * You can back up the contents of this directory elsewhere and return the directory to an empty state -when a new version of SC4S is released to pick up any new changes provided by Splunk. Upon a restart, -the direcory will populate as it did when you first installed SC4S. Your previous changes can then -be merged back in and will take effect after another restart. - -* Create the subdirectory ``/opt/sc4s/disk-buffer``. This will be used as a mount point for local disk buffering -of events in the event of network failure to the Splunk infrastructure. - - * This directory will populate with the disk buffer files upon SC4S startup. If SC4S restarts for any reason, a new -set of files will be created in addition to the original ones. _The original ones will not be removed_. -If you are sure, after stopping SC4S, that all data has been sent, these files can be removed. They will be created -again upon restart + * In the `local/config` directory, there are example log path files (`lp-example.*`) and a filter (`example.conf`) in the +appropriate subdirectories. These should _not_ be used directly, but copied as examples for your own log path development. +They _will_ get overwritten at each SC4S start. + * In the `local/context` directory, if you change the "non-example" version of a file (e.g. `splunk_index.csv`) the changes +will be preserved on a restart. However, the "example" files _themselves_ (e.g. `splunk_index.csv.example`) will be updated +regularly, and should be used as a template to merge new/changed functionality into existing context files. + * Create the subdirectory ``/opt/sc4s/archive``. This will be used as a mount point for local storage of syslog events (if the optional mount is uncommented above). The events will be written in the syslog-ng EWMM format. See the "configuration" document for details on the directory structure the archive uses. @@ -131,26 +135,30 @@ documentation to identify the specific environment variables that are mapped to ```ini [Unit] Description=SC4S Container -After=network.service -Requires=network.service +Wants=NetworkManager.service network-online.target +After=NetworkManager.service network-online.target + +[Install] +WantedBy=multi-user.target [Service] Environment="SC4S_IMAGE=splunk/scs:latest" +# Required mount point for syslog-ng persist data (including disk buffer) +Environment="SC4S_PERSIST_VOLUME=-v splunk-sc4s-var:/opt/syslog-ng/var" + # Optional mount point for local overrides and configurations; see notes in docs Environment="SC4S_LOCAL_CONFIG_MOUNT=-v /opt/sc4s/local:/opt/syslog-ng/etc/conf.d/local:z" # Optional mount point for local disk archive (EWMM output) files # Environment="SC4S_LOCAL_ARCHIVE_MOUNT=-v /opt/sc4s/archive:/opt/syslog-ng/var/archive:z" -# Mount point for local disk buffer (required) -Environment="SC4S_LOCAL_DISK_BUFFER_MOUNT=-v /opt/sc4s/disk-buffer:/opt/syslog-ng/var/data/disk-buffer:z" - # Uncomment the following line if custom TLS certs are provided # Environment="SC4S_TLS_DIR=-v /opt/sc4s/tls:/opt/syslog-ng/tls" TimeoutStartSec=0 Restart=always + ExecStartPre=/usr/bin/podman pull $SC4S_IMAGE ExecStartPre=/usr/bin/podman run \ --env-file=/opt/sc4s/env_file \ @@ -159,8 +167,8 @@ ExecStartPre=/usr/bin/podman run \ --rm $SC4S_IMAGE -s ExecStart=/usr/bin/podman run -p 514:514 -p 514:514/udp -p 6514:6514 -p 5000-5020:5000-5020 -p 5000-5020:5000-5020/udp \ --env-file=/opt/sc4s/env_file \ + "$SC4S_PERSIST_VOLUME" \ "$SC4S_LOCAL_CONFIG_MOUNT" \ - "$SC4S_LOCAL_DISK_BUFFER_MOUNT" \ "$SC4S_LOCAL_ARCHIVE_MOUNT" \ "$SC4S_TLS_DIR" \ --name SC4S \ @@ -266,3 +274,61 @@ Oct 1 05:29:55 77cd4776af41 syslog-ng[1]: Syslog connection closed; fd='49', cl If you see http server errors such as 4xx or 5xx responses from the http (HEC) endpoint, one or more of the items above are likely set incorrectly. If validating/fixing the configuration fails to correct the problem, proceed to the "Troubleshooting" section for more information. + +# SC4S non-root operation + +To operate SC4S as a user other than root, follow the instructions above, with these modifications: + +## Prepare SC4S user + +Create a non-root user in which to run SC4S and prepare podman for non-root operation: + +```bash +sudo useradd -m -d /home/sc4s -s /bin/bash sc4s +sudo su - sc4s +mkdir -p /home/sc4s/local +mkdir -p /home/sc4s/archive +mkdir -p /home/sc4s/tls +podman system migrate +``` + +## Initial Setup + +NOTE: Be sure to exectute all instructions below as the SC4S user created above with the exception of changes to the unit file, +which requires sudo access. + +Make the following changes to the unit file(s) configured in the main section: + +* Add the name of the user created above immediately after the Service declaration, as shown in the snippet below: + +``` +[Service] +User=sc4s +``` + +* Replace all references to `/opt/sc4s` in the "Environment" declarations with `/home/sc4s`. Make sure _not_ to change the +right-hand-side of the mount. For example: + +``` +Environment="SC4S_LOCAL_CONFIG_MOUNT=-v /home/sc4s/local:/opt/syslog-ng/etc/conf.d/local:z" +``` + +* Replace all references to standard UDP/TCP outside listening ports (typically 514) on the _left hand side only_ of the port pairs +with arbirtrary high-numbered (> 1024) ports so that the container can listen without root privleges. The right hand side of the pairs +(also typically 514) should remain unchanged: + +``` +ExecStart=/usr/bin/podman run -p 2514:514 -p 2514:514/udp -p 6514:6514 +``` + +If not done in the "Prepare SC4S user" above, create the three local mount directories as instructed in the main instructions, +replacing the head of the directory (`/opt/sc4s`) with the sc4s service user's home directory as shown below: +``` +mkdir /home/sc4s/local +mkdir /home/sc4s/archive +mkdir /home/sc4s/tls +``` + +## Remaining Setup + +The remainder of the setup can be followed directly from the main setup instructions. diff --git a/docs/upgrade.md b/docs/upgrade.md new file mode 100644 index 0000000..38b1a0d --- /dev/null +++ b/docs/upgrade.md @@ -0,0 +1,12 @@ +# Upgrading Splunk Connect for Syslog + +Splunk Connect for Syslog is updated regularly using a CI/CD development process. The notes below outline significant changes that +must be taken into account prior and after an upgrade. Ensure to follow specific instructions below to ensure a smooth transition to +a new version of SC4S in production. + +## Version 1.9.0 + +Example context files have been added to the local mount `context` directory. These example files will be updated at each release +to outline support for new data sources, which can be added to existing context files (those without the `.example` extension). +Existing context files will _not_ be overwritten on subsequent SC4S starts/upgrades, so ensure that any new content from these example +files is incorporated into existing context files. diff --git a/mkdocs.yml b/mkdocs.yml index dcf986c..8f18791 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -32,6 +32,7 @@ nav: - 'Demo Lab': 'demo.md' - Performance: 'performance.md' - Troubleshooting: 'troubleshooting.md' + - 'Upgrading SC4S': 'upgrade.md' markdown_extensions: - toc: