diff --git a/docs/gettingstarted/byoe-rhel7.md b/docs/gettingstarted/byoe-rhel7.md index e91f5b1..2cb4540 100644 --- a/docs/gettingstarted/byoe-rhel7.md +++ b/docs/gettingstarted/byoe-rhel7.md @@ -21,7 +21,16 @@ for the reason why syslog-ng builds are so dated in almost all RHEL/Debian distr # BYOE Installation Instructions -* Install CentOS or RHEL 7.7 +These installation instructions assume a recent RHEL or CentOS-based release. Minor adjustments may have to be made for +Debian/Ubuntu. In addition, almost _all_ pre-compiled binaries for syslog-ng assume installation in `etc/syslog-ng`; these instructions +will reflect that. + +The following installation instructions are summarized from a +[blog](https://www.syslog-ng.com/community/b/blog/posts/introducing-the-syslog-ng-stable-rpm-repositories) +maintained by a developer at One Identity (formerly Balabit), who is the owner of the syslog-ng Open Source project. +It is always adivisable to review the blog for the latest changes to the repo(s), as changes here are quite dynamic. + +* Install CentOS or RHEL 7.7/8.0 * Enable EPEL (Centos 7) @@ -55,15 +64,10 @@ sudo systemctl stop syslog-ng sudo systemctl disable syslog-ng ``` -* Download the latest bare_metal.tar from [releases](https://github.com/splunk/splunk-connect-for-syslog/releases) on github and untar the package +* Download the latest bare_metal.tar from [releases](https://github.com/splunk/splunk-connect-for-syslog/releases) on github and untar the package in `/etc/syslog-ng` ```bash -cd /tmp -sudo wget https://github.com/splunk/splunk-connect-for-syslog/releases/download/0.12.1/baremetal.tar -tar -xf baremetal.tar -sudo mkdir -p /opt/syslog-ng/etc -sudo mkdir -p /opt/syslog-ng/var -sudo cp -R etc/* /opt/syslog-ng/etc/ +sudo wget -c https://github.com/splunk/splunk-connect-for-syslog/releases/download/latest/baremetal.tar -O - | sudo tar -x -C /etc/syslog-ng ``` * Install gomplate and confirm that the version is 3.5.0 or newer @@ -103,13 +107,20 @@ Restart=on-failure WantedBy=multi-user.target ``` -* create the file ``/opt/sc4s/bin/preconfig.sh``. This file should be made executable according to your file permission standards. Add the following content: +* NOTE: The `wget` process above will download a file called `entrypoint.sh` and place it in `/etc/syslog-ng`. This is the +preconfiguration file that is used for the container version of sc4s, and forms the foundation of the BYOE version of the file you will +create below. Do _not_ use it verbatim as there are differences between them (most notably the install location). However, it does include +the "latest and greatest" updates from the container, and should be used (with appropriate modifications) as the basis of the contents of +`preconfig.sh` below. + +* create the file ``/opt/sc4s/bin/preconfig.sh``. This file should be made executable according to your file permission standards. +Add the following content (but be sure to check the note above to ensure the latest updates are included): ```bash #!/usr/bin/env bash source scl_source enable rh-python36 -cd /opt/syslog-ng +cd /etc/syslog-ng #The following is no longer needed but retained as a comment just in case we run into command line length issues #for d in $(find /opt/syslog-ng/etc -type d) #do @@ -120,15 +131,14 @@ cd /opt/syslog-ng # --exclude=*.conf --exclude=*.csv --exclude=*.t --exclude=.*\ # --output-map="$d/{{ .in | strings.ReplaceAll \".conf.tmpl\" \".conf\" }}" #done + gomplate $(find . -name *.tmpl | sed -E 's/^(\/.*\/)*(.*)\..*$/--file=\2.tmpl --out=\2/') --template t=etc/go_templates/ -mkdir -p /opt/syslog-ng/etc/conf.d/local/context/ -mkdir -p /opt/syslog-ng/etc/conf.d/local/config/ -cp --verbose -n /opt/syslog-ng/etc/context_templates/* /opt/syslog-ng/etc/conf.d/local/context/ -cp --verbose -R -n /opt/syslog-ng/etc/local_config/* /opt/syslog-ng/etc/conf.d/local/config/ -mkdir -p /opt/syslog-ng/var/data/disk-buffer/ -mkdir -p /opt/syslog-ng/var/archive/ -mkdir -p /opt/syslog-ng/tls/ +mkdir -p /etc/syslog-ng/etc/conf.d/local/context/ +mkdir -p /etc/syslog-ng/etc/conf.d/local/config/ +cp /etc/syslog-ng/etc/context_templates/* /etc/syslog-ng/etc/conf.d/local/context/ +for file in /etc/syslog-ng/etc/conf.d/local/context/*.example ; do cp -v -n $file ${file%.example}; done +cp -v -R /etc/syslog-ng/etc/local_config/* /etc/syslog-ng/etc/conf.d/local/config/ ``` * (Optional) Execute the preconfiguration shell script created above. You may also optionally execute it as part of the unit @@ -142,7 +152,7 @@ sudo bash /opt/sc4s/bin/preconfig.sh * Create the file ``/opt/sc4s/env_file`` and add the following environment variables: ```dotenv -SYSLOGNG_OPTS=-f /opt/syslog-ng/etc/syslog-ng.conf +SYSLOGNG_OPTS=-f /etc/syslog-ng/etc/syslog-ng.conf SPLUNK_HEC_URL=https://splunk.smg.aws:8088 SPLUNK_HEC_TOKEN=a778f63a-5dff-4e3c-a72c-a03183659e94 SC4S_DEST_SPLUNK_HEC_WORKERS=6 diff --git a/docs/gettingstarted/docker-swarm-general.md b/docs/gettingstarted/docker-swarm-general.md index a26a1c1..9e1a85f 100644 --- a/docs/gettingstarted/docker-swarm-general.md +++ b/docs/gettingstarted/docker-swarm-general.md @@ -55,33 +55,34 @@ services: - /opt/sc4s/env_file volumes: - /opt/sc4s/local:/opt/syslog-ng/etc/conf.d/local:z - - /opt/sc4s/disk-buffer:/opt/syslog-ng/var/data/disk-buffer:z + - splunk-sc4s-var:/opt/syslog-ng/var # Uncomment the following line if local disk archiving is desired # - /opt/sc4s/archive:/opt/syslog-ng/var/archive:z # Uncomment the following line if custom TLS certs are provided # - /opt/sc4s/tls:/opt/syslog-ng/tls:z ``` +* Execute the following command to create a local volume that will contain the disk buffer files in the event of a communication +failure to the upstream destination(s). This will also be used to keep track of the state of syslog-ng between restarts, and in +particular the state of the disk buffer. This is a required step. +``` +sudo docker volume create splunk-sc4s-var +``` + * Create the subdirectory ``/opt/sc4s/local``. This will be used as a mount point for local overrides and configurations. - * The empty ``local`` directory created above will populate with templates at the first invocation -of SC4S for local configurations and overrides. Changes made to these files will be preserved on subsequent -restarts (i.e. a "no-clobber" copy is performed for any missing files). _Do not_ change the directory structure of + * The empty ``local`` directory created above will populate with defaults and examples at the first invocation +of SC4S for local configurations and context overrides. _Do not_ change the directory structure of the files that are laid down; change (or add) only individual files if desired. SC4S depends on the directory layout -to read the local configurations properly. - - * You can back up the contents of this directory elsewhere and return the directory to an empty state -when a new version of SC4S is released to pick up any new changes provided by Splunk. Upon a restart, -the direcory will populate as it did when you first installed SC4S. Your previous changes can then -be merged back in and will take effect after another restart. +to read the local configurations properly. See the notes below for which files will be preserved on restarts. -* Create the subdirectory ``/opt/sc4s/disk-buffer``. This will be used as a mount point for local disk buffering -of events in the event of network failure to the Splunk infrastructure. + * In the `local/config` directory, there are example log path files (`lp-example.*`) and a filter (`example.conf`) in the +appropriate subdirectories. These should _not_ be used directly, but copied as examples for your own log path development. +They _will_ get overwritten at each SC4S start. - * This directory will populate with the disk buffer files upon SC4S startup. If SC4S restarts for any reason, a new - set of files will be created in addition to the original ones. _The original ones will not be removed_. - If you are sure, after stopping SC4S, that all data has been sent, these files can be removed. They will be created - again upon restart. + * In the `local/context` directory, if you change the "non-example" version of a file (e.g. `splunk_index.csv`) the changes +will be preserved on a restart. However, the "example" files _themselves_ (e.g. `splunk_index.csv.example`) will be updated +regularly, and should be used as a template to merge new/changed functionality into existing context files. * Create the subdirectory ``/opt/sc4s/archive``. This will be used as a mount point for local storage of syslog events (if the optional mount is uncommented above). The events will be written in the syslog-ng EWMM format. See the "configuration" @@ -183,7 +184,7 @@ services: - /opt/sc4s/env_file volumes: - /opt/sc4s/local:/opt/syslog-ng/etc/conf.d/local:z - - /opt/sc4s/disk-buffer:/opt/syslog-ng/var/data/disk-buffer:z + - splunk-sc4s-var:/opt/syslog-ng/var # Uncomment the following line if local disk archiving is desired # - /opt/sc4s/archive:/opt/syslog-ng/var/archive:z # Uncomment the following line if custom TLS certs are provided diff --git a/docs/gettingstarted/docker-swarm-rhel7.md b/docs/gettingstarted/docker-swarm-rhel7.md index 251c9fa..c276f64 100644 --- a/docs/gettingstarted/docker-swarm-rhel7.md +++ b/docs/gettingstarted/docker-swarm-rhel7.md @@ -63,34 +63,35 @@ services: - /opt/sc4s/env_file volumes: - /opt/sc4s/local:/opt/syslog-ng/etc/conf.d/local:z - - /opt/sc4s/disk-buffer:/opt/syslog-ng/var/data/disk-buffer:z + - splunk-sc4s-var:/opt/syslog-ng/var # Uncomment the following line if local disk archiving is desired # - /opt/sc4s/archive:/opt/syslog-ng/var/archive:z # Uncomment the following line if custom TLS certs are provided # - /opt/sc4s/tls:/opt/syslog-ng/tls:z ``` +* Execute the following command to create a local volume that will contain the disk buffer files in the event of a communication +failure to the upstream destination(s). This will also be used to keep track of the state of syslog-ng between restarts, and in +particular the state of the disk buffer. This is a required step. +``` +sudo docker volume create splunk-sc4s-var +``` + * Create the subdirectory ``/opt/sc4s/local``. This will be used as a mount point for local overrides and configurations. - * The empty ``local`` directory created above will populate with templates at the first invocation -of SC4S for local configurations and overrides. Changes made to these files will be preserved on subsequent -restarts (i.e. a "no-clobber" copy is performed for any missing files). _Do not_ change the directory structure of + * The empty ``local`` directory created above will populate with defaults and examples at the first invocation +of SC4S for local configurations and context overrides. _Do not_ change the directory structure of the files that are laid down; change (or add) only individual files if desired. SC4S depends on the directory layout -to read the local configurations properly. +to read the local configurations properly. See the notes below for which files will be preserved on restarts. - * You can back up the contents of this directory elsewhere and return the directory to an empty state -when a new version of SC4S is released to pick up any new changes provided by Splunk. Upon a restart, -the direcory will populate as it did when you first installed SC4S. Your previous changes can then -be merged back in and will take effect after another restart. - -* Create the subdirectory ``/opt/sc4s/disk-buffer``. This will be used as a mount point for local disk buffering -of events in the event of network failure to the Splunk infrastructure. - - * This directory will populate with the disk buffer files upon SC4S startup. If SC4S restarts for any reason, a new -set of files will be created in addition to the original ones. _The original ones will not be removed_. -If you are sure, after stopping SC4S, that all data has been sent, these files can be removed. They will be created -again upon restart + * In the `local/config` directory, there are example log path files (`lp-example.*`) and a filter (`example.conf`) in the +appropriate subdirectories. These should _not_ be used directly, but copied as examples for your own log path development. +They _will_ get overwritten at each SC4S start. + * In the `local/context` directory, if you change the "non-example" version of a file (e.g. `splunk_index.csv`) the changes +will be preserved on a restart. However, the "example" files _themselves_ (e.g. `splunk_index.csv.example`) will be updated +regularly, and should be used as a template to merge new/changed functionality into existing context files. + * Create the subdirectory ``/opt/sc4s/archive``. This will be used as a mount point for local storage of syslog events (if the optional mount is uncommented above). The events will be written in the syslog-ng EWMM format. See the "configuration" document for details on the directory structure the archive uses. @@ -191,7 +192,7 @@ services: - /opt/sc4s/env_file volumes: - /opt/sc4s/local:/opt/syslog-ng/etc/conf.d/local:z - - /opt/sc4s/disk-buffer:/opt/syslog-ng/var/data/disk-buffer:z + - splunk-sc4s-var:/opt/syslog-ng/var # Uncomment the following line if local disk archiving is desired # - /opt/sc4s/archive:/opt/syslog-ng/var/archive:z # Uncomment the following line if custom TLS certs are provided diff --git a/docs/gettingstarted/docker-systemd-general.md b/docs/gettingstarted/docker-systemd-general.md index 55a4788..4c3dbee 100644 --- a/docs/gettingstarted/docker-systemd-general.md +++ b/docs/gettingstarted/docker-systemd-general.md @@ -28,21 +28,24 @@ net.ipv4.ip_forward=1 ```ini [Unit] Description=SC4S Container -Wants=network.target network-online.target -After=network.target network-online.target +Wants=NetworkManager.service network-online.target +After=NetworkManager.service network-online.target + +[Install] +WantedBy=multi-user.target [Service] Environment="SC4S_IMAGE=splunk/scs:latest" +# Required mount point for syslog-ng persist data (including disk buffer) +Environment="SC4S_PERSIST_VOLUME=-v splunk-sc4s-var:/opt/syslog-ng/var" + # Optional mount point for local overrides and configurations; see notes in docs Environment="SC4S_LOCAL_CONFIG_MOUNT=-v /opt/sc4s/local:/opt/syslog-ng/etc/conf.d/local:z" # Optional mount point for local disk archive (EWMM output) files # Environment="SC4S_LOCAL_ARCHIVE_MOUNT=-v /opt/sc4s/archive:/opt/syslog-ng/var/archive:z" -# Mount point for local disk buffer (required) -Environment="SC4S_LOCAL_DISK_BUFFER_MOUNT=-v /opt/sc4s/disk-buffer:/opt/syslog-ng/var/data/disk-buffer:z" - # Uncomment the following line if custom TLS certs are provided # Environment="SC4S_TLS_DIR=-v /opt/sc4s/tls:/opt/syslog-ng/tls:z" @@ -50,42 +53,38 @@ TimeoutStartSec=0 Restart=always ExecStartPre=/usr/bin/docker pull $SC4S_IMAGE -ExecStartPre=/usr/bin/docker run \ - --env-file=/opt/sc4s/env_file \ - "$SC4S_LOCAL_CONFIG_MOUNT" \ - --name SC4S_preflight \ - --rm $SC4S_IMAGE -s ExecStart=/usr/bin/docker run -p 514:514 -p 514:514/udp -p 6514:6514 \ --env-file=/opt/sc4s/env_file \ + "$SC4S_PERSIST_VOLUME" \ "$SC4S_LOCAL_CONFIG_MOUNT" \ - "$SC4S_LOCAL_DISK_BUFFER_MOUNT" \ "$SC4S_LOCAL_ARCHIVE_MOUNT" \ "$SC4S_TLS_DIR" \ --name SC4S \ --rm $SC4S_IMAGE ``` +* Execute the following command to create a local volume that will contain the disk buffer files in the event of a communication +failure to the upstream destination(s). This will also be used to keep track of the state of syslog-ng between restarts, and in +particular the state of the disk buffer. This is a required step. +``` +sudo docker volume create splunk-sc4s-var +``` + * Create the subdirectory ``/opt/sc4s/local``. This will be used as a mount point for local overrides and configurations. - * The empty ``local`` directory created above will populate with templates at the first invocation -of SC4S for local configurations and overrides. Changes made to these files will be preserved on subsequent -restarts (i.e. a "no-clobber" copy is performed for any missing files). _Do not_ change the directory structure of + * The empty ``local`` directory created above will populate with defaults and examples at the first invocation +of SC4S for local configurations and context overrides. _Do not_ change the directory structure of the files that are laid down; change (or add) only individual files if desired. SC4S depends on the directory layout -to read the local configurations properly. - - * You can back up the contents of this directory elsewhere and return the directory to an empty state -when a new version of SC4S is released to pick up any new changes provided by Splunk. Upon a restart, -the direcory will populate as it did when you first installed SC4S. Your previous changes can then -be merged back in and will take effect after another restart. +to read the local configurations properly. See the notes below for which files will be preserved on restarts. -* Create the subdirectory ``/opt/sc4s/disk-buffer``. This will be used as a mount point for local disk buffering -of events in the event of network failure to the Splunk infrastructure. - - * This directory will populate with the disk buffer files upon SC4S startup. If SC4S restarts for any reason, a new -set of files will be created in addition to the original ones. _The original ones will not be removed_. -If you are sure, after stopping SC4S, that all data has been sent, these files can be removed. They will be created -again upon restart + * In the `local/config` directory, there are example log path files (`lp-example.*`) and a filter (`example.conf`) in the +appropriate subdirectories. These should _not_ be used directly, but copied as examples for your own log path development. +They _will_ get overwritten at each SC4S start. + * In the `local/context` directory, if you change the "non-example" version of a file (e.g. `splunk_index.csv`) the changes +will be preserved on a restart. However, the "example" files _themselves_ (e.g. `splunk_index.csv.example`) will be updated +regularly, and should be used as a template to merge new/changed functionality into existing context files. + * Create the subdirectory ``/opt/sc4s/archive``. This will be used as a mount point for local storage of syslog events (if the optional mount is uncommented above). The events will be written in the syslog-ng EWMM format. See the "configuration" document for details on the directory structure the archive uses. @@ -149,36 +148,35 @@ documentation to identify the specific environment variables that are mapped to ```ini [Unit] Description=SC4S Container -After=network.service -Requires=network.service +Wants=NetworkManager.service network-online.target +After=NetworkManager.service network-online.target + +[Install] +WantedBy=multi-user.target [Service] Environment="SC4S_IMAGE=splunk/scs:latest" +# Required mount point for syslog-ng persist data (including disk buffer) +Environment="SC4S_PERSIST_VOLUME=-v splunk-sc4s-var:/opt/syslog-ng/var" + # Optional mount point for local overrides and configurations; see notes in docs Environment="SC4S_LOCAL_CONFIG_MOUNT=-v /opt/sc4s/local:/opt/syslog-ng/etc/conf.d/local:z" # Optional mount point for local disk archive (EWMM output) files # Environment="SC4S_LOCAL_ARCHIVE_MOUNT=-v /opt/sc4s/archive:/opt/syslog-ng/var/archive:z" -# Mount point for local disk buffer (required) -Environment="SC4S_LOCAL_DISK_BUFFER_MOUNT=-v /opt/sc4s/disk-buffer:/opt/syslog-ng/var/data/disk-buffer:z" - # Uncomment the following line if custom TLS certs are provided # Environment="SC4S_TLS_DIR=-v /opt/sc4s/tls:/opt/syslog-ng/tls" TimeoutStartSec=0 Restart=always + ExecStartPre=/usr/bin/docker pull $SC4S_IMAGE -ExecStartPre=/usr/bin/docker run \ - --env-file=/opt/sc4s/env_file \ - "$SC4S_LOCAL_CONFIG_MOUNT" \ - --name SC4S_preflight \ - --rm $SC4S_IMAGE -s ExecStart=/usr/bin/docker run -p 514:514 -p 514:514/udp -p 6514:6514 -p 5000-5020:5000-5020 -p 5000-5020:5000-5020/udp \ --env-file=/opt/sc4s/env_file \ + "$SC4S_PERSIST_VOLUME" \ "$SC4S_LOCAL_CONFIG_MOUNT" \ - "$SC4S_LOCAL_DISK_BUFFER_MOUNT" \ "$SC4S_LOCAL_ARCHIVE_MOUNT" \ "$SC4S_TLS_DIR" \ --name SC4S \ diff --git a/docs/gettingstarted/podman-systemd-general.md b/docs/gettingstarted/podman-systemd-general.md index 7b49762..0d76496 100644 --- a/docs/gettingstarted/podman-systemd-general.md +++ b/docs/gettingstarted/podman-systemd-general.md @@ -10,64 +10,63 @@ Refer to [Installation](https://podman.io/getting-started/installation) ```ini [Unit] Description=SC4S Container -Wants=network.target network-online.target -After=network.target network-online.target +Wants=NetworkManager.service network-online.target +After=NetworkManager.service network-online.target + +[Install] +WantedBy=multi-user.target [Service] Environment="SC4S_IMAGE=splunk/scs:latest" +# Required mount point for syslog-ng persist data (including disk buffer) +Environment="SC4S_PERSIST_VOLUME=-v splunk-sc4s-var:/opt/syslog-ng/var" + # Optional mount point for local overrides and configurations; see notes in docs Environment="SC4S_LOCAL_CONFIG_MOUNT=-v /opt/sc4s/local:/opt/syslog-ng/etc/conf.d/local:z" # Optional mount point for local disk archive (EWMM output) files # Environment="SC4S_LOCAL_ARCHIVE_MOUNT=-v /opt/sc4s/archive:/opt/syslog-ng/var/archive:z" -# Mount point for local disk buffer (required) -Environment="SC4S_LOCAL_DISK_BUFFER_MOUNT=-v /opt/sc4s/disk-buffer:/opt/syslog-ng/var/data/disk-buffer:z" - # Uncomment the following line if custom TLS certs are provided # Environment="SC4S_TLS_DIR=-v /opt/sc4s/tls:/opt/syslog-ng/tls:z" TimeoutStartSec=0 Restart=always -ExecStartPre=/usr/bin/podman pull $SC4S_IMAGE -ExecStartPre=/usr/bin/podman run \ - --env-file=/opt/sc4s/env_file \ - "$SC4S_LOCAL_CONFIG_MOUNT" \ - --name SC4S_preflight \ - --rm $SC4S_IMAGE -s -ExecStart=/usr/bin/podman run -p 514:514 -p 514:514/udp -p 6514:6514 \ +ExecStartPre=/usr/bin/docker pull $SC4S_IMAGE +ExecStart=/usr/bin/docker run -p 514:514 -p 514:514/udp -p 6514:6514 \ --env-file=/opt/sc4s/env_file \ + "$SC4S_PERSIST_VOLUME" \ "$SC4S_LOCAL_CONFIG_MOUNT" \ - "$SC4S_LOCAL_DISK_BUFFER_MOUNT" \ "$SC4S_LOCAL_ARCHIVE_MOUNT" \ "$SC4S_TLS_DIR" \ --name SC4S \ --rm $SC4S_IMAGE ``` +* Execute the following command to create a local volume that will contain the disk buffer files in the event of a communication +failure to the upstream destination(s). This will also be used to keep track of the state of syslog-ng between restarts, and in +particular the state of the disk buffer. This is a required step. +``` +sudo docker volume create splunk-sc4s-var +``` + * Create the subdirectory ``/opt/sc4s/local``. This will be used as a mount point for local overrides and configurations. - * The empty ``local`` directory created above will populate with templates at the first invocation -of SC4S for local configurations and overrides. Changes made to these files will be preserved on subsequent -restarts (i.e. a "no-clobber" copy is performed for any missing files). _Do not_ change the directory structure of + * The empty ``local`` directory created above will populate with defaults and examples at the first invocation +of SC4S for local configurations and context overrides. _Do not_ change the directory structure of the files that are laid down; change (or add) only individual files if desired. SC4S depends on the directory layout -to read the local configurations properly. - - * You can back up the contents of this directory elsewhere and return the directory to an empty state -when a new version of SC4S is released to pick up any new changes provided by Splunk. Upon a restart, -the direcory will populate as it did when you first installed SC4S. Your previous changes can then -be merged back in and will take effect after another restart. +to read the local configurations properly. See the notes below for which files will be preserved on restarts. -* Create the subdirectory ``/opt/sc4s/disk-buffer``. This will be used as a mount point for local disk buffering -of events in the event of network failure to the Splunk infrastructure. - - * This directory will populate with the disk buffer files upon SC4S startup. If SC4S restarts for any reason, a new -set of files will be created in addition to the original ones. _The original ones will not be removed_. -If you are sure, after stopping SC4S, that all data has been sent, these files can be removed. They will be created -again upon restart + * In the `local/config` directory, there are example log path files (`lp-example.*`) and a filter (`example.conf`) in the +appropriate subdirectories. These should _not_ be used directly, but copied as examples for your own log path development. +They _will_ get overwritten at each SC4S start. + * In the `local/context` directory, if you change the "non-example" version of a file (e.g. `splunk_index.csv`) the changes +will be preserved on a restart. However, the "example" files _themselves_ (e.g. `splunk_index.csv.example`) will be updated +regularly, and should be used as a template to merge new/changed functionality into existing context files. + * Create the subdirectory ``/opt/sc4s/archive``. This will be used as a mount point for local storage of syslog events (if the optional mount is uncommented above). The events will be written in the syslog-ng EWMM format. See the "configuration" document for details on the directory structure the archive uses. @@ -131,36 +130,35 @@ documentation to identify the specific environment variables that are mapped to ```ini [Unit] Description=SC4S Container -After=network.service -Requires=network.service +Wants=NetworkManager.service network-online.target +After=NetworkManager.service network-online.target + +[Install] +WantedBy=multi-user.target [Service] Environment="SC4S_IMAGE=splunk/scs:latest" +# Required mount point for syslog-ng persist data (including disk buffer) +Environment="SC4S_PERSIST_VOLUME=-v splunk-sc4s-var:/opt/syslog-ng/var" + # Optional mount point for local overrides and configurations; see notes in docs Environment="SC4S_LOCAL_CONFIG_MOUNT=-v /opt/sc4s/local:/opt/syslog-ng/etc/conf.d/local:z" # Optional mount point for local disk archive (EWMM output) files # Environment="SC4S_LOCAL_ARCHIVE_MOUNT=-v /opt/sc4s/archive:/opt/syslog-ng/var/archive:z" -# Mount point for local disk buffer (required) -Environment="SC4S_LOCAL_DISK_BUFFER_MOUNT=-v /opt/sc4s/disk-buffer:/opt/syslog-ng/var/data/disk-buffer:z" - # Uncomment the following line if custom TLS certs are provided # Environment="SC4S_TLS_DIR=-v /opt/sc4s/tls:/opt/syslog-ng/tls" TimeoutStartSec=0 Restart=always -ExecStartPre=/usr/bin/podman pull $SC4S_IMAGE -ExecStartPre=/usr/bin/podman run \ - --env-file=/opt/sc4s/env_file \ - "$SC4S_LOCAL_CONFIG_MOUNT" \ - --name SC4S_preflight \ - --rm $SC4S_IMAGE -s -ExecStart=/usr/bin/podman run -p 514:514 -p 514:514/udp -p 6514:6514 -p 5000-5020:5000-5020 -p 5000-5020:5000-5020/udp \ + +ExecStartPre=/usr/bin/docker pull $SC4S_IMAGE +ExecStart=/usr/bin/docker run -p 514:514 -p 514:514/udp -p 6514:6514 -p 5000-5020:5000-5020 -p 5000-5020:5000-5020/udp \ --env-file=/opt/sc4s/env_file \ + "$SC4S_PERSIST_VOLUME" \ "$SC4S_LOCAL_CONFIG_MOUNT" \ - "$SC4S_LOCAL_DISK_BUFFER_MOUNT" \ "$SC4S_LOCAL_ARCHIVE_MOUNT" \ "$SC4S_TLS_DIR" \ --name SC4S \ diff --git a/docs/release.md b/docs/release.md new file mode 100644 index 0000000..fdf5c6b --- /dev/null +++ b/docs/release.md @@ -0,0 +1,12 @@ +# Release Notes + +Splunk Connect for Syslog is updated regularly using a CI/CD development process. Check back often for the latest on each release +here. Changes that will affect current installations, as well as new and deprecated features, will be outlined below. + +## Version 1.9.0 + +* Version string added +* Example context files added to local mount. These example files will be updated at each release to add support for new data sources, +and the new content can be added to existing context files (without the `.example` extension). Existing context files will _not_ be +overwritten on subsequent SC4S starts/upgrades, so ensure that content for new data sources is incorporated in existing context files. +* Support for Cisco devices sending events with no hostname has been added. diff --git a/mkdocs.yml b/mkdocs.yml index dcf986c..8ae29c6 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -32,6 +32,7 @@ nav: - 'Demo Lab': 'demo.md' - Performance: 'performance.md' - Troubleshooting: 'troubleshooting.md' + - 'Release Notes': 'release.md' markdown_extensions: - toc: