From d7f01617ac19039fd65b7a74e3131f4f087d4c66 Mon Sep 17 00:00:00 2001 From: l-kutch <55860089+l-kutch@users.noreply.github.com> Date: Fri, 4 Oct 2019 12:04:38 -0700 Subject: [PATCH] removing optional parts (#109) * Revising to make the optional parts mandatory for other getting started files --- docs/gettingstarted/docker-swarm-general.md | 40 ++++++++--------- docs/gettingstarted/docker-swarm-rhel7.md | 39 ++++++++-------- docs/gettingstarted/docker-systemd-general.md | 38 +++++++--------- docs/gettingstarted/podman-systemd-general.md | 45 +++++++++---------- 4 files changed, 74 insertions(+), 88 deletions(-) diff --git a/docs/gettingstarted/docker-swarm-general.md b/docs/gettingstarted/docker-swarm-general.md index 9fef61c..f9a6e12 100644 --- a/docs/gettingstarted/docker-swarm-general.md +++ b/docs/gettingstarted/docker-swarm-general.md @@ -35,21 +35,11 @@ services: - /opt/sc4s/tls:/opt/syslog-ng/tls ``` -* NOTE: While strictly optional, it is recommended that you create and/or download all files and directories referenced in the yml template -above (`volumes` declarations) according to the configuration steps that follow. The TLS options are described in the "Configuration" section. -Failure to match the volume specification in the `yml` file with what exists locally will result in startup errors. +* NOTE: If you use the default `volumes` declarations as-is from the `docker-compose.yml` file template example, you must create and/or download all files and directories referenced in the file according to the configuration steps that follow. The TLS-specific options are described in the "Configure the sc4s Environment" section. Failure to match the volume specification in the `yml` file with what exists locally will result in startup errors. ## Configure the SC4S environment -Create the following file ``/opt/sc4s/env_file`` and add the environment variables below: - -* Update ``SPLUNK_HEC_URL`` and ``SPLUNK_HEC_TOKEN`` to reflect the correct values for your environment - -* Set `SC4S_DEST_SPLUNK_HEC_WORKERS` to match the number of indexers and/or HWFs with HEC endpoints. If the endpoint is a VIP, -match this value to the total number of indexers behind the load balancer. - -* NOTE: Splunk Connect for Syslog defaults to secure configurations. If you are not using trusted SSL certificates, be sure to -uncomment the last line in the example below. +Create a file named ``/opt/sc4s/env_file`` and add the following environment variables: ```dotenv SPLUNK_HEC_URL=https://splunk.smg.aws:8088/services/collector/event @@ -62,32 +52,38 @@ SPLUNK_METRICS_INDEX=em_metrics #SC4S_DEST_SPLUNK_HEC_TLS_VERIFY=no ``` +* Update ``SPLUNK_HEC_URL`` and ``SPLUNK_HEC_TOKEN`` to reflect the correct values for your environment. + +* Set `SC4S_DEST_SPLUNK_HEC_WORKERS` to match the number of indexers and/or HWFs with HEC endpoints. If the endpoint is a VIP, +match this value to the total number of indexers behind the load balancer. + +* NOTE: Splunk Connect for Syslog defaults to secure configurations. If you are not using trusted SSL certificates, be sure to +uncomment the last line in the example above. + ## Configure index destinations for Splunk -Log paths are preconfigured to utilize a convention of index destinations that is suitable for most customers. This step is optional to allow -customization of index destinations. +Log paths are preconfigured to utilize a convention of index destinations that is suitable for most customers. -* Create a subdirectory called ``default`` in the directory (e.g. ``/opt/sc4s/``) created in the first step above. From this directory, +* Create a subdirectory called ``default`` in the directory that you created in the previous step (e.g. ``/opt/sc4s/``). Make sure the local directory volume references in the `yml` file match the directory you create here. From this directory, execute the command below to download the index context file: ```bash sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/master/package/etc/context-local/splunk_index.csv ``` -* Edit splunk_index.csv review the index configuration and revise as required for sourcetypes utilized in your environment. +* Edit splunk_index.csv to review the index configuration and revise as required for the sourcetypes utilized in your environment. ## Configure sources by source IP or host name -Legacy sources and non-standard-compliant source require configuration by source IP or hostname as included in the event. The following steps -apply to support such sources. To identify sources which require this step refer to the "sources" section of this documentation. +Legacy sources and non-standard-compliant sources require configuration by source IP or hostname as included in the event. The following steps +apply to support such sources. To identify sources that require this step, refer to the "sources" section of this documentation. -* If not already done in the step immediately above, create a subdirectory called ``default`` in the directory (e.g. ``/opt/sc4s/``) -created in the first step above. From this directory, execute the commands below to download the vendor context files: +* If not already done, create a subdirectory called ``default`` in the ``/opt/sc4s/`` directory. Make sure the local directory volume references in the `yml` file match the directory you create here. From this directory, execute the following commands to download the vendor context files: ```bash sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/master/package/etc/context-local/vendor_product_by_source.conf sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/master/package/etc/context-local/vendor_product_by_source.csv ``` -* Edit the file to identify appropriate vendor products by host glob or network mask using syslog-ng filter syntax. +* If you have legacy sources and non-standard-compliant sources, edit the file to properly identify these products by host glob or network mask using syslog-ng filter syntax. ## Start/Restart SC4S @@ -224,4 +220,4 @@ Oct 1 05:29:55 77cd4776af41 syslog-ng[1]: Syslog connection closed; fd='49', cl ``` If you see http server errors such as 4xx or 5xx responses from the http (HEC) endpoint, one or more of the items above are likely set incorrectly. If validating/fixing the configuration fails to correct the problem, proceed to the "Troubleshooting" section for more -information. \ No newline at end of file +information. diff --git a/docs/gettingstarted/docker-swarm-rhel7.md b/docs/gettingstarted/docker-swarm-rhel7.md index eba329c..008dd65 100644 --- a/docs/gettingstarted/docker-swarm-rhel7.md +++ b/docs/gettingstarted/docker-swarm-rhel7.md @@ -63,21 +63,12 @@ services: - /opt/sc4s/tls:/opt/syslog-ng/tls ``` -* NOTE: While strictly optional, it is recommended that you create and/or download all files and directories referenced in the yml template -above (`volumes` declarations) according to the configuration steps that follow. The TLS options are described in the "Configuration" section. -Failure to match the volume specification in the `yml` file with what exists locally will result in startup errors. +* NOTE: If you use the default `volumes` declarations as-is from the `docker-compose.yml` file template example, do create and/or download all files and directories referenced in the file according to the configuration steps that follow. The TLS-specific options are described in the "Configure the sc4s environment" section. Failure to match the volume specification in the `yml` file with what exists locally will result in startup errors. -## Configure the SC4S environment - -Create the following file ``/opt/sc4s/env_file`` and add the environment variables below: -* Update ``SPLUNK_HEC_URL`` and ``SPLUNK_HEC_TOKEN`` to reflect the correct values for your environment - -* Set `SC4S_DEST_SPLUNK_HEC_WORKERS` to match the number of indexers and/or HWFs with HEC endpoints. If the endpoint is a VIP, -match this value to the total number of indexers behind the load balancer. +## Configure the SC4S environment -* NOTE: Splunk Connect for Syslog defaults to secure configurations. If you are not using trusted SSL certificates, be sure to -uncomment the last line in the example below. +Create a file named ``/opt/sc4s/env_file`` and add the following environment variables: ```dotenv SPLUNK_HEC_URL=https://splunk.smg.aws:8088/services/collector/event @@ -90,32 +81,40 @@ SPLUNK_METRICS_INDEX=em_metrics #SC4S_DEST_SPLUNK_HEC_TLS_VERIFY=no ``` +* Update ``SPLUNK_HEC_URL`` and ``SPLUNK_HEC_TOKEN`` to reflect the correct values for your environment. + +* Set `SC4S_DEST_SPLUNK_HEC_WORKERS` to match the number of indexers and/or HWFs with HEC endpoints. If the endpoint is a VIP, +match this value to the total number of indexers behind the load balancer. + +* NOTE: Splunk Connect for Syslog defaults to secure configurations. If you are not using trusted SSL certificates, be sure to +uncomment the last line in the example below. + + + ## Configure index destinations for Splunk -Log paths are preconfigured to utilize a convention of index destinations that is suitable for most customers. This step is optional to allow -customization of index destinations. +Log paths are preconfigured to utilize a convention of index destinations that is suitable for most customers. -* Create a subdirectory called ``default`` in the directory (e.g. ``/opt/sc4s/``) created in the first step above. From this directory, +* Create a subdirectory called ``default`` in the directory that you created in the previous step (e.g. ``/opt/sc4s/``). Make sure the local directory volume references in the `yml` file match the directory you create here. From this directory, execute the command below to download the index context file: ```bash sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/master/package/etc/context-local/splunk_index.csv ``` -* Edit splunk_index.csv review the index configuration and revise as required for sourcertypes utilized in your environment. +* Edit splunk_index.csv to review the index configuration and revise as required for the sourcertypes utilized in your environment. ## Configure sources by source IP or host name Legacy sources and non-standard-compliant sources require configuration by source IP or hostname as included in the event. The following steps -apply to support such sources. To identify sources which require this step refer to the "sources" section of this documentation. +apply to support such sources. To identify sources that require this step, refer to the "sources" section of this documentation. -* If not already done in the step immediately above, create a subdirectory called ``default`` in the directory (e.g. ``/opt/sc4s/``) -created in the first step above. From this directory, execute the commands below to download the vendor context files: +* If not already done, create a subdirectory called ``default`` in the ``/opt/sc4s/`` directory. Make sure the local directory volume references in the `yml` file match the directory you create here. From this directory, execute the following commands to download the vendor context files: ```bash sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/master/package/etc/context-local/vendor_product_by_source.conf sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/master/package/etc/context-local/vendor_product_by_source.csv ``` -* Edit the file to identify appropriate vendor products by host glob or network mask using syslog-ng filter syntax. +* If you have legacy sources and non-standard-compliant sources, edit the file to properly identify these products by host glob or network mask using syslog-ng filter syntax. ## Start/Restart SC4S diff --git a/docs/gettingstarted/docker-systemd-general.md b/docs/gettingstarted/docker-systemd-general.md index d4cd5c4..0e1fc58 100644 --- a/docs/gettingstarted/docker-systemd-general.md +++ b/docs/gettingstarted/docker-systemd-general.md @@ -42,20 +42,11 @@ ExecStart=/usr/bin/docker run -p 514:514 -p 514:514/udp \ $SC4S_IMAGE ``` -* NOTE: While strictly optional, it is recommended that you create and/or download all files and directories referenced in the unit -file above (the `Environment` assignments) according to the configuration steps that follow. The TLS options are described in the "Configuration" section. +* NOTE: If you use the default `Environment` assignments as-is from the `sc4s.service` unit file template example, do create and/or download all files and directories referenced in the file's Service stanza according to the configuration steps that follow. The TLS-specific options are described in the "Configure the sc4s environment" section. ## Configure the SC4S environment -Create the following file ``/opt/sc4s/default/env_file`` and add the environment variables below: - -* Update ``SPLUNK_HEC_URL`` and ``SPLUNK_HEC_TOKEN`` to reflect the correct values for your environment - -* Set `SC4S_DEST_SPLUNK_HEC_WORKERS` to match the number of indexers and/or HWFs with HEC endpoints. If the endpoint is a VIP, -match this value to the total number of indexers behind the load balancer. - -* NOTE: Splunk Connect for Syslog defaults to secure configurations. If you are not using trusted SSL certificates, be sure to -uncomment the last line in the example below. +Create a file named ``/opt/sc4s/default/env_file`` and add the following environment variables: ```dotenv SPLUNK_HEC_URL=https://splunk.smg.aws:8088/services/collector/event @@ -68,33 +59,38 @@ SPLUNK_METRICS_INDEX=em_metrics #SC4S_DEST_SPLUNK_HEC_TLS_VERIFY=no ``` +* Update ``SPLUNK_HEC_URL`` and ``SPLUNK_HEC_TOKEN`` to reflect the correct values for your environment + +* Set `SC4S_DEST_SPLUNK_HEC_WORKERS` to match the number of indexers and/or HWFs with HEC endpoints. If the endpoint is a VIP, +match this value to the total number of indexers behind the load balancer. + +* NOTE: Splunk Connect for Syslog defaults to secure configurations. If you are not using trusted SSL certificates, be sure to +uncomment the last line in the example. + + ## Configure index destinations for Splunk -Log paths are preconfigured to utilize a convention of index destinations that is suitable for most customers. This step is optional to allow -customization of index destinations. +Log paths are preconfigured to utilize a convention of index destinations that is suitable for most customers. -* Create a directory (e.g. ``/opt/sc4s/default/`` ). Make sure the local directory references in the unit file above (the ``-v`` variables) -match the directory you created above. From this directory, execute the following to download the latest index context file: +* Create a directory (e.g. ``/opt/sc4s/default/`` ). Make sure the local directory references in the `sc4s.service` unit file match the directory you create here (the ``-v`` variables). From this directory, execute the following to download the latest index context file: ```bash sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/master/package/etc/context-local/splunk_index.csv ``` -* Edit splunk_index.csv review the index configuration and revise as required for sourcertypes utilized in your environment. +* Edit splunk_index.csv to review the index configuration and revise as required for the sourcertypes utilized in your environment. ## Configure sources by source IP or host name Legacy sources and non-standard-compliant sources require configuration by source IP or hostname as included in the event. The following steps -apply to support such sources. To identify sources which require this step refer to the "sources" section of this documentation. +apply to support such sources. To identify sources that require this step refer to the "sources" section of this documentation. -* If not already done in the step immediately above, create a directory (e.g. ``/opt/sc4s/default/`` ). Make sure the local directory -references in the unit file above (the ``-v`` variables) match the directory you created above. From this directory, execute the following to -download the latest vendor context files: +* If not already done, create a directory (e.g. ``/opt/sc4s/default/`` ). Make sure the local directory references in the `sc4s.service` unit file match the directory you create here (the ``-v`` variables). From this directory, execute the following to download the latest vendor context files: ```bash sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/master/package/etc/context-local/vendor_product_by_source.conf sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/master/package/etc/context-local/vendor_product_by_source.csv ``` -* Edit the file to identify appropriate vendor products by host glob or network mask using syslog-ng filter syntax. +* If you have legacy sources and non-standard-compliant sources, edit the file to properly identify these products by host glob or network mask using syslog-ng filter syntax. ## Configure SC4S for systemd and start SC4S diff --git a/docs/gettingstarted/podman-systemd-general.md b/docs/gettingstarted/podman-systemd-general.md index 74b6855..361ac12 100644 --- a/docs/gettingstarted/podman-systemd-general.md +++ b/docs/gettingstarted/podman-systemd-general.md @@ -20,8 +20,8 @@ Environment="SC4S_IMAGE=splunk/scs:latest" Environment="SC4S_UNIT_SPLUNK_INDEX=-v /opt/sc4s/default/splunk_index.csv:/opt/syslog-ng/etc/context-local/splunk_index.csv" #Note Uncomment the following two lines for host and ip based source type mapping AND download the two file templates per getting started -#Environment="SC4S_UNIT_VP_CSV=-v /opt/sc4s/default/vendor_product_by_source.csv:/opt/syslog-ng/etc/context-local/vendor_product_by_source.csv" -#Environment="SC4S_UNIT_VP_CONF=-v /opt/sc4s/default/vendor_product_by_source.conf:/opt/syslog-ng/etc/context-local/vendor_product_by_source.conf" +Environment="SC4S_UNIT_VP_CSV=-v /opt/sc4s/default/vendor_product_by_source.csv:/opt/syslog-ng/etc/context-local/vendor_product_by_source.csv" +Environment="SC4S_UNIT_VP_CONF=-v /opt/sc4s/default/vendor_product_by_source.conf:/opt/syslog-ng/etc/context-local/vendor_product_by_source.conf" #Uncomment the following line if custom TLS certs are provided #Environment="SC4S_TLS_DIR=-v /opt/sc4s/tls:/opt/syslog-ng/tls" @@ -42,20 +42,11 @@ ExecStart=/usr/bin/podman run -p 514:514 -p 514:514/udp \ $SC4S_IMAGE ``` -* NOTE: While strictly optional, it is recommended that you create and/or download all files and directories referenced in the unit -file above (the `Environment` assignments) according to the configuration steps that follow. The TLS options are described in the "Configuration" section. +* NOTE: If you use the default `Environment` assignments as-is from the `sc4s.service` unit file template example, do create and/or download all files and directories referenced in the file's Service stanza according to the configuration steps that follow. The TLS-specific options are described in the "Configure the sc4s environment" section. ## Configure the sc4s environment -Create the following file ``/opt/sc4s/default/env_file`` and add the environment variables below: - -* Update ``SPLUNK_HEC_URL`` and ``SPLUNK_HEC_TOKEN`` to reflect the correct values for your environment - -* Set `SC4S_DEST_SPLUNK_HEC_WORKERS` to match the number of indexers and/or HWFs with HEC endpoints. If the endpoint is a VIP, -match this value to the total number of indexers behind the load balancer. - -* NOTE: Splunk Connect for Syslog defaults to secure configurations. If you are not using trusted SSL certificates, be sure to -uncomment the last line in the example below. +Create a file named ``/opt/sc4s/default/env_file`` and add the following environment variables: ```dotenv SPLUNK_HEC_URL=https://splunk.smg.aws:8088/services/collector/event @@ -68,33 +59,37 @@ SPLUNK_METRICS_INDEX=em_metrics #SC4S_DEST_SPLUNK_HEC_TLS_VERIFY=no ``` +* Update ``SPLUNK_HEC_URL`` and ``SPLUNK_HEC_TOKEN`` to reflect the correct values for your environment + +* Set `SC4S_DEST_SPLUNK_HEC_WORKERS` to match the number of indexers and/or HWFs with HEC endpoints. If the endpoint is a VIP, +match this value to the total number of indexers behind the load balancer. + +* NOTE: Splunk Connect for Syslog defaults to secure configurations. If you are not using trusted SSL certificates, be sure to +uncomment the last line in the example. + + ## Configure index destinations for Splunk -Log paths are preconfigured to utilize a convention of index destinations that is suitable for most customers. This step is optional to allow -customization of index destinations. +Log paths are preconfigured to utilize a convention of index destinations that are suitable for most customers. -* Create a directory (e.g. ``/opt/sc4s/default/`` ). Make sure the local directory references in the unit file above (the ``-v`` variables) -match the directory you created above. From this directory, execute the following to download the latest index context file: +* Create a directory (e.g. ``/opt/sc4s/default/`` ). Make sure the local directory references in the `sc4s.service` unit file match the directory you create here (the ``-v`` variables). From this directory, execute the following to download the latest index context file: ```bash sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/master/package/etc/context-local/splunk_index.csv ``` -* Edit splunk_index.csv review the index configuration and revise as required for sourcertypes utilized in your environment. +* Edit splunk_index.csv to review the index configuration and revise as required for the sourcertypes utilized in your environment. ## Configure sources by source IP or host name -Legacy sources and non-standard-compliant sources require configuration by source IP or hostname as included in the event. The following steps -apply to support such sources. To identify sources which require this step refer to the "sources" section of this documentation. +Legacy sources and non-standard-compliant sources require configuration by source IP or hostname as included in the event. The following steps apply to support such sources. To identify sources that require this step, refer to the "sources" section of this documentation. -* If not already done in the step immediately above, create a directory (e.g. ``/opt/sc4s/default/`` ). Make sure the local directory -references in the unit file above (the ``-v`` variables) match the directory you created above. From this directory, execute the following to -download the latest vendor context files: +* If not already done, create a directory (e.g. ``/opt/sc4s/default/`` ). Make sure the local directory references in the `sc4s.service` unit file match the directory you create here (the ``-v`` variables). From this directory, execute the following to download the latest vendor context files: ```bash sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/master/package/etc/context-local/vendor_product_by_source.conf sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/master/package/etc/context-local/vendor_product_by_source.csv ``` -* Edit the file to identify appropriate vendor products by host glob or network mask using syslog-ng filter syntax. +* If you have legacy sources and non-standard-compliant sources, edit the file to properly identify these products by host glob or network mask using syslog-ng filter syntax. ## Configure SC4S for systemd and start SC4S @@ -235,4 +230,4 @@ Oct 1 05:29:55 77cd4776af41 syslog-ng[1]: Syslog connection closed; fd='49', cl ``` If you see http server errors such as 4xx or 5xx responses from the http (HEC) endpoint, one or more of the items above are likely set incorrectly. If validating/fixing the configuration fails to correct the problem, proceed to the "Troubleshooting" section for more -information. \ No newline at end of file +information.