diff --git a/docs/gettingstarted.md b/docs/gettingstarted.md index cfcc7ba..6c4dffd 100644 --- a/docs/gettingstarted.md +++ b/docs/gettingstarted.md @@ -2,8 +2,8 @@ Splunk Connect for Syslog is a containerized distribution of syslog-ng with a configuration framework designed to simplify getting syslog data into Splunk Enterprise and Splunk Cloud. Our approach is -to provide a container runtime agnostic solution allowing customers to follow our guidance or adapt the solution -to their enterprise container strategy. +to provide a runtime-agnostic solution allowing customers to deploy using the container runtime +environment of choice. # Planning Deployment @@ -13,11 +13,11 @@ transmission of events between computer systems over UDP, TCP, or TLS. The proto overhead on the sender favoring performance over reliability. This fundamental choice means any instability or resource constraint will cause data to be lost in transmission. -* When practical and cost effective considering the importance of completeness as a requirement, place the SC4s +* When practical and cost effective considering the importance of completeness as a requirement, place the sc4s instance in the same VLAN as the source device. * Avoid crossing a Wireless network, WAN, Firewall, Load Balancer, or inline IDS. -* When High Availability of a single instance of SC4S is required, implement multi node clustering of the container +* When High Availability of a single instance of SC4S is required, implement multi node clustering of the container environment. * Avoid TCP except where the source is unable to contain the event to a single UDP packet. * Avoid TLS except where the event may cross a untrusted network. @@ -25,7 +25,9 @@ environment. # Implementation -## Setup indexes in Splunk +## Splunk Setup + +### Create Indexes SC4S is pre-configured to map each sourcetype to a typical index, for new installations best practice is to create the following indexes in Splunk. The indexes can be customized easily if desired. If using defaults create the following indexes on Splunk: @@ -38,28 +40,33 @@ indexes in Splunk. The indexes can be customized easily if desired. If using def * netipam * em_metrics (ensure this is created as a metrics index) -## Install Related Splunk Apps +### Install Related Splunk Apps Install the following: * [Splunk App for Infrastructure](https://splunkbase.splunk.com/app/3975/) * [Splunk Add-on for Infrastructure](https://splunkbase.splunk.com/app/4217/) -* [Splunk Metrics Workspace](https://splunkbase.splunk.com/app/4192/) *NOTE Included in Splunk 7.3.0 and above* +* [Splunk Metrics Workspace](https://splunkbase.splunk.com/app/4192/) *NOTE Included in Splunk 7.3.0 and above* -## Setup Splunk HTTP Event Collector +### Configure the Splunk HTTP Event Collector -- Set up the Splunk HTTP Event Collector with the HEC endpoints behind a load balancer (VIP) configured for https round robin *WITHOUT* sticky session. Alternatively, a list of HEC endpoint URLs can be configured in SC4S if no load balancer is in place. In either case, it is recommended that SC4S traffic be sent to HEC endpoints configured directly on the indexers rather than an intermediate tier of HWFs. -- Create a HEC token that will be used by SC4S and ensure the token has access to place events in main, em_metrics, and all indexes used as event destinations - -### Splunk Cloud +- Set up the Splunk HTTP Event Collector with the HEC endpoints behind a load balancer (VIP) configured for https round robin *WITHOUT* sticky +session. Alternatively, a list of HEC endpoint URLs can be configured in SC4S if no load balancer is in place. In either case, it is +recommended that SC4S traffic be sent to HEC endpoints configured directly on the indexers rather than an intermediate tier of HWFs. +- Create a HEC token that will be used by SC4S and ensure the token has access to place events in main, em_metrics, and all indexes used as +event destinations +- Refer to [Splunk Cloud](http://docs.splunk.com/Documentation/Splunk/7.3.1/Data/UsetheHTTPEventCollector#Configure_HTTP_Event_Collector_on_managed_Splunk_Cloud) +or [Splunk Enterprise](http://dev.splunk.com/view/event-collector/SP-CAAAE6Q) for specific HEC configuration instructions based on your +Splunk type. -Refer to [Splunk Cloud](http://docs.splunk.com/Documentation/Splunk/7.3.1/Data/UsetheHTTPEventCollector#Configure_HTTP_Event_Collector_on_managed_Splunk_Cloud) +## Implement a Container Runtime and SC4S -### Splunk Enterprise +### Prerequisites -Refer to [Splunk Enterprise](http://dev.splunk.com/view/event-collector/SP-CAAAE6Q) +* Linux host with Docker (CE 19.x or greater with Docker Swarm) or Podman enabled, depending on runtime choice (below). +* A network load balancer (NLB) configured for round robin. Note: Special consideration may be required when more advanced products are used. The optimal configuration of the load balancer will round robin each http POST request (not each connection). -## Implement a container run time and sc4s +### Select a Container Runtime and SC4S Configuration | Container and Orchestration | Notes | |-----------------------------|-------| @@ -68,21 +75,9 @@ Refer to [Splunk Enterprise](http://dev.splunk.com/view/event-collector/SP-CAAAE | [Docker CE + Swarm single node](gettingstarted/docker-swarm-general.md) | Option for Debian, Ubuntu, and CentOS desiring swarm orchestration | | [Docker CE + Swarm single node RHEL 7.7](gettingstarted/docker-swarm-rhel7.md) | Option for RedHat 7.7 desiring swarm orchestration | -# Pre-req - -* Linux host with Docker 19.x or newer with Docker Swarm enabled - * [Getting Started](https://docs.docker.com/get-started/) -* A Splunk index for metrics typically "em_metrics" -* One or more Splunk indexes for events collected by SC4S -* Splunk HTTP event collector enabled with a token dedicated for SC4S - * [Splunk Enterprise](http://dev.splunk.com/view/event-collector/SP-CAAAE6Q) - * [Splunk Cloud](http://docs.splunk.com/Documentation/Splunk/7.3.1/Data/UsetheHTTPEventCollector#Configure_HTTP_Event_Collector_on_managed_Splunk_Cloud) -* A network load balancer (NLB) configured for round robin. Note: Special consideration may be required when more advanced products are used. The optimal configuration of the load balancer will round robin each http POST request (not each connection) - -## Scale out +# Scale out Additional hosts can be deployed for syslog collection from additional network zones and locations ![SC4S deployment diagram](SC4S%20deployment.png) - diff --git a/docs/gettingstarted/docker-swarm-general.md b/docs/gettingstarted/docker-swarm-general.md index 123a303..d7beaa7 100644 --- a/docs/gettingstarted/docker-swarm-general.md +++ b/docs/gettingstarted/docker-swarm-general.md @@ -3,11 +3,11 @@ Refer to [Getting Started](https://docs.docker.com/get-started/) -# Setup +# SC4S Configuration * Create a directory on the server for configuration. This should be available to all administrators, for example: ``/opt/sc4s/`` -* Create a docker-compose.yml file and place it in the directory created above based on the following template: +* Create a docker-compose.yml file in the directory created above, based on the following template: ```yaml version: "3.7" @@ -29,10 +29,10 @@ services: - /opt/sc4s/env_file volumes: #Uncomment the following line if overriding index destinations -# - ./sc4s-juniper/splunk_index.csv:/opt/syslog-ng/etc/context-local/splunk_index.csv +# - ./context-local/splunk_index.csv:/opt/syslog-ng/etc/context-local/splunk_index.csv #Uncomment the following lines if using a host or network based filter and log_path -# - ./sc4s-juniper/vendor_product_by_source.csv:/opt/syslog-ng/etc/context-local/vendor_product_by_source.csv -# - ./sc4s-juniper/vendor_product_by_source.conf:/opt/syslog-ng/etc/context-local/vendor_product_by_source.conf +# - ./context-local/vendor_product_by_source.csv:/opt/syslog-ng/etc/context-local/vendor_product_by_source.csv +# - ./context-local/vendor_product_by_source.conf:/opt/syslog-ng/etc/context-local/vendor_product_by_source.conf ``` @@ -52,9 +52,11 @@ SPLUNK_METRICS_INDEX=em_metrics ## Configure index destinations for Splunk -Log paths are preconfigured to utilize a convention of index destinations that is suitable for most customers. This step is optional to allow customization of index destinations. +Log paths are preconfigured to utilize a convention of index destinations that is suitable for most customers. This step is optional to allow +customization of index destinations. -* Download the latest context.csv file to a subdirectory SC4S below the docker-compose.yml file created above. +* Create a subdirectory called ``context-local`` in the directory (e.g. ``/opt/sc4s/``) created in the first step above. From this directory, +execute the command below to download the index context file: ```bash sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/master/package/etc/context-local/splunk_index.csv @@ -63,16 +65,19 @@ sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/mas ## Configure sources by source IP or host name -Legacy sources and non-standard-compliant source require configuration by source IP or hostname as included in the event. The following steps apply to support such sources. To identify sources which require this step refer to the "sources" section of this documentation. +Legacy sources and non-standard-compliant source require configuration by source IP or hostname as included in the event. The following steps +apply to support such sources. To identify sources which require this step refer to the "sources" section of this documentation. + +* If not already done in the step immediately above, create a subdirectory called ``context-local`` in the directory (e.g. ``/opt/sc4s/``) +created in the first step above. From this directory, execute the commands below to download the vendor context files: -* Download the latest vendor_product_by_source.conf file to a subdirectory SC4S below the docker-compose.yml file created above. ```bash sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/master/package/etc/context-local/vendor_product_by_source.conf sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/master/package/etc/context-local/vendor_product_by_source.csv ``` * Edit the file to identify appropriate vendor products by host glob or network mask using syslog-ng filter syntax. -* Start SC4S. +# Start SC4S ```bash docker stack deploy --compose-file docker-compose.yml sc4s @@ -91,7 +96,10 @@ of such legacy nonstandard sources we provide a means of dedicating a container an alternate port. Refer to the Sources documentation to identify the specific variable used to enable a specific port for the technology in use. -In the following example ``-p 5000-5020:5000-5020`` allows for up to 21 technology specific ports modify the range as appropriate +Refer to the "Sources" documentation to identify the specific variable used to enable a specific port for the technology in use. + +In the following example ``-p 5000-5020:5000-5020`` allows for up to 21 technology-specific ports. Modify the individual ports or a +range as appropriate for your network. * Modify the unit file ``/opt/sc4s/docker-compose.yml`` ```yaml diff --git a/docs/gettingstarted/docker-swarm-rhel7.md b/docs/gettingstarted/docker-swarm-rhel7.md index 6342c71..e57bdb5 100644 --- a/docs/gettingstarted/docker-swarm-rhel7.md +++ b/docs/gettingstarted/docker-swarm-rhel7.md @@ -3,7 +3,7 @@ *Warning* this method of installing docker on RHEL does not appear to be supported: -Enable required repositories +## Enable required repositories ```bash subscription-manager repos --enable=rhel-7-server-rpms subscription-manager repos --enable=rhel-7-server-extras-rpms @@ -11,7 +11,7 @@ subscription-manager repos --enable=rhel-7-server-optional-rpms yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo ``` -Enable EPEL +## Enable EPEL ```bash yum install wget -y cd /tmp @@ -19,23 +19,23 @@ wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm rpm -Uvh epel-release-latest-7.noarch.rpm ``` -Install required packages and enable docker +## Install required packages and enable docker ```bash yum install docker-ce -y systemctl enable docker.service systemctl start docker.service ``` -Initialize Docker Swarm +## Initialize Docker Swarm ```bash sudo docker swarm init ``` -# Setup +# SC4S Configuration * Create a directory on the server for configuration. This should be available to all administrators, for example: ``/opt/sc4s/`` -* Create a docker-compose.yml file and place it in the directory created above based on the following template: +* Create a docker-compose.yml file in the directory created above, based on the following template: ```yaml version: "3.7" @@ -57,10 +57,10 @@ services: - /opt/sc4s/env_file volumes: #Uncomment the following line if overriding index destinations -# - ./sc4s-juniper/splunk_index.csv:/opt/syslog-ng/etc/context-local/splunk_index.csv +# - ./context-local/splunk_index.csv:/opt/syslog-ng/etc/context-local/splunk_index.csv #Uncomment the following lines if using a host or network based filter and log_path -# - ./sc4s-juniper/vendor_product_by_source.csv:/opt/syslog-ng/etc/context-local/vendor_product_by_source.csv -# - ./sc4s-juniper/vendor_product_by_source.conf:/opt/syslog-ng/etc/context-local/vendor_product_by_source.conf +# - ./context-local/vendor_product_by_source.csv:/opt/syslog-ng/etc/context-local/vendor_product_by_source.csv +# - ./context-local/vendor_product_by_source.conf:/opt/syslog-ng/etc/context-local/vendor_product_by_source.conf ``` @@ -81,9 +81,11 @@ SPLUNK_METRICS_INDEX=em_metrics ## Configure index destinations for Splunk -Log paths are preconfigured to utilize a convention of index destinations that is suitable for most customers. This step is optional to allow customization of index destinations. +Log paths are preconfigured to utilize a convention of index destinations that is suitable for most customers. This step is optional to allow +customization of index destinations. -* Download the latest context.csv file to a subdirectory SC4S below the docker-compose.yml file created above. +* Create a subdirectory called ``context-local`` in the directory (e.g. ``/opt/sc4s/``) created in the first step above. From this directory, +execute the command below to download the index context file: ```bash sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/master/package/etc/context-local/splunk_index.csv @@ -92,16 +94,19 @@ sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/mas ## Configure sources by source IP or host name -Legacy sources and non-standard-compliant sources require configuration by source IP or hostname as included in the event. The following steps apply to support such sources. To identify sources which require this step refer to the "sources" section of this documentation. +Legacy sources and non-standard-compliant sources require configuration by source IP or hostname as included in the event. The following steps +apply to support such sources. To identify sources which require this step refer to the "sources" section of this documentation. + +* If not already done in the step immediately above, create a subdirectory called ``context-local`` in the directory (e.g. ``/opt/sc4s/``) +created in the first step above. From this directory, execute the commands below to download the vendor context files: -* Download the latest vendor_product_by_source.conf file to a subdirectory SC4S below the docker-compose.yml file created above ```bash sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/master/package/etc/context-local/vendor_product_by_source.conf sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/master/package/etc/context-local/vendor_product_by_source.csv ``` * Edit the file to identify appropriate vendor products by host glob or network mask using syslog-ng filter syntax. -* Start SC4S. +# Start SC4S ```bash sudo docker stack deploy --compose-file docker-compose.yml sc4s @@ -119,7 +124,8 @@ of such legacy nonstandard sources we provide a means of dedicating a container an alternate port. Refer to the Sources documentation to identify the specific variable used to enable a specific port for the technology in use. -In the following example ``-p 5000-5020:5000-5020`` allows for up to 21 technology specific ports modify the range as appropriate +In the following example ``-p 5000-5020:5000-5020`` allows for up to 21 technology-specific ports. Modify the individual ports or a +range as appropriate for your network. * Modify the unit file ``/opt/sc4s/docker-compose.yml`` ```yaml diff --git a/docs/gettingstarted/docker-systemd-general.md b/docs/gettingstarted/docker-systemd-general.md index 808cbf6..1ab3909 100644 --- a/docs/gettingstarted/docker-systemd-general.md +++ b/docs/gettingstarted/docker-systemd-general.md @@ -7,7 +7,7 @@ Refer to [Getting Started](https://docs.docker.com/get-started/) * Create a systemd unit file use to start the container with the host os. ``/lib/systemd/system/sc4s.service`` -*NOTE*: The 3 volumes "-v" are optional and should be omited if the customization options are not used +*NOTE*: In a future release, the mechanism to support mounted volumes (the three -v arguments in the unit file below) will change. For now, do not change these arguments, and ensure that the three files they reference are downloaded to the proper directory per the configuration instructions below. *NOTE-2*: Replace the URL and HEC tokens with the appropriate values for our environment @@ -58,11 +58,16 @@ SPLUNK_METRICS_INDEX=em_metrics #SC4S_DEST_SPLUNK_HEC_TLS_VERIFY=no ``` +*NOTE*: While optional, until the final mechanism for mounted volumes in containers is released, do _not_ skip the download (wget) steps in +the configuration steps below. Ensure these three files are in place in the directory specified, even if there is no intent to modify them. + ## Configure index destinations for Splunk -Log paths are preconfigured to utilize a convention of index destinations that is suitable for most customers. This step is optional to allow customization of index destinations. +Log paths are preconfigured to utilize a convention of index destinations that is suitable for most customers. This step is optional to allow +customization of index destinations. -* Download the latest context.csv file to a directory ``/opt/sc4s/default/`` +* Create a directory (e.g. ``/opt/sc4s/default/`` ). Make sure the local directory references in the unit file above (the ``-v`` arguments) +match the directory you created above. From this directory, execute the following to download the latest index context file: ```bash sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/master/package/etc/context-local/splunk_index.csv @@ -71,16 +76,20 @@ sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/mas ## Configure sources by source IP or host name -Legacy sources and non-standard-compliant sources require configuration by source IP or hostname as included in the event. The following steps apply to support such sources. To identify sources which require this step refer to the "sources" section of this documentation. +Legacy sources and non-standard-compliant sources require configuration by source IP or hostname as included in the event. The following steps +apply to support such sources. To identify sources which require this step refer to the "sources" section of this documentation. + +* If not already done in the step immeidately above, create a directory (e.g. ``/opt/sc4s/default/`` ). Make sure the local directory +references in the unit file above (the ``-v`` arguments) match the directory you created above. From this directory, execute the following to +download the latest vendor context files: -* Download the latest vendor_product_by_source.conf file to a directory ``/opt/sc4s/default/`` ```bash sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/master/package/etc/context-local/vendor_product_by_source.conf sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/master/package/etc/context-local/vendor_product_by_source.csv ``` * Edit the file to identify appropriate vendor products by host glob or network mask using syslog-ng filter syntax. -* Start SC4S. +# Start SC4S ```bash sudo systemctl daemon-reload @@ -97,9 +106,10 @@ an alternate port. Refer to the Sources documentation to identify the specific variable used to enable a specific port for the technology in use. -In the following example ``-p 5000-5020:5000-5020`` allows for up to 21 technology specific ports modify the range as appropriate +In the following example ``-p 5000-5020:5000-5020`` allows for up to 21 technology-specific ports. Modify the individual ports or a +range as appropriate for your network. -* Modify the unit file ``/opt/sc4s/default/env_file`` +* Modify the unit file ``/lib/systemd/system/sc4s.service`` ```ini [Unit] Description=SC4S Container diff --git a/docs/gettingstarted/podman-systemd-general.md b/docs/gettingstarted/podman-systemd-general.md index e6f500c..4693995 100644 --- a/docs/gettingstarted/podman-systemd-general.md +++ b/docs/gettingstarted/podman-systemd-general.md @@ -3,13 +3,11 @@ Refer to [Installation](https://podman.io/getting-started/installation) -# Configure sc4s - # Setup * Create a systemd unit file use to start the container with the host os. ``/lib/systemd/system/sc4s.service`` -*NOTE*: The 3 volumes "-v" are optional and should be omited if the customization options are not used +*NOTE*: In a future release, the mechanism to support mounted volumes (the three -v arguments in the unit file below) will change. For now, do not change these arguments, and ensure that the three files they reference are downloaded to the proper directory per the configuration instructions below. *NOTE-2*: Replace the URL and HEC tokens with the appropriate values for our environment @@ -45,6 +43,9 @@ $SC4S_IMAGE ``` +*NOTE*: While optional, until the final mechanism for mounted volumes in containers is released, do _not_ skip the download (wget) steps in +the configuration steps below. Ensure these three files are in place in the directory specified, even if there is no intent to modify them. + ## Configure the SC4S environment Create the following file ``/opt/sc4s/default/env_file`` @@ -63,9 +64,11 @@ SPLUNK_METRICS_INDEX=em_metrics ## Configure index destinations for Splunk -Log paths are preconfigured to utilize a convention of index destinations that is suitable for most customers. This step is optional to allow customization of index destinations. +Log paths are preconfigured to utilize a convention of index destinations that is suitable for most customers. This step is optional to allow +customization of index destinations. -* Download the latest context.csv file to a directory ``/opt/sc4s/default/`` +* Create a directory (e.g. ``/opt/sc4s/default/`` ). Make sure the local directory references in the unit file above (the ``-v`` arguments) +match the directory you created above. From this directory, execute the following to download the latest index context file: ```bash sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/master/package/etc/context-local/splunk_index.csv @@ -74,16 +77,20 @@ sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/mas ## Configure sources by source IP or host name -Legacy sources and non-standard-compliant sources require configuration by source IP or hostname as included in the event. The following steps apply to support such sources. To identify sources which require this step refer to the "sources" section of this documentation. +Legacy sources and non-standard-compliant sources require configuration by source IP or hostname as included in the event. The following steps +apply to support such sources. To identify sources which require this step refer to the "sources" section of this documentation. + +* If not already done in the step immeidately above, create a directory (e.g. ``/opt/sc4s/default/`` ). Make sure the local directory +references in the unit file above (the ``-v`` arguments) match the directory you created above. From this directory, execute the following to +download the latest vendor context files: -* Download the latest vendor_product_by_source.conf file to a directory ``/opt/sc4s/default/`` ```bash sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/master/package/etc/context-local/vendor_product_by_source.conf sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/master/package/etc/context-local/vendor_product_by_source.csv ``` * Edit the file to identify appropriate vendor products by host glob or network mask using syslog-ng filter syntax. -* Start SC4S. +# Start SC4S ```bash sudo systemctl daemon-reload @@ -99,7 +106,8 @@ an alternate port. Refer to the Sources documentation to identify the specific variable used to enable a specific port for the technology in use. -In the following example ``-p 5000-5020:5000-5020`` allows for up to 21 technology specific ports modify the range as appropriate +In the following example ``-p 5000-5020:5000-5020`` allows for up to 21 technology-specific ports. Modify the individual ports or a +range as appropriate for your network. * Modify the unit file ``/lib/systemd/system/sc4s.service`` ```ini