Skip to content

Commit

Permalink
Feature/cleanup scs (#84)
Browse files Browse the repository at this point in the history
Change SCS-->SC4S in docs
  • Loading branch information
Ryan Faircloth authored and GitHub committed Sep 20, 2019
1 parent 1f9ac59 commit 48f0d7c
Show file tree
Hide file tree
Showing 29 changed files with 228 additions and 230 deletions.
4 changes: 2 additions & 2 deletions docker-compose-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,8 @@ services:
- SPLUNK_CONNECT_METHOD=${SPLUNK_CONNECT_METHOD}
- SPLUNK_DEFAULT_INDEX=${SPLUNK_DEFAULT_INDEX}
- SPLUNK_METRICS_INDEX=${SPLUNK_DEFAULT_INDEX}
- SCS_DEST_SPLUNK_HEC_TLS_VERIFY=no
- SCS_LISTEN_JUNIPER_NETSCREEN_TCP_PORT=5000
- SC4S_DEST_SPLUNK_HEC_TLS_VERIFY=no
- SC4S_LISTEN_JUNIPER_NETSCREEN_TCP_PORT=5000
splunk:
image: splunk/splunk:latest
hostname: splunk
Expand Down
2 changes: 1 addition & 1 deletion docker-compose-debug.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ services:
- SPLUNK_CONNECT_METHOD=${SPLUNK_CONNECT_METHOD}
- SPLUNK_DEFAULT_INDEX=${SPLUNK_DEFAULT_INDEX}
- SPLUNK_METRICS_INDEX=${SPLUNK_DEFAULT_INDEX}
- SCS_DEST_SPLUNK_HEC_TLS_VERIFY=no
- SC4S_DEST_SPLUNK_HEC_TLS_VERIFY=no
splunk:
image: splunk/splunk:latest
hostname: splunk
Expand Down
6 changes: 3 additions & 3 deletions docker-compose-demo.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,8 @@ services:
- SPLUNK_CONNECT_METHOD=${SPLUNK_CONNECT_METHOD}
- SPLUNK_DEFAULT_INDEX=${SPLUNK_DEFAULT_INDEX}
- SPLUNK_METRICS_INDEX=${SPLUNK_DEFAULT_INDEX}
- SCS_DEST_SPLUNK_HEC_TLS_VERIFY=no
- SCS_LISTEN_JUNIPER_NETSCREEN_TCP_PORT=5000
- SC4S_DEST_SPLUNK_HEC_TLS_VERIFY=no
- SC4S_LISTEN_JUNIPER_NETSCREEN_TCP_PORT=5000
splunk:
image: splunk/splunk:latest
hostname: splunk
Expand All @@ -65,4 +65,4 @@ volumes:
sc4s-results:
external: true
splunk-etc:
external: true
external: true
5 changes: 2 additions & 3 deletions docker-compose-perf.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,8 @@ services:
- SPLUNK_CONNECT_METHOD=${SPLUNK_CONNECT_METHOD}
- SPLUNK_DEFAULT_INDEX=${SPLUNK_DEFAULT_INDEX}
- SPLUNK_METRICS_INDEX=${SPLUNK_DEFAULT_INDEX}
- SCS_DEST_SPLUNK_HEC_TLS_VERIFY=no
- SCS_LISTEN_JUNIPER_NETSCREEN_TCP_PORT=5000
- SC4S_DEST_SPLUNK_HEC_TLS_VERIFY=no
- SC4S_LISTEN_JUNIPER_NETSCREEN_TCP_PORT=5000

splunk:
image: splunk/splunk:latest
Expand Down Expand Up @@ -61,4 +61,3 @@ services:
links:
- egbundles
- sc4s

6 changes: 3 additions & 3 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,8 @@ services:
- SPLUNK_CONNECT_METHOD=${SPLUNK_CONNECT_METHOD}
- SPLUNK_DEFAULT_INDEX=${SPLUNK_DEFAULT_INDEX}
- SPLUNK_METRICS_INDEX=${SPLUNK_DEFAULT_INDEX}
- SCS_DEST_SPLUNK_HEC_TLS_VERIFY=no
- SCS_LISTEN_JUNIPER_NETSCREEN_TCP_PORT=5000
- SC4S_DEST_SPLUNK_HEC_TLS_VERIFY=no
- SC4S_LISTEN_JUNIPER_NETSCREEN_TCP_PORT=5000
splunk:
image: splunk/splunk:latest
hostname: splunk
Expand All @@ -68,4 +68,4 @@ volumes:
sc4s-results:
external: true
splunk-etc:
external: true
external: true
8 changes: 4 additions & 4 deletions docs/Configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
|----------|---------------|-------------|
| SPLUNK_HEC_URL | url | URL(s) of the Splunk endpoint, can be a single URL space seperated list |
| SPLUNK_HEC_TOKEN | string | Splunk HTTP Event Collector Token |
| SCS_DEST_SPLUNK_HEC_TLS_VERIFY | yes(default) or no | verify HTTP(s) certificate |
| SCS_DEST_SPLUNK_HEC_CIPHER_SUITE | comma separated list | Open SSL cipher suite list |
| SCS_DEST_SPLUNK_HEC_SSL_VERSION | comma separated list | Open SSL version list |
| SCS_DEST_SPLUNK_HEC_TLS_CA_FILE | path | Custom trusted cert file |
| SC4S_DEST_SPLUNK_HEC_TLS_VERIFY | yes(default) or no | verify HTTP(s) certificate |
| SC4S_DEST_SPLUNK_HEC_CIPHER_SUITE | comma separated list | Open SSL cipher suite list |
| SC4S_DEST_SPLUNK_HEC_SSL_VERSION | comma separated list | Open SSL version list |
| SC4S_DEST_SPLUNK_HEC_TLS_CA_FILE | path | Custom trusted cert file |
10 changes: 5 additions & 5 deletions docs/gettingstarted.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,11 @@ transmission of events between computer systems over UDP, TCP, or TLS. The proto
overhead on the sender favoring performance over reliability. This fundamental choice means any instability
or resource constraint will cause data to be lost in transmission.

* When practical and cost effective considering the importance of completeness as a requirement, place the scs
* When practical and cost effective considering the importance of completeness as a requirement, place the SC4s
instance in the same VLAN as the source device.

* Avoid crossing a Wireless network, WAN, Firewall, Load Balancer, or inline IDS.
* When High Availability of a single instance of SCS is required, implement multi node clustering of the container
* When High Availability of a single instance of SC4S is required, implement multi node clustering of the container
environment.
* Avoid TCP except where the source is unable to contain the event to a single UDP packet.
* Avoid TLS except where the event may cross a untrusted network.
Expand All @@ -27,7 +27,7 @@ environment.

## Setup indexes in Splunk

SCS is pre-configured to map each sourcetype to a typical index, for new installations best practice is to create the following
SC4S is pre-configured to map each sourcetype to a typical index, for new installations best practice is to create the following
indexes in Splunk. The indexes can be customized easily if desired. If using defaults create the following indexes on Splunk:

* netauth
Expand All @@ -49,7 +49,7 @@ Install the following:
## Setup Splunk HTTP Event Collector

- Set up the Splunk HTTP Event Collector with the HEC endpoints behind a load balancer (VIP) configured for https round robin *WITHOUT* sticky session. Alternatively, a list of HEC endpoint URLs can be configured in SC4S if no load balancer is in place. In either case, it is recommended that SC4S traffic be sent to HEC endpoints configured directly on the indexers rather than an intermediate tier of HWFs.
- Create a HEC token that will be used by SCS and ensure the token has access to place events in main, em_metrics, and all indexes used as event destinations
- Create a HEC token that will be used by SC4S and ensure the token has access to place events in main, em_metrics, and all indexes used as event destinations

### Splunk Cloud

Expand All @@ -59,7 +59,7 @@ Refer to [Splunk Cloud](http://docs.splunk.com/Documentation/Splunk/7.3.1/Data/U

Refer to [Splunk Enterprise](http://dev.splunk.com/view/event-collector/SP-CAAAE6Q)

## Implement a container run time and SCS
## Implement a container run time and sc4s

| Container and Orchestration | Notes |
|-----------------------------|-------|
Expand Down
30 changes: 15 additions & 15 deletions docs/gettingstarted/docker-swarm-general.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Refer to [Getting Started](https://docs.docker.com/get-started/)
# Setup

* Create a directory on the server for configuration. This should be available to all administrators, for example:
``/opt/scs/``
``/opt/sc4s/``
* Create a docker-compose.yml file and place it in the directory created above based on the following template:

```yaml
Expand All @@ -26,7 +26,7 @@ services:
#Comment the following line out if using docker-compose
mode: host
env_file:
- /opt/scs/env_file
- /opt/sc4s/env_file
volumes:
#Uncomment the following line if overriding index destinations
# - ./sc4s-juniper/splunk_index.csv:/opt/syslog-ng/etc/context-local/splunk_index.csv
Expand All @@ -36,9 +36,9 @@ services:

```

## Configure the SCS environment
## Configure the SC4S environment

Create the following file ``/opt/scs/env_file``
Create the following file ``/opt/sc4s/env_file``

* Update ``SPLUNK_HEC_URL`` and ``SPLUNK_HEC_TOKEN`` to reflect the correct values for your environment

Expand All @@ -50,11 +50,11 @@ SPLUNK_DEFAULT_INDEX=main
SPLUNK_METRICS_INDEX=em_metrics
```

## Configure index destinations for Splunk
## Configure index destinations for Splunk

Log paths are preconfigured to utilize a convention of index destinations that is suitable for most customers. This step is optional to allow customization of index destinations.

* Download the latest context.csv file to a subdirectory sc4s below the docker-compose.yml file created above.
* Download the latest context.csv file to a subdirectory SC4S below the docker-compose.yml file created above.

```bash
sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/master/package/etc/context-local/splunk_index.csv
Expand All @@ -63,9 +63,9 @@ sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/mas

## Configure sources by source IP or host name

Legacy sources and non-standard-compliant source require configuration by source IP or hostname as included in the event. The following steps apply to support such sources. To identify sources which require this step refer to the "sources" section of this documentation.
Legacy sources and non-standard-compliant source require configuration by source IP or hostname as included in the event. The following steps apply to support such sources. To identify sources which require this step refer to the "sources" section of this documentation.

* Download the latest vendor_product_by_source.conf file to a subdirectory sc4s below the docker-compose.yml file created above.
* Download the latest vendor_product_by_source.conf file to a subdirectory SC4S below the docker-compose.yml file created above.
```bash
sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/master/package/etc/context-local/vendor_product_by_source.conf
sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/master/package/etc/context-local/vendor_product_by_source.csv
Expand All @@ -86,14 +86,14 @@ Additional hosts can be deployed for syslog collection from additional network z

# Single Source Technology instance

For certain source technologies message categorization by content is impossible to support collection
For certain source technologies message categorization by content is impossible to support collection
of such legacy nonstandard sources we provide a means of dedicating a container to a specific source using
an alternate port.
an alternate port.
Refer to the Sources documentation to identify the specific variable used to enable a specific port for the technology in use.

In the following example ``-p 5000-5020:5000-5020`` allows for up to 21 technology specific ports modify the range as appropriate

* Modify the unit file ``/opt/scs/docker-compose.yml``
* Modify the unit file ``/opt/sc4s/docker-compose.yml``
```yaml
version: "3.7"
services:
Expand Down Expand Up @@ -121,7 +121,7 @@ services:
#Comment the following line out if using docker-compose
mode: host
env_file:
- /opt/scs/env_file
- /opt/sc4s/env_file
volumes:
#Uncomment the following line if overriding index destinations
# - ./sc4s-juniper/splunk_index.csv:/opt/syslog-ng/etc/context-local/splunk_index.csv
Expand All @@ -130,7 +130,7 @@ services:
# - ./sc4s-juniper/vendor_product_by_source.conf:/opt/syslog-ng/etc/context-local/vendor_product_by_source.conf
```

Modify the following file ``/opt/scs/default/env_file``
Modify the following file ``/opt/sc4s/default/env_file``

* Update ``SPLUNK_HEC_URL`` and ``SPLUNK_HEC_TOKEN`` to reflect the correct values for your environment

Expand All @@ -140,9 +140,9 @@ SPLUNK_HEC_TOKEN=a778f63a-5dff-4e3c-a72c-a03183659e94
SPLUNK_CONNECT_METHOD=hec
SPLUNK_DEFAULT_INDEX=main
SPLUNK_METRICS_INDEX=em_metrics
SCS_LISTEN_JUNIPER_NETSCREEN_TCP_PORT=5000
SC4S_LISTEN_JUNIPER_NETSCREEN_TCP_PORT=5000
#Uncomment the following line if using untrusted SSL certificates
#SCS_DEST_SPLUNK_HEC_TLS_VERIFY=no
#SC4S_DEST_SPLUNK_HEC_TLS_VERIFY=no
```

* Start SC4S.
Expand Down
30 changes: 15 additions & 15 deletions docs/gettingstarted/docker-swarm-rhel7.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ sudo docker swarm init
# Setup

* Create a directory on the server for configuration. This should be available to all administrators, for example:
``/opt/scs/``
``/opt/sc4s/``
* Create a docker-compose.yml file and place it in the directory created above based on the following template:

```yaml
Expand All @@ -54,7 +54,7 @@ services:
#Comment the following line out if using docker-compose
mode: host
env_file:
- /opt/scs/env_file
- /opt/sc4s/env_file
volumes:
#Uncomment the following line if overriding index destinations
# - ./sc4s-juniper/splunk_index.csv:/opt/syslog-ng/etc/context-local/splunk_index.csv
Expand All @@ -64,9 +64,9 @@ services:

```

## Configure the SCS environment
## Configure the SC4S environment

Create the following file ``/opt/scs/env_file``
Create the following file ``/opt/sc4s/env_file``

* Update ``SPLUNK_HEC_URL`` and ``SPLUNK_HEC_TOKEN`` to reflect the correct values for your environment

Expand All @@ -79,11 +79,11 @@ SPLUNK_METRICS_INDEX=em_metrics
```


## Configure index destinations for Splunk
## Configure index destinations for Splunk

Log paths are preconfigured to utilize a convention of index destinations that is suitable for most customers. This step is optional to allow customization of index destinations.

* Download the latest context.csv file to a subdirectory sc4s below the docker-compose.yml file created above.
* Download the latest context.csv file to a subdirectory SC4S below the docker-compose.yml file created above.

```bash
sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/master/package/etc/context-local/splunk_index.csv
Expand All @@ -92,9 +92,9 @@ sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/mas

## Configure sources by source IP or host name

Legacy sources and non-standard-compliant sources require configuration by source IP or hostname as included in the event. The following steps apply to support such sources. To identify sources which require this step refer to the "sources" section of this documentation.
Legacy sources and non-standard-compliant sources require configuration by source IP or hostname as included in the event. The following steps apply to support such sources. To identify sources which require this step refer to the "sources" section of this documentation.

* Download the latest vendor_product_by_source.conf file to a subdirectory sc4s below the docker-compose.yml file created above
* Download the latest vendor_product_by_source.conf file to a subdirectory SC4S below the docker-compose.yml file created above
```bash
sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/master/package/etc/context-local/vendor_product_by_source.conf
sudo wget https://raw.githubusercontent.com/splunk/splunk-connect-for-syslog/master/package/etc/context-local/vendor_product_by_source.csv
Expand All @@ -114,14 +114,14 @@ Additional hosts can be deployed for syslog collection from additional network z

# Single Source Technology instance

For certain source technologies message categorization by content is impossible to support collection
For certain source technologies message categorization by content is impossible to support collection
of such legacy nonstandard sources we provide a means of dedicating a container to a specific source using
an alternate port.
an alternate port.
Refer to the Sources documentation to identify the specific variable used to enable a specific port for the technology in use.

In the following example ``-p 5000-5020:5000-5020`` allows for up to 21 technology specific ports modify the range as appropriate

* Modify the unit file ``/opt/scs/docker-compose.yml``
* Modify the unit file ``/opt/sc4s/docker-compose.yml``
```yaml
version: "3.7"
services:
Expand Down Expand Up @@ -149,7 +149,7 @@ services:
#Comment the following line out if using docker-compose
mode: host
env_file:
- /opt/scs/env_file
- /opt/sc4s/env_file
volumes:
#Uncomment the following line if overriding index destinations
# - ./sc4s-juniper/splunk_index.csv:/opt/syslog-ng/etc/context-local/splunk_index.csv
Expand All @@ -158,7 +158,7 @@ services:
# - ./sc4s-juniper/vendor_product_by_source.conf:/opt/syslog-ng/etc/context-local/vendor_product_by_source.conf
```

Modify the following file ``/opt/scs/default/env_file``
Modify the following file ``/opt/sc4s/default/env_file``

* Update ``SPLUNK_HEC_URL`` and ``SPLUNK_HEC_TOKEN`` to reflect the correct values for your environment

Expand All @@ -168,9 +168,9 @@ SPLUNK_HEC_TOKEN=a778f63a-5dff-4e3c-a72c-a03183659e94
SPLUNK_CONNECT_METHOD=hec
SPLUNK_DEFAULT_INDEX=main
SPLUNK_METRICS_INDEX=em_metrics
SCS_LISTEN_JUNIPER_NETSCREEN_TCP_PORT=5000
SC4S_LISTEN_JUNIPER_NETSCREEN_TCP_PORT=5000
#Uncomment the following line if using untrusted SSL certificates
#SCS_DEST_SPLUNK_HEC_TLS_VERIFY=no
#SC4S_DEST_SPLUNK_HEC_TLS_VERIFY=no
```
* Start SC4S.

Expand Down
Loading

0 comments on commit 48f0d7c

Please sign in to comment.