From 666bbd246f541b658a5c6d84674a33d2edc5e1ee Mon Sep 17 00:00:00 2001 From: rfaircloth-splunk Date: Mon, 7 Oct 2019 14:18:14 -0400 Subject: [PATCH] Fixes #112 --- .../{SC4S deployment.png => SC4Sdeployment.png} | Bin docs/gettingstarted.md | 3 ++- docs/performance.md | 16 ++++++++++++++++ 3 files changed, 18 insertions(+), 1 deletion(-) rename docs/{SC4S deployment.png => SC4Sdeployment.png} (100%) diff --git a/docs/SC4S deployment.png b/docs/SC4Sdeployment.png similarity index 100% rename from docs/SC4S deployment.png rename to docs/SC4Sdeployment.png diff --git a/docs/gettingstarted.md b/docs/gettingstarted.md index 7a29b37..a43e906 100644 --- a/docs/gettingstarted.md +++ b/docs/gettingstarted.md @@ -21,6 +21,7 @@ instance in the same VLAN as the source device. environment. * Avoid TCP except where the source is unable to contain the event to a single UDP packet. * Avoid TLS except where the event may cross a untrusted network. +* Plan for appropriately sized hardware (see)[performance.md] # Implementation @@ -81,4 +82,4 @@ Splunk type. Additional hosts can be deployed for syslog collection from additional network zones and locations: -![SC4S deployment diagram](SC4S%20deployment.png) \ No newline at end of file +![SC4S deployment diagram](SC4Sdeployment.png) \ No newline at end of file diff --git a/docs/performance.md b/docs/performance.md index 966a318..53831b2 100644 --- a/docs/performance.md +++ b/docs/performance.md @@ -24,3 +24,19 @@ average rate = 9717.58 msg/sec, count=1749420, time=180.026, (average) msg size= Splunk Enterprise's implementation of the http event collection server responds to the client with a status code 200 and fails to commit the events to disk during a rolling restart. In our testing, 20-30 events per indexer are lost. +## Guidance on sizing hardware + +The following reference deployment hardware specifications are based on Splunk performance testing results in AWS +The overall load on your deployment HW will vary based on the percentage of events which are not handled by a filter or +custom filters added not accounted for here. While we consider the following conservative actual hardware performance will vary +due to network interface card, driver, kernel version, exact CPU even type of memory and configuration. SYSLOG is a fire +and forget protocol making it sensitive to performance. Given this it is highly recommended you should validate +performance with your hardware and production data samples. The syslog-ng loggen tool available in the SC4S container +and the commands above can be utilized in this effort. + +Deployment Size | Hardware Spec | Average EPS with average msg size 800 k +-- | -- | -- +Small | 2 X 3.1 ghz cores1 GB of memory | 2K msg/sec +Medium | 4 X 3.1 ghz cores2 GB of memory | 4.5K msg/sec +Large | 8 X 3.1 ghz cores4 GB of memory | 9K msg/sec +XL | 16 X 3.1 ghz cores8 GB of memory | 18K msg/sec