diff --git a/docs/configuration.md b/docs/configuration.md index 75b7e82..9860a3c 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -10,6 +10,10 @@ and variables needed to properly configure SC4S for your environment. | SPLUNK_HEC_URL | url | URL(s) of the Splunk endpoint, can be a single URL space seperated list | | SPLUNK_HEC_TOKEN | string | Splunk HTTP Event Collector Token | +* NOTE: Do _not_ configure HEC Acknowledgement when deploying the HEC token on the Splunk side; the underlying syslog-ng http +destination does not support this feature. Moreover, HEC Ack would significantly degrade performance for streaming data such as +syslog. + ## Splunk HEC Destination Configuration @@ -26,15 +30,24 @@ and variables needed to properly configure SC4S for your environment. Disk buffers in SC4S are allocated _per destination_. In the future as more destinations are supported, a separate list of variables will be used for each. This is why you see the `DEST_SPLUNK_HEC` in the variable names below. + * NOTE: "Reliable" disk buffering offers little advantage over "normal" disk buffering, at a significant performance penalty. For this reason, normal disk buffering is recommended. + * NOTE: If you add destinations locally in your configuration, pay attention to the _cumulative_ buffer requirements when allocating local disk. -* Be sure to factor in the syslog-ng overhead (approx. 1.7x) when calculating the total buffer size needed. See the "Data Resilience" section below for more information. + +* NOTE: Disk buffer storage is configured via container volumes and is persistent between restarts of the conatiner. +Be sure to account for disk space requirements on the local sc4s host when creating the container volumes in your respective +runtime environment (outlined in the "getting started" runtime docs). These volumes can grow significantly if there is +an extended outage to the SC4S destinations (HEC endpoints). See the "SC4S Disk Buffer Configuration" section on the Configruation +page for more info. + * NOTE: The values for the variables below represent the _total_ sizes of the buffers for the destination. These sizes are divded by the number of workers (threads) when setting the actual syslog-ng buffer options, because the buffer options apply to each worker rather than the entire destination. Pay careful attention to this when using the "BYOE" version of SC4S, where direct access to the syslog-ng config files -may hide this nuance. To determine the proper size of the disk buffer, consult the "Data Resilience" section below. +may hide this nuance. Lastly, be sure to factor in the syslog-ng data structure overhead (approx. 2x raw message size) when calculating the +total buffer size needed. To determine the proper size of the disk buffer, consult the "Data Resilience" section below. | Variable | Values/Default | Description | |----------|---------------|-------------| diff --git a/docs/developing/index.md b/docs/developing/index.md index 5cf9063..d462d1d 100644 --- a/docs/developing/index.md +++ b/docs/developing/index.md @@ -2,17 +2,17 @@ ## Get Docker -Development requires Docker desktop available for windows + and mac or Docker CE available for Linux visit (Docker)[https://www.docker.com/get-started] +Development requires Docker desktop available for windows + and mac or Docker CE available for Linux. Visit (Docker)[https://www.docker.com/get-started] for download instructions ## Setup VS Code IDE -VS Code provides a free IDE experience that is effective for daily development with SC4S visit (Microsoft)[https://code.visualstudio.com/docs/introvideos/basics] +VS Code provides a free IDE experience that is effective for daily development with SC4S. Visit (Microsoft)[https://code.visualstudio.com/docs/introvideos/basics] to download and install for your plaform (windows/mac/linux) ## Fork and Clone the github repository -Visit our repository at (Github)[https://github.com/splunk/splunk-connect-for-syslog] and "fork" our repository this will allow you to make changes and submit pull requests. +Visit our repository at (Github)[https://github.com/splunk/splunk-connect-for-syslog] and "fork" our repository. This will allow you to make changes and submit pull requests. ![How to Fork](gh_fork.png) @@ -22,23 +22,24 @@ Click the clone icon and select the location ## Setup the project and install requirements -The follow steps are only required on the first time run. +The following steps are required _only_ on the first time run. * Install VS Code Extensions S * Python * Test Explorer * "Python Test Explorer" -* From the terminal menu select Run Task +* From the terminal menu select "Run Task" * Select "Setup step 1: python venv" then "go without scanning output" -* From the terminal menu select Run Task +* From the terminal menu select "Run Task" * Select "Setup step 2: python requirements" then "go without scanning output" ![VS Code setup](vsc_run.png) ## Click the test lab icon -* Run all tests this will appear to do nothing for a period system fan may spin loud whiletests are run icons on each test will turn green -or red to indicate pass fail however VS Code does not show the status of status until the tests complete +* Run all tests. Icons on each test will turn green or red to indicate pass or fail. Though VS Code does not show the status +of any given test until all tests complete in the test tree, you can select "Show test output" near the top of the test +directory tree to see the terminal output of each test as it runs in the "Output" pane. ![VS Code Debug](vsc_debug.png) diff --git a/docs/gettingstarted/docker-swarm-general.md b/docs/gettingstarted/docker-swarm-general.md index 0d7dc62..4d38ea2 100644 --- a/docs/gettingstarted/docker-swarm-general.md +++ b/docs/gettingstarted/docker-swarm-general.md @@ -69,6 +69,10 @@ particular the state of the disk buffer. This is a required step. sudo docker volume create splunk-sc4s-var ``` +* NOTE: Be sure to account for disk space requirements for the docker volume created above. This volume is located in +`/var/lib/docker/volumes/` and could grow significantly if there is an extended outage to the SC4S destinations +(typically HEC endpoints). See the "SC4S Disk Buffer Configuration" section on the Configruation page for more info. + * Create the subdirectory ``/opt/sc4s/local``. This will be used as a mount point for local overrides and configurations. * The empty ``local`` directory created above will populate with defaults and examples at the first invocation @@ -107,7 +111,9 @@ SC4S_DEST_SPLUNK_HEC_WORKERS=6 #SC4S_DEST_SPLUNK_HEC_TLS_VERIFY=no ``` -* Update ``SPLUNK_HEC_URL`` and ``SPLUNK_HEC_TOKEN`` to reflect the correct values for your environment. +* Update ``SPLUNK_HEC_URL`` and ``SPLUNK_HEC_TOKEN`` to reflect the correct values for your environment. Do _not_ configure HEC +Acknowledgement when deploying the HEC token on the Splunk side; the underlying syslog-ng http destination does not support this +feature. Moreover, HEC Ack would significantly degrade performance for streaming data such as syslog. * Set `SC4S_DEST_SPLUNK_HEC_WORKERS` to match the number of indexers and/or HWFs with HEC endpoints, up to a maxiumum of 32. If the endpoint is a VIP, match this value to the total number of indexers behind the load balancer. diff --git a/docs/gettingstarted/docker-swarm-rhel7.md b/docs/gettingstarted/docker-swarm-rhel7.md index af6d519..1c02bbb 100644 --- a/docs/gettingstarted/docker-swarm-rhel7.md +++ b/docs/gettingstarted/docker-swarm-rhel7.md @@ -77,6 +77,10 @@ particular the state of the disk buffer. This is a required step. sudo docker volume create splunk-sc4s-var ``` +* NOTE: Be sure to account for disk space requirements for the docker volume created above. This volume is located in +`/var/lib/docker/volumes/` and could grow significantly if there is an extended outage to the SC4S destinations +(typically HEC endpoints). See the "SC4S Disk Buffer Configuration" section on the Configruation page for more info. + * Create the subdirectory ``/opt/sc4s/local``. This will be used as a mount point for local overrides and configurations. * The empty ``local`` directory created above will populate with defaults and examples at the first invocation @@ -115,7 +119,9 @@ SC4S_DEST_SPLUNK_HEC_WORKERS=6 #SC4S_DEST_SPLUNK_HEC_TLS_VERIFY=no ``` -* Update ``SPLUNK_HEC_URL`` and ``SPLUNK_HEC_TOKEN`` to reflect the correct values for your environment. +* Update ``SPLUNK_HEC_URL`` and ``SPLUNK_HEC_TOKEN`` to reflect the correct values for your environment. Do _not_ configure HEC +Acknowledgement when deploying the HEC token on the Splunk side; the underlying syslog-ng http destination does not support this +feature. Moreover, HEC Ack would significantly degrade performance for streaming data such as syslog. * Set `SC4S_DEST_SPLUNK_HEC_WORKERS` to match the number of indexers and/or HWFs with HEC endpoints, up to a maxiumum of 32. If the endpoint is a VIP, match this value to the total number of indexers behind the load balancer. diff --git a/docs/gettingstarted/docker-systemd-general.md b/docs/gettingstarted/docker-systemd-general.md index 132e80b..bdbb78d 100644 --- a/docs/gettingstarted/docker-systemd-general.md +++ b/docs/gettingstarted/docker-systemd-general.md @@ -75,6 +75,10 @@ particular the state of the disk buffer. This is a required step. sudo docker volume create splunk-sc4s-var ``` +* NOTE: Be sure to account for disk space requirements for the docker volume created above. This volume is located in +`/var/lib/docker/volumes/` and could grow significantly if there is an extended outage to the SC4S destinations +(typically HEC endpoints). See the "SC4S Disk Buffer Configuration" section on the Configruation page for more info. + * Create the subdirectory ``/opt/sc4s/local``. This will be used as a mount point for local overrides and configurations. * The empty ``local`` directory created above will populate with defaults and examples at the first invocation @@ -113,7 +117,9 @@ SC4S_DEST_SPLUNK_HEC_WORKERS=6 #SC4S_DEST_SPLUNK_HEC_TLS_VERIFY=no ``` -* Update ``SPLUNK_HEC_URL`` and ``SPLUNK_HEC_TOKEN`` to reflect the correct values for your environment +* Update ``SPLUNK_HEC_URL`` and ``SPLUNK_HEC_TOKEN`` to reflect the correct values for your environment. Do _not_ configure HEC +Acknowledgement when deploying the HEC token on the Splunk side; the underlying syslog-ng http destination does not support this +feature. Moreover, HEC Ack would significantly degrade performance for streaming data such as syslog. * Set `SC4S_DEST_SPLUNK_HEC_WORKERS` to match the number of indexers and/or HWFs with HEC endpoints, up to a maxiumum of 32. If the endpoint is a VIP, match this value to the total number of indexers behind the load balancer. diff --git a/docs/gettingstarted/podman-systemd-general.md b/docs/gettingstarted/podman-systemd-general.md index 8988b50..8c79df8 100644 --- a/docs/gettingstarted/podman-systemd-general.md +++ b/docs/gettingstarted/podman-systemd-general.md @@ -57,6 +57,10 @@ particular the state of the disk buffer. This is a required step. sudo podman volume create splunk-sc4s-var ``` +* NOTE: Be sure to account for disk space requirements for the podman volume created above. This volume is located in +`/var/lib/containers/storage/volumes/` and could grow significantly if there is an extended outage to the SC4S destinations +(typically HEC endpoints). See the "SC4S Disk Buffer Configuration" section on the Configruation page for more info. + * Create the subdirectory ``/opt/sc4s/local``. This will be used as a mount point for local overrides and configurations. * The empty ``local`` directory created above will populate with defaults and examples at the first invocation @@ -95,7 +99,9 @@ SC4S_DEST_SPLUNK_HEC_WORKERS=6 #SC4S_DEST_SPLUNK_HEC_TLS_VERIFY=no ``` -* Update ``SPLUNK_HEC_URL`` and ``SPLUNK_HEC_TOKEN`` to reflect the correct values for your environment +* Update ``SPLUNK_HEC_URL`` and ``SPLUNK_HEC_TOKEN`` to reflect the correct values for your environment. Do _not_ configure HEC +Acknowledgement when deploying the HEC token on the Splunk side; the underlying syslog-ng http destination does not support this +feature. Moreover, HEC Ack would significantly degrade performance for streaming data such as syslog. * Set `SC4S_DEST_SPLUNK_HEC_WORKERS` to match the number of indexers and/or HWFs with HEC endpoints, up to a maxiumum of 32. If the endpoint is a VIP, match this value to the total number of indexers behind the load balancer. diff --git a/docs/sources/Citrix/index.md b/docs/sources/Citrix/index.md index 3c2fc13..3dedf1a 100644 --- a/docs/sources/Citrix/index.md +++ b/docs/sources/Citrix/index.md @@ -29,7 +29,7 @@ MSG Parse: This filter parses message content * Install the Splunk Add-on on the search head(s) for the user communities interested in this data source. If SC4S is exclusively used the addon is not required on the indexer. * Review and update the splunk_index.csv file and set the index and sourcetype as required for the data source. -* Follow vendor configuration steps per Product Manual above. Ensure the data format selected is "MMDDYYYY" +* Follow vendor configuration steps per Product Manual above. Ensure the data format selected is "DDMMYYYY" ### Options diff --git a/docs/upgrade.md b/docs/upgrade.md index 2491f14..a0a3dde 100644 --- a/docs/upgrade.md +++ b/docs/upgrade.md @@ -13,3 +13,7 @@ files is incorporated into existing context files. * UNIT FILE CHANGES: Make sure to update the unit file used to start the sc4s service with the changes included in this release. It includes updates for proper operation with RHEL 8, and is backward-compatible with RHEL 7.7. + +## Version 1.10.0 + +* The "Development" section outlines new instructions for operation with the vscode IDE. \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index 6a73268..98c30ca 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -10,7 +10,7 @@ nav: - "Docker CE + Swarm RHEL 7.7": "gettingstarted/docker-swarm-rhel7.md" - "Bring your own Envionment": "gettingstarted/byoe-rhel7.md" - Configuration: "configuration.md" - - Developing: "docs/developing/index.md" + - Development: "docs/developing/index.md" - Sources: - About: sources/index.md - Checkpoint: sources/Checkpoint/index.md @@ -29,7 +29,7 @@ nav: - Proofpoint: sources/Proofpoint/index.md - Symantec: sources/Symantec/index.md - Ubiquiti: sources/Ubiquiti/index.md - - VMware: sources/VMWare/index.md + - VMware: sources/VMware/index.md - Zscaler: sources/Zscaler/index.md - "Demo Lab": "demo.md" - Performance: "performance.md" diff --git a/package/etc/conf.d/filters/cisco/meraki.conf b/package/etc/conf.d/filters/cisco/meraki.conf index c0573b5..bf1eec3 100644 --- a/package/etc/conf.d/filters/cisco/meraki.conf +++ b/package/etc/conf.d/filters/cisco/meraki.conf @@ -15,7 +15,9 @@ parser p_cisco_meraki { }; parser { date-parser(format('%s') - template("${EPOCH}")); + template("${EPOCH}") + flags(guess-timezone) + ); }; }; diff --git a/package/etc/conf.d/filters/citrix/netscaler.conf.tmpl b/package/etc/conf.d/filters/citrix/netscaler.conf.tmpl index 76c0334..57a1dbf 100644 --- a/package/etc/conf.d/filters/citrix/netscaler.conf.tmpl +++ b/package/etc/conf.d/filters/citrix/netscaler.conf.tmpl @@ -9,14 +9,14 @@ filter f_citrix_netscaler_message { }; parser p_citrix_netscaler_date { -{{- if (conv.ToBool (getenv "SC4S_SOURCE_CITRIX_NETSCALER_USEALT_DATE_FORMAT" "yes")) }} - #01/10/2001:01:01:01 GMT - date-parser(format('%d/%m/%Y:%H:%M:%S %Z') - template("$2")); -{{- else }} +{{- if (conv.ToBool (getenv "SC4S_SOURCE_CITRIX_NETSCALER_USEALT_DATE_FORMAT" "no")) }} #10/01/2001:01:01:01 GMT date-parser(format('%m/%d/%Y:%H:%M:%S %Z') template("$2")); +{{- else }} + #01/10/2001:01:01:01 GMT + date-parser(format('%d/%m/%Y:%H:%M:%S %Z') + template("$2")); {{- end }} }; diff --git a/package/etc/conf.d/log_paths/lp-common_event_format.conf.tmpl b/package/etc/conf.d/log_paths/lp-common_event_format.conf.tmpl index 52faaa7..5c25810 100644 --- a/package/etc/conf.d/log_paths/lp-common_event_format.conf.tmpl +++ b/package/etc/conf.d/log_paths/lp-common_event_format.conf.tmpl @@ -16,11 +16,15 @@ parser p_cef_header { }; parser p_cef_ts_rt { - date-parser(format("%s") template("${.cef.rt}") + date-parser(format('%s') + template("${.cef.rt}") + flags(guess-timezone) ); }; parser p_cef_ts_end { - date-parser(format("%s") template("${.cef.end}") + date-parser(format('%s') + template("${.cef.end}") + flags(guess-timezone) ); }; diff --git a/package/etc/conf.d/log_paths/lp-zscaler_nss.conf.tmpl b/package/etc/conf.d/log_paths/lp-zscaler_nss.conf.tmpl index 17782bf..3959dc2 100644 --- a/package/etc/conf.d/log_paths/lp-zscaler_nss.conf.tmpl +++ b/package/etc/conf.d/log_paths/lp-zscaler_nss.conf.tmpl @@ -20,7 +20,7 @@ log { }; }; - parser { date-parser(format("%Y-%m-%d %H:%M:%S") template('$(substr "$LEGACY_MSGHDR$MSG" "0" "19")')); }; + parser { date-parser(format("%Y-%m-%d %H:%M:%S") template('$(substr "$LEGACY_MSGHDR$MSG" "0" "19")') flags(guess-timezone)); }; rewrite { set("zscaler_nss", value("fields.sc4s_vendor_product")); subst("^[^\t]+\t", "", value("MESSAGE"), flags("global")); diff --git a/tests/conftest.py b/tests/conftest.py index a02c8a2..7a9c7bf 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -195,7 +195,7 @@ def sc4s_docker(docker_services): docker_services.start('sc4s') ports = {514: docker_services.port_for("sc4s", 514)} - for x in range(5000, 5006): + for x in range(5000, 5007): ports.update({x: docker_services.port_for("sc4s", x)}) return docker_services.docker_ip, ports diff --git a/tests/test_checkpoint.py b/tests/test_checkpoint.py index 9e1a9a0..efd52e0 100644 --- a/tests/test_checkpoint.py +++ b/tests/test_checkpoint.py @@ -18,12 +18,12 @@ def test_checkpoint_splunk_ips(record_property, setup_wordlist, setup_splunk, se host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} time={% now 'utc', '%s' %}|hostname={{ host }}|severity=Informational|confidence_level=Unknown|product=IPS|action=Drop|ifdir=inbound|ifname=bond2|loguid={0x5d9cdcc9,0x8d159f,0x5f19f392,0x1897a828}|origin=1.1.1.1|time={% now 'utc', '%s' %}|version=1|attack=Streaming Engine: TCP Segment Limit Enforcement|attack_info=TCP segment out of maximum allowed sequence. Packet dropped.|chassis_bladed_system=[ 1_3 ]|dst=10.10.10.10|origin_sic_name=CN=something_03_local,O=devicename.domain.com.p7fdbt|performance_impact=0|protection_id=tcp_segment_limit|protection_name=TCP Segment Limit Enforcement|protection_type=settings_tcp|proto=6|rule=393|rule_name=10.384_..|rule_uid={9F77F944-8DD5-4ADF-803A-785D03B3A2E8}|s_port=46455|service=443|smartdefense_profile=Recommended_Protection_ded9e8d8ee89d|src=1.1.1.2|\n") + "{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} time={% now 'local', '%s' %}|hostname={{ host }}|severity=Informational|confidence_level=Unknown|product=IPS|action=Drop|ifdir=inbound|ifname=bond2|loguid={0x5d9cdcc9,0x8d159f,0x5f19f392,0x1897a828}|origin=1.1.1.1|time={% now 'local', '%s' %}|version=1|attack=Streaming Engine: TCP Segment Limit Enforcement|attack_info=TCP segment out of maximum allowed sequence. Packet dropped.|chassis_bladed_system=[ 1_3 ]|dst=10.10.10.10|origin_sic_name=CN=something_03_local,O=devicename.domain.com.p7fdbt|performance_impact=0|protection_id=tcp_segment_limit|protection_name=TCP Segment Limit Enforcement|protection_type=settings_tcp|proto=6|rule=393|rule_name=10.384_..|rule_uid={9F77F944-8DD5-4ADF-803A-785D03B3A2E8}|s_port=46455|service=443|smartdefense_profile=Recommended_Protection_ded9e8d8ee89d|src=1.1.1.2|\n") message = mt.render(mark="<111>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netids host=\"{{ host }}\" sourcetype=\"cp_log\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netids host=\"{{ host }}\" sourcetype=\"cp_log\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -39,12 +39,12 @@ def test_checkpoint_splunk_firewall(record_property, setup_wordlist, setup_splun host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} time={% now 'utc', '%s' %}|hostname={{ host }}|product=Firewall|action=Drop|ifdir=inbound|ifname=bond1|loguid={0x5d9ce80f,0x8d0555,0x5f19f392,0x18982828}|origin=1.1.1.1|time={% now 'utc', '%s' %}|version=1|chassis_bladed_system=[ 1_1 ]|dst=10.10.10.10|inzone=External|origin_sic_name=CN=something_03_local,O=devicename.domain.com.p7fdbt|outzone=Internal|proto=6|rule=402|rule_name=11_..|rule_uid={C8CD796E-7BD5-47B6-90CA-B250D062D5E5}|s_port=33687|service=23|src=1.1.1.2|\n") + "{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} time={% now 'local', '%s' %}|hostname={{ host }}|product=Firewall|action=Drop|ifdir=inbound|ifname=bond1|loguid={0x5d9ce80f,0x8d0555,0x5f19f392,0x18982828}|origin=1.1.1.1|time={% now 'local', '%s' %}|version=1|chassis_bladed_system=[ 1_1 ]|dst=10.10.10.10|inzone=External|origin_sic_name=CN=something_03_local,O=devicename.domain.com.p7fdbt|outzone=Internal|proto=6|rule=402|rule_name=11_..|rule_uid={C8CD796E-7BD5-47B6-90CA-B250D062D5E5}|s_port=33687|service=23|src=1.1.1.2|\n") message = mt.render(mark="<111>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netfw host=\"{{ host }}\" sourcetype=\"cp_log\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netfw host=\"{{ host }}\" sourcetype=\"cp_log\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -60,12 +60,12 @@ def test_checkpoint_splunk_mds(record_property, setup_wordlist, setup_splunk, se host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} |hostname={{ host }}|time={% now 'utc', '%s' %}|product=mds-query-tool|action=Accept|ifdir=outbound|origin=1.1.1.1|2.2.2.2|originsicname=cn\=cp_mgmt,o\=DEVICENAME.domain.com.p7fdbt|sequencenum=1|time={% now 'utc', '%s' %}|version=5|administrator=localhost|client_ip=3.3.3.3|machine=DEVICENAME|operation=Log Out|operation_number=12|subject=Administrator Login|\n") + "{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} |hostname={{ host }}|time={% now 'local', '%s' %}|product=mds-query-tool|action=Accept|ifdir=outbound|origin=1.1.1.1|2.2.2.2|originsicname=cn\=cp_mgmt,o\=DEVICENAME.domain.com.p7fdbt|sequencenum=1|time={% now 'local', '%s' %}|version=5|administrator=localhost|client_ip=3.3.3.3|machine=DEVICENAME|operation=Log Out|operation_number=12|subject=Administrator Login|\n") message = mt.render(mark="<111>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netops host=\"{{ host }}\" sourcetype=\"cp_log\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netops host=\"{{ host }}\" sourcetype=\"cp_log\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -81,12 +81,12 @@ def test_checkpoint_splunk_cpmi(record_property, setup_wordlist, setup_splunk, s host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} |hostname={{ host }}|product=CPMI Client|action=Accept|ifdir=outbound|origin=1.1.1.1|2.2.2.2|originsicname=cn\=cp_mgmt,o\=DEVICENAME.domain.com.p7fdbt|sequencenum=1|time={% now 'utc', '%s' %}|version=5|administrator=localhost|client_ip=3.3.3.3|machine=DEVICENAME|operation=Log Out|operation_number=12|subject=Administrator Login\n") + "{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} |hostname={{ host }}|product=CPMI Client|action=Accept|ifdir=outbound|origin=1.1.1.1|2.2.2.2|originsicname=cn\=cp_mgmt,o\=DEVICENAME.domain.com.p7fdbt|sequencenum=1|time={% now 'local', '%s' %}|version=5|administrator=localhost|client_ip=3.3.3.3|machine=DEVICENAME|operation=Log Out|operation_number=12|subject=Administrator Login\n") message = mt.render(mark="<111>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netops host=\"{{ host }}\" sourcetype=\"cp_log\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netops host=\"{{ host }}\" sourcetype=\"cp_log\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -102,12 +102,12 @@ def test_checkpoint_splunk_web_api(record_property, setup_wordlist, setup_splunk host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} |hostname={{ host }}|product=WEB_API|action=Accept|ifdir=outbound|origin=1.1.1.1|2.2.2.2|originsicname=cn\=cp_mgmt,o\=DEVICENAME.domain.com.p7fdbt|sequencenum=1|time={% now 'utc', '%s' %}|version=5|administrator=tufinapi|client_ip=3.3.3.3|machine=DEVICENAME|operation=Log Out|operation_number=12|subject=Administrator Login\n") + "{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} |hostname={{ host }}|product=WEB_API|action=Accept|ifdir=outbound|origin=1.1.1.1|2.2.2.2|originsicname=cn\=cp_mgmt,o\=DEVICENAME.domain.com.p7fdbt|sequencenum=1|time={% now 'local', '%s' %}|version=5|administrator=tufinapi|client_ip=3.3.3.3|machine=DEVICENAME|operation=Log Out|operation_number=12|subject=Administrator Login\n") message = mt.render(mark="<111>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netops host=\"{{ host }}\" sourcetype=\"cp_log\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netops host=\"{{ host }}\" sourcetype=\"cp_log\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -123,12 +123,12 @@ def test_checkpoint_splunk_smartconsole(record_property, setup_wordlist, setup_s host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} time={% now 'utc', '%s' %}|hostname={{ host }}|product=SmartConsole|action=Accept|ifdir=outbound|origin=1.1.1.1|4.4.4.4|sequencenum=1|time={% now 'utc', '%s' %}|version=5|additional_info=Authentication method: Password based application token|administrator=psanadhya|client_ip=3.3.3.3|machine=DEVICENAME|operation=Log In|operation_number=10|subject=Administrator Login|\n") + "{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} time={% now 'local', '%s' %}|hostname={{ host }}|product=SmartConsole|action=Accept|ifdir=outbound|origin=1.1.1.1|4.4.4.4|sequencenum=1|time={% now 'local', '%s' %}|version=5|additional_info=Authentication method: Password based application token|administrator=psanadhya|client_ip=3.3.3.3|machine=DEVICENAME|operation=Log In|operation_number=10|subject=Administrator Login|\n") message = mt.render(mark="<111>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netops host=\"{{ host }}\" sourcetype=\"cp_log\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netops host=\"{{ host }}\" sourcetype=\"cp_log\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -152,7 +152,7 @@ def test_checkpoint_splunk_os(record_property, setup_wordlist, setup_splunk, set sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=osnix \"0x{{ pid }}\" sourcetype=\"nix:syslog\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=osnix \"0x{{ pid }}\" sourcetype=\"nix:syslog\" | head 2") search = st.render(pid=pid) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -161,4 +161,4 @@ def test_checkpoint_splunk_os(record_property, setup_wordlist, setup_splunk, set record_property("resultCount", resultCount) record_property("message", message) - assert resultCount == 1 \ No newline at end of file + assert resultCount == 1 diff --git a/tests/test_cisco_acs.py b/tests/test_cisco_acs.py index 190e985..283c696 100644 --- a/tests/test_cisco_acs.py +++ b/tests/test_cisco_acs.py @@ -4,6 +4,7 @@ # license that can be found in the LICENSE-BSD2 file or at # https://opensource.org/licenses/BSD-2-Clause import random +import datetime from jinja2 import Environment @@ -12,16 +13,22 @@ env = Environment(extensions=['jinja2_time.TimeExtension']) - def test_cisco_acs_single(record_property, setup_wordlist, setup_splunk, setup_sc4s): host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) +# Generate an ISO 8601 compliant timestamp with local timezone offset (2020-02-12 12:46:39.323-08:00) + dt = datetime.datetime.utcnow().replace(tzinfo=datetime.timezone.utc).astimezone().isoformat(sep=' ', timespec='milliseconds') + +# Function to insert a space between the time and TZ offset (2020-02-12 12:46:39.323 -08:00) + def insert_space(string, integer): + return string[0:integer] + ' ' + string[integer:] + mt = env.from_string( - "{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} CSCOacs_Passed_Authentications 0765855540 1 0 2019-10-24 21:01:05.028 +00:00 0178632943 5202 NOTICE Device-Administration: Command Authorization succeeded, ACSVersion=acs-5.8.1.4-B.462.x86_64, ConfigVersionId=16489, Device IP Address=10.0.0.93, DestinationIPAddress=10.0.0.10, DestinationPort=49, UserName=nsdevman, CmdSet=[ CmdAV=show CmdArgAV=vpn-sessiondb CmdArgAV=full CmdArgAV=ra-ikev2-ipsec ], Protocol=Tacacs, MatchedCommandSet=fw3, RequestLatency=11, Type=Authorization, Privilege-Level=15, Authen-Type=ASCII, Service=None, User=nsdevman, Port=443, Remote-Address=10.0.0.15, Authen-Method=TacacsPlus, Service-Argument=shell, AcsSessionID=mnsvdcfpiuac03/359448835/9871764, AuthenticationIdentityStore=AD1, AuthenticationMethod=Lookup, SelectedAccessService=Default Device Admin, SelectedCommandSet=fw3, IdentityGroup=IdentityGroup:All Groups:SystemID, Step=13005 , Step=15008 , Step=15004 , Step=15012 , Step=15041 , Step=15004 , Step=15013 , Step=24210 , Step=24212 , Step=24432 , Step=24325 , Step=24313 , Step=24319 , Step=24323 , Step=24420 , Step=24355 , Step=24416 , Step=22037 , Step=15044 , Step=15035 , Step=15042 , Step=15036 , Step=15004 , Step=15018 , Step=13024 , Step=13034 , SelectedAuthenticationIdentityStores=Internal Users, NetworkDeviceName=devicenamehere, NetworkDeviceGroups=Device Type:All Device Types:Firewall:Cisco Systems:Firewall:ASA5545, NetworkDeviceGroups=Location:All Locations:MN, ServiceSelectionMatchedRule=TACACS, IdentityPolicyMatchedRule=Firewall, AuthorizationPolicyMatchedRule=nsdevman, AD-User-Candidate-Identities=nsdevman@ent.example.corp, AD-User-DNS-Domain=ent.example.corp, AD-User-NetBios-Name=AD-ENT, AD-User-Resolved-Identities=nsdevman@ent.example.corp, AD-User-Join-Point=ENT.example.CORP, AD-User-Resolved-DNs=CN=nsdevman\,OU=Service Accounts\,OU=CAO\,OU=ENT\,DC=ent\,DC=wfb\,DC=example\,DC=corp, StepData=10=nsdevman, StepData=11=ent.example.corp, StepData=12=example.corp, StepData=15=ent.example.corp, AD-Domain=ent.example.corp, IdentityAccessRestricted=false, UserIdentityGroup=IdentityGroup:All Groups:SystemID, Cisco-Firewall=Superuser, Firewall=Superuser, NetSec-CSM=User, NetSec-Logging=Engineer, Response={Type=Authorization; Author-Reply-Status=PassAdd; ExternalIdentityStoreName=AD1; }\n") - message = mt.render(mark="<165>", host=host) + "{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} CSCOacs_Passed_Authentications 0765855540 1 0 {{ dt }} 0178632943 5202 NOTICE Device-Administration: Command Authorization succeeded, ACSVersion=acs-5.8.1.4-B.462.x86_64, ConfigVersionId=16489, Device IP Address=10.0.0.93, DestinationIPAddress=10.0.0.10, DestinationPort=49, UserName=nsdevman, CmdSet=[ CmdAV=show CmdArgAV=vpn-sessiondb CmdArgAV=full CmdArgAV=ra-ikev2-ipsec ], Protocol=Tacacs, MatchedCommandSet=fw3, RequestLatency=11, Type=Authorization, Privilege-Level=15, Authen-Type=ASCII, Service=None, User=nsdevman, Port=443, Remote-Address=10.0.0.15, Authen-Method=TacacsPlus, Service-Argument=shell, AcsSessionID=mnsvdcfpiuac03/359448835/9871764, AuthenticationIdentityStore=AD1, AuthenticationMethod=Lookup, SelectedAccessService=Default Device Admin, SelectedCommandSet=fw3, IdentityGroup=IdentityGroup:All Groups:SystemID, Step=13005 , Step=15008 , Step=15004 , Step=15012 , Step=15041 , Step=15004 , Step=15013 , Step=24210 , Step=24212 , Step=24432 , Step=24325 , Step=24313 , Step=24319 , Step=24323 , Step=24420 , Step=24355 , Step=24416 , Step=22037 , Step=15044 , Step=15035 , Step=15042 , Step=15036 , Step=15004 , Step=15018 , Step=13024 , Step=13034 , SelectedAuthenticationIdentityStores=Internal Users, NetworkDeviceName=devicenamehere, NetworkDeviceGroups=Device Type:All Device Types:Firewall:Cisco Systems:Firewall:ASA5545, NetworkDeviceGroups=Location:All Locations:MN, ServiceSelectionMatchedRule=TACACS, IdentityPolicyMatchedRule=Firewall, AuthorizationPolicyMatchedRule=nsdevman, AD-User-Candidate-Identities=nsdevman@ent.example.corp, AD-User-DNS-Domain=ent.example.corp, AD-User-NetBios-Name=AD-ENT, AD-User-Resolved-Identities=nsdevman@ent.example.corp, AD-User-Join-Point=ENT.example.CORP, AD-User-Resolved-DNs=CN=nsdevman\,OU=Service Accounts\,OU=CAO\,OU=ENT\,DC=ent\,DC=wfb\,DC=example\,DC=corp, StepData=10=nsdevman, StepData=11=ent.example.corp, StepData=12=example.corp, StepData=15=ent.example.corp, AD-Domain=ent.example.corp, IdentityAccessRestricted=false, UserIdentityGroup=IdentityGroup:All Groups:SystemID, Cisco-Firewall=Superuser, Firewall=Superuser, NetSec-CSM=User, NetSec-Logging=Engineer, Response={Type=Authorization; Author-Reply-Status=PassAdd; ExternalIdentityStoreName=AD1; }\n") + message = mt.render(mark="<165>", host=host, dt=insert_space(dt,23)) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netauth host=\"{{ host }}\" sourcetype=\"cisco:acs\" | head 11") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netauth host=\"{{ host }}\" sourcetype=\"cisco:acs\" | head 11") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -35,17 +42,24 @@ def test_cisco_acs_single(record_property, setup_wordlist, setup_splunk, setup_s def test_cisco_acs_multi(record_property, setup_wordlist, setup_splunk, setup_sc4s): host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) +# Generate an ISO 8601 compliant timestamp with local timezone offset (2020-02-12 12:46:39.323-08:00) + dt = datetime.datetime.utcnow().replace(tzinfo=datetime.timezone.utc).astimezone().isoformat(sep=' ', timespec='milliseconds') + +# Function to insert a space between the time and TZ offset (2020-02-12 12:46:39.323 -08:00) + def insert_space(string, integer): + return string[0:integer] + ' ' + string[integer:] + mt = env.from_string( - "{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} CSCOacs_Passed_Authentications 0000000002 2 0 2011-08-01 22:32:53.032 +00:00 0000008450 5203 NOTICE Device-Administration: Session Authorization succeeded, ACSVersion=acs-5.2.0.26-B.3075, ConfigVersionId=117, Device IP Address=192.168.26.137, UserName=edward, CmdSet=[ CmdAV= ], Protocol=Tacacs, RequestLatency=10, NetworkDeviceName=switch, Type=Authorization, Privilege-Level=1, Authen-Type=ASCII, Service=Login, User=edward, Port=tty2, Remote-Address=10.78.167.190, Authen-Method=TacacsPlus, Service-Argument=shell, AcsSessionID=ACS41/101085887/112, AuthenticationIdentityStore=Internal Users, AuthenticationMethod=Lookup, SelectedAccessService=Default Device Admin, SelectedShellProfile=Permit Access, IdentityGroup=IdentityGroup:All Groups, Step=13005 , Step=15008 , Step=15004 , Step=15012 , Step=15041 , Step=15006 , Step=15013 , Step=24210 , Step=24212 , Step=22037 , Step=15044 , Step=15035 , Step=15042 , Step=15036 , Step=15004 , Step=15017 , Step=13034 ,\n") - message = mt.render(mark="<165>", host=host) + "{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} CSCOacs_Passed_Authentications 0000000002 2 0 {{ dt }} 0000008450 5203 NOTICE Device-Administration: Session Authorization succeeded, ACSVersion=acs-5.2.0.26-B.3075, ConfigVersionId=117, Device IP Address=192.168.26.137, UserName=edward, CmdSet=[ CmdAV= ], Protocol=Tacacs, RequestLatency=10, NetworkDeviceName=switch, Type=Authorization, Privilege-Level=1, Authen-Type=ASCII, Service=Login, User=edward, Port=tty2, Remote-Address=10.78.167.190, Authen-Method=TacacsPlus, Service-Argument=shell, AcsSessionID=ACS41/101085887/112, AuthenticationIdentityStore=Internal Users, AuthenticationMethod=Lookup, SelectedAccessService=Default Device Admin, SelectedShellProfile=Permit Access, IdentityGroup=IdentityGroup:All Groups, Step=13005 , Step=15008 , Step=15004 , Step=15012 , Step=15041 , Step=15006 , Step=15013 , Step=24210 , Step=24212 , Step=22037 , Step=15044 , Step=15035 , Step=15042 , Step=15036 , Step=15004 , Step=15017 , Step=13034 ,\n") + message = mt.render(mark="<165>", host=host, dt=insert_space(dt,23)) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) mt = env.from_string( - "{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} CSCOacs_Passed_Authentications 0000000002 2 1 Step=13015 , SelectedAuthenticationIdentityStores=Internal Users, NetworkDeviceGroups=s1Migrated_NDGs:All s1Migrated_NDGs, NetworkDeviceGroups=Device Type:All Device Types, NetworkDeviceGroups=Location:All Locations, ServiceSelectionMatchedRule=Rule-2, IdentityPolicyMatchedRule=Default, AuthorizationPolicyMatchedRule=Rule-0, Action=Login, Privilege-Level=1, Authen-Type=ASCII, Service=Login, Remote-Address=10.78.167.190, UserIdentityGroup=IdentityGroup:All\n") + "{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} CSCOacs_Passed_Authentications 0000000002 2 1 Step=13015 , SelectedAuthenticationIdentityStores=Internal Users, NetworkDeviceGroups=s1Migrated_NDGs:All s1Migrated_NDGs, NetworkDeviceGroups=Device Type:All Device Types, NetworkDeviceGroups=Location:All Locations, ServiceSelectionMatchedRule=Rule-2, IdentityPolicyMatchedRule=Default, AuthorizationPolicyMatchedRule=Rule-0, Action=Login, Privilege-Level=1, Authen-Type=ASCII, Service=Login, Remote-Address=10.78.167.190, UserIdentityGroup=IdentityGroup:All\n") message = mt.render(mark="<165>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netauth host=\"{{ host }}\" sourcetype=\"cisco:acs\" \"Step=13015\" | head 11") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netauth host=\"{{ host }}\" sourcetype=\"cisco:acs\" \"Step=13015\" | head 11") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -54,4 +68,4 @@ def test_cisco_acs_multi(record_property, setup_wordlist, setup_splunk, setup_sc record_property("resultCount", resultCount) record_property("message", message) - assert resultCount == 1 \ No newline at end of file + assert resultCount == 1 diff --git a/tests/test_cisco_apic.py b/tests/test_cisco_apic.py index 9526f55..4599650 100644 --- a/tests/test_cisco_apic.py +++ b/tests/test_cisco_apic.py @@ -17,11 +17,11 @@ def test_cisco_aci(record_property, setup_wordlist, setup_splunk, setup_sc4s): host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} %LOG_LOCAL0-2-SYSTEM_MSG [F0110][soaking][node-failed][critical][topology/pod-1/node-102/fault-F0110]\n") + "{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} %LOG_LOCAL0-2-SYSTEM_MSG [F0110][soaking][node-failed][critical][topology/pod-1/node-102/fault-F0110]\n") message = mt.render(mark="<165>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netops host=\"{{ host }}\" sourcetype=\"cisco:apic:events\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netops host=\"{{ host }}\" sourcetype=\"cisco:apic:events\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -37,11 +37,11 @@ def test_cisco_aci_acl(record_property, setup_wordlist, setup_splunk, setup_sc4s host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} %ACLLOG-5-ACLLOG_PKTLOG unable to locate real message\n") + "{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} %ACLLOG-5-ACLLOG_PKTLOG unable to locate real message\n") message = mt.render(mark="<165>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netfw host=\"{{ host }}\" sourcetype=\"cisco:apic:acl\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netfw host=\"{{ host }}\" sourcetype=\"cisco:apic:acl\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) diff --git a/tests/test_cisco_asa.py b/tests/test_cisco_asa.py index 32d0c00..4d71789 100644 --- a/tests/test_cisco_asa.py +++ b/tests/test_cisco_asa.py @@ -20,12 +20,12 @@ def test_cisco_asa_traditional(record_property, setup_wordlist, setup_splunk, se host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} : %ASA-3-003164: TCP access denied by ACL from 179.236.133.160/3624 to outside:72.142.18.38/23\n") + "{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} : %ASA-3-003164: TCP access denied by ACL from 179.236.133.160/3624 to outside:72.142.18.38/23\n") message = mt.render(mark="<111>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netfw host=\"{{ host }}\" sourcetype=\"cisco:asa\" \"%ASA-3-003164\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netfw host=\"{{ host }}\" sourcetype=\"cisco:asa\" \"%ASA-3-003164\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -41,12 +41,12 @@ def test_cisco_asa_traditional_nohost(record_property, setup_wordlist, setup_spl host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %}: %ASA-4-402119: IPSEC: Received an ESP packet (SPI= 0x0C190BF9, sequence number= 0x598243) from {{host}} (user= 192.0.0.1) to 192.0.0.2 that failed anti-replay checking.\n") + "{{ mark }} {% now 'local', '%b %d %H:%M:%S' %}: %ASA-4-402119: IPSEC: Received an ESP packet (SPI= 0x0C190BF9, sequence number= 0x598243) from {{host}} (user= 192.0.0.1) to 192.0.0.2 that failed anti-replay checking.\n") message = mt.render(mark="<111>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netfw sourcetype=\"cisco:asa\" \"%ASA-4-402119\" \"{{ host }}\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netfw sourcetype=\"cisco:asa\" \"%ASA-4-402119\" \"{{ host }}\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -68,7 +68,7 @@ def test_cisco_asa_rfc5424(record_property, setup_wordlist, setup_splunk, setup_ sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netfw host=\"{{ host }}\" sourcetype=\"cisco:asa\" \"%ASA-3-005424\"| head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netfw host=\"{{ host }}\" sourcetype=\"cisco:asa\" \"%ASA-3-005424\"| head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) diff --git a/tests/test_cisco_ios.py b/tests/test_cisco_ios.py index f3c3046..587559c 100644 --- a/tests/test_cisco_ios.py +++ b/tests/test_cisco_ios.py @@ -17,12 +17,12 @@ def test_cisco_ios(record_property, setup_wordlist, get_host_key, setup_splunk, host = get_host_key mt = env.from_string( - "{{ mark }}{{ seq }}: {{ host }}: *{% now 'utc', '%b %d %H:%M:%S' %}.100: CET: %SEC-6-IPACCESSLOGP: list 110 denied tcp 54.122.123.124(8932) -> 10.1.0.1(22), 1 packet\n") + "{{ mark }}{{ seq }}: {{ host }}: *{% now 'local', '%b %d %H:%M:%S' %}.100: CET: %SEC-6-IPACCESSLOGP: list 110 denied tcp 54.122.123.124(8932) -> 10.1.0.1(22), 1 packet\n") message = mt.render(mark="<166>", seq=20, host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netops host=\"{{ host }}\" sourcetype=\"cisco:ios\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netops host=\"{{ host }}\" sourcetype=\"cisco:ios\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) diff --git a/tests/test_cisco_ise.py b/tests/test_cisco_ise.py index 000e5fb..c3ff69c 100644 --- a/tests/test_cisco_ise.py +++ b/tests/test_cisco_ise.py @@ -21,26 +21,26 @@ def test_cisco_ise(record_property, setup_wordlist, setup_splunk, setup_sc4s): host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} CISE_Passed_Authentications 0001939187 4 0 {% now 'utc', '%Y-%m-%d %H:%M:%S' %}.610 +00:00 0042009748 5200 NOTICE Passed-Authentication: Authentication succeeded, ConfigVersionId=128, Device IP Address=10.6.64.15, DestinationIPAddress=10.16.20.23, DestinationPort=1812, UserName=90-1B-0E-34-EA-92, Protocol=Radius, RequestLatency=8, NetworkDeviceName=ICPAV2-SW15, User-Name=901b0e34ea92, NAS-IP-Address=10.6.64.15, NAS-Port=50104, Service-Type=Call Check, Framed-IP-Address=10.6.226.138, Framed-MTU=1500, Called-Station-ID=B0-FA-EB-11-70-04, Calling-Station-ID=90-1B-0E-34-EA-92, NAS-Port-Type=Ethernet, NAS-Port-Id=GigabitEthernet0/4, EAP-Key-Name=, cisco-av-pair=service-type=Call Check, cisco-av-pair=audit-session-id=0A06400F000006AA6F83C371, cisco-av-pair=method=mab, OriginalUserName=901b0e34ea92, NetworkDeviceProfileName=Cisco, NetworkDeviceProfileId=b2652f13-5b3f-41ba-ada2-8385c8870809, IsThirdPartyDeviceFlow=false, RadiusFlowType=WiredMAB, SSID=B0-FA-EB-11-70-04,\n") + "{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} CISE_Passed_Authentications 0001939187 4 0 {% now 'utc', '%Y-%m-%d %H:%M:%S' %}.610 +00:00 0042009748 5200 NOTICE Passed-Authentication: Authentication succeeded, ConfigVersionId=128, Device IP Address=10.6.64.15, DestinationIPAddress=10.16.20.23, DestinationPort=1812, UserName=90-1B-0E-34-EA-92, Protocol=Radius, RequestLatency=8, NetworkDeviceName=ICPAV2-SW15, User-Name=901b0e34ea92, NAS-IP-Address=10.6.64.15, NAS-Port=50104, Service-Type=Call Check, Framed-IP-Address=10.6.226.138, Framed-MTU=1500, Called-Station-ID=B0-FA-EB-11-70-04, Calling-Station-ID=90-1B-0E-34-EA-92, NAS-Port-Type=Ethernet, NAS-Port-Id=GigabitEthernet0/4, EAP-Key-Name=, cisco-av-pair=service-type=Call Check, cisco-av-pair=audit-session-id=0A06400F000006AA6F83C371, cisco-av-pair=method=mab, OriginalUserName=901b0e34ea92, NetworkDeviceProfileName=Cisco, NetworkDeviceProfileId=b2652f13-5b3f-41ba-ada2-8385c8870809, IsThirdPartyDeviceFlow=false, RadiusFlowType=WiredMAB, SSID=B0-FA-EB-11-70-04,\n") message = mt.render(mark="<165>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) mt = env.from_string( - "{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} CISE_Passed_Authentications 0001939187 4 1 AcsSessionID=ICDC-ISE03/341048949/1407358, AuthenticationIdentityStore=Internal Endpoints, AuthenticationMethod=Lookup, SelectedAccessService=Default Network Access, SelectedAuthorizationProfiles=WIRED_GUEST_REDIRECT, UseCase=Host Lookup, IdentityGroup=Endpoint Identity Groups:Unknown, Step=11001, Step=11017, Step=11027, Step=15049, Step=15008, Step=15048, Step=15048, Step=15041, Step=15013, Step=24209, Step=24211, Step=22037, Step=24715, Step=15036, Step=15048, Step=15048, Step=15016, Step=11002, SelectedAuthenticationIdentityStores=Internal Endpoints, AuthenticationStatus=AuthenticationPassed, NetworkDeviceGroups=Location#All Locations#Provo, NetworkDeviceGroups=Device Type#All Device Types#Switches, NetworkDeviceGroups=Migrated NDGs#All Migrated NDGs, IdentityPolicyMatchedRule=Default, AuthorizationPolicyMatchedRule=Guest Web Auth, UserType=Host, CPMSessionID=0A06400F000006AA6F83C371, EndPointMACAddress=90-1B-0E-34-EA-92,\n") + "{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} CISE_Passed_Authentications 0001939187 4 1 AcsSessionID=ICDC-ISE03/341048949/1407358, AuthenticationIdentityStore=Internal Endpoints, AuthenticationMethod=Lookup, SelectedAccessService=Default Network Access, SelectedAuthorizationProfiles=WIRED_GUEST_REDIRECT, UseCase=Host Lookup, IdentityGroup=Endpoint Identity Groups:Unknown, Step=11001, Step=11017, Step=11027, Step=15049, Step=15008, Step=15048, Step=15048, Step=15041, Step=15013, Step=24209, Step=24211, Step=22037, Step=24715, Step=15036, Step=15048, Step=15048, Step=15016, Step=11002, SelectedAuthenticationIdentityStores=Internal Endpoints, AuthenticationStatus=AuthenticationPassed, NetworkDeviceGroups=Location#All Locations#Provo, NetworkDeviceGroups=Device Type#All Device Types#Switches, NetworkDeviceGroups=Migrated NDGs#All Migrated NDGs, IdentityPolicyMatchedRule=Default, AuthorizationPolicyMatchedRule=Guest Web Auth, UserType=Host, CPMSessionID=0A06400F000006AA6F83C371, EndPointMACAddress=90-1B-0E-34-EA-92,\n") message = mt.render(mark="<111>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) mt = env.from_string( - "{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} CISE_Passed_Authentications 0001939187 4 2 PostureAssessmentStatus=NotApplicable, EndPointMatchedProfile=Unknown, DeviceRegistrationStatus=notRegistered, ISEPolicySetName=Wired MAB, IdentitySelectionMatchedRule=Default, StepData=5= Aruba.Aruba-Essid-Name, StepData=6= DEVICE.Device Type, StepData=8=Internal Endpoints, StepData=14= EndPoints.LogicalProfile, StepData=15= Network Access.Protocol, allowEasyWiredSession=false, DTLSSupport=Unknown, HostIdentityGroup=Endpoint Identity Groups:Unknown, Network Device Profile=Cisco, Location=Location#All Locations#Provo, Device Type=Device Type#All Device Types#Switches, Migrated NDGs=Migrated NDGs#All Migrated NDGs, Name=Endpoint Identity Groups:Unknown,\n") + "{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} CISE_Passed_Authentications 0001939187 4 2 PostureAssessmentStatus=NotApplicable, EndPointMatchedProfile=Unknown, DeviceRegistrationStatus=notRegistered, ISEPolicySetName=Wired MAB, IdentitySelectionMatchedRule=Default, StepData=5= Aruba.Aruba-Essid-Name, StepData=6= DEVICE.Device Type, StepData=8=Internal Endpoints, StepData=14= EndPoints.LogicalProfile, StepData=15= Network Access.Protocol, allowEasyWiredSession=false, DTLSSupport=Unknown, HostIdentityGroup=Endpoint Identity Groups:Unknown, Network Device Profile=Cisco, Location=Location#All Locations#Provo, Device Type=Device Type#All Device Types#Switches, Migrated NDGs=Migrated NDGs#All Migrated NDGs, Name=Endpoint Identity Groups:Unknown,\n") message = mt.render(mark="<111>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) mt = env.from_string( - "{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} CISE_Passed_Authentications 0001939187 4 3 Response={UserName=90:1B:0E:34:EA:92; User-Name=90-1B-0E-34-EA-92; State=ReauthSession:0A06400F000006AA6F83C371; Class=CACS:0A06400F000006AA6F83C371:ICDC-ISE03/341048949/1407358; Session-Timeout=300; Idle-Timeout=240; Termination-Action=RADIUS-Request; cisco-av-pair=url-redirect-acl=ACL-WEBAUTH-REDIRECT; cisco-av-pair=url-redirect=https://ICDC-ISE03.example.com:8443/portal/gateway?sessionId=0A06400F000006AA6F83C371&portal=c5a76cb0-6150-11e5-b062-0050568d954e&action=cwa&type=drw&token=6ded1943789b345f7afdd09e91549047; cisco-av-pair=profile-name=Unknown; LicenseTypes=1; },\n") + "{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} CISE_Passed_Authentications 0001939187 4 3 Response={UserName=90:1B:0E:34:EA:92; User-Name=90-1B-0E-34-EA-92; State=ReauthSession:0A06400F000006AA6F83C371; Class=CACS:0A06400F000006AA6F83C371:ICDC-ISE03/341048949/1407358; Session-Timeout=300; Idle-Timeout=240; Termination-Action=RADIUS-Request; cisco-av-pair=url-redirect-acl=ACL-WEBAUTH-REDIRECT; cisco-av-pair=url-redirect=https://ICDC-ISE03.example.com:8443/portal/gateway?sessionId=0A06400F000006AA6F83C371&portal=c5a76cb0-6150-11e5-b062-0050568d954e&action=cwa&type=drw&token=6ded1943789b345f7afdd09e91549047; cisco-av-pair=profile-name=Unknown; LicenseTypes=1; },\n") message = mt.render(mark="<111>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netauth host=\"{{ host }}\" sourcetype=\"cisco:ise:syslog\" | head 11") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netauth host=\"{{ host }}\" sourcetype=\"cisco:ise:syslog\" | head 11") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -51,16 +51,16 @@ def test_cisco_ise(record_property, setup_wordlist, setup_splunk, setup_sc4s): assert resultCount == 1 -#<181>Oct 24 21:00:02 ciscohost CISE_RADIUS_Accounting 0006028545 1 0 2019-10-24 21:00:02.879 +00:00 0088472694 3002 NOTICE Radius-Accounting: RADIUS Accounting watchdog update, ConfigVersionId=336, Device IP Address=10.0.0.3, RequestLatency=3, NetworkDeviceName=nc-aaa-aaa1, User-Name=U100000.ent.corp, NAS-IP-Address=10.0.0.3, NAS-Port=50047, Service-Type=Framed, Framed-IP-Address=10.0.0.80, Class=CACS:0AEF12345677832097B3F362:ncsilsepsuie212/356139633/9969901, Called-Station-ID=00-08-00-00-1B-AF, Calling-Station-ID=00-00-00-00-A0-7E, Acct-Status-Type=Interim-Update, Acct-Delay-Time=0, Acct-Input-Octets=653293631, Acct-Output-Octets=1497972244, Acct-Session-Id=00000B68, Acct-Authentic=RADIUS, Acct-Session-Time=241598, Acct-Input-Packets=2656224, Acct-Output-Packets=7614179, Acct-Input-Gigawords=0, Acct-Output-Gigawords=1, NAS-Port-Type=Ethernet, NAS-Port-Id=FastEthernet0/47, undefined-151=31D7AADD, cisco-av-pair=audit-session-id=0AEF10030000032097B3F362, cisco-av-pair=connect-progress=Auth Open, AcsSessionID=ncsilsepsuie205/359238109/4017186, SelectedAccessService=Default Network Access, Step=11004, Step=11017, Step=15049, Step=15008, Step=15048, Step=15048, Step=15048, Step=15004, Step=22094, Step=11005, NetworkDeviceGroups=Location#All Locations#NC, NetworkDeviceGroups=Device Type#All Device Types#Switch#2960-Switches, NetworkDeviceGroups=All Network Device Groups#All Network Device Groups, CPMSessionID=0AEF10030000032097B3F362, AllowedProtocolMatchedRule=EAP-TLS, All Network Device Groups=All Network Device Groups#All Network Device Groups, Location=Location#All Locations#NC, Device Type=Device Type#All Device Types#Switch#2960-Switches, Network Device Profile=Cisco, +#<181>Oct 24 21:00:02 ciscohost CISE_RADIUS_Accounting 0006028545 1 0 2019-10-24 21:00:02.305 +00:00 0088472694 3002 NOTICE Radius-Accounting: RADIUS Accounting watchdog update, ConfigVersionId=336, Device IP Address=10.0.0.3, RequestLatency=3, NetworkDeviceName=nc-aaa-aaa1, User-Name=U100000.ent.corp, NAS-IP-Address=10.0.0.3, NAS-Port=50047, Service-Type=Framed, Framed-IP-Address=10.0.0.80, Class=CACS:0AEF12345677832097B3F362:ncsilsepsuie212/356139633/9969901, Called-Station-ID=00-08-00-00-1B-AF, Calling-Station-ID=00-00-00-00-A0-7E, Acct-Status-Type=Interim-Update, Acct-Delay-Time=0, Acct-Input-Octets=653293631, Acct-Output-Octets=1497972244, Acct-Session-Id=00000B68, Acct-Authentic=RADIUS, Acct-Session-Time=241598, Acct-Input-Packets=2656224, Acct-Output-Packets=7614179, Acct-Input-Gigawords=0, Acct-Output-Gigawords=1, NAS-Port-Type=Ethernet, NAS-Port-Id=FastEthernet0/47, undefined-151=31D7AADD, cisco-av-pair=audit-session-id=0AEF10030000032097B3F362, cisco-av-pair=connect-progress=Auth Open, AcsSessionID=ncsilsepsuie205/359238109/4017186, SelectedAccessService=Default Network Access, Step=11004, Step=11017, Step=15049, Step=15008, Step=15048, Step=15048, Step=15048, Step=15004, Step=22094, Step=11005, NetworkDeviceGroups=Location#All Locations#NC, NetworkDeviceGroups=Device Type#All Device Types#Switch#2960-Switches, NetworkDeviceGroups=All Network Device Groups#All Network Device Groups, CPMSessionID=0AEF10030000032097B3F362, AllowedProtocolMatchedRule=EAP-TLS, All Network Device Groups=All Network Device Groups#All Network Device Groups, Location=Location#All Locations#NC, Device Type=Device Type#All Device Types#Switch#2960-Switches, Network Device Profile=Cisco, def test_cisco_ise_single(record_property, setup_wordlist, setup_splunk, setup_sc4s): host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} CISE_RADIUS_Accounting 0006028545 1 0 2019-10-24 21:00:02.879 +00:00 0088472694 3002 NOTICE Radius-Accounting: RADIUS Accounting watchdog update, ConfigVersionId=336, Device IP Address=10.0.0.3, RequestLatency=3, NetworkDeviceName=nc-aaa-aaa1, User-Name=U100000.ent.corp, NAS-IP-Address=10.0.0.3, NAS-Port=50047, Service-Type=Framed, Framed-IP-Address=10.0.0.80, Class=CACS:0AEF12345677832097B3F362:ncsilsepsuie212/356139633/9969901, Called-Station-ID=00-08-00-00-1B-AF, Calling-Station-ID=00-00-00-00-A0-7E, Acct-Status-Type=Interim-Update, Acct-Delay-Time=0, Acct-Input-Octets=653293631, Acct-Output-Octets=1497972244, Acct-Session-Id=00000B68, Acct-Authentic=RADIUS, Acct-Session-Time=241598, Acct-Input-Packets=2656224, Acct-Output-Packets=7614179, Acct-Input-Gigawords=0, Acct-Output-Gigawords=1, NAS-Port-Type=Ethernet, NAS-Port-Id=FastEthernet0/47, undefined-151=31D7AADD, cisco-av-pair=audit-session-id=0AEF10030000032097B3F362, cisco-av-pair=connect-progress=Auth Open, AcsSessionID=ncsilsepsuie205/359238109/4017186, SelectedAccessService=Default Network Access, Step=11004, Step=11017, Step=15049, Step=15008, Step=15048, Step=15048, Step=15048, Step=15004, Step=22094, Step=11005, NetworkDeviceGroups=Location#All Locations#NC, NetworkDeviceGroups=Device Type#All Device Types#Switch#2960-Switches, NetworkDeviceGroups=All Network Device Groups#All Network Device Groups, CPMSessionID=0AEF10030000032097B3F362, AllowedProtocolMatchedRule=EAP-TLS, All Network Device Groups=All Network Device Groups#All Network Device Groups, Location=Location#All Locations#NC, Device Type=Device Type#All Device Types#Switch#2960-Switches, Network Device Profile=Cisco,\n") + "{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} CISE_RADIUS_Accounting 0006028545 1 0 {% now 'utc', '%Y-%m-%d %H:%M:%S' %}.305 +00:00 0088472694 3002 NOTICE Radius-Accounting: RADIUS Accounting watchdog update, ConfigVersionId=336, Device IP Address=10.0.0.3, RequestLatency=3, NetworkDeviceName=nc-aaa-aaa1, User-Name=U100000.ent.corp, NAS-IP-Address=10.0.0.3, NAS-Port=50047, Service-Type=Framed, Framed-IP-Address=10.0.0.80, Class=CACS:0AEF12345677832097B3F362:ncsilsepsuie212/356139633/9969901, Called-Station-ID=00-08-00-00-1B-AF, Calling-Station-ID=00-00-00-00-A0-7E, Acct-Status-Type=Interim-Update, Acct-Delay-Time=0, Acct-Input-Octets=653293631, Acct-Output-Octets=1497972244, Acct-Session-Id=00000B68, Acct-Authentic=RADIUS, Acct-Session-Time=241598, Acct-Input-Packets=2656224, Acct-Output-Packets=7614179, Acct-Input-Gigawords=0, Acct-Output-Gigawords=1, NAS-Port-Type=Ethernet, NAS-Port-Id=FastEthernet0/47, undefined-151=31D7AADD, cisco-av-pair=audit-session-id=0AEF10030000032097B3F362, cisco-av-pair=connect-progress=Auth Open, AcsSessionID=ncsilsepsuie205/359238109/4017186, SelectedAccessService=Default Network Access, Step=11004, Step=11017, Step=15049, Step=15008, Step=15048, Step=15048, Step=15048, Step=15004, Step=22094, Step=11005, NetworkDeviceGroups=Location#All Locations#NC, NetworkDeviceGroups=Device Type#All Device Types#Switch#2960-Switches, NetworkDeviceGroups=All Network Device Groups#All Network Device Groups, CPMSessionID=0AEF10030000032097B3F362, AllowedProtocolMatchedRule=EAP-TLS, All Network Device Groups=All Network Device Groups#All Network Device Groups, Location=Location#All Locations#NC, Device Type=Device Type#All Device Types#Switch#2960-Switches, Network Device Profile=Cisco,\n") message = mt.render(mark="<165>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netauth host=\"{{ host }}\" sourcetype=\"cisco:ise:syslog\" | head 11") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netauth host=\"{{ host }}\" sourcetype=\"cisco:ise:syslog\" | head 11") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) diff --git a/tests/test_cisco_meraki.py b/tests/test_cisco_meraki.py index 34adcbc..62c4ab6 100644 --- a/tests/test_cisco_meraki.py +++ b/tests/test_cisco_meraki.py @@ -17,12 +17,12 @@ def test_cisco_meraki_security_event(record_property, setup_wordlist, setup_splu host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{{ mark }}1 {% now 'utc', '%s' %}.123456789 testcm-{{ host }} security_event ids_alerted signature=1:28423:1 priority=1 timestamp={% now 'utc', '%s' %}.123456 dhost=98:5A:EB:E1:81:2F direction=ingress protocol=tcp/ip src=151.101.52.238:80 dst=192.168.128.2:53023 message: EXPLOIT-KIT Multiple exploit kit single digit exe detection\n") + "{{ mark }}1 {% now 'local', '%s' %}.123456789 testcm-{{ host }} security_event ids_alerted signature=1:28423:1 priority=1 timestamp={% now 'local', '%s' %}.123456 dhost=98:5A:EB:E1:81:2F direction=ingress protocol=tcp/ip src=151.101.52.238:80 dst=192.168.128.2:53023 message: EXPLOIT-KIT Multiple exploit kit single digit exe detection\n") message = mt.render(mark="<134>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netfw host=\"testcm-{{ host }}\" sourcetype=\"meraki\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netfw host=\"testcm-{{ host }}\" sourcetype=\"meraki\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) diff --git a/tests/test_cisco_nx_os.py b/tests/test_cisco_nx_os.py index f9c267d..1f9ed65 100644 --- a/tests/test_cisco_nx_os.py +++ b/tests/test_cisco_nx_os.py @@ -16,12 +16,12 @@ def test_cisco_nx_os(record_property, setup_wordlist, get_host_key, setup_splunk host = get_host_key mt = env.from_string( - "{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} csconx-{{ host }} %MODULE-5-MOD_OK: Module 1 is online") + "{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} csconx-{{ host }} %MODULE-5-MOD_OK: Module 1 is online") message = mt.render(mark="<111>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netops host=\"csconx-{{ host }}\" sourcetype=\"cisco:ios\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netops host=\"csconx-{{ host }}\" sourcetype=\"cisco:ios\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -38,12 +38,12 @@ def test_cisco_nx_os(record_property, setup_wordlist, get_host_key, setup_splunk # host = get_host_key # # mt = env.from_string( -# "{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} %MODULE-5-MOD_OK: Module 1 is online") +# "{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} %MODULE-5-MOD_OK: Module 1 is online") # message = mt.render(mark="<23>", host=host) # # sendsingle(message, host="sc4s-nx-os") # -# st = env.from_string("search index=main host=\"{{ host }}\" sourcetype=\"cisco:ios\" | head 2") +# st = env.from_string("search earliest=-1m@m latest=+1m@m index=main host=\"{{ host }}\" sourcetype=\"cisco:ios\" | head 2") # search = st.render(host=host) # # resultCount, eventCount = splunk_single(setup_splunk, search) @@ -52,4 +52,4 @@ def test_cisco_nx_os(record_property, setup_wordlist, get_host_key, setup_splunk # record_property("resultCount", resultCount) # record_property("message", message) # -# assert resultCount == 1 \ No newline at end of file +# assert resultCount == 1 diff --git a/tests/test_cisco_ucm.py b/tests/test_cisco_ucm.py index 1a155cd..bbe24cf 100644 --- a/tests/test_cisco_ucm.py +++ b/tests/test_cisco_ucm.py @@ -27,7 +27,7 @@ def test_cisco_ucm_nohost_auditlog(record_property, setup_wordlist, setup_splunk sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) st = env.from_string( - "search index=main host=\"{{ host }}\" sourcetype=\"cisco:ucm\" | head 11") + "search earliest=-1m@m latest=+1m@m index=main host=\"{{ host }}\" sourcetype=\"cisco:ucm\" | head 11") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -50,7 +50,7 @@ def test_cisco_ucm_nohost_rtmt(record_property, setup_wordlist, setup_splunk, se sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) st = env.from_string( - "search index=main host=\"{{ host }}\" sourcetype=\"cisco:ucm\" | head 11") + "search earliest=-1m@m latest=+1m@m index=main host=\"{{ host }}\" sourcetype=\"cisco:ucm\" | head 11") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -74,7 +74,7 @@ def test_cisco_ucm_host_auditlog(record_property, setup_wordlist, setup_splunk, sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) st = env.from_string( - "search index=main host=\"{{ host }}\" sourcetype=\"cisco:ucm\" | head 11") + "search earliest=-1m@m latest=+1m@m index=main host=\"{{ host }}\" sourcetype=\"cisco:ucm\" | head 11") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -98,7 +98,7 @@ def test_cisco_ucm_nohost_alert(record_property, setup_wordlist, setup_splunk, s sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) st = env.from_string( - "search index=main host=\"{{ host }}\" sourcetype=\"cisco:ucm\" | head 11") + "search earliest=-1m@m latest=+1m@m index=main host=\"{{ host }}\" sourcetype=\"cisco:ucm\" | head 11") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) diff --git a/tests/test_citrix_netscaler.py b/tests/test_citrix_netscaler.py index 038eede..a382acf 100644 --- a/tests/test_citrix_netscaler.py +++ b/tests/test_citrix_netscaler.py @@ -20,12 +20,12 @@ def test_citrix_netscaler(record_property, setup_wordlist, setup_splunk, setup_s host = "test-ctitrixns-{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) pid = random.randint(1000, 32000) - mt = env.from_string("{{ mark }} {% now 'utc', '%m/%d/%Y:%H:%M:%S' %} GMT {{ host }} ABC-D : SSLVPN HTTPREQUEST 1234567 : Context username@192.0.2.1 - SessionId: 12345- example.com User username : Group(s) groupname : Vserver a1b2:c3d4:e5f6:a7b8:c9d0:e1f2:a3b4:c5d6:123 - 01/01/2001:01:01:01 GMT GET file/path.gif - -\n") + mt = env.from_string("{{ mark }} {% now 'utc', '%d/%m/%Y:%H:%M:%S' %} GMT {{ host }} ABC-D : SSLVPN HTTPREQUEST 1234567 : Context username@192.0.2.1 - SessionId: 12345- example.com User username : Group(s) groupname : Vserver a1b2:c3d4:e5f6:a7b8:c9d0:e1f2:a3b4:c5d6:123 - 01/01/2001:01:01:01 GMT GET file/path.gif - -\n") message = mt.render(mark="<12>", host=host, pid=pid) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netfw host={{ host }} sourcetype=\"citrix:netscaler:syslog\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netfw host={{ host }} sourcetype=\"citrix:netscaler:syslog\" | head 2") search = st.render(host=host, pid=pid) resultCount, eventCount = splunk_single(setup_splunk, search) diff --git a/tests/test_common.py b/tests/test_common.py index ea82d77..42207ae 100644 --- a/tests/test_common.py +++ b/tests/test_common.py @@ -17,12 +17,12 @@ def test_defaultroute(record_property, setup_wordlist, setup_splunk, setup_sc4s): host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) - mt = env.from_string("{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} test something else\n") + mt = env.from_string("{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} test something else\n") message = mt.render(mark="<111>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=main host=\"{{ host }}\" sourcetype=\"sc4s:fallback\" PROGRAM=\"test\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=main host=\"{{ host }}\" sourcetype=\"sc4s:fallback\" PROGRAM=\"test\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -36,12 +36,12 @@ def test_defaultroute(record_property, setup_wordlist, setup_splunk, setup_sc4s) def test_internal(record_property, setup_wordlist, setup_splunk, setup_sc4s): host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) - mt = env.from_string("{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} sc4sdefault[0]: test\n") + mt = env.from_string("{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} sc4sdefault[0]: test\n") message = mt.render(mark="<111>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=main NOT host=\"{{ host }}\" sourcetype=\"sc4s:events\" | head 1") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=main NOT host=\"{{ host }}\" sourcetype=\"sc4s:events\" | head 1") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -55,12 +55,12 @@ def test_internal(record_property, setup_wordlist, setup_splunk, setup_sc4s): def test_fallback(record_property, setup_wordlist, setup_splunk, setup_sc4s): host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) - mt = env.from_string("{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} testvp-{{ host }} test\n") + mt = env.from_string("{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} testvp-{{ host }} test\n") message = mt.render(mark="<111>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=main host=\"testvp-{{ host }}\" sourcetype=\"sc4s:fallback\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=main host=\"testvp-{{ host }}\" sourcetype=\"sc4s:fallback\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -93,7 +93,7 @@ def test_tz_guess(record_property, setup_wordlist, setup_splunk, setup_sc4s): sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netfw host=\"{{ host }}\" sourcetype=\"cisco:asa\" \"%ASA-3-003164\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netfw host=\"{{ host }}\" sourcetype=\"cisco:asa\" \"%ASA-3-003164\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -109,14 +109,17 @@ def test_tz_fix_hst(record_property, setup_wordlist, setup_splunk, setup_sc4s): host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) - dt = datetime.datetime.utcnow() - datetime.timedelta(hours=10, minutes=10) +# 10 minute offset (reserved for future use) +# dt = datetime.datetime.utcnow() - datetime.timedelta(hours=10, minutes=10) + + dt = datetime.datetime.utcnow() - datetime.timedelta(hours=10) mt = env.from_string( "{{ mark }} {{ dt }} tzfhst-{{ host }} : %ASA-3-003164: TCP access denied by ACL from 179.236.133.160/3624 to outside:72.142.18.38/23\n") message = mt.render(mark="<111>", host=host, dt=dt.strftime('%b %d %H:%M:%S')) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netfw host=\"tzfhst-{{ host }}\" sourcetype=\"cisco:asa\"") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netfw host=\"tzfhst-{{ host }}\" sourcetype=\"cisco:asa\"") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -132,14 +135,18 @@ def test_tz_fix_ny(record_property, setup_wordlist, setup_splunk, setup_sc4s): host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) tz_NY = pytz.timezone('America/New_York') - dt = datetime.datetime.now(tz_NY) - datetime.timedelta(minutes=10) + +# 10 minute offset (reserved for future use) +# dt = datetime.datetime.now(tz_NY) - datetime.timedelta(minutes=10) + + dt = datetime.datetime.now(tz_NY) mt = env.from_string( "{{ mark }} {{ dt }} tzfny-{{ host }} : %ASA-3-003164: TCP access denied by ACL from 179.236.133.160/3624 to outside:72.142.18.38/23\n") message = mt.render(mark="<111>", host=host, dt=dt.strftime('%b %d %H:%M:%S')) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netfw host=\"tzfny-{{ host }}\" sourcetype=\"cisco:asa\"") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netfw host=\"tzfny-{{ host }}\" sourcetype=\"cisco:asa\"") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -153,7 +160,7 @@ def test_tz_fix_ny(record_property, setup_wordlist, setup_splunk, setup_sc4s): def test_check_config_version(record_property, setup_wordlist, setup_splunk, setup_sc4s): - st = env.from_string("search index=main sourcetype=\"sc4s:events:startup:err\" \"Configuration file format is too old\" ") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=main sourcetype=\"sc4s:events:startup:err\" \"Configuration file format is too old\" ") search = st.render() resultCount, eventCount = splunk_single(setup_splunk, search) @@ -164,7 +171,7 @@ def test_check_config_version(record_property, setup_wordlist, setup_splunk, set def test_check_config_version_multiple(record_property, setup_wordlist, setup_splunk, setup_sc4s): - st = env.from_string("search index=main sourcetype=\"sc4s:events:startup:err\" \"you have multiple @version directives\" ") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=main sourcetype=\"sc4s:events:startup:err\" \"you have multiple @version directives\" ") search = st.render() resultCount, eventCount = splunk_single(setup_splunk, search) @@ -175,11 +182,11 @@ def test_check_config_version_multiple(record_property, setup_wordlist, setup_sp def test_check_sc4s_version(record_property, setup_wordlist, setup_splunk, setup_sc4s): - st = env.from_string("search index=main sourcetype=\"sc4s:events:startup:out\" \"sc4s version=\" NOT \"UNKNOWN\"") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=main sourcetype=\"sc4s:events:startup:out\" \"sc4s version=\" NOT \"UNKNOWN\"") search = st.render() resultCount, eventCount = splunk_single(setup_splunk, search) record_property("resultCount", resultCount) - assert resultCount == 0 \ No newline at end of file + assert resultCount == 0 diff --git a/tests/test_cyberark.py b/tests/test_cyberark.py index 5380d92..ce6d916 100644 --- a/tests/test_cyberark.py +++ b/tests/test_cyberark.py @@ -17,12 +17,12 @@ def test_cyberark_epv_5424(record_property, setup_wordlist, setup_splunk, setup_ host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{{ mark }}1 {% now 'utc', '%Y-%m-%dT%H:%M:%S' %}Z {{ host }} CEF:0|Cyber-Ark|Vault|9.20.0000|7|Logon|5|act=\"Logon\" suser=##USER_NAME## fname= dvc= shost=##SOURCE_IP## dhost= duser= externalId= app= reason= cs1Label=\"Affected User Name\" cs1= cs2Label=\"Safe Name\" cs2= cs3Label=\"Device Type\" cs3=11111 cs4Label=\"Database\" cs4=222222 cs5Label=\"Other info\" cs5= cn1Label=\"Request Id\" cn1= cn2Label=\"Ticket Id\" cn2= msg=\n") + "{{ mark }}1 {% now 'local', '%Y-%m-%dT%H:%M:%S' %}Z {{ host }} CEF:0|Cyber-Ark|Vault|9.20.0000|7|Logon|5|act=\"Logon\" suser=##USER_NAME## fname= dvc= shost=##SOURCE_IP## dhost= duser= externalId= app= reason= cs1Label=\"Affected User Name\" cs1= cs2Label=\"Safe Name\" cs2= cs3Label=\"Device Type\" cs3=11111 cs4Label=\"Database\" cs4=222222 cs5Label=\"Other info\" cs5= cn1Label=\"Request Id\" cn1= cn2Label=\"Ticket Id\" cn2= msg=\n") message = mt.render(mark="<111>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netauth host=\"{{ host }}\" sourcetype=\"cyberark:epv:cef\"| head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netauth host=\"{{ host }}\" sourcetype=\"cyberark:epv:cef\"| head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -38,12 +38,12 @@ def test_cyberark_epv(record_property, setup_wordlist, setup_splunk, setup_sc4s) host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{{ mark }}{% now 'utc', '%b %d %H:%M:%S' %} {{ host }} CEF:0|Cyber-Ark|Vault|9.20.0000|7|Logon|5|act=\"Logon\" suser=##USER_NAME## fname= dvc= shost=##SOURCE_IP## dhost= duser= externalId= app= reason= cs1Label=\"Affected User Name\" cs1= cs2Label=\"Safe Name\" cs2= cs3Label=\"Device Type\" cs3=11111 cs4Label=\"Database\" cs4=222222 cs5Label=\"Other info\" cs5= cn1Label=\"Request Id\" cn1= cn2Label=\"Ticket Id\" cn2= msg=\n") + "{{ mark }}{% now 'local', '%b %d %H:%M:%S' %} {{ host }} CEF:0|Cyber-Ark|Vault|9.20.0000|7|Logon|5|act=\"Logon\" suser=##USER_NAME## fname= dvc= shost=##SOURCE_IP## dhost= duser= externalId= app= reason= cs1Label=\"Affected User Name\" cs1= cs2Label=\"Safe Name\" cs2= cs3Label=\"Device Type\" cs3=11111 cs4Label=\"Database\" cs4=222222 cs5Label=\"Other info\" cs5= cn1Label=\"Request Id\" cn1= cn2Label=\"Ticket Id\" cn2= msg=\n") message = mt.render(mark="<111>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netauth host=\"{{ host }}\" sourcetype=\"cyberark:epv:cef\"| head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netauth host=\"{{ host }}\" sourcetype=\"cyberark:epv:cef\"| head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -59,12 +59,12 @@ def test_cyberark_pta(record_property, setup_wordlist, setup_splunk, setup_sc4s) host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{{ mark }}{% now 'utc', '%b %d %H:%M:%S' %} {{ host }} CEF:0|CyberArk|PTA|2.6.1|20|Privileged account anomaly|8|cs1Label=incidentId cs1=55a32ed8e4b0e4a90114e12c start=1436755482000 deviceCustomDate1Label=detectionDate deviceCustomDate1=1436759065017 msg=Incident updated. Now contains 7 anomalies cs2Label=link cs2=https://10.0.0.1/incidents/55a32ed8e4b0e4a90114e12c\n") + "{{ mark }}{% now 'local', '%b %d %H:%M:%S' %} {{ host }} CEF:0|CyberArk|PTA|2.6.1|20|Privileged account anomaly|8|cs1Label=incidentId cs1=55a32ed8e4b0e4a90114e12c start=1436755482000 deviceCustomDate1Label=detectionDate deviceCustomDate1=1436759065017 msg=Incident updated. Now contains 7 anomalies cs2Label=link cs2=https://10.0.0.1/incidents/55a32ed8e4b0e4a90114e12c\n") message = mt.render(mark="<111>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=main host=\"{{ host }}\" sourcetype=\"cyberark:pta:cef\"| head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=main host=\"{{ host }}\" sourcetype=\"cyberark:pta:cef\"| head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) diff --git a/tests/test_forcepoint_web.py b/tests/test_forcepoint_web.py index 4f4a410..44c5df0 100644 --- a/tests/test_forcepoint_web.py +++ b/tests/test_forcepoint_web.py @@ -17,12 +17,12 @@ def test_forcepoint_webprotect_kv(record_property, setup_wordlist, setup_splunk, host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{{ mark }}{% now 'utc', '%b %d %H:%M:%S' %} {{ host }} vendor=Websense 9f product=Security product_version=7.7.0 action=permitted severity=7 category=755 user=LDAP://user7 OU=Users,OU=Beijing,DC=com/TEST\, TEST_NAME src_host=10.0.0.4 src_port=61435 dst_host=HOST-013 dst_ip=10.0.0.19 dst_port=25404 bytes_out=4074 bytes_in=12328 http_response=200 http_method=POST http_content_type=image/gif;charset=UTF-8 http_user_agent=Mozilla/3.0 (Windows; U; Windows NT 6.1; es-def; rv:1.7.0.11) Gecko/2009060215 Firefox/8.0.11 (.NET CLR 8.5.30729) http_proxy_status_code=200 reason=- disposition=2573 policy=role-8**Default role=4 duration=63 url=http://test_web.com/contents/content1.jpg unknownfield=-\n") + "{{ mark }}{% now 'local', '%b %d %H:%M:%S' %} {{ host }} vendor=Websense 9f product=Security product_version=7.7.0 action=permitted severity=7 category=755 user=LDAP://user7 OU=Users,OU=Beijing,DC=com/TEST\, TEST_NAME src_host=10.0.0.4 src_port=61435 dst_host=HOST-013 dst_ip=10.0.0.19 dst_port=25404 bytes_out=4074 bytes_in=12328 http_response=200 http_method=POST http_content_type=image/gif;charset=UTF-8 http_user_agent=Mozilla/3.0 (Windows; U; Windows NT 6.1; es-def; rv:1.7.0.11) Gecko/2009060215 Firefox/8.0.11 (.NET CLR 8.5.30729) http_proxy_status_code=200 reason=- disposition=2573 policy=role-8**Default role=4 duration=63 url=http://test_web.com/contents/content1.jpg unknownfield=-\n") message = mt.render(mark="<134>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netproxy host=\"{{ host }}\" sourcetype=\"websense:cg:kv\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netproxy host=\"{{ host }}\" sourcetype=\"websense:cg:kv\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) diff --git a/tests/test_fortinet_ngfw.py b/tests/test_fortinet_ngfw.py index 892f87d..20c3d7b 100644 --- a/tests/test_fortinet_ngfw.py +++ b/tests/test_fortinet_ngfw.py @@ -17,12 +17,12 @@ def test_fortinet_fgt_event(record_property, setup_wordlist, setup_splunk, setup host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{{ mark }}date={% now 'utc', '%Y-%m-%d' %} time={% now 'utc', '%H:%M:%S' %} devname={{ host }} devid=FGT60D4614044725 logid=0100040704 type=event subtype=system level=notice vd=root logdesc=\"System performance statistics\" action=\"perf-stats\" cpu=2 mem=35 totalsession=61 disk=2 bandwidth=158/138 setuprate=2 disklograte=0 fazlograte=0 msg=\"Performance statistics: average CPU: 2, memory: 35, concurrent sessions: 61, setup-rate: 2\"\n") + "{{ mark }}date={% now 'local', '%Y-%m-%d' %} time={% now 'local', '%H:%M:%S' %} devname={{ host }} devid=FGT60D4614044725 logid=0100040704 type=event subtype=system level=notice vd=root logdesc=\"System performance statistics\" action=\"perf-stats\" cpu=2 mem=35 totalsession=61 disk=2 bandwidth=158/138 setuprate=2 disklograte=0 fazlograte=0 msg=\"Performance statistics: average CPU: 2, memory: 35, concurrent sessions: 61, setup-rate: 2\"\n") message = mt.render(mark="<13>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netops host=\"{{ host }}\" sourcetype=\"fgt_event\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netops host=\"{{ host }}\" sourcetype=\"fgt_event\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -38,11 +38,11 @@ def test_fortinet_fgt_traffic(record_property, setup_wordlist, setup_splunk, set host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{{ mark }}date={% now 'utc', '%Y-%m-%d' %} time={% now 'utc', '%H:%M:%S' %} devname={{ host }} devid=FG800C3912801080 logid=0004000017 type=traffic subtype=sniffer level=notice vd=root srcip=fe80::20c:29ff:fe77:20d4 srcintf=\"port3\" dstip=ff02::1:ff77:20d4 dstintf=\"port3\" sessionid=408903 proto=58 action=accept policyid=2 dstcountry=\"Reserved\" srccountry=\"Reserved\" trandisp=snat transip=:: transport=0 service=\"icmp6/131/0\" duration=36 sentbyte=0 rcvdbyte=40 sentpkt=0 rcvdpkt=0 appid=16321 app=\"IPv6.ICMP\" appcat=\"Network.Service\" apprisk=elevated applist=\"sniffer-profile\" appact=detected utmaction=allow countapp=1\n") + "{{ mark }}date={% now 'local', '%Y-%m-%d' %} time={% now 'local', '%H:%M:%S' %} devname={{ host }} devid=FG800C3912801080 logid=0004000017 type=traffic subtype=sniffer level=notice vd=root srcip=fe80::20c:29ff:fe77:20d4 srcintf=\"port3\" dstip=ff02::1:ff77:20d4 dstintf=\"port3\" sessionid=408903 proto=58 action=accept policyid=2 dstcountry=\"Reserved\" srccountry=\"Reserved\" trandisp=snat transip=:: transport=0 service=\"icmp6/131/0\" duration=36 sentbyte=0 rcvdbyte=40 sentpkt=0 rcvdpkt=0 appid=16321 app=\"IPv6.ICMP\" appcat=\"Network.Service\" apprisk=elevated applist=\"sniffer-profile\" appact=detected utmaction=allow countapp=1\n") message = mt.render(mark="<13>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netfw host=\"{{ host }}\" sourcetype=\"fgt_traffic\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netfw host=\"{{ host }}\" sourcetype=\"fgt_traffic\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -58,11 +58,11 @@ def test_fortinet_fgt_utm(record_property, setup_wordlist, setup_splunk, setup_s host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{{ mark }}date={% now 'utc', '%Y-%m-%d' %} time={% now 'utc', '%H:%M:%S' %} devname={{ host }} devid=FGT37D4613800138 logid=0317013312 type=utm subtype=webfilter eventtype=ftgd_allow level=notice vd=root sessionid=1490845588 user=\"\" srcip=172.30.16.119 srcport=53235 srcintf=\"Internal\" dstip=114.112.67.75 dstport=80 dstintf=\"External-SDC\" proto=6 service=HTTP hostname=\"popo.wan.ijinshan.com\" profile=\"scan\" action=passthrough reqtype=direct url=\"/popo/launch?c=cHA9d29vZHMxOTgyQGhvdG1haWwuY29tJnV1aWQ9NDBiNDkyZDRmNzdhNjFmOTNlMjQwMjhiYjE3ZGRlYTYmY29tcGl\" sentbyte=525 rcvdbyte=325 direction=outgoing msg=\"URL belongs to an allowed category in policy\" method=domain cat=52 catdesc=\"Information Technology\"\n") + "{{ mark }}date={% now 'local', '%Y-%m-%d' %} time={% now 'local', '%H:%M:%S' %} devname={{ host }} devid=FGT37D4613800138 logid=0317013312 type=utm subtype=webfilter eventtype=ftgd_allow level=notice vd=root sessionid=1490845588 user=\"\" srcip=172.30.16.119 srcport=53235 srcintf=\"Internal\" dstip=114.112.67.75 dstport=80 dstintf=\"External-SDC\" proto=6 service=HTTP hostname=\"popo.wan.ijinshan.com\" profile=\"scan\" action=passthrough reqtype=direct url=\"/popo/launch?c=cHA9d29vZHMxOTgyQGhvdG1haWwuY29tJnV1aWQ9NDBiNDkyZDRmNzdhNjFmOTNlMjQwMjhiYjE3ZGRlYTYmY29tcGl\" sentbyte=525 rcvdbyte=325 direction=outgoing msg=\"URL belongs to an allowed category in policy\" method=domain cat=52 catdesc=\"Information Technology\"\n") message = mt.render(mark="<13>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netids host=\"{{ host }}\" sourcetype=\"fgt_utm\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netids host=\"{{ host }}\" sourcetype=\"fgt_utm\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) diff --git a/tests/test_imperva.py b/tests/test_imperva.py index a3ec535..a827dfe 100644 --- a/tests/test_imperva.py +++ b/tests/test_imperva.py @@ -16,12 +16,12 @@ def test_imperva_incapsula(record_property, setup_wordlist, setup_splunk, setup_ host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{% now 'utc', '%b %d %H:%M:%S' %} {{ host }} " + 'CEF:0|Incapsula|SIEMintegration|1|1|Illegal Resource Access|3| fileid=3412341160002518171 sourceServiceName=site123.abcd.info siteid=1509732 suid=50005477 requestClientApplication=Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.0 deviceFacility=mia cs2=true cs2Label=Javascript Support cs3=true cs3Label=CO Support src=12.12.12.12 caIP=13.13.13.13 ccode=IL tag=www.elvis.com cn1=200 in=54 xff=44.44.44.44 cs1=NOT_SUPPORTED cs1Label=Cap Support cs4=c2e72124-0e8a-4dd8-b13b-3da246af3ab2 cs4Label=VID cs5=de3c633ac428e0678f3aac20cf7f239431e54cbb8a17e8302f53653923305e1835a9cd871db32aa4fc7b8a9463366cc4 cs5Label=clappsigdproc=Browser cs6=Firefox cs6Label=clapp ccode=IL cicode=Rehovot cs7=31.8969 cs7Label=latitude cs8=34.8186 cs8Label=longitude Customer=CEFcustomer123 ver=TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 start=1453290121336 request=site123.abcd.info/ requestmethod=GET qstr=p\=%2fetc%2fpasswd app=HTTP act=REQ_CHALLENGE_CAPTCHA deviceExternalID=33411452762204224 cpt=443 filetype=30037,1001, filepermission=2,1, cs9=Block Malicious User,High Risk Resources, cs9Label=Rule name' + "\n") + "{% now 'local', '%b %d %H:%M:%S' %} {{ host }} " + 'CEF:0|Incapsula|SIEMintegration|1|1|Illegal Resource Access|3| fileid=3412341160002518171 sourceServiceName=site123.abcd.info siteid=1509732 suid=50005477 requestClientApplication=Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.0 deviceFacility=mia cs2=true cs2Label=Javascript Support cs3=true cs3Label=CO Support src=12.12.12.12 caIP=13.13.13.13 ccode=IL tag=www.elvis.com cn1=200 in=54 xff=44.44.44.44 cs1=NOT_SUPPORTED cs1Label=Cap Support cs4=c2e72124-0e8a-4dd8-b13b-3da246af3ab2 cs4Label=VID cs5=de3c633ac428e0678f3aac20cf7f239431e54cbb8a17e8302f53653923305e1835a9cd871db32aa4fc7b8a9463366cc4 cs5Label=clappsigdproc=Browser cs6=Firefox cs6Label=clapp ccode=IL cicode=Rehovot cs7=31.8969 cs7Label=latitude cs8=34.8186 cs8Label=longitude Customer=CEFcustomer123 ver=TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 start=1453290121336 request=site123.abcd.info/ requestmethod=GET qstr=p\=%2fetc%2fpasswd app=HTTP act=REQ_CHALLENGE_CAPTCHA deviceExternalID=33411452762204224 cpt=443 filetype=30037,1001, filepermission=2,1, cs9=Block Malicious User,High Risk Resources, cs9Label=Rule name' + "\n") message = mt.render(mark="<111>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netwaf host=\"{{ host }}\" sourcetype=\"cef\" source=\"Imperva:Incapsula\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netwaf host=\"{{ host }}\" sourcetype=\"cef\" source=\"Imperva:Incapsula\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) diff --git a/tests/test_infoblox.py b/tests/test_infoblox.py index f9f29ef..ebcf366 100644 --- a/tests/test_infoblox.py +++ b/tests/test_infoblox.py @@ -20,12 +20,12 @@ def test_infoblox_dns(record_property, setup_wordlist, setup_splunk, setup_sc4s) host = "vib-{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) pid = random.randint(1000, 32000) - mt = env.from_string("{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} named[{{ pid }}]: FORMERR resolving 'www.google.com/AAAA/IN': 209.2.230.6#53\n") + mt = env.from_string("{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} named[{{ pid }}]: FORMERR resolving 'www.google.com/AAAA/IN': 209.2.230.6#53\n") message = mt.render(mark="<111>", host=host, pid=pid) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netdns host={{ host }} sourcetype=\"infoblox:dns\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netdns host={{ host }} sourcetype=\"infoblox:dns\" | head 2") search = st.render(host=host, pid=pid) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -41,12 +41,12 @@ def test_infoblox_dhcp(record_property, setup_wordlist, setup_splunk, setup_sc4s host = "vib-{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) pid = random.randint(1000, 32000) - mt = env.from_string("{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} dhcpd[{{ pri }}]: DHCPREQUEST for 10.00.00.62 from 80:00:00:00:0e:70 (EXAMPLE) via eth2 TransID 802c562c uid 01:80:00:00:00:00:70 (RENEW)\n") + mt = env.from_string("{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} dhcpd[{{ pri }}]: DHCPREQUEST for 10.00.00.62 from 80:00:00:00:0e:70 (EXAMPLE) via eth2 TransID 802c562c uid 01:80:00:00:00:00:70 (RENEW)\n") message = mt.render(mark="<150>", host=host, pid=pid) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netipam host={{ host }} sourcetype=\"infoblox:dhcp\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netipam host={{ host }} sourcetype=\"infoblox:dhcp\" | head 2") search = st.render(host=host, pid=pid) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -55,4 +55,4 @@ def test_infoblox_dhcp(record_property, setup_wordlist, setup_splunk, setup_sc4s record_property("resultCount", resultCount) record_property("message", message) - assert resultCount == 1 \ No newline at end of file + assert resultCount == 1 diff --git a/tests/test_juniper_junos_rfc3164.py b/tests/test_juniper_junos_rfc3164.py index 9bac164..8288954 100644 --- a/tests/test_juniper_junos_rfc3164.py +++ b/tests/test_juniper_junos_rfc3164.py @@ -15,12 +15,12 @@ def test_juniper_utm_standard(record_property, setup_wordlist, get_host_key, set host = get_host_key mt = env.from_string( - "{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} RT_UTM: WEBFILTER_URL_PERMITTED: WebFilter: ACTION=\"URL Permitted\" 192.168.32.1(62054)->1.1.1.1(443) CATEGORY=\"Enhanced_Information_Technology\" REASON=\"BY_PRE_DEFINED\" PROFILE=\"UTM-Wireless-Profile\" URL=ent-shasta-rrs.symantec.com OBJ=/ username N/A roles N/A") + "{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} RT_UTM: WEBFILTER_URL_PERMITTED: WebFilter: ACTION=\"URL Permitted\" 192.168.32.1(62054)->1.1.1.1(443) CATEGORY=\"Enhanced_Information_Technology\" REASON=\"BY_PRE_DEFINED\" PROFILE=\"UTM-Wireless-Profile\" URL=ent-shasta-rrs.symantec.com OBJ=/ username N/A roles N/A") message = mt.render(mark="<23>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netids host=\"{{ host }}\" sourcetype=\"juniper:junos:firewall\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netids host=\"{{ host }}\" sourcetype=\"juniper:junos:firewall\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -36,12 +36,12 @@ def test_juniper_firewall_standard(record_property, setup_wordlist, get_host_key host = get_host_key mt = env.from_string( - "{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} RT_FLOW: RT_FLOW_SESSION_CREATE: session created 192.168.1.102/58662->8.8.8.8/53 junos-dns-udp 68.144.1.1/55893->8.8.8.8/53 TRUST-INET-ACCESS None 17 OUTBOUND-INTERNET-ACCESS TRUST INTERNET 6316 N/A(N/A) vlan.192") + "{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} RT_FLOW: RT_FLOW_SESSION_CREATE: session created 192.168.1.102/58662->8.8.8.8/53 junos-dns-udp 68.144.1.1/55893->8.8.8.8/53 TRUST-INET-ACCESS None 17 OUTBOUND-INTERNET-ACCESS TRUST INTERNET 6316 N/A(N/A) vlan.192") message = mt.render(mark="<23>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netfw host=\"{{ host }}\" sourcetype=\"juniper:junos:firewall\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netfw host=\"{{ host }}\" sourcetype=\"juniper:junos:firewall\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -56,12 +56,12 @@ def test_juniper_idp_standard(record_property, setup_wordlist, get_host_key, set host = get_host_key mt = env.from_string( - "{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} RT_IDP: IDP_ATTACK_LOG_EVENT: IDP: at 1303673404, ANOMALY Attack log <64.1.2.1/48397->198.87.233.110/80> for TCP protocol and service HTTP application NONE by rule 3 of rulebase IPS in policy Recommended. attack: repeat=0, action=DROP, threat-severity=HIGH, name=HTTP:INVALID:MSNG-HTTP-VER, NAT <46.0.3.254:55870->0.0.0.0:0>, time-elapsed=0, inbytes=0, outbytes=0, inpackets=0, outpackets=0, intf:trust:fe-0/0/2.0->untrust:fe-0/0/3.0, packet-log-id: 0 and misc-message -") + "{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} RT_IDP: IDP_ATTACK_LOG_EVENT: IDP: at 1303673404, ANOMALY Attack log <64.1.2.1/48397->198.87.233.110/80> for TCP protocol and service HTTP application NONE by rule 3 of rulebase IPS in policy Recommended. attack: repeat=0, action=DROP, threat-severity=HIGH, name=HTTP:INVALID:MSNG-HTTP-VER, NAT <46.0.3.254:55870->0.0.0.0:0>, time-elapsed=0, inbytes=0, outbytes=0, inpackets=0, outpackets=0, intf:trust:fe-0/0/2.0->untrust:fe-0/0/3.0, packet-log-id: 0 and misc-message -") message = mt.render(mark="<23>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netids host=\"{{ host }}\" sourcetype=\"juniper:junos:idp\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netids host=\"{{ host }}\" sourcetype=\"juniper:junos:idp\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) diff --git a/tests/test_juniper_junos_rfc5124.py b/tests/test_juniper_junos_rfc5124.py index 188e99f..5e4565f 100644 --- a/tests/test_juniper_junos_rfc5124.py +++ b/tests/test_juniper_junos_rfc5124.py @@ -22,7 +22,7 @@ def test_juniper_junos_structured(record_property, setup_wordlist, get_host_key, sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netops host=\"{{ host }}\" sourcetype=\"juniper:structured\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netops host=\"{{ host }}\" sourcetype=\"juniper:structured\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -44,7 +44,7 @@ def test_juniper_junos_idp_structured(record_property, setup_wordlist, get_host_ sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netids host=\"{{ host }}\" sourcetype=\"juniper:junos:idp:structured\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netids host=\"{{ host }}\" sourcetype=\"juniper:junos:idp:structured\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -66,7 +66,7 @@ def test_juniper_junos_fw_structured(record_property, setup_wordlist, get_host_k sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netfw host=\"{{ host }}\" sourcetype=\"juniper:junos:firewall:structured\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netfw host=\"{{ host }}\" sourcetype=\"juniper:junos:firewall:structured\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) diff --git a/tests/test_juniper_legacy.py b/tests/test_juniper_legacy.py index f9696d9..4367883 100644 --- a/tests/test_juniper_legacy.py +++ b/tests/test_juniper_legacy.py @@ -17,12 +17,12 @@ def test_juniper_nsm_standard(record_property, setup_wordlist, get_host_key, set host = get_host_key mt = env.from_string( - "{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} jnpnsm-{{ host }} 65.197.254.193 20090320, 17331, 2009/03/20 14:47:45, 2009/03/20 14:47:50, global, 53, [FW NAME], [FW IP], traffic, traffic log, trust, (NULL), 10.1.1.20, 1725, 82.2.19.2, 2383, untrust, (NULL), 84.5.78.4, 80, 84.53.178.64, 80, tcp, global, 53, [FW NAME], fw/vpn, 4, accepted, info, no, Creation, (NULL), (NULL), (NULL), 0, 0, 0, 0, 0, 0, 0, 1, no, 0, Not Set, sos") + "{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} jnpnsm-{{ host }} 65.197.254.193 20090320, 17331, 2009/03/20 14:47:45, 2009/03/20 14:47:50, global, 53, [FW NAME], [FW IP], traffic, traffic log, trust, (NULL), 10.1.1.20, 1725, 82.2.19.2, 2383, untrust, (NULL), 84.5.78.4, 80, 84.53.178.64, 80, tcp, global, 53, [FW NAME], fw/vpn, 4, accepted, info, no, Creation, (NULL), (NULL), (NULL), 0, 0, 0, 0, 0, 0, 0, 1, no, 0, Not Set, sos") message = mt.render(mark="<134>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netfw host=\"jnpnsm-{{ host }}\" sourcetype=\"juniper:nsm\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netfw host=\"jnpnsm-{{ host }}\" sourcetype=\"juniper:nsm\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -39,12 +39,12 @@ def test_juniper_nsm_idp_standard(record_property, setup_wordlist, get_host_key, host = get_host_key mt = env.from_string( - "{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} jnpnsmidp-{{ host }} 65.197.254.193 20090320, 17331, 2009/03/20 14:47:45, 2009/03/20 14:47:50, global, 53, [IDP NAME], [IDP IP], predefined, rule, trust, (NULL), 10.1.1.20, 1725, 82.2.19.2, 2383, untrust, (NULL), 84.5.78.4, 80, 84.53.178.64, 80, tcp, global, 53, [IDP NAME], fw/vpn, 4, accepted, info, no, Creation, (NULL), (NULL), (NULL), 0, 0, 0, 0, 0, 0, 0, 1, no, 0, Not Set, sos") + "{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} jnpnsmidp-{{ host }} 65.197.254.193 20090320, 17331, 2009/03/20 14:47:45, 2009/03/20 14:47:50, global, 53, [IDP NAME], [IDP IP], predefined, rule, trust, (NULL), 10.1.1.20, 1725, 82.2.19.2, 2383, untrust, (NULL), 84.5.78.4, 80, 84.53.178.64, 80, tcp, global, 53, [IDP NAME], fw/vpn, 4, accepted, info, no, Creation, (NULL), (NULL), (NULL), 0, 0, 0, 0, 0, 0, 0, 1, no, 0, Not Set, sos") message = mt.render(mark="<134>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netids host=\"jnpnsmidp-{{ host }}\" sourcetype=\"juniper:nsm:idp\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netids host=\"jnpnsmidp-{{ host }}\" sourcetype=\"juniper:nsm:idp\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -61,12 +61,12 @@ def test_juniper_netscreen_fw(record_property, setup_wordlist, get_host_key, set host = get_host_key mt = env.from_string( - "{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} jnpns-{{ host }} ns204: NetScreen device_id=netscreen2 [Root]system-notification-00257(traffic): start_time=\"2009-03-18 16:07:06\" duration=0 policy_id=320001 service=msrpc Endpoint Mapper(tcp) proto=6 src zone=Null dst zone=self action=Deny sent=0 rcvd=16384 src=21.10.90.125 dst=23.16.1.1\n") + "{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} jnpns-{{ host }} ns204: NetScreen device_id=netscreen2 [Root]system-notification-00257(traffic): start_time=\"2009-03-18 16:07:06\" duration=0 policy_id=320001 service=msrpc Endpoint Mapper(tcp) proto=6 src zone=Null dst zone=self action=Deny sent=0 rcvd=16384 src=21.10.90.125 dst=23.16.1.1\n") message = mt.render(mark="<23>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netfw host=\"jnpns-{{ host }}\" sourcetype=\"netscreen:firewall\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netfw host=\"jnpns-{{ host }}\" sourcetype=\"netscreen:firewall\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -91,7 +91,7 @@ def test_juniper_idp_structured(record_property, setup_wordlist, get_host_key, s sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netids host=\"{{ host }}\" sourcetype=\"juniper:idp\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netids host=\"{{ host }}\" sourcetype=\"juniper:idp\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -108,12 +108,12 @@ def test_juniper_netscreen_fw_singleport(record_property, setup_wordlist, get_ho host = get_host_key mt = env.from_string( - "{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} ns204: NetScreen device_id=netscreen2 [Root]system-notification-00257(traffic): start_time=\"2009-03-18 16:07:06\" duration=0 policy_id=320001 service=msrpc Endpoint Mapper(tcp) proto=6 src zone=Null dst zone=self action=Deny sent=0 rcvd=16384 src=21.10.90.125 dst=23.16.1.1 singleport=5000\n") + "{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} ns204: NetScreen device_id=netscreen2 [Root]system-notification-00257(traffic): start_time=\"2009-03-18 16:07:06\" duration=0 policy_id=320001 service=msrpc Endpoint Mapper(tcp) proto=6 src zone=Null dst zone=self action=Deny sent=0 rcvd=16384 src=21.10.90.125 dst=23.16.1.1 singleport=5000\n") message = mt.render(mark="<23>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][5000]) - st = env.from_string("search index=netfw host=\"{{ host }}\" sourcetype=\"netscreen:firewall\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netfw host=\"{{ host }}\" sourcetype=\"netscreen:firewall\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -122,4 +122,4 @@ def test_juniper_netscreen_fw_singleport(record_property, setup_wordlist, get_ho record_property("resultCount", resultCount) record_property("message", message) - assert resultCount == 1 \ No newline at end of file + assert resultCount == 1 diff --git a/tests/test_juniper_sslvpn.py b/tests/test_juniper_sslvpn.py index 867a082..c4a6512 100644 --- a/tests/test_juniper_sslvpn.py +++ b/tests/test_juniper_sslvpn.py @@ -17,12 +17,12 @@ def test_juniper_sslvpn_standard(record_property, setup_wordlist, get_host_key, host = get_host_key mt = env.from_string( - "{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} Juniper: {% now 'utc', '%Y-%m-%d %H:%M:%S' %} - ive - [000.000.000.000] SAMPLE::xxx@xxx.xxx(Users)[User_Role] - Remote address for user xxx@xxx.xxx/Users changed from 000.000.000.000 to 000.000.000.000. Access denied.") + "{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} Juniper: {% now 'local', '%Y-%m-%d %H:%M:%S' %} - ive - [000.000.000.000] SAMPLE::xxx@xxx.xxx(Users)[User_Role] - Remote address for user xxx@xxx.xxx/Users changed from 000.000.000.000 to 000.000.000.000. Access denied.") message = mt.render(mark="<23>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netfw host=\"{{ host }}\" sourcetype=\"juniper:sslvpn\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netfw host=\"{{ host }}\" sourcetype=\"juniper:sslvpn\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) diff --git a/tests/test_linux_syslog.py b/tests/test_linux_syslog.py index e467eea..9e98dc2 100644 --- a/tests/test_linux_syslog.py +++ b/tests/test_linux_syslog.py @@ -20,12 +20,12 @@ def test_linux__nohost_program_as_path(record_property, setup_wordlist, setup_sp host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) pid = random.randint(1000, 32000) - mt = env.from_string("{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} /usr/sbin/cron[{{ pid }}]: (root) CMD (/usr/libexec/atrun)\n") + mt = env.from_string("{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} /usr/sbin/cron[{{ pid }}]: (root) CMD (/usr/libexec/atrun)\n") message = mt.render(mark="<111>", host=host, pid=pid) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=osnix \"[{{ pid }}]\" sourcetype=\"nix:syslog\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=osnix \"[{{ pid }}]\" sourcetype=\"nix:syslog\" | head 2") search = st.render(host=host, pid=pid) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -40,12 +40,12 @@ def test_linux__host_program_as_path(record_property, setup_wordlist, setup_splu host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) pid = random.randint(1000, 32000) - mt = env.from_string("{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} /usr/sbin/cron[{{ pid }}]: (root) CMD (/usr/libexec/atrun)\n") + mt = env.from_string("{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} /usr/sbin/cron[{{ pid }}]: (root) CMD (/usr/libexec/atrun)\n") message = mt.render(mark="<111>", host=host, pid=pid) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=osnix \"[{{ pid }}]\" host={{ host }} sourcetype=\"nix:syslog\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=osnix \"[{{ pid }}]\" host={{ host }} sourcetype=\"nix:syslog\" | head 2") search = st.render(host=host, pid=pid) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -60,12 +60,12 @@ def test_linux__nohost_program_conforms(record_property, setup_wordlist, setup_s host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) pid = random.randint(1000, 32000) - mt = env.from_string("{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} cron[{{ pid }}]: (root) CMD (/usr/libexec/atrun)\n") + mt = env.from_string("{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} cron[{{ pid }}]: (root) CMD (/usr/libexec/atrun)\n") message = mt.render(mark="<111>", host=host, pid=pid) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=osnix \"[{{ pid }}]\" sourcetype=\"nix:syslog\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=osnix \"[{{ pid }}]\" sourcetype=\"nix:syslog\" | head 2") search = st.render(host=host, pid=pid) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -80,12 +80,12 @@ def test_linux__host_program_conforms(record_property, setup_wordlist, setup_spl host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) pid = random.randint(1000, 32000) - mt = env.from_string("{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} cron[{{ pid }}]: (root) CMD (/usr/libexec/atrun)\n") + mt = env.from_string("{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} cron[{{ pid }}]: (root) CMD (/usr/libexec/atrun)\n") message = mt.render(mark="<111>", host=host, pid=pid) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=osnix \"[{{ pid }}]\" host={{ host }} sourcetype=\"nix:syslog\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=osnix \"[{{ pid }}]\" host={{ host }} sourcetype=\"nix:syslog\" | head 2") search = st.render(host=host, pid=pid) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -94,4 +94,4 @@ def test_linux__host_program_conforms(record_property, setup_wordlist, setup_spl record_property("resultCount", resultCount) record_property("message", message) - assert resultCount == 1 \ No newline at end of file + assert resultCount == 1 diff --git a/tests/test_microfocus_arcsight.py b/tests/test_microfocus_arcsight.py index 0825d05..1126de5 100644 --- a/tests/test_microfocus_arcsight.py +++ b/tests/test_microfocus_arcsight.py @@ -20,12 +20,12 @@ def test_microfocus_arcsight_ts_rt(record_property, setup_wordlist, setup_splunk host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{% now 'utc', '%b %d %H:%M:%S' %} {{ host }} " + 'CEF:0|ArcSight|ArcSight|7.9.0.8084.0|agent:016|Device connection up|Low| eventId=77 msg=Connected to Host mrt=1539321047341 categorySignificance=/Normal categoryBehavior=/Access/Start categoryDeviceGroup=/Application catdt=Security Management categoryOutcome=/Success categoryObject=/Host/Application art=1539321049259 cat=/Agent/Connection/Device?Success deviceSeverity=Warning rt=' + "{% now 'utc', '%s' %}" + ' dhost=WIN-PAN1 dst=192.168.13.152 destinationZoneURI=/All Zones/ArcSight System/Private Address Space Zones/RFC1918: 192.168.0.0-192.168.255.255 fileType=Agent cs2= cs2Label=Configuration Resource ahost=win-pan1 agt=192.168.13.152 agentZoneURI=/All Zones/ArcSight System/Private Address Space Zones/RFC1918: 192.168.0.0-192.168.255.255 amac=00-0C-29-98-8D-D7 av=7.9.0.8084.0 atz=Asia/Riyadh at=windowsfg dvchost=win-pan1 dvc=192.168.13.152 deviceZoneURI=/All Zones/ArcSight System/Private Address Space Zones/RFC1918: 192.168.0.0-192.168.255.255 dvcmac=00-0C-29-98-8D-D7 dtz=Asia/Riyadh _cefVer=0.1 aid=3o0OiZmYBABCACGN9CiyuGQ\\=\\=' + "\n") + "{% now 'local', '%b %d %H:%M:%S' %} {{ host }} " + 'CEF:0|ArcSight|ArcSight|7.9.0.8084.0|agent:016|Device connection up|Low| eventId=77 msg=Connected to Host mrt=1539321047341 categorySignificance=/Normal categoryBehavior=/Access/Start categoryDeviceGroup=/Application catdt=Security Management categoryOutcome=/Success categoryObject=/Host/Application art=1539321049259 cat=/Agent/Connection/Device?Success deviceSeverity=Warning rt=' + "{% now 'local', '%s' %}" + ' dhost=WIN-PAN1 dst=192.168.13.152 destinationZoneURI=/All Zones/ArcSight System/Private Address Space Zones/RFC1918: 192.168.0.0-192.168.255.255 fileType=Agent cs2= cs2Label=Configuration Resource ahost=win-pan1 agt=192.168.13.152 agentZoneURI=/All Zones/ArcSight System/Private Address Space Zones/RFC1918: 192.168.0.0-192.168.255.255 amac=00-0C-29-98-8D-D7 av=7.9.0.8084.0 atz=Asia/Riyadh at=windowsfg dvchost=win-pan1 dvc=192.168.13.152 deviceZoneURI=/All Zones/ArcSight System/Private Address Space Zones/RFC1918: 192.168.0.0-192.168.255.255 dvcmac=00-0C-29-98-8D-D7 dtz=Asia/Riyadh _cefVer=0.1 aid=3o0OiZmYBABCACGN9CiyuGQ\\=\\=' + "\n") message = mt.render(mark="<111>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=main host=\"{{ host }}\" sourcetype=\"cef\" source=ArcSight:ArcSight | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=main host=\"{{ host }}\" sourcetype=\"cef\" source=ArcSight:ArcSight | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -40,12 +40,12 @@ def test_microfocus_arcsight_ts_end(record_property, setup_wordlist, setup_splun host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{% now 'utc', '%b %d %H:%M:%S' %} {{ host }} " + 'CEF:0|ArcSight|ArcSight|7.9.0.8084.0|agent:016|Device connection up|Low| eventId=77 msg=Connected to Host mrt=1539321047341 categorySignificance=/Normal categoryBehavior=/Access/Start categoryDeviceGroup=/Application catdt=Security Management categoryOutcome=/Success categoryObject=/Host/Application art=1539321049259 cat=/Agent/Connection/Device?Success deviceSeverity=Warning end=' + "{% now 'utc', '%s' %}" + ' dhost=WIN-PAN1 dst=192.168.13.152 destinationZoneURI=/All Zones/ArcSight System/Private Address Space Zones/RFC1918: 192.168.0.0-192.168.255.255 fileType=Agent cs2= cs2Label=Configuration Resource ahost=win-pan1 agt=192.168.13.152 agentZoneURI=/All Zones/ArcSight System/Private Address Space Zones/RFC1918: 192.168.0.0-192.168.255.255 amac=00-0C-29-98-8D-D7 av=7.9.0.8084.0 atz=Asia/Riyadh at=windowsfg dvchost=win-pan1 dvc=192.168.13.152 deviceZoneURI=/All Zones/ArcSight System/Private Address Space Zones/RFC1918: 192.168.0.0-192.168.255.255 dvcmac=00-0C-29-98-8D-D7 dtz=Asia/Riyadh _cefVer=0.1 aid=3o0OiZmYBABCACGN9CiyuGQ\\=\\=' + "\n") + "{% now 'local', '%b %d %H:%M:%S' %} {{ host }} " + 'CEF:0|ArcSight|ArcSight|7.9.0.8084.0|agent:016|Device connection up|Low| eventId=77 msg=Connected to Host mrt=1539321047341 categorySignificance=/Normal categoryBehavior=/Access/Start categoryDeviceGroup=/Application catdt=Security Management categoryOutcome=/Success categoryObject=/Host/Application art=1539321049259 cat=/Agent/Connection/Device?Success deviceSeverity=Warning end=' + "{% now 'local', '%s' %}" + ' dhost=WIN-PAN1 dst=192.168.13.152 destinationZoneURI=/All Zones/ArcSight System/Private Address Space Zones/RFC1918: 192.168.0.0-192.168.255.255 fileType=Agent cs2= cs2Label=Configuration Resource ahost=win-pan1 agt=192.168.13.152 agentZoneURI=/All Zones/ArcSight System/Private Address Space Zones/RFC1918: 192.168.0.0-192.168.255.255 amac=00-0C-29-98-8D-D7 av=7.9.0.8084.0 atz=Asia/Riyadh at=windowsfg dvchost=win-pan1 dvc=192.168.13.152 deviceZoneURI=/All Zones/ArcSight System/Private Address Space Zones/RFC1918: 192.168.0.0-192.168.255.255 dvcmac=00-0C-29-98-8D-D7 dtz=Asia/Riyadh _cefVer=0.1 aid=3o0OiZmYBABCACGN9CiyuGQ\\=\\=' + "\n") message = mt.render(mark="<111>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=main host=\"{{ host }}\" sourcetype=\"cef\" source=ArcSight:ArcSight| head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=main host=\"{{ host }}\" sourcetype=\"cef\" source=ArcSight:ArcSight| head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -60,12 +60,12 @@ def test_microfocus_arcsight_ts_syslog(record_property, setup_wordlist, setup_sp host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{% now 'utc', '%b %d %H:%M:%S' %} {{ host }} " + 'CEF:0|ArcSight|ArcSight|7.9.0.8084.0|agent:016|Device connection up|Low| eventId=77 msg=Connected to Host mrt=1539321047341 categorySignificance=/Normal categoryBehavior=/Access/Start categoryDeviceGroup=/Application catdt=Security Management categoryOutcome=/Success categoryObject=/Host/Application art=1539321049259 cat=/Agent/Connection/Device?Success deviceSeverity=Warning dhost=WIN-PAN1 dst=192.168.13.152 destinationZoneURI=/All Zones/ArcSight System/Private Address Space Zones/RFC1918: 192.168.0.0-192.168.255.255 fileType=Agent cs2= cs2Label=Configuration Resource ahost=win-pan1 agt=192.168.13.152 agentZoneURI=/All Zones/ArcSight System/Private Address Space Zones/RFC1918: 192.168.0.0-192.168.255.255 amac=00-0C-29-98-8D-D7 av=7.9.0.8084.0 atz=Asia/Riyadh at=windowsfg dvchost=win-pan1 dvc=192.168.13.152 deviceZoneURI=/All Zones/ArcSight System/Private Address Space Zones/RFC1918: 192.168.0.0-192.168.255.255 dvcmac=00-0C-29-98-8D-D7 dtz=Asia/Riyadh _cefVer=0.1 aid=3o0OiZmYBABCACGN9CiyuGQ\\=\\=' + "\n") + "{% now 'local', '%b %d %H:%M:%S' %} {{ host }} " + 'CEF:0|ArcSight|ArcSight|7.9.0.8084.0|agent:016|Device connection up|Low| eventId=77 msg=Connected to Host mrt=1539321047341 categorySignificance=/Normal categoryBehavior=/Access/Start categoryDeviceGroup=/Application catdt=Security Management categoryOutcome=/Success categoryObject=/Host/Application art=1539321049259 cat=/Agent/Connection/Device?Success deviceSeverity=Warning dhost=WIN-PAN1 dst=192.168.13.152 destinationZoneURI=/All Zones/ArcSight System/Private Address Space Zones/RFC1918: 192.168.0.0-192.168.255.255 fileType=Agent cs2= cs2Label=Configuration Resource ahost=win-pan1 agt=192.168.13.152 agentZoneURI=/All Zones/ArcSight System/Private Address Space Zones/RFC1918: 192.168.0.0-192.168.255.255 amac=00-0C-29-98-8D-D7 av=7.9.0.8084.0 atz=Asia/Riyadh at=windowsfg dvchost=win-pan1 dvc=192.168.13.152 deviceZoneURI=/All Zones/ArcSight System/Private Address Space Zones/RFC1918: 192.168.0.0-192.168.255.255 dvcmac=00-0C-29-98-8D-D7 dtz=Asia/Riyadh _cefVer=0.1 aid=3o0OiZmYBABCACGN9CiyuGQ\\=\\=' + "\n") message = mt.render(mark="<111>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=main host=\"{{ host }}\" sourcetype=\"cef\" source=ArcSight:ArcSight | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=main host=\"{{ host }}\" sourcetype=\"cef\" source=ArcSight:ArcSight | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -80,12 +80,12 @@ def test_microfocus_arcsight_windows(record_property, setup_wordlist, setup_splu host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{% now 'utc', '%b %d %H:%M:%S' %} {{ host }} " + 'CEF:0|Microsoft|Microsoft Windows||Microsoft-Windows-Security-Auditing:4634|An account was logged off.|Low| eventId=1 externalId=4634 msg=Network: A user or computer logged on to this computer from the network. rawEvent=EventlogType\=Security&&EventIndex\=1031&&WindowsVersion\=Windows Server 2012 R2&&WindowsKeyMapFamily\=Windows 2012 R2&&WindowsParserFamily\=Windows 2012 R2|2012|8&&DetectTime\=2018-10-12 7:25:11&&EventSource\=Microsoft-Windows-Security-Auditing&&EventID\=4634&&EventType\=Audit_success&&EventCategory\=12545&&User\=null&&ComputerName\=WIN-PAN1&&Description\=An account was logged off.&&Message\=This event is generated when a logon session is destroyed. It may be positively correlated with a logon event using the Logon ID value. Logon IDs are only unique between reboots on the same computer.&&Subject:Security ID\=S-1-5-21-750061412-3179291162-3140434184-500&&Subject:Account Name\=Administrator&&Subject:Account Domain\=WIN-PAN1&&Subject:Logon ID\=0x373c2&&Logon Type\=3 categorySignificance=/Informational categoryBehavior=/Access/Stop categoryDeviceGroup=/Operating System catdt=Operating System categoryOutcome=/Success categoryObject=/Host/Operating System art=1539321047369 cat=Security deviceSeverity=Audit_success rt=' + "{% now 'utc', '%s' %}" + ' dhost=WIN-PAN1 dst=192.168.13.152 destinationZoneURI=/All Zones/ArcSight System/Private Address Space Zones/RFC1918: 192.168.0.0-192.168.255.255 dntdom=WIN-PAN1 duser=Administrator duid=0x373c2 cs2=Logon/Logoff:Logoff cn1=3 cs1Label=Accesses cs2Label=EventlogCategory cs4Label=Reason or Error Code cs5Label=Authentication Package Name cn1Label=LogonType cn2Label=CrashOnAuditFail cn3Label=Count ahost=win-pan1 agt=192.168.13.152 agentZoneURI=/All Zones/ArcSight System/Private Address Space Zones/RFC1918: 192.168.0.0-192.168.255.255 amac=00-0C-29-98-8D-D7 av=7.9.0.8084.0 atz=Asia/Riyadh at=windowsfg dvchost=WIN-PAN1 dvc=192.168.13.152 deviceZoneURI=/All Zones/ArcSight System/Private Address Space Zones/RFC1918: 192.168.0.0-192.168.255.255 deviceNtDomain=WIN-PAN1 dtz=Asia/Riyadh _cefVer=0.1 ad.WindowsVersion=Windows Server 2012 R2 ad.WindowsParserFamily=Windows 2012 R2|2012|8 ad.WindowsKeyMapFamily=Windows 2012 R2 ad.EventIndex=1031 aid=3o0OiZmYBABCACGN9CiyuGQ\=\=' + "\n") + "{% now 'local', '%b %d %H:%M:%S' %} {{ host }} " + 'CEF:0|Microsoft|Microsoft Windows||Microsoft-Windows-Security-Auditing:4634|An account was logged off.|Low| eventId=1 externalId=4634 msg=Network: A user or computer logged on to this computer from the network. rawEvent=EventlogType\=Security&&EventIndex\=1031&&WindowsVersion\=Windows Server 2012 R2&&WindowsKeyMapFamily\=Windows 2012 R2&&WindowsParserFamily\=Windows 2012 R2|2012|8&&DetectTime\=2018-10-12 7:25:11&&EventSource\=Microsoft-Windows-Security-Auditing&&EventID\=4634&&EventType\=Audit_success&&EventCategory\=12545&&User\=null&&ComputerName\=WIN-PAN1&&Description\=An account was logged off.&&Message\=This event is generated when a logon session is destroyed. It may be positively correlated with a logon event using the Logon ID value. Logon IDs are only unique between reboots on the same computer.&&Subject:Security ID\=S-1-5-21-750061412-3179291162-3140434184-500&&Subject:Account Name\=Administrator&&Subject:Account Domain\=WIN-PAN1&&Subject:Logon ID\=0x373c2&&Logon Type\=3 categorySignificance=/Informational categoryBehavior=/Access/Stop categoryDeviceGroup=/Operating System catdt=Operating System categoryOutcome=/Success categoryObject=/Host/Operating System art=1539321047369 cat=Security deviceSeverity=Audit_success rt=' + "{% now 'local', '%s' %}" + ' dhost=WIN-PAN1 dst=192.168.13.152 destinationZoneURI=/All Zones/ArcSight System/Private Address Space Zones/RFC1918: 192.168.0.0-192.168.255.255 dntdom=WIN-PAN1 duser=Administrator duid=0x373c2 cs2=Logon/Logoff:Logoff cn1=3 cs1Label=Accesses cs2Label=EventlogCategory cs4Label=Reason or Error Code cs5Label=Authentication Package Name cn1Label=LogonType cn2Label=CrashOnAuditFail cn3Label=Count ahost=win-pan1 agt=192.168.13.152 agentZoneURI=/All Zones/ArcSight System/Private Address Space Zones/RFC1918: 192.168.0.0-192.168.255.255 amac=00-0C-29-98-8D-D7 av=7.9.0.8084.0 atz=Asia/Riyadh at=windowsfg dvchost=WIN-PAN1 dvc=192.168.13.152 deviceZoneURI=/All Zones/ArcSight System/Private Address Space Zones/RFC1918: 192.168.0.0-192.168.255.255 deviceNtDomain=WIN-PAN1 dtz=Asia/Riyadh _cefVer=0.1 ad.WindowsVersion=Windows Server 2012 R2 ad.WindowsParserFamily=Windows 2012 R2|2012|8 ad.WindowsKeyMapFamily=Windows 2012 R2 ad.EventIndex=1031 aid=3o0OiZmYBABCACGN9CiyuGQ\=\=' + "\n") message = mt.render(mark="<111>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=oswinsec host=\"{{ host }}\" sourcetype=\"cef\" source=\"CEFEventLog:Microsoft Windows\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=oswinsec host=\"{{ host }}\" sourcetype=\"cef\" source=\"CEFEventLog:Microsoft Windows\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -100,12 +100,12 @@ def test_microfocus_arcsight_windows_system(record_property, setup_wordlist, set host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{% now 'utc', '%b %d %H:%M:%S' %} {{ host }} " + 'CEF:0|Microsoft|System or Application Event||Software Protection Platform Service:902|Software Protection Platform Service|Unknown| eventId=39 externalId=902 rawEvent=EventlogType\=Application&&EventIndex\=1604&&WindowsVersion\=Windows Server 2012 R2&&WindowsKeyMapFamily\=Windows 2012 R2&&WindowsParserFamily\=Windows 2012 R2|2012|8&&DetectTime\=2018-10-12 8:12:31&&EventSource\=Software Protection Platform Service&&EventID\=902&&EventType\=null&&EventCategory\=0&&User\=null&&ComputerName\=WIN-PAN1&&Key[0]\=6.3.9600.16384 art=1539321151610 cat=Application rt=' + "{% now 'utc', '%s' %}" + ' cs2=0 cs3=Software Protection Platform Service cs2Label=EventlogCategory cs3Label=EventSource ahost=win-pan1 agt=192.168.13.152 agentZoneURI=/All Zones/ArcSight System/Private Address Space Zones/RFC1918: 192.168.0.0-192.168.255.255 amac=00-0C-29-98-8D-D7 av=7.9.0.8084.0 atz=Asia/Riyadh at=windowsfg dvchost=WIN-PAN1 dvc=192.168.13.152 deviceZoneURI=/All Zones/ArcSight System/Private Address Space Zones/RFC1918: 192.168.0.0-192.168.255.255 dtz=Asia/Riyadh _cefVer=0.1 ad.WindowsVersion=Windows Server 2012 R2 ad.WindowsParserFamily=Windows 2012 R2|2012|8 ad.WindowsKeyMapFamily=Windows 2012 R2 ad.Key[0]=6.3.9600.16384 ad.EventIndex=1604 aid=3o0OiZmYBABCACGN9CiyuGQ\=\=' + "\n") + "{% now 'local', '%b %d %H:%M:%S' %} {{ host }} " + 'CEF:0|Microsoft|System or Application Event||Software Protection Platform Service:902|Software Protection Platform Service|Unknown| eventId=39 externalId=902 rawEvent=EventlogType\=Application&&EventIndex\=1604&&WindowsVersion\=Windows Server 2012 R2&&WindowsKeyMapFamily\=Windows 2012 R2&&WindowsParserFamily\=Windows 2012 R2|2012|8&&DetectTime\=2018-10-12 8:12:31&&EventSource\=Software Protection Platform Service&&EventID\=902&&EventType\=null&&EventCategory\=0&&User\=null&&ComputerName\=WIN-PAN1&&Key[0]\=6.3.9600.16384 art=1539321151610 cat=Application rt=' + "{% now 'local', '%s' %}" + ' cs2=0 cs3=Software Protection Platform Service cs2Label=EventlogCategory cs3Label=EventSource ahost=win-pan1 agt=192.168.13.152 agentZoneURI=/All Zones/ArcSight System/Private Address Space Zones/RFC1918: 192.168.0.0-192.168.255.255 amac=00-0C-29-98-8D-D7 av=7.9.0.8084.0 atz=Asia/Riyadh at=windowsfg dvchost=WIN-PAN1 dvc=192.168.13.152 deviceZoneURI=/All Zones/ArcSight System/Private Address Space Zones/RFC1918: 192.168.0.0-192.168.255.255 dtz=Asia/Riyadh _cefVer=0.1 ad.WindowsVersion=Windows Server 2012 R2 ad.WindowsParserFamily=Windows 2012 R2|2012|8 ad.WindowsKeyMapFamily=Windows 2012 R2 ad.Key[0]=6.3.9600.16384 ad.EventIndex=1604 aid=3o0OiZmYBABCACGN9CiyuGQ\=\=' + "\n") message = mt.render(mark="<111>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=oswin host=\"{{ host }}\" sourcetype=\"cef\" source=\"CEFEventLog:System or Application Event\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=oswin host=\"{{ host }}\" sourcetype=\"cef\" source=\"CEFEventLog:System or Application Event\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) diff --git a/tests/test_palo_alto.py b/tests/test_palo_alto.py index 400442a..de6b31a 100644 --- a/tests/test_palo_alto.py +++ b/tests/test_palo_alto.py @@ -18,12 +18,12 @@ def test_palo_alto_traffic(record_property, setup_wordlist, setup_splunk, setup_ host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} 1,{% now 'utc', '%Y/%m/%d %H:%M:%S' %},007200001056,TRAFFIC,end,1,{% now 'utc', '%Y/%m/%d %H:%M:%S' %},192.168.41.30,192.168.41.255,10.193.16.193,192.168.41.255,allow-all,,,netbios-ns,vsys1,Trust,Untrust,ethernet1/1,ethernet1/2,To-Panorama,2014/01/28 01:28:34,8720,1,137,137,11637,137,0x400000,udp,allow,276,276,0,3,2014/01/28 01:28:02,2,any,0,2076326,0x0,192.168.0.0-192.168.255.255,192.168.0.0-192.168.255.255,0,3,0\n") + "{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} 1,{% now 'local', '%Y/%m/%d %H:%M:%S' %},007200001056,TRAFFIC,end,1,{% now 'local', '%Y/%m/%d %H:%M:%S' %},192.168.41.30,192.168.41.255,10.193.16.193,192.168.41.255,allow-all,,,netbios-ns,vsys1,Trust,Untrust,ethernet1/1,ethernet1/2,To-Panorama,2014/01/28 01:28:34,8720,1,137,137,11637,137,0x400000,udp,allow,276,276,0,3,2014/01/28 01:28:02,2,any,0,2076326,0x0,192.168.0.0-192.168.255.255,192.168.0.0-192.168.255.255,0,3,0\n") message = mt.render(mark="<111>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netfw host=\"{{ host }}\" sourcetype=\"pan:traffic\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netfw host=\"{{ host }}\" sourcetype=\"pan:traffic\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -40,12 +40,12 @@ def test_palo_alto_threat(record_property, setup_wordlist, setup_splunk, setup_s host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} 1,{% now 'utc', '%Y/%m/%d %H:%M:%S' %},01606001116,THREAT,url,1,{% now 'utc', '%Y/%m/%d %H:%M:%S' %},192.168.0.2,204.232.231.46,0.0.0.0,0.0.0.0,rule1,crusher,,web-browsing,vsys1,trust,untrust,ethernet1/2,ethernet1/1,forwardAll,2012/04/10 04:39:57,22860,1,59303,80,0,0,0x208000,tcp,alert,\"litetopdetect.cn/index.php\",(9999),not-resolved,informational,client-to-server,0,0x0,192.168.0.0-192.168.255.255,United States,0,text/html\n") + "{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} 1,{% now 'local', '%Y/%m/%d %H:%M:%S' %},01606001116,THREAT,url,1,{% now 'local', '%Y/%m/%d %H:%M:%S' %},192.168.0.2,204.232.231.46,0.0.0.0,0.0.0.0,rule1,crusher,,web-browsing,vsys1,trust,untrust,ethernet1/2,ethernet1/1,forwardAll,2012/04/10 04:39:57,22860,1,59303,80,0,0,0x208000,tcp,alert,\"litetopdetect.cn/index.php\",(9999),not-resolved,informational,client-to-server,0,0x0,192.168.0.0-192.168.255.255,United States,0,text/html\n") message = mt.render(mark="<111>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netproxy host=\"{{ host }}\" sourcetype=\"pan:threat\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netproxy host=\"{{ host }}\" sourcetype=\"pan:threat\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -60,12 +60,12 @@ def test_palo_alto_traffic_badietf(record_property, setup_wordlist, setup_splunk host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{{ mark }}1 {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} 1,{% now 'utc', '%Y/%m/%d %H:%M:%S' %},007200001056,TRAFFIC,end,1,{% now 'utc', '%Y/%m/%d %H:%M:%S' %},192.168.41.30,192.168.41.255,10.193.16.193,192.168.41.255,allow-all,,,netbios-ns,vsys1,Trust,Untrust,ethernet1/1,ethernet1/2,To-Panorama,2014/01/28 01:28:34,8720,1,137,137,11637,137,0x400000,udp,allow,276,276,0,3,2014/01/28 01:28:02,2,any,0,2076326,0x0,192.168.0.0-192.168.255.255,192.168.0.0-192.168.255.255,0,3,0\n") + "{{ mark }}1 {% now 'local', '%b %d %H:%M:%S' %} {{ host }} 1,{% now 'local', '%Y/%m/%d %H:%M:%S' %},007200001056,TRAFFIC,end,1,{% now 'local', '%Y/%m/%d %H:%M:%S' %},192.168.41.30,192.168.41.255,10.193.16.193,192.168.41.255,allow-all,,,netbios-ns,vsys1,Trust,Untrust,ethernet1/1,ethernet1/2,To-Panorama,2014/01/28 01:28:34,8720,1,137,137,11637,137,0x400000,udp,allow,276,276,0,3,2014/01/28 01:28:02,2,any,0,2076326,0x0,192.168.0.0-192.168.255.255,192.168.0.0-192.168.255.255,0,3,0\n") message = mt.render(mark="<111>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netfw host=\"{{ host }}\" sourcetype=\"pan:traffic\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netfw host=\"{{ host }}\" sourcetype=\"pan:traffic\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -82,12 +82,12 @@ def test_palo_alto_traffic_mstime(record_property, setup_wordlist, setup_splunk, host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} 1,{% now 'utc', '%Y/%m/%d %H:%M:%S.%f' %},007200001056,TRAFFIC,end,1,{% now 'utc', '%Y/%m/%d %H:%M:%S.%f' %},192.168.41.30,192.168.41.255,10.193.16.193,192.168.41.255,allow-all,,,netbios-ns,vsys1,Trust,Untrust,ethernet1/1,ethernet1/2,To-Panorama,2014/01/28 01:28:34,8720,1,137,137,11637,137,0x400000,udp,allow,276,276,0,3,{% now 'utc', '%Y/%m/%d %H:%M:%S.%f' %},2,any,0,2076326,0x0,192.168.0.0-192.168.255.255,192.168.0.0-192.168.255.255,0,3,0\n") + "{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} 1,{% now 'local', '%Y/%m/%d %H:%M:%S.%f' %},007200001056,TRAFFIC,end,1,{% now 'local', '%Y/%m/%d %H:%M:%S.%f' %},192.168.41.30,192.168.41.255,10.193.16.193,192.168.41.255,allow-all,,,netbios-ns,vsys1,Trust,Untrust,ethernet1/1,ethernet1/2,To-Panorama,2014/01/28 01:28:34,8720,1,137,137,11637,137,0x400000,udp,allow,276,276,0,3,{% now 'local', '%Y/%m/%d %H:%M:%S.%f' %},2,any,0,2076326,0x0,192.168.0.0-192.168.255.255,192.168.0.0-192.168.255.255,0,3,0\n") message = mt.render(mark="<111>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netfw host=\"{{ host }}\" sourcetype=\"pan:traffic\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netfw host=\"{{ host }}\" sourcetype=\"pan:traffic\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) diff --git a/tests/test_pfsense.py b/tests/test_pfsense.py index 2f8ee6a..8b33bc4 100644 --- a/tests/test_pfsense.py +++ b/tests/test_pfsense.py @@ -16,11 +16,11 @@ def test_pfsense_filterlog(record_property, setup_wordlist, setup_splunk, setup_ host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{{mark}}{% now 'utc', '%b %d %H:%M:%S' %} filterlog: 82,,,1000002666,mvneta2,match,pass,out,6,0x00,0x00000,64,ICMPv6,58,8,{{key}},\n") + "{{mark}}{% now 'local', '%b %d %H:%M:%S' %} filterlog: 82,,,1000002666,mvneta2,match,pass,out,6,0x00,0x00000,64,ICMPv6,58,8,{{key}},\n") message = mt.render(mark="<27>", key=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][5006]) - st = env.from_string("search index=netfw sourcetype=pfsense:filterlog \"{{key}}\" earliest=-2m | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netfw sourcetype=pfsense:filterlog \"{{key}}\" earliest=-2m | head 2") search = st.render(key=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -36,11 +36,11 @@ def test_pfsense_other(record_property, setup_wordlist, setup_splunk, setup_sc4s host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{{mark}}{% now 'utc', '%b %d %H:%M:%S' %} kqueue error: {{key}}\n") + "{{mark}}{% now 'local', '%b %d %H:%M:%S' %} kqueue error: {{key}}\n") message = mt.render(mark="<27>", key=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][5006]) - st = env.from_string("search index=netops sourcetype=pfsense:* \"{{key}}\" earliest=-2m | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netops sourcetype=pfsense:* \"{{key}}\" earliest=-2m | head 2") search = st.render(key=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -56,11 +56,11 @@ def test_pfsense_syslogd(record_property, setup_wordlist, setup_splunk, setup_sc host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{{mark}}{% now 'utc', '%b %d %H:%M:%S' %} syslogd: restart {{key}}\n") + "{{mark}}{% now 'local', '%b %d %H:%M:%S' %} syslogd: restart {{key}}\n") message = mt.render(mark="<27>", key=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][5006]) - st = env.from_string("search index=netops sourcetype=pfsense:syslogd \"{{key}}\" earliest=-2m | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netops sourcetype=pfsense:syslogd \"{{key}}\" earliest=-2m | head 2") search = st.render(key=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -69,4 +69,4 @@ def test_pfsense_syslogd(record_property, setup_wordlist, setup_splunk, setup_sc record_property("resultCount", resultCount) record_property("message", message) - assert resultCount == 1 \ No newline at end of file + assert resultCount == 1 diff --git a/tests/test_plugin_example.py b/tests/test_plugin_example.py index b510ca8..5c21e35 100644 --- a/tests/test_plugin_example.py +++ b/tests/test_plugin_example.py @@ -15,12 +15,12 @@ def test_plugin_local_example(record_property, setup_wordlist, setup_splunk, setup_sc4s): host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) - mt = env.from_string("{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} sc4splugin[0]: test\n") + mt = env.from_string("{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} sc4splugin[0]: test\n") message = mt.render(mark="<111>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=main host=\"{{ host }}\" sourcetype=\"sc4s:local_example\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=main host=\"{{ host }}\" sourcetype=\"sc4s:local_example\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -29,4 +29,4 @@ def test_plugin_local_example(record_property, setup_wordlist, setup_splunk, set record_property("resultCount", resultCount) record_property("message", message) - assert resultCount == 1 \ No newline at end of file + assert resultCount == 1 diff --git a/tests/test_proofpoint.py b/tests/test_proofpoint.py index 512a406..f16e45d 100644 --- a/tests/test_proofpoint.py +++ b/tests/test_proofpoint.py @@ -17,12 +17,12 @@ def test_proofpoint_pps_filter(record_property, setup_wordlist, get_host_key, se host = get_host_key mt = env.from_string( - "{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} filter_instance1[195529]: rprt s=2hdryp02r6 m=1 x=2hdryp02r6-1 cmd=send profile=mail qid=w3HMWjG3039079 rcpts=rfaircloth@splunk.com\n") + "{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} filter_instance1[195529]: rprt s=2hdryp02r6 m=1 x=2hdryp02r6-1 cmd=send profile=mail qid=w3HMWjG3039079 rcpts=rfaircloth@splunk.com\n") message = mt.render(mark="<166>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=email host=\"{{ host }}\" sourcetype=\"pps_filter_log\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=email host=\"{{ host }}\" sourcetype=\"pps_filter_log\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -38,12 +38,12 @@ def test_proofpoint_pps_mail(record_property, setup_wordlist, get_host_key, setu host = get_host_key mt = env.from_string( - "{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} pps-{{ host }} sendmail[195529]: w3HMZPVT056101: to=, delay=00:00:01, xdelay=00:00:01, mailer=esmtp, tls_verify=FAIL, pri=133527, relay=mx1.splunk.iphmx.com. [216.71.153.223], dsn=2.0.0, stat=Sent (ok: Message 22675962 accepted)\n") + "{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} pps-{{ host }} sendmail[195529]: w3HMZPVT056101: to=, delay=00:00:01, xdelay=00:00:01, mailer=esmtp, tls_verify=FAIL, pri=133527, relay=mx1.splunk.iphmx.com. [216.71.153.223], dsn=2.0.0, stat=Sent (ok: Message 22675962 accepted)\n") message = mt.render(mark="<166>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=email host=\"pps-{{ host }}\" sourcetype=\"pps_mail_log\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=email host=\"pps-{{ host }}\" sourcetype=\"pps_mail_log\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) diff --git a/tests/test_symantec_brightmail.py b/tests/test_symantec_brightmail.py index d73bae8..130ad78 100644 --- a/tests/test_symantec_brightmail.py +++ b/tests/test_symantec_brightmail.py @@ -17,11 +17,11 @@ def test_symantec_brightmail(record_property, setup_wordlist, setup_splunk, setu host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{{ mark }}{% now 'utc', '%b %d %H:%M:%S' %} {{host}} conduit: [Brightmail] (NOTICE:7500.3119331456): [12066] 'BrightSig3 Newsletter Rules' were updated successfully.") + "{{ mark }}{% now 'local', '%b %d %H:%M:%S' %} {{host}} conduit: [Brightmail] (NOTICE:7500.3119331456): [12066] 'BrightSig3 Newsletter Rules' were updated successfully.") message = mt.render(mark="<134>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=email host=\"{{ host }}\" sourcetype=\"symantec:smg\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=email host=\"{{ host }}\" sourcetype=\"symantec:smg\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -36,28 +36,28 @@ def test_symantec_brightmail_msg(record_property, setup_wordlist, setup_splunk, host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) msgid = uuid.uuid4() - mt = env.from_string("""{{ mark }}{% now 'utc', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195989|{{ MSGID }}|VERDICT|someone@example.com|none|default|default\n -{{ mark }}{% now 'utc', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195989|{{ MSGID }}|FIRED|someone@example.com|none\n -{{ mark }}{% now 'utc', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195989|{{ MSGID }}|UNTESTED|someone@example.com|safe|opl|content_1574820902092|content_1574820956288|content_1574821059194|content_1574821017042|sys_deny_ip|sys_allow_ip|sys_deny_email|dns_allow|dns_deny|user_allow|user_deny|freq_va|freq_dha|freq_sa|connection_class_0|connection_class_1|connection_class_2|connection_class_3|connection_class_4|connection_class_5|connection_class_6|connection_class_7|connection_class_8|connection_class_9|blockedlang|knownlang\n -{{ mark }}{% now 'utc', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195989|{{ MSGID }}|LOGICAL_IP|200.200.200.154\n -{{ mark }}{% now 'utc', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195989|{{ MSGID }}|ATTACHFILTER|google-play_111-33.png\n -{{ mark }}{% now 'utc', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195989|{{ MSGID }}|ATTACHFILTER|mac_appstore_136_33.png\n -{{ mark }}{% now 'utc', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195989|{{ MSGID }}|ATTACHFILTER|6f0c8ad8-e0da-4bcc-aac9-8fa71ba43bb6.jpg\n -{{ mark }}{% now 'utc', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195989|{{ MSGID }}|ATTACHFILTER|b340ec99-9c66-4a19-a2f4-ee467e0d63e1.jpg\n -{{ mark }}{% now 'utc', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195989|{{ MSGID }}|ATTACHFILTER|product-logo.update.png\n -{{ mark }}{% now 'utc', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195989|{{ MSGID }}|ATTACHFILTER|header-research.png\n -{{ mark }}{% now 'utc', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195989|{{ MSGID }}|ATTACHFILTER|ms-logo-138.png\n -{{ mark }}{% now 'utc', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195988|{{ MSGID }}|ATTACH|ms-logo-138.png|header-research.png|product-logo.update.png|b340ec99-9c66-4a19-a2f4-ee467e0d63e1.jpg|6f0c8ad8-e0da-4bcc-aac9-8fa71ba43bb6.jpg|mac_appstore_136_33.png|google-play_111-33.png\n -{{ mark }}{% now 'utc', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195988|{{ MSGID }}|EHLO|mail6.bemta23.messagelabs.com\n -{{ mark }}{% now 'utc', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195988|{{ MSGID }}|MSG_SIZE|94239\n -{{ mark }}{% now 'utc', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195988|{{ MSGID }}|MSGID| <7jszytr60wmja@example.com>\n -{{ mark }}{% now 'utc', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195988|{{ MSGID }}|SUBJECT|pulse: this is a subject\n -{{ mark }}{% now 'utc', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195988|{{ MSGID }}|SOURCE|external\n -{{ mark }}{% now 'utc', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195987|{{ MSGID }}|VERDICT||connection_class_1|default|static connection class 1\n""") + mt = env.from_string("""{{ mark }}{% now 'local', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195989|{{ MSGID }}|VERDICT|someone@example.com|none|default|default\n +{{ mark }}{% now 'local', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195989|{{ MSGID }}|FIRED|someone@example.com|none\n +{{ mark }}{% now 'local', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195989|{{ MSGID }}|UNTESTED|someone@example.com|safe|opl|content_1574820902092|content_1574820956288|content_1574821059194|content_1574821017042|sys_deny_ip|sys_allow_ip|sys_deny_email|dns_allow|dns_deny|user_allow|user_deny|freq_va|freq_dha|freq_sa|connection_class_0|connection_class_1|connection_class_2|connection_class_3|connection_class_4|connection_class_5|connection_class_6|connection_class_7|connection_class_8|connection_class_9|blockedlang|knownlang\n +{{ mark }}{% now 'local', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195989|{{ MSGID }}|LOGICAL_IP|200.200.200.154\n +{{ mark }}{% now 'local', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195989|{{ MSGID }}|ATTACHFILTER|google-play_111-33.png\n +{{ mark }}{% now 'local', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195989|{{ MSGID }}|ATTACHFILTER|mac_appstore_136_33.png\n +{{ mark }}{% now 'local', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195989|{{ MSGID }}|ATTACHFILTER|6f0c8ad8-e0da-4bcc-aac9-8fa71ba43bb6.jpg\n +{{ mark }}{% now 'local', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195989|{{ MSGID }}|ATTACHFILTER|b340ec99-9c66-4a19-a2f4-ee467e0d63e1.jpg\n +{{ mark }}{% now 'local', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195989|{{ MSGID }}|ATTACHFILTER|product-logo.update.png\n +{{ mark }}{% now 'local', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195989|{{ MSGID }}|ATTACHFILTER|header-research.png\n +{{ mark }}{% now 'local', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195989|{{ MSGID }}|ATTACHFILTER|ms-logo-138.png\n +{{ mark }}{% now 'local', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195988|{{ MSGID }}|ATTACH|ms-logo-138.png|header-research.png|product-logo.update.png|b340ec99-9c66-4a19-a2f4-ee467e0d63e1.jpg|6f0c8ad8-e0da-4bcc-aac9-8fa71ba43bb6.jpg|mac_appstore_136_33.png|google-play_111-33.png\n +{{ mark }}{% now 'local', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195988|{{ MSGID }}|EHLO|mail6.bemta23.messagelabs.com\n +{{ mark }}{% now 'local', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195988|{{ MSGID }}|MSG_SIZE|94239\n +{{ mark }}{% now 'local', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195988|{{ MSGID }}|MSGID| <7jszytr60wmja@example.com>\n +{{ mark }}{% now 'local', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195988|{{ MSGID }}|SUBJECT|pulse: this is a subject\n +{{ mark }}{% now 'local', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195988|{{ MSGID }}|SOURCE|external\n +{{ mark }}{% now 'local', '%b %d %H:%M:%S' %} {{host}} bmserver: 1576195987|{{ MSGID }}|VERDICT||connection_class_1|default|static connection class 1\n""") message = mt.render(mark="<1>", host=host, MSGID=msgid) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=email host=\"{{ host }}\" sourcetype=\"symantec:smg:mail\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=email host=\"{{ host }}\" sourcetype=\"symantec:smg:mail\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) diff --git a/tests/test_symantec_proxy.py b/tests/test_symantec_proxy.py index 8f29a59..4f965ca 100644 --- a/tests/test_symantec_proxy.py +++ b/tests/test_symantec_proxy.py @@ -20,7 +20,7 @@ def test_bluecoatproxySG_kv(record_property, setup_wordlist, setup_splunk, setup message = mt.render(mark="<134>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netproxy host=\"{{ host }}\" sourcetype=\"bluecoat:proxysg:access:kv\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netproxy host=\"{{ host }}\" sourcetype=\"bluecoat:proxysg:access:kv\" | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) diff --git a/tests/test_ubiquiti_unifi.py b/tests/test_ubiquiti_unifi.py index 3298a58..2b79eb7 100644 --- a/tests/test_ubiquiti_unifi.py +++ b/tests/test_ubiquiti_unifi.py @@ -17,11 +17,11 @@ def test_ubiquiti_unifi_us8p60(record_property, setup_wordlist, setup_splunk, se host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{{mark}}{% now 'utc', '%b %d %H:%M:%S' %} US8P60,18e8294876c3,v4.0.66.10832 switch: DOT1S: dot1sBpduReceive(): Discarding the BPDU on port 0/7, since it is an invalid BPDU type {{key}}") + "{{mark}}{% now 'local', '%b %d %H:%M:%S' %} US8P60,18e8294876c3,v4.0.66.10832 switch: DOT1S: dot1sBpduReceive(): Discarding the BPDU on port 0/7, since it is an invalid BPDU type {{key}}") message = mt.render(mark="<27>", key=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netops sourcetype=ubnt:switch \"{{key}}\" earliest=-2m | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netops sourcetype=ubnt:switch \"{{key}}\" earliest=-2m | head 2") search = st.render(key=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -37,11 +37,11 @@ def test_ubiquiti_unifi_switch_us24p250(record_property, setup_wordlist, setup_s host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{{mark}}{% now 'utc', '%b %d %H:%M:%S' %} US24P250,f09fc26f4419,v4.0.54.10625 switch: TRAPMGR: Cold Start: Unit: {{key}}") + "{{mark}}{% now 'local', '%b %d %H:%M:%S' %} US24P250,f09fc26f4419,v4.0.54.10625 switch: TRAPMGR: Cold Start: Unit: {{key}}") message = mt.render(mark="<27>", key=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netops sourcetype=ubnt:switch \"{{key}}\" earliest=-2m | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netops sourcetype=ubnt:switch \"{{key}}\" earliest=-2m | head 2") search = st.render(key=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -57,11 +57,11 @@ def test_ubiquiti_unifi_ap_u7pg2(record_property, setup_wordlist, setup_splunk, host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{{mark}}{% now 'utc', '%b %d %H:%M:%S' %} U7PG2,788a2056b181,v4.0.66.10832: logread[5495]: Logread connected to 10.2.0.9:514") + "{{mark}}{% now 'local', '%b %d %H:%M:%S' %} U7PG2,788a2056b181,v4.0.66.10832: logread[5495]: Logread connected to 10.2.0.9:514") message = mt.render(mark="<27>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netops sourcetype=ubnt:wireless earliest=-2m | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netops sourcetype=ubnt:wireless earliest=-2m | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -77,11 +77,11 @@ def test_ubiquiti_unifi_usg(record_property, setup_wordlist, setup_splunk, setup host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{{mark}}{% now 'utc', '%b %d %H:%M:%S' %} usg-{{host}} kernel: [LAN_LOCAL-default-A]IN=eth0.2004 OUT= MAC= SRC=10.254.3.1 DST=224.0.0.251 LEN=348 TOS=0x00 PREC=0x00 TTL=255 ID=32463 DF PROTO=UDP SPT=5353 DPT=5353 LEN=328") + "{{mark}}{% now 'local', '%b %d %H:%M:%S' %} usg-{{host}} kernel: [LAN_LOCAL-default-A]IN=eth0.2004 OUT= MAC= SRC=10.254.3.1 DST=224.0.0.251 LEN=348 TOS=0x00 PREC=0x00 TTL=255 ID=32463 DF PROTO=UDP SPT=5353 DPT=5353 LEN=328") message = mt.render(mark="<27>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netfw sourcetype=ubnt:fw host=usg-{{host}} | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netfw sourcetype=ubnt:fw host=usg-{{host}} | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) diff --git a/tests/test_vmware.py b/tests/test_vmware.py index 8649ade..9c16f27 100644 --- a/tests/test_vmware.py +++ b/tests/test_vmware.py @@ -25,7 +25,7 @@ def test_linux_vmware(record_property, setup_wordlist, setup_splunk, setup_sc4s) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=main {{ pid }} sourcetype=\"vmware:vsphere:esx\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=main {{ pid }} sourcetype=\"vmware:vsphere:esx\" | head 2") search = st.render(host=host, pid=pid) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -46,7 +46,7 @@ def test_linux_vmware_nsx_ietf(record_property, setup_wordlist, setup_splunk, se sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=main host={{ host }} sourcetype=\"vmware:vsphere:nsx\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=main host={{ host }} sourcetype=\"vmware:vsphere:nsx\" | head 2") search = st.render(host=host, pid=pid) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -62,12 +62,12 @@ def test_linux_vmware_nsx_fw(record_property, setup_wordlist, setup_splunk, setu host = "testvmw-{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) pid = random.randint(1000, 32000) - mt = env.from_string("{{ mark }} {% now 'utc', '%b %d %H:%M:%S' %} {{ host }} dfwpktlogs: {{ pid }} INET match PASS domain-c7/1001 IN 60 TCP 10.33.24.50/45926->10.33.24.9/8140 S\n") + mt = env.from_string("{{ mark }} {% now 'local', '%b %d %H:%M:%S' %} {{ host }} dfwpktlogs: {{ pid }} INET match PASS domain-c7/1001 IN 60 TCP 10.33.24.50/45926->10.33.24.9/8140 S\n") message = mt.render(mark="<144>", host=host, pid=pid) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=main host={{ host }} {{ pid }} sourcetype=\"vmware:vsphere:nsx\" | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=main host={{ host }} {{ pid }} sourcetype=\"vmware:vsphere:nsx\" | head 2") search = st.render(host=host, pid=pid) resultCount, eventCount = splunk_single(setup_splunk, search) diff --git a/tests/test_zscaler_proxy.py b/tests/test_zscaler_proxy.py index e5581bb..595eb08 100644 --- a/tests/test_zscaler_proxy.py +++ b/tests/test_zscaler_proxy.py @@ -17,11 +17,11 @@ def test_zscaler_proxy(record_property, setup_wordlist, setup_splunk, setup_sc4s host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{% now 'utc', '%Y-%m-%d %H:%M:%S' %}\treason=Allowed\tevent_id=6748427317914894361\tprotocol=HTTPS\taction=Allowed\ttransactionsize=663\tresponsesize=65\trequestsize=598\turlcategory=UK_ALLOW_Pharmacies\tserverip=216.58.204.70\tclienttranstime=0\trequestmethod=CONNECT\trefererURL=None\tuseragent=Windows Windows 10 Enterprise ZTunnel/1.0\tproduct=NSS\tlocation=UK_Wynyard_VPN->other\tClientIP=192.168.0.38\tstatus=200\tuser=first.last@example.com\turl=4171764.fls.doubleclick.net:443\tvendor=Zscaler\thostname={{host}}.fls.doubleclick.net\tclientpublicIP=213.86.221.94\tthreatcategory=None\tthreatname=None\tfiletype=None\tappname=DoubleClick\tpagerisk=0\tdepartment=Procurement, Generics\turlsupercategory=User-defined\tappclass=Sales and Marketing\tdlpengine=None\turlclass=Bandwidth Loss\tthreatclass=None\tdlpdictionaries=None\tfileclass=None\tbwthrottle=NO\tservertranstime=0\tmd5=None") + "{% now 'local', '%Y-%m-%d %H:%M:%S' %}\treason=Allowed\tevent_id=6748427317914894361\tprotocol=HTTPS\taction=Allowed\ttransactionsize=663\tresponsesize=65\trequestsize=598\turlcategory=UK_ALLOW_Pharmacies\tserverip=216.58.204.70\tclienttranstime=0\trequestmethod=CONNECT\trefererURL=None\tuseragent=Windows Windows 10 Enterprise ZTunnel/1.0\tproduct=NSS\tlocation=UK_Wynyard_VPN->other\tClientIP=192.168.0.38\tstatus=200\tuser=first.last@example.com\turl=4171764.fls.doubleclick.net:443\tvendor=Zscaler\thostname={{host}}.fls.doubleclick.net\tclientpublicIP=213.86.221.94\tthreatcategory=None\tthreatname=None\tfiletype=None\tappname=DoubleClick\tpagerisk=0\tdepartment=Procurement, Generics\turlsupercategory=User-defined\tappclass=Sales and Marketing\tdlpengine=None\turlclass=Bandwidth Loss\tthreatclass=None\tdlpdictionaries=None\tfileclass=None\tbwthrottle=NO\tservertranstime=0\tmd5=None") message = mt.render(mark="<134>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netproxy sourcetype=\"zscalernss-web\" hostname={{host}}.fls.doubleclick.net | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netproxy sourcetype=\"zscalernss-web\" hostname={{host}}.fls.doubleclick.net | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search) @@ -37,11 +37,11 @@ def test_zscaler_proxy_pri(record_property, setup_wordlist, setup_splunk, setup_ host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist)) mt = env.from_string( - "{{mark}}{% now 'utc', '%Y-%m-%d %H:%M:%S' %}\treason=Allowed\tevent_id=6748427317914894362\tprotocol=HTTPS\taction=Allowed\ttransactionsize=663\tresponsesize=65\trequestsize=598\turlcategory=UK_ALLOW_Pharmacies\tserverip=216.58.204.70\tclienttranstime=0\trequestmethod=CONNECT\trefererURL=None\tuseragent=Windows Windows 10 Enterprise ZTunnel/1.0\tproduct=NSS\tlocation=UK_Wynyard_VPN->other\tClientIP=192.168.0.38\tstatus=200\tuser=first.last@example.com\turl=4171764.fls.doubleclick.net:443\tvendor=Zscaler\thostname={{host}}.fls.doubleclick.net\tclientpublicIP=213.86.221.94\tthreatcategory=None\tthreatname=None\tfiletype=None\tappname=DoubleClick\tpagerisk=0\tdepartment=Procurement, Generics\turlsupercategory=User-defined\tappclass=Sales and Marketing\tdlpengine=None\turlclass=Bandwidth Loss\tthreatclass=None\tdlpdictionaries=None\tfileclass=None\tbwthrottle=NO\tservertranstime=0\tmd5=None") + "{{mark}}{% now 'local', '%Y-%m-%d %H:%M:%S' %}\treason=Allowed\tevent_id=6748427317914894362\tprotocol=HTTPS\taction=Allowed\ttransactionsize=663\tresponsesize=65\trequestsize=598\turlcategory=UK_ALLOW_Pharmacies\tserverip=216.58.204.70\tclienttranstime=0\trequestmethod=CONNECT\trefererURL=None\tuseragent=Windows Windows 10 Enterprise ZTunnel/1.0\tproduct=NSS\tlocation=UK_Wynyard_VPN->other\tClientIP=192.168.0.38\tstatus=200\tuser=first.last@example.com\turl=4171764.fls.doubleclick.net:443\tvendor=Zscaler\thostname={{host}}.fls.doubleclick.net\tclientpublicIP=213.86.221.94\tthreatcategory=None\tthreatname=None\tfiletype=None\tappname=DoubleClick\tpagerisk=0\tdepartment=Procurement, Generics\turlsupercategory=User-defined\tappclass=Sales and Marketing\tdlpengine=None\turlclass=Bandwidth Loss\tthreatclass=None\tdlpdictionaries=None\tfileclass=None\tbwthrottle=NO\tservertranstime=0\tmd5=None") message = mt.render(mark="<134>", host=host) sendsingle(message, setup_sc4s[0], setup_sc4s[1][514]) - st = env.from_string("search index=netproxy sourcetype=\"zscalernss-web\" hostname={{host}}.fls.doubleclick.net | head 2") + st = env.from_string("search earliest=-1m@m latest=+1m@m index=netproxy sourcetype=\"zscalernss-web\" hostname={{host}}.fls.doubleclick.net | head 2") search = st.render(host=host) resultCount, eventCount = splunk_single(setup_splunk, search)