Skip to content

Commit

Permalink
Merge branch 'develop' into feature/netscaler2
Browse files Browse the repository at this point in the history
  • Loading branch information
Ryan Faircloth authored and GitHub committed Feb 10, 2020
2 parents eeb0d87 + 651db37 commit 5c21e67
Show file tree
Hide file tree
Showing 13 changed files with 430 additions and 3 deletions.
95 changes: 95 additions & 0 deletions docs/sources/Cisco/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,54 @@ index=<asconfigured> sourcetype=cisco:acs

Verify timestamp, and host values match as expected

## Product - APIC (ACI)

| Ref | Link |
|----------------|---------------------------------------------------------------------------------------------------------|
| Splunk Add-on | No current add-on for syslog events |
| Product Manual | https://community.cisco.com/t5/security-documents/acs-5-x-configuring-the-external-syslog-server/ta-p/3143143 |


### Sourcetypes

| sourcetype | notes |
|----------------|---------------------------------------------------------------------------------------------------------|
| cisco:apic:acl: | APIC events from leaf switches |
| cisco:apic:events | APIC events from any component used |

### Sourcetype and Index Configuration

| key | sourcetype | index | notes |
|----------------|----------------|----------------|----------------|
| cisco_apic_acl | cisco:apic:acl | netfw | None |
| cisco_apic_events | cisco:apic:events | netops | None |

### Filter type

PATTERN MATCH

### Setup and Configuration

* No special steps required

### Options

| Variable | default | description |
|----------------|----------------|----------------|
| SC4S_LISTEN_CISCO_APIC_TCP_PORT | empty string | Enable a TCP port for this specific vendor product using the number defined |
| SC4S_LISTEN_CISCO_APIC_UDP_PORT | empty string | Enable a UDP port for this specific vendor product using the number defined |
| SC4S_ARCHIVE_CISCO_APIC | no | Enable archive to disk for this specific source |
| SC4S_DEST_CISCO_APIC_HEC | no | When Splunk HEC is disabled globally set to yes to enable this specific source |

### Verification

Use the following search to validate events are present

```
index=<asconfigured> sourcetype=cisco:apic:*
```

Verify timestamp, and host values match as expected

## Product - ASA (Pre Firepower)

Expand Down Expand Up @@ -283,3 +331,50 @@ index=<asconfigured> sourcetype=merkai

Verify timestamp, and host values match as expected

## Product - UCM

| Ref | Link |
|----------------|---------------------------------------------------------------------------------------------------------|
| Splunk Add-on | na |
| Product Manual | multiple |


### Sourcetypes

| sourcetype | notes |
|----------------|---------------------------------------------------------------------------------------------------------|
| cisco:ucm | None |

### Sourcetype and Index Configuration

| key | sourcetype | index | notes |
|----------------|----------------|----------------|----------------|
| cisco_ucm | cisco:ucm | ucm | None |


### Filter type

PATTERN MATCH

### Setup and Configuration

* Refer to Cisco support web site

### Options

| Variable | default | description |
|----------------|----------------|----------------|
| SC4S_LISTEN_CISCO_UCM_TCP_PORT | empty string | Enable a TCP port for this specific vendor product using the number defined |
| SC4S_LISTEN_CISCO_UCM_UDP_PORT | empty string | Enable a UDP port for this specific vendor product using the number defined |
| SC4S_ARCHIVE_CISCO_UCM | no | Enable archive to disk for this specific source |
| SC4S_DEST_CISCO_UCM_HEC | no | When Splunk HEC is disabled globally set to yes to enable this specific source |

### Verification

Use the following search to validate events are present

```
index=<asconfigured> sourcetype=cisco:ucm
```

Verify timestamp, and host values match as expected
6 changes: 6 additions & 0 deletions package/etc/conf.d/filters/cisco/apic.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@

filter f_cisco_apic {
program('^%LOG_LOCAL\d-\d-');
or
program('^%ACLLOG-\d-ACLLOG_PKTLOG');
};
32 changes: 32 additions & 0 deletions package/etc/conf.d/filters/cisco/ucm.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@

filter f_cisco_ucm {
message("^%UC\_")
or
message("^%CCM\_")
};

filter f_cisco_ucm_message {
message(
'^(<\d{1,3}>)\d*: (?:([^:]+): )?(.*) : (%.*)'
flags(store-matches)
);
};

parser p_cisco_ucm_date {
#Oct 14 2015 05:50:19 AM.484 UTC
#Apr 21 19:01:35.638 UTC
date-parser(format(
'%b %d %Y %I:%M:%S %p.%f %Z',
'%b %d %H:%M:%S.%f %Z'
)
template("$3")
);
};

rewrite r_cisco_ucm_message {
set("cisco_ucm" value("fields.sc4s_syslog_format"));
set("cisco_ucm" value("fields.sc4s_vendor_product"));
set("$HOST_FROM" value("HOST") );
set("$2" value("HOST") condition(match("^..." template("${2}"))) );
set("$4" value("MESSAGE"));
};
56 changes: 56 additions & 0 deletions package/etc/conf.d/log_paths/lp-cisco_apic.conf.tmpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Cisco APIC
{{- /* The following provides a unique port source configuration if env var(s) are set */}}
{{- $context := dict "port_id" "CISCO_APIC" "parser" "rfc3164" }}
{{- tmpl.Exec "t/source_network.t" $context }}

log {
junction {
{{- if or (or (getenv (print "SC4S_LISTEN_CISCO_APIC_TCP_PORT")) (getenv (print "SC4S_LISTEN_CISCO_APIC_UDP_PORT"))) (getenv (print "SC4S_LISTEN_CISCO_APIC_TLS_PORT")) }}
channel {
# Listen on the specified dedicated port(s) for CISCO_APIC traffic
source (s_CISCO_APIC);
flags (final);
};
{{- end}}
channel {
# Listen on the default port (typically 514) for CISCO_APIC traffic
source (s_DEFAULT);
filter(f_cisco_apic);
flags(final);
};
};

rewrite {
guess-time-zone();
};
if {
filter {
program('^%ACLLOG-\d-ACLLOG_PKTLOG')
};
rewrite {
set("cisco_APIC_acl", value("fields.sc4s_vendor_product"));
r_set_splunk_dest_default(sourcetype("cisco:apic:acl"), index("netfw"), template("t_hdr_msg"))
};
parser { p_add_context_splunk(key("cisco_apic_acl")); };

} elif {
rewrite {
set("cisco_APIC_events", value("fields.sc4s_vendor_product"));
r_set_splunk_dest_default(sourcetype("cisco:apic:events"), index("netops"), template("t_hdr_msg"))
};
parser { p_add_context_splunk(key("cisco_apic_events")); };
};

parser (compliance_meta_by_source);
rewrite { set("$(template ${.splunk.sc4s_template} $(template t_hdr_msg))" value("MSG")); };

{{- if or (conv.ToBool (getenv "SC4S_DEST_SPLUNK_HEC_GLOBAL" "yes")) (conv.ToBool (getenv "SC4S_DEST_CISCO_APIC_HEC" "no")) }}
destination(d_hec);
{{- end}}

{{- if or (conv.ToBool (getenv "SC4S_ARCHIVE_GLOBAL" "no")) (conv.ToBool (getenv "SC4S_ARCHIVE_CISCO_APIC" "no")) }}
destination(d_archive);
{{- end}}

flags(flow-control,final);
};
62 changes: 62 additions & 0 deletions package/etc/conf.d/log_paths/lp-cisco_ucm.conf.tmpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# Cisco UCM
{{- /* The following provides a unique port source configuration if env var(s) are set */}}
{{- $context := dict "port_id" "CISCO_UCM" "parser" "cisco_ucm" }}
{{- tmpl.Exec "t/source_network.t" $context }}

log {
junction {
{{- if or (or (getenv (print "SC4S_LISTEN_CISCO_UCM_TCP_PORT")) (getenv (print "SC4S_LISTEN_CISCO_UCM_UDP_PORT"))) (getenv (print "SC4S_LISTEN_CISCO_UCM_TLS_PORT")) }}
channel {
# Listen on the specified dedicated port(s) for CISCO_UCM traffic
source (s_CISCO_UCM);
flags (final);
};
{{- end}}
channel {
# Listen on the default port (typically 514) for CISCO_UCM traffic
source (s_DEFAULT);
filter(f_cisco_ucm);
flags(final);
};
};

if {
filter {
message(
'Node ?ID(?:\:|\=)([^ \]]+)'
flags(store-matches)
);
};
rewrite {
set("$1" value("HOST") );
};
} elif {
filter {
message(
' on node ([^ ]+\. |[^ ]+ )'
flags(store-matches)
);
};
rewrite {
set("$1" value("HOST") );
};
};

rewrite {
set("cisco_ucm", value("fields.sc4s_vendor_product"));
r_set_splunk_dest_default(sourcetype("cisco:ucm"), index("main"))
};
parser {p_add_context_splunk(key("cisco_ucm")); };
parser (compliance_meta_by_source);
rewrite { set("$(template ${.splunk.sc4s_template} $(template t_hdr_msg))" value("MSG")); };

{{- if or (conv.ToBool (getenv "SC4S_DEST_SPLUNK_HEC_GLOBAL" "yes")) (conv.ToBool (getenv "SC4S_DEST_CISCO_UCM_HEC" "no")) }}
destination(d_hec);
{{- end}}

{{- if or (conv.ToBool (getenv "SC4S_ARCHIVE_GLOBAL" "no")) (conv.ToBool (getenv "SC4S_ARCHIVE_CISCO_UCM" "no")) }}
destination(d_archive);
{{- end}}

flags(flow-control,final);
};
3 changes: 3 additions & 0 deletions package/etc/context_templates/splunk_index.csv.example
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,14 @@
#checkpoint_splunk_web,index,netproxy
#checkpoint_splunk,index,netops
#checkpoint_splunk,index,netops
#cisco_apic_acl,index,netfw
#cisco_apic_events,index,netops
#cisco_acs,index,netauth
#cisco_asa,index,netfw
#cisco_ios,index,netops
#cisco_ise,index,netauth
#cisco_nx_os,index,netops
#cisco_ucm,index,main
#citrix_netscaler,index,netfw
#local_example,index,main
#forcepoint_webprotect,index,netproxy
Expand Down
7 changes: 7 additions & 0 deletions package/etc/go_templates/source_network.t
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,9 @@ source s_{{ .port_id }} {
{{ else if eq .parser "citrix_netscaler" }}
parser(p_citrix_netscaler_date);
rewrite(r_citrix_netscaler_message);
{{ else if eq .parser "cisco_ucm" }}
parser (p_cisco_ucm_date);
rewrite (r_cisco_ucm_message);
{{ else if eq .parser "no_parse" }}
rewrite(set_no_parse);
{{ else }}
Expand All @@ -107,6 +110,10 @@ source s_{{ .port_id }} {
} elif {
parser (p_cisco_meraki);
rewrite(set_rfc5424_epochtime);
} elif {
filter(f_cisco_ucm_message);
parser (p_cisco_ucm_date);
rewrite (r_cisco_ucm_message);
} elif {
filter(f_rfc3164_version);
rewrite(set_rfc3164_no_version_string);
Expand Down
1 change: 1 addition & 0 deletions tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
import splunklib.client as client



@pytest.fixture(scope="module")
def setup_wordlist():
path_to_current_file = os.path.realpath(__file__)
Expand Down
1 change: 1 addition & 0 deletions tests/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ services:
- SC4S_LISTEN_PALOALTO_PANOS_TCP_PORT=5005
- SC4S_LISTEN_PFSENSE_TCP_PORT=5006
- SC4S_ARCHIVE_GLOBAL=no
- SC4S_SOURCE_STORE_RAWMSG=yes
splunk:
build:
context: ../splunk
Expand Down
5 changes: 3 additions & 2 deletions tests/splunkutils.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,13 @@ def splunk_single(service, search):

if stats["isDone"] == "1":
break
sleep(2)
else:
sleep(2)

# Get the results and display them
resultCount = stats["resultCount"]
eventCount = stats["eventCount"]
if resultCount > 0 or tried > 15:
if resultCount > 0 or tried > 5:
break
else:
tried += 1
Expand Down
53 changes: 53 additions & 0 deletions tests/test_cisco_apic.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Copyright 2019 Splunk, Inc.
#
# Use of this source code is governed by a BSD-2-clause-style
# license that can be found in the LICENSE-BSD2 file or at
# https://opensource.org/licenses/BSD-2-Clause
import random

from jinja2 import Environment

from .sendmessage import *
from .splunkutils import *

env = Environment(extensions=['jinja2_time.TimeExtension'])

#<11>July 22 22:45:28 apic1 %LOG_LOCAL0-2-SYSTEM_MSG [F0110][soaking][node-failed][critical][topology/pod-1/node-102/fault-F0110] Node 102 not reachable. unknown
def test_cisco_aci(record_property, setup_wordlist, setup_splunk, setup_sc4s):
host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist))

mt = env.from_string(
"{{ mark }}{% now 'utc', '%b %d %H:%M:%S' %} {{ host }} %LOG_LOCAL0-2-SYSTEM_MSG [F0110][soaking][node-failed][critical][topology/pod-1/node-102/fault-F0110]\n")
message = mt.render(mark="<165>", host=host)
sendsingle(message, setup_sc4s[0], setup_sc4s[1][514])

st = env.from_string("search index=netops host=\"{{ host }}\" sourcetype=\"cisco:apic:events\" | head 2")
search = st.render(host=host)

resultCount, eventCount = splunk_single(setup_splunk, search)

record_property("host", host)
record_property("resultCount", resultCount)
record_property("message", message)

assert resultCount == 1

#%ACLLOG-5-ACLLOG_PKTLOG
def test_cisco_aci_acl(record_property, setup_wordlist, setup_splunk, setup_sc4s):
host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist))

mt = env.from_string(
"{{ mark }}{% now 'utc', '%b %d %H:%M:%S' %} {{ host }} %ACLLOG-5-ACLLOG_PKTLOG unable to locate real message\n")
message = mt.render(mark="<165>", host=host)
sendsingle(message, setup_sc4s[0], setup_sc4s[1][514])

st = env.from_string("search index=netfw host=\"{{ host }}\" sourcetype=\"cisco:apic:acl\" | head 2")
search = st.render(host=host)

resultCount, eventCount = splunk_single(setup_splunk, search)

record_property("host", host)
record_property("resultCount", resultCount)
record_property("message", message)

assert resultCount == 1
2 changes: 1 addition & 1 deletion tests/test_cisco_asa.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ def test_cisco_asa_traditional_nohost(record_property, setup_wordlist, setup_spl

sendsingle(message, setup_sc4s[0], setup_sc4s[1][514])

st = env.from_string("search index=netfw sourcetype=\"cisco:asa\" \"%ASA-4-402119\" \"{host}\" | head 2")
st = env.from_string("search index=netfw sourcetype=\"cisco:asa\" \"%ASA-4-402119\" \"{{ host }}\" | head 2")
search = st.render(host=host)

resultCount, eventCount = splunk_single(setup_splunk, search)
Expand Down
Loading

0 comments on commit 5c21e67

Please sign in to comment.