Skip to content

Commit

Permalink
FortiWeb initial commit
Browse files Browse the repository at this point in the history
  • Loading branch information
kenny committed Feb 12, 2020
1 parent 6cb882c commit 802d853
Show file tree
Hide file tree
Showing 5 changed files with 241 additions and 1 deletion.
92 changes: 91 additions & 1 deletion docs/sources/Fortinet/index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@

| key | sourcetype | index | notes |
|----------------|----------------|----------------|----------------|
| fortinet_fortios_traffic | fgt_traffic | netops | none |
| fortinet_fortios_traffic | fgt_traffic | netfw | none |
| fortinet_fortios_utm | fgt_utm | netids | none |
| fortinet_fortios_event | fgt_event | netops | none |
| fortinet_fortios_log | fgt_log | netops | none |
Expand Down Expand Up @@ -105,4 +105,94 @@ index=<asconfigured> (sourcetype=fgt_log OR sourcetype=fgt_traffic OR sourcetype
###Event Message Type
![FortiGate Event message](FortiGate_event.png)

Verify timestamp, and host values match as expected

## Product - FortiWeb

| Ref | Link |
|----------------|---------------------------------------------------------------------------------------------------------|
| Splunk Add-on | https://splunkbase.splunk.com/app/4679/ |
| Product Manual | https://docs.fortinet.com/product/fortiweb/6.3 |


### Sourcetypes

| sourcetype | notes |
|----------------|---------------------------------------------------------------------------------------------------------|
| fweb_log | The catch all sourcetype is not used |
| fwb_traffic | None |
| fwb_attack | None |
| fwb_event | None


### Sourcetype and Index Configuration

| key | sourcetype | index | notes |
|----------------|----------------|----------------|----------------|
| fortinet_fortiweb_traffic | fwb_traffic | netfw | none |
| fortinet_fortiweb_attack | fwb_attack | netids | none |
| fortinet_fortiweb_event | fwb_event | netops | none |
| fortinet_fortiweb_log | fwb_log | netops | none |


### Filter type

MSG Parse: This filter parses message content

### Setup and Configuration

* Install the Splunk Add-on on the search head(s) for the user communities interested in this data source. If SC4S is exclusively used the addon is not required on the indexer.
* Review and update the splunk_index.csv file and set the index and sourcetype as required for the data source.
* Refer to the admin manual for specific details of configuration to send Reliable syslog using RFC 3195 format, a typical logging configuration will include the following features.

```
config log syslog-policy
edit splunk
config syslog-server-list
edit 1
set server x.x.x.x
set port 514 (Example. Should be the same as in data input of Splunk server)
end
end
config log syslogd
set policy splunk
set status enable
end
```

### Options

| Variable | default | description |
|----------------|----------------|----------------|
| SC4S_LISTEN_FORTINET_FORTIWEB_TCP_PORT | empty string | Enable a UDP port for this specific vendor product using the number defined |
| SC4S_LISTEN_FORTINET_FORTIWEB_UDP_PORT | empty string | Enable a UDP port for this specific vendor product using the number defined |
| SC4S_ARCHIVE_FORTINET_FORTIWEB | no | Enable archive to disk for this specific source |
| SC4S_DEST_FORTINET_FORTIWEB_HEC | no | When Splunk HEC is disabled globally set to yes to enable this specific source |

### Verification

An active firewall will generate frequent events, in addition fortigate has the ability to test logging functionality using a built in command

```
diag log test
```

Verify timestamp, and host values match as expected

```
index=<asconfigured> (sourcetype=fwb_log OR sourcetype=fwb_traffic OR sourcetype=fwb_attack OR sourcetype=fwb_event)
```

Verify timestamp, and host values match as expected
3 changes: 3 additions & 0 deletions package/etc/conf.d/filters/fortinet/fortiweb.conf
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
filter f_fortinet_fortiweb {
message('device_id=\"?FV.+type=\"?(traffic|attack|event)');
};
71 changes: 71 additions & 0 deletions package/etc/conf.d/log_paths/lp-fortinet_fortiweb.conf.tmpl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# Fortinet FortiWeb
{{ $context := dict "port_id" "FORTINET_FORTIWEB" "parser" "common" }}
{{ tmpl.Exec "t/source_network.t" $context }}

# The following is an inline template; we will use this to generate the actual log path
{{ define "log_path" }}
log {
{{- if eq (.) "yes"}}
source(s_DEFAULT);
filter(f_fortinet_fortiweb);
{{- end}}
{{- if eq (.) "no"}}
source (s_FORTINET_FORTIWEB);
{{- end}}

parser {
kv-parser(prefix(".kv.") pair-separator(",") template("${MSGHDR} ${MSG}"));
};

rewrite {
set("${.kv.devname}", value("HOST"));
set("fortigate_fortiweb", value("fields.sc4s_vendor_product"));
};

if (match("traffic" value(".kv.type"))) {
rewrite { r_set_splunk_dest_default(sourcetype("fwb_traffic"), index("netfw"))};
parser {p_add_context_splunk(key("fortinet_fortiweb_traffic")); };
} elif (match("attack" value(".kv.type"))) {
rewrite { r_set_splunk_dest_default(sourcetype("fwb_attack"), index("netids"))};
parser {p_add_context_splunk(key("fortinet_fortiweb_attack")); };
} elif (match("event" value(".kv.type"))) {
rewrite { r_set_splunk_dest_default(sourcetype("fwb_event"), index("netops"))};
parser {p_add_context_splunk(key("fortinet_fortiweb_event")); };
} else {
rewrite { r_set_splunk_dest_default(sourcetype("fwb_log"), index("netops"))};
parser {p_add_context_splunk(key("fortinet_fortiweb_log")); };
};

parser (compliance_meta_by_source);

#We want to unset the fields we won't need, as this is copied into the
#disk queue for network destinations. This can be very disk expensive
#if we don't
rewrite {
set("$(template ${fields.sc4s_template} $(template t_hdr_msg))" value("MSG"));
unset(value("RAWMSG"));
unset(value("PROGRAM"));
unset(value("LEGACY_MSGHDR"));
groupunset(values(".kv.*"));
};

{{- if ((getenv "SC4S_DEST_SPLUNK_HEC_GLOBAL" "yes") | conv.ToBool) or (conv.ToBool (getenv "SC4S_DEST_FORTINET_FORTIWEB_HEC" "no") | conv.ToBool) }}
destination(d_hec);
{{- end}}


{{- if (getenv "SC4S_ARCHIVE_GLOBAL") or (getenv "SC4S_ARCHIVE_FORTINET_FORTIWEB") }}
destination(d_archive);
{{- end}}

flags(flow-control);
};
{{- end}}

{{- if or (or (getenv (print "SC4S_LISTEN_FORTINET_FORTIWEB_TCP_PORT")) (getenv (print "SC4S_LISTEN_FORTINET_FORTIWEB_UDP_PORT"))) (getenv (print "SC4S_LISTEN_FORTINET_FORTIWEB_TLS_PORT")) }}
# Listen on the specified dedicated port(s) for FORTINET_FORTIWEB traffic
{{ tmpl.Exec "log_path" "no" }}
{{- end}}

# Listen on the default port (typically 514) for FORTINET_FORTIWEB traffic
{{ tmpl.Exec "log_path" "yes" }}
3 changes: 3 additions & 0 deletions package/etc/context_templates/splunk_index.csv.example
View file
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,9 @@
#fortinet_fortios_log,index,netops
#fortinet_fortios_traffic,index,netfw
#fortinet_fortios_utm,index,netids
#fortinet_fortweb_log,index,netops
#fortinet_fortweb_traffic,index,netfw
#fortinet_fortweb_attack,index,netids
#infoblox_dns,index,netdns
#infoblox_dhcp,index,netipam
#infoblox_threat,index,netids
Expand Down
73 changes: 73 additions & 0 deletions tests/test_fortinet_web.py
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# Copyright 2019 Splunk, Inc.
#
# Use of this source code is governed by a BSD-2-clause-style
# license that can be found in the LICENSE-BSD2 file or at
# https://opensource.org/licenses/BSD-2-Clause
import random

from jinja2 import Environment

from .sendmessage import *
from .splunkutils import *

env = Environment(extensions=['jinja2_time.TimeExtension'])

#<111> 2020-02-12,23:13:33,devname=FortiWeb-A,log_id=11005607,msg_id=000377260939,device_id=FV-1111111800222,vd=\"root\",\"timezone=\"\"(GMT+3:00)Kuwait,Riyadh\"\"\",type=event,subtype=\"system\",pri=notice,trigger_policy=\"Splunk_policy\",user=daemon,ui=daemon,action=check-resource,status=success,\"msg=\"\"The logdisk usage is too high\"\"\"
def test_fortinet_fwb_event(record_property, setup_wordlist, setup_splunk, setup_sc4s):
host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist))

mt = env.from_string(
"{{ mark }}{% now 'utc', '%Y-%m-%d' %},{% now 'utc', '%H:%M:%S' %},devname={{ host }},log_id=11005607,msg_id=000377260939,device_id=FV-1111111800222,vd=\"root\",\"timezone=\"\"(GMT+3:00)Kuwait,Riyadh\"\"\",type=event,subtype=\"system\",pri=notice,trigger_policy=\"Splunk_policy\",user=daemon,ui=daemon,action=check-resource,status=success,\"msg=\"\"The logdisk usage is too high\"\"\"\n")
message = mt.render(mark="<13>", host=host)

sendsingle(message, setup_sc4s[0], setup_sc4s[1][514])

st = env.from_string("search index=netops host=\"{{ host }}\" sourcetype=\"fwb_event\" | head 2")
search = st.render(host=host)

resultCount, eventCount = splunk_single(setup_splunk, search)

record_property("host", host)
record_property("resultCount", resultCount)
record_property("message", message)

assert resultCount == 1
#<111> 2020-02-12,23:16:41,devname=FortiWeb-A,log_id=30001000,msg_id=000377262759,device_id=FV-1111111800222,vd="root","timezone=""(GMT+3:00)Kuwait,Riyadh""",type=traffic,subtype="https",pri=notice,proto=tcp,service=https/tls1.2,status=success,reason=none,policy=Phome_Policy,original_src=1.107.71.90,src=1.107.71.90,src_port=28799,dst=1.16.16.11,dst_port=80,http_request_time=0,http_response_time=0,http_request_bytes=623,http_response_bytes=15660,http_method=get,"http_url=""/publish/templates/images/bluebottom.jpg""","http_host=""splunk.infigo.hr""","http_agent=""Mozilla/5.0 (Linux; Android 9; SM-J415F) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.87 Mobile Safari/537.36""",http_retcode=200,"msg=""HTTPS get request from 1.107.71.90:28799 to 1.16.16.11:80""",original_srccountry="Saudi Arabia",srccountry="Saudi Arabia",content_switch_name="none",server_pool_name="PHOME","user_name=""Unknown""","http_refer=""https://splunk.infigo.hr/publish/templates/CSS/sc4s.css""",http_version="1.x",dev_id=none,cipher_suite="TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384"
def test_fortinet_fwb_traffic(record_property, setup_wordlist, setup_splunk, setup_sc4s):
host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist))

mt = env.from_string(
"{{ mark }}{% now 'utc', '%Y-%m-%d' %},{% now 'utc', '%H:%M:%S' %},devname={{ host }},log_id=30001000,msg_id=000377262759,device_id=FV-1111111800222,vd=\"root\",\"timezone=\"\"(GMT+3:00)Kuwait,Riyadh\"\"\",type=traffic,subtype=\"https\",pri=notice,proto=tcp,service=https/tls1.2,status=success,reason=none,policy=Phome_Policy,original_src=1.107.71.90,src=1.107.71.90,src_port=28799,dst=1.16.16.11,dst_port=80,http_request_time=0,http_response_time=0,http_request_bytes=623,http_response_bytes=15660,http_method=get,\"http_url=\"\"/publish/templates/images/bluebottom.jpg\"\"\",\"http_host=\"\"splunk.infigo.hr\"\"\",\"http_agent=\"\"Mozilla/5.0 (Linux; Android 9; SM-J415F) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.87 Mobile Safari/537.36\"\"\",http_retcode=200,\"msg=\"\"HTTPS get request from 1.107.71.90:28799 to 1.16.16.11:80\"\"\",original_srccountry=\"Saudi Arabia\",srccountry=\"Saudi Arabia\",content_switch_name=\"none\",server_pool_name=\"PHOME\",\"user_name=\"\"Unknown\"\"\",\"http_refer=\"\"https://splunk.infigo.hr/publish/templates/CSS/sc4s.css\"\"\",http_version=\"1.x\",dev_id=none,cipher_suite=\"TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384\"\n")
message = mt.render(mark="<13>", host=host)
sendsingle(message, setup_sc4s[0], setup_sc4s[1][514])

st = env.from_string("search index=netfw host=\"{{ host }}\" sourcetype=\"fwb_traffic\" | head 2")
search = st.render(host=host)

resultCount, eventCount = splunk_single(setup_splunk, search)

record_property("host", host)
record_property("resultCount", resultCount)
record_property("message", message)

assert resultCount == 1

#<111> 2020-02-12,23:16:41,devname=FortiWeb-A,log_id=20000008,msg_id=000377262743,device_id=FV-1111111800222,vd="root","timezone=""(GMT+3:00)Kuwait,Riyadh""",type=attack,pri=alert,main_type="Signature Detection",sub_type="Information Disclosure",trigger_policy="",severity_level=Low,proto=tcp,service=https/tls1.2,backend_service=https/tls1.2,action=Alert,policy="MobApp_policy",src=1.70.8.51,src_port=20894,dst=1.16.220.15,dst_port=443,http_method=post,"http_url=""/mfp/api/abc""","http_host=""splunk.infigo.hr""","http_agent=""WLNativeAPI(HWSTK-HF; STK-L21MDV 9.1.0.336(C185E3R2P1); STK-L21; SDK 28; Android 9)""",http_session_id=ASDSADSA,"msg=""HTTP Header triggered signature ID 080200004 of Signatures policy Alert Only""",signature_subclass="HTTP Header Leakage",signature_id="080200004",signature_cve_id="N/A",srccountry="Kuwait",content_switch_name="none",server_pool_name="MObApp_pool",false_positive_mitigation="none","user_name=""Unknown""",monitor_status="Enabled","http_refer=""none""",http_version="1.x",dev_id="none",es=1,threat_weight=5,history_threat_weight=0,threat_level=Low,ftp_mode="N/A",ftp_cmd="N/A",cipher_suite="TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256"ml_log_hmm_probability=0.000000,ml_log_sample_prob_mean=0.000000,ml_log_sample_arglen_mean=0.000000,ml_log_arglen=0,ml_svm_log_main_types=0,ml_svm_log_match_types="none",ml_svm_accuracy="none",ml_domain_index=0,ml_url_dbid=0,ml_arg_dbid=0,ml_allow_method="none",owasp_top10="A3:2017-Sensitive Data Exposure",bot_info="none",matched_field="header","matched_pattern=""X-Powered-By: Servlet/3.1"""
def test_fortinet_fwb_attack(record_property, setup_wordlist, setup_splunk, setup_sc4s):
host = "{}-{}".format(random.choice(setup_wordlist), random.choice(setup_wordlist))

mt = env.from_string(
"{{ mark }}{% now 'utc', '%Y-%m-%d' %},{% now 'utc', '%H:%M:%S' %},devname={{ host }},log_id=20000008,msg_id=000377262743,device_id=FV-1111111800222,vd=\"root\",\"timezone=\"\"(GMT+3:00)Kuwait,Riyadh\"\"\",type=attack,pri=alert,main_type=\"Signature Detection\",sub_type=\"Information Disclosure\",trigger_policy=\"\",severity_level=Low,proto=tcp,service=https/tls1.2,backend_service=https/tls1.2,action=Alert,policy=\"MobApp_policy\",src=1.70.8.51,src_port=20894,dst=1.16.220.15,dst_port=443,http_method=post,\"http_url=\"\"/mfp/api/abc\"\"\",\"http_host=\"\"splunk.infigo.hr\"\"\",\"http_agent=\"\"WLNativeAPI(HWSTK-HF; STK-L21MDV 9.1.0.336(C185E3R2P1); STK-L21; SDK 28; Android 9)\"\"\",http_session_id=ASDSADSA,\"msg=\"\"HTTP Header triggered signature ID 080200004 of Signatures policy Alert Only\"\"\",signature_subclass=\"HTTP Header Leakage\",signature_id=\"080200004\",signature_cve_id=\"N/A\",srccountry=\"Kuwait\",content_switch_name=\"none\",server_pool_name=\"MObApp_pool\",false_positive_mitigation=\"none\",\"user_name=\"\"Unknown\"\"\",monitor_status=\"Enabled\",\"http_refer=\"\"none\"\"\",http_version=\"1.x\",dev_id=\"none\",es=1,threat_weight=5,history_threat_weight=0,threat_level=Low,ftp_mode=\"N/A\",ftp_cmd=\"N/A\",cipher_suite=\"TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256\"ml_log_hmm_probability=0.000000,ml_log_sample_prob_mean=0.000000,ml_log_sample_arglen_mean=0.000000,ml_log_arglen=0,ml_svm_log_main_types=0,ml_svm_log_match_types=\"none\",ml_svm_accuracy=\"none\",ml_domain_index=0,ml_url_dbid=0,ml_arg_dbid=0,ml_allow_method=\"none\",owasp_top10=\"A3:2017-Sensitive Data Exposure\",bot_info=\"none\",matched_field=\"header\",\"matched_pattern=\"\"X-Powered-By: Servlet/3.1\"\"\"\n")
message = mt.render(mark="<13>", host=host)
sendsingle(message, setup_sc4s[0], setup_sc4s[1][514])

st = env.from_string("search index=netids host=\"{{ host }}\" sourcetype=\"fwb_attack\" | head 2")
search = st.render(host=host)

resultCount, eventCount = splunk_single(setup_splunk, search)

record_property("host", host)
record_property("resultCount", resultCount)
record_property("message", message)

assert resultCount == 1

0 comments on commit 802d853

Please sign in to comment.