Skip to content

Commit

Permalink
Merge pull request #325 from splunk/pr/323
Browse files Browse the repository at this point in the history
Support Fortigate Fortiweb product line
  • Loading branch information
Ryan Faircloth authored and GitHub committed Feb 21, 2020
2 parents 4f07d28 + 1ceb5a1 commit 7deb4f9
Show file tree
Hide file tree
Showing 11 changed files with 560 additions and 119 deletions.
10 changes: 9 additions & 1 deletion docs/gettingstarted/byoe-rhel7.md
Original file line number Diff line number Diff line change
Expand Up @@ -188,4 +188,12 @@ the data. In other cases, a unique listening port is required for certain devic
For collection of such sources we provide a means of dedicating a unique listening port to a specific source.

Refer to the "Sources" documentation to identify the specific environment variables used to enable unique listening ports for the technology
in use.
in use.

## Unique Ports for Device "Families"

Certain technology "families", such as CEF and Fortinet, are handled by a single log path in SC4S. To set unique ports for individual
devices in a family (e.g. one each for Fortiweb and FortiOS), the container version of SC4S uses "container networking" (detailed
in the source document for the respective device families). This, of course, is not avaialble in BYOE. For this reason, the syslog-ng source
configuration for the extra ports that need to be mapped will need to be added manually to either the template or final "conf" version of the
respective log path file.
152 changes: 142 additions & 10 deletions docs/sources/Fortinet/index.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,41 @@
# Vendor - Fortinet

There are two Fortinet device flavors (FortiOS and Fortiweb) that are supported by a single log path
in SC4S. Therefore, both Fortinet variants use "FORTINET" as the core of the unique port and
archive environment variable settings (rather than a unique one per product), as the Fortinet log path
handles either variant sending events to SC4S. Therefore, the FORTINET environment variables for unique
port, archive, etc. should be set only _once_, regardless of how many unique ports or Fortinet appliance
variants are in use.

If your deployment has multiple Fortinet devices that send to more than one port,
set the FORTINET unique port variable(s) to just one of the ports in use. Then, map the others with
container networking to the port chosen, similar to the way default ports are configured (see the
"Getting Started" runtime documents for more details).

Example: If you have three Fortinet devices, sending on TCP ports 2000,2001, and 2002, set
`SC4S_LISTEN_FORTINET_TCP_PORT=2000`. Then, change the unit/compose files to route the three external
ports to the single port 2000 on the container. Here is the example for podman/systemd:

```
ExecStart=/usr/bin/podman -p 514:514 -p 514:514/udp -p 6514:6514 -p 2000-2002:2000 \
```

or this, for docker-compose/swarm installations:

```
# Comment the following line out if using docker-compose
mode: host
- target: 2000
published: 2000-2002
protocol: tcp
```

These changes will route all three ports to TCP port 2000 inside the container, and the single Fortinet log
path will properly process data from all three devices.

The source documentation included below includes settings for both appliance types (FortiOS and Fortigate)
supported by SC4S.

## Product - Fortigate

| Ref | Link |
Expand All @@ -12,17 +48,17 @@

| sourcetype | notes |
|----------------|---------------------------------------------------------------------------------------------------------|
| fgt_log | The catch all sourcetype is not used |
| fgt_traffic | None |
| fgt_utm | None |
| fgt_event | None
| fgt_log | Catch-all sourcetype; not used by the TA |
| fgt_traffic | None |
| fgt_utm | None |
| fgt_event | None |


### Sourcetype and Index Configuration

| key | sourcetype | index | notes |
|----------------|----------------|----------------|----------------|
| fortinet_fortios_traffic | fgt_traffic | netops | none |
| fortinet_fortios_traffic | fgt_traffic | netfw | none |
| fortinet_fortios_utm | fgt_utm | netids | none |
| fortinet_fortios_event | fgt_event | netops | none |
| fortinet_fortios_log | fgt_log | netops | none |
Expand Down Expand Up @@ -73,12 +109,15 @@ end

### Options

* NOTE: Remember to set the variable(s) below only _once_, regardless of how many unique ports and/or Fortinet device types
are in use. See the introductory note above for more details.

| Variable | default | description |
|----------------|----------------|----------------|
| SC4S_LISTEN_FORTINET_FORTIOS_TCP_PORT | empty string | Enable a UDP port for this specific vendor product using the number defined |
| SC4S_LISTEN_FORTINET_FORTIOS_UDP_PORT | empty string | Enable a UDP port for this specific vendor product using the number defined |
| SC4S_ARCHIVE_FORTINET_FORTIOS | no | Enable archive to disk for this specific source |
| SC4S_DEST_FORTINET_FORTIOS_HEC | no | When Splunk HEC is disabled globally set to yes to enable this specific source |
| SC4S_LISTEN_FORTINET_TCP_PORT | empty string | Enable a UDP port for this specific vendor product using the number defined |
| SC4S_LISTEN_FORTINET_UDP_PORT | empty string | Enable a UDP port for this specific vendor product using the number defined |
| SC4S_ARCHIVE_FORTINET | no | Enable archive to disk for this specific source |
| SC4S_DEST_FORTINET_HEC | no | When Splunk HEC is disabled globally set to yes to enable this specific source |

### Verification

Expand All @@ -105,4 +144,97 @@ index=<asconfigured> (sourcetype=fgt_log OR sourcetype=fgt_traffic OR sourcetype
###Event Message Type
![FortiGate Event message](FortiGate_event.png)

Verify timestamp, and host values match as expected
Verify timestamp, and host values match as expected

## Product - FortiWeb

| Ref | Link |
|----------------|---------------------------------------------------------------------------------------------------------|
| Splunk Add-on | https://splunkbase.splunk.com/app/4679/ |
| Product Manual | https://docs.fortinet.com/product/fortiweb/6.3 |


### Sourcetypes

| sourcetype | notes |
|----------------|---------------------------------------------------------------------------------------------------------|
| fgt_log | Catch-all sourcetype; not used by the TA |
| fwb_traffic | None |
| fwb_attack | None |
| fwb_event | None |


### Sourcetype and Index Configuration

| key | sourcetype | index | notes |
|----------------|----------------|----------------|----------------|
| fortinet_fortiweb_traffic | fwb_traffic | netfw | none |
| fortinet_fortiweb_attack | fwb_attack | netids | none |
| fortinet_fortiweb_event | fwb_event | netops | none |
| fortinet_fortiweb_log | fwb_log | netops | none |


### Filter type

MSG Parse: This filter parses message content

### Setup and Configuration

* Install the Splunk Add-on on the search head(s) for the user communities interested in this data source. If SC4S is exclusively used the addon is not required on the indexer.
* Review and update the splunk_index.csv file and set the index and sourcetype as required for the data source.
* Refer to the admin manual for specific details of configuration to send Reliable syslog using RFC 3195 format, a typical logging configuration will include the following features.

```
config log syslog-policy
edit splunk
config syslog-server-list
edit 1
set server x.x.x.x
set port 514 (Example. Should be the same as default or dedicated port selected for sc4s)
end
end
config log syslogd
set policy splunk
set status enable
end
```

### Options

* NOTE: Remember to set the variable(s) below only _once_, regardless of how many unique ports and/or Fortinet device types
are in use. See the introductory note above for more details.

| Variable | default | description |
|----------------|----------------|----------------|
| SC4S_LISTEN_FORTINET_TCP_PORT | empty string | Enable a UDP port for this specific vendor product using the number defined |
| SC4S_LISTEN_FORTINET_UDP_PORT | empty string | Enable a UDP port for this specific vendor product using the number defined |
| SC4S_ARCHIVE_FORTINET | no | Enable archive to disk for this specific source |
| SC4S_DEST_FORTINET_HEC | no | When Splunk HEC is disabled globally set to yes to enable this specific source |

### Verification

An active firewall will generate frequent events, in addition fortigate has the ability to test logging functionality using a built in command

```
diag log test
```

Verify timestamp, and host values match as expected

```
index=<asconfigured> (sourcetype=fwb_log OR sourcetype=fwb_traffic OR sourcetype=fwb_attack OR sourcetype=fwb_event)
```

Verify timestamp, and host values match as expected
4 changes: 2 additions & 2 deletions package/etc/conf.d/filters/cisco/meraki.conf
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ parser p_cisco_meraki {
);
};
parser {
date-parser(format('%s')
template("${EPOCH}")
date-parser(format('%s.%f')
template("${EPOCH}.${TIMESECFRAC}")
flags(guess-timezone)
);
};
Expand Down
8 changes: 8 additions & 0 deletions package/etc/conf.d/filters/fortinet/fortinet.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
filter f_fortinet {
message('devid=\"?F[G|W|6K].+type=\"?(traffic|utm|event)') or
message('device_id=\"?FV.+type=\"?(traffic|attack|event)');
};

filter f_fortinet_fortiweb {
message('device_id=\"?FV.+type=\"?(traffic|attack|event)');
};
3 changes: 0 additions & 3 deletions package/etc/conf.d/filters/fortinet/fortios.conf

This file was deleted.

118 changes: 118 additions & 0 deletions package/etc/conf.d/log_paths/lp-fortinet.conf.tmpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
# Fortinet Fortigate and Fortiweb
{{- /* The following provides a unique port source configuration if env var(s) are set */}}
{{- $context := dict "port_id" "FORTINET" "parser" "rfc3164" }}
{{- tmpl.Exec "t/source_network.t" $context }}

log {
junction {
{{- if or (or (getenv (print "SC4S_LISTEN_FORTINET_TCP_PORT")) (getenv (print "SC4S_LISTEN_FORTINET_UDP_PORT"))) (getenv (print "SC4S_LISTEN_FORTINET_TLS_PORT")) }}
channel {
# Listen on the specified dedicated port(s) for FORTINET traffic
source (s_FORTINET);
flags (final);
};
{{- end}}
channel {
# Listen on the default port (typically 514) for FORTINET traffic
source (s_DEFAULT);
filter(f_is_rfc3164);
filter(f_fortinet);
flags(final);
};
};

parser {
kv-parser(prefix(".kv.") template("${LEGACY_MSGHDR} ${MSG}"));
};
if {
filter(f_fortinet_fortiweb);
# Fetch timezone from timezone nv pair and parse unique format (no zero padding, e.g. "-8:00" rather than "-08:00"
# Reformat to "-08:00"
rewrite {
subst('.*([\+-]\d+:\d+).*', $1, value(".kv.timezone"));
subst('([\+-])(\d)(?=:)(:\d+)', "${1}0${2}${3}", value(".kv.timezone"));
};
parser {
date-parser(
format("%Y-%m-%d:%H:%M:%S%z")
template('${.kv.date}:${.kv.time}${.kv.timezone}')
flags(guess-timezone)
);
};
} elif {
filter { match('.{5}' value (".kv.tz")) };
parser {
date-parser(
format("%Y-%m-%d:%H:%M:%S%z")
template("${.kv.date}:${.kv.time}${.kv.tz}")
flags(guess-timezone)
);
};
} elif {
parser {
date-parser(
format("%Y-%m-%d:%H:%M:%S")
template("${.kv.date}:${.kv.time}")
time-zone({{- getenv "SC4S_DEFAULT_TIMEZONE" "GMT"}})
flags(guess-timezone)
);
};
} else {
rewrite { set("date/time parser failed", value("fields.sc4s_error")); };
};

# Fortiweb
if {
filter(f_fortinet_fortiweb);
rewrite {
set("fortigate_fortiweb", value("fields.sc4s_vendor_product"));
set("${.kv.devname}", value("HOST"));
};
if (match("traffic" value(".kv.type"))) {
rewrite { r_set_splunk_dest_default(sourcetype("fwb_traffic"), index("netfw"))};
parser {p_add_context_splunk(key("fortinet_fortiweb_traffic")); };
} elif (match("attack" value(".kv.type"))) {
rewrite { r_set_splunk_dest_default(sourcetype("fwb_attack"), index("netids"))};
parser {p_add_context_splunk(key("fortinet_fortiweb_attack")); };
} elif (match("event" value(".kv.type"))) {
rewrite { r_set_splunk_dest_default(sourcetype("fwb_event"), index("netops"))};
parser {p_add_context_splunk(key("fortinet_fortiweb_event")); };
} else {
rewrite { r_set_splunk_dest_default(sourcetype("fwb_log"), index("netops"))};
parser {p_add_context_splunk(key("fortinet_fortiweb_log")); };
};
#FortiOS
} else {
rewrite {
set("fortigate_fortios", value("fields.sc4s_vendor_product"));
set("${.kv.devname}", value("HOST"));
};
if (match("traffic" value(".kv.type"))) {
rewrite { r_set_splunk_dest_default(sourcetype("fgt_traffic"), index("netfw"))};
parser {p_add_context_splunk(key("fortinet_fortios_traffic")); };
} elif (match("utm" value(".kv.type"))) {
rewrite { r_set_splunk_dest_default(sourcetype("fgt_utm"), index("netids"))};
parser {p_add_context_splunk(key("fortinet_fortios_utm")); };
} elif (match("event" value(".kv.type"))) {
rewrite { r_set_splunk_dest_default(sourcetype("fgt_event"), index("netops"))};
parser {p_add_context_splunk(key("fortinet_fortios_event")); };
} else {
rewrite { r_set_splunk_dest_default(sourcetype("fgt_log"), index("netops"))};
parser {p_add_context_splunk(key("fortinet_fortios_log")); };
};
};

parser (compliance_meta_by_source);
rewrite { set("$(template ${.splunk.sc4s_template} $(template t_hdr_msg))" value("MSG")); };

{{- if or (conv.ToBool (getenv "SC4S_DEST_SPLUNK_HEC_GLOBAL" "yes")) (conv.ToBool (getenv "SC4S_DEST_FORTINET_HEC" "no")) }}
destination(d_hec);
{{- end}}


{{- if or (conv.ToBool (getenv "SC4S_ARCHIVE_GLOBAL" "no")) (conv.ToBool (getenv "SC4S_ARCHIVE_FORTINET" "no")) }}
destination(d_archive);
{{- end}}

flags(flow-control,final);
};
Loading

0 comments on commit 7deb4f9

Please sign in to comment.