Skip to content

Commit

Permalink
Develop (#36)
Browse files Browse the repository at this point in the history
* test using compose

* Update config.yml

* Update config.yml

* Update config.yml

* Update config.yml

* Update config.yml

* Update config.yml

* suppress docker up failures

* don't cleanup volumes

* Update config.yml

* Update config.yml

* handle other containers not yet ready

* Update test_poc.py

* Update test_poc.py

* Update test_poc.py

* Update test_poc.py

* env shift

* Code Cleanup

* add flake 8

* Update config.yml

* Update config.yml

* Update config.yml

* Update config.yml

* Update config.yml

* Update config.yml

* remove flake8 will try again later

* Cleanup local system Fixes #1 (#5)

* Feature/#6 add clair scanner to ci (#7)

Fixes #6 adds vuln scanner and current approved vulns to CI

* demo experience

* Adds copyright and license information (#10)

This update Fixes #9 adding copyright and license information

* Feature/baseconfig (#13)

This change implements the configuration framework allowing plug of filter modules for the primary deployment type
Fixes #12 
Fixes #11 
This change adds a filter for Palo Alto networks devices

* Support Cisco ASA and misc fixes (#17)

* Support Cisco ASA and misc fixes

Fixes #14
Fixes #15
Fixes #16

* fix typo in rfc number

* Feature/filter cisco ios (#21)

* Support Cisco ASA and misc fixes

Fixes #14
Fixes #15
Fixes #16

* Test Refactor

Fixes #18
Fixes #19

* New filter cisco IOS

Fixes #20

* Update .env.template

* Feature/add fortinet filter (#24)

* Add filter with tests for Fortinet Fortigate

* CI CD CLEANUP

* Feature/gitignore (#26)

* Add filter with tests for Fortinet Fortigate

Fixes #23

* Update fortinet_fgt.conf

remove extra line

* Update config.yml

ci fix

* Update config.yml

update

* fix typo in build and update ci process

* Update docker-compose.yml

fix  app install typo

* Update conftest.py

extend time waiting for splunk start

* some tests not marked flaky

* Update .gitignore

Fixes #25

* Update filter based on wire samples (#31)

* Add filter with tests for Fortinet Fortigate

* Contributing and owners (#33)

repo only change

* Feature/perftest (#35)

This merge will restore broken metrics functionality
Define metrics using the a hierarchy syslogng.${SourceName} emulating Splunk Connect for Kafka
Configure persistant queuing
Define a new dest for metrics
Remove early metrics collection scripts

* Update syslog-ng.conf

* Update splunk_index.csv
  • Loading branch information
Ryan Faircloth authored and GitHub committed Aug 6, 2019
1 parent 82cccc7 commit 19b805c
Show file tree
Hide file tree
Showing 31 changed files with 612 additions and 87 deletions.
48 changes: 40 additions & 8 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,13 +28,48 @@ jobs:
command: docker login -u $DOCKER_USER -p $DOCKER_PASS
- run:
name: Build Docker image
command: pushd package; docker build -f Dockerfile --build-arg RH_ORG=$RH_ORG --build-arg RH_ACTIVATION=$RH_ACTIVATION -t $IMAGE_NAME:build .
command: docker build -f package/Dockerfile --build-arg RH_ORG=$RH_ORG --build-arg RH_ACTIVATION=$RH_ACTIVATION -t $IMAGE_NAME:build package
- run:
name: Tag Docker image
command: docker tag $IMAGE_NAME:build $IMAGE_NAME:$CIRCLE_SHA1
- run:
name: Tag Docker image
command: docker tag $IMAGE_NAME:build $IMAGE_NAME:edge
- run:
name: Push Docker image
command: docker push $IMAGE_NAME:$CIRCLE_SHA1
- run:
name: Push Docker image
command: docker push $IMAGE_NAME:edge
build-egb:
environment:
IMAGE_NAME: rfaircloth/scs
docker:
- image: circleci/buildpack-deps:stretch
steps:
- checkout
- run: git submodule sync
- run: git submodule update --init --recursive
- setup_remote_docker:
docker_layer_caching: true
- run:
name: Docker Login
command: docker login -u $DOCKER_USER -p $DOCKER_PASS
- run:
name: Build Docker image
command: docker build -f perftests/bundlesrv/Dockerfile -t $IMAGE_NAME:egb-build perftests/bundlesrv
- run:
name: Tag Docker image
command: docker tag $IMAGE_NAME:egb-build $IMAGE_NAME:egb-$CIRCLE_SHA1
- run:
name: Tag Docker image
command: docker tag $IMAGE_NAME:egb-build $IMAGE_NAME:egb-edge
- run:
name: Push Docker image
command: docker push $IMAGE_NAME:egb-$CIRCLE_SHA1
- run:
name: Push Docker image
command: docker push $IMAGE_NAME:egb-edge
dgoss:
environment:
IMAGE_NAME: rfaircloth/scs
Expand Down Expand Up @@ -103,9 +138,7 @@ jobs:
- run:
name: Docker Compose build
command: |
docker-compose build \
--build-arg RH_ACTIVATION=$RH_ACTIVATION \
--build-arg RH_ORG=$RH_ORG
docker-compose build
- run:
name: Docker Compose up
command: docker-compose up --abort-on-container-exit
Expand Down Expand Up @@ -136,10 +169,8 @@ jobs:
pip install -r requirements.txt
python clair_to_junit_parser.py "/clair-reports/$IMAGE_NAME:$CIRCLE_SHA1.json" --output test-results/results.xml
when: on_fail
# - store_test_results:
# path: test-results/results.xml
- store_artifacts:
path: test-results
- store_test_results:
path: test-results/results.xml
- store_artifacts:
path: /clair-reports

Expand Down Expand Up @@ -168,6 +199,7 @@ workflows:
build-publish:
jobs:
- build
- build-egb
- dgoss:
requires:
- build
Expand Down
2 changes: 1 addition & 1 deletion .env.template
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,6 @@ SPLUNK_HEC_STATSURL=https://splunk:8088/services/collector/event
SPLUNK_CONNECT_METHOD=hec
SPLUNK_DEFAULT_INDEX=main
SPLUNK_METRICS_INDEX=metrics
SPLUNK_APPS_URL=https://splunkbase.splunk.com/app/2757/release/6.1.1/download,https://splunkbase.splunk.com/app/3245/release/1.0/download,https://splunkbase.splunk.com/app/1620/release/3.4.0/download,https://splunkbase.splunk.com/app/1467/release/2.5.8/download
SPLUNK_APPS_URL=https://splunkbase.splunk.com/app/2757/release/6.1.1/download,https://splunkbase.splunk.com/app/3245/release/1.0/download,https://splunkbase.splunk.com/app/1620/release/3.4.0/download,https://splunkbase.splunk.com/app/1467/release/2.5.8/download,https://splunkbase.splunk.com/app/2846/release/1.6.0/download
SPLUNKBASE_USERNAME=username
SPLUNKBASE_PASSWORD=password
2 changes: 2 additions & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
* @rfaircloth-splunk
package/etc/ @rfaircloth @mbonsack
77 changes: 77 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -303,9 +303,86 @@ tags
# Persistent undo
[._]*.un~

<<<<<<< HEAD
# Covers JetBrains IDEs: IntelliJ, RubyMine, PhpStorm, AppCode, PyCharm, CLion, Android Studio and WebStorm
# Reference: https://intellij-support.jetbrains.com/hc/en-us/articles/206544839

# User-specific stuff
.idea/**/workspace.xml
.idea/**/tasks.xml
.idea/**/usage.statistics.xml
.idea/**/dictionaries
.idea/**/shelf

# Generated files
.idea/**/contentModel.xml

# Sensitive or high-churn files
.idea/**/dataSources/
.idea/**/dataSources.ids
.idea/**/dataSources.local.xml
.idea/**/sqlDataSources.xml
.idea/**/dynamic.xml
.idea/**/uiDesigner.xml
.idea/**/dbnavigator.xml

# Gradle
.idea/**/gradle.xml
.idea/**/libraries

# Gradle and Maven with auto-import
# When using Gradle or Maven with auto-import, you should exclude module files,
# since they will be recreated, and may cause churn. Uncomment if using
# auto-import.
# .idea/modules.xml
# .idea/*.iml
# .idea/modules
# *.iml
# *.ipr

# CMake
cmake-build-*/

# Mongo Explorer plugin
.idea/**/mongoSettings.xml

# File-based project format
*.iws

# IntelliJ
out/

# mpeltonen/sbt-idea plugin
.idea_modules/

# JIRA plugin
atlassian-ide-plugin.xml

# Cursive Clojure plugin
.idea/replstate.xml

# Crashlytics plugin (for Android Studio and IntelliJ)
com_crashlytics_export_strings.xml
crashlytics.properties
crashlytics-build.properties
fabric.properties

# Editor-based Rest Client
.idea/httpRequests

# Android studio 3.1+ serialized cache file
.idea/caches/build_file_checksums.ser

/test-results/
!/package/scripts/switch_transport.sh
!/package/scripts/splunkmetrics.sh
!/package/scripts/
.ecs
=======
/test-results/
/.idea/workspace.xml
/.idea/tasks.xml
!/package/scripts/switch_transport.sh
!/package/scripts/splunkmetrics.sh
!/package/scripts/
>>>>>>> master
10 changes: 10 additions & 0 deletions .idea/vcs.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

100 changes: 100 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
# Contributing

### Past / Active(marked as *) Contributors
rfaircloth*
mbonsack*
rfipro*
Hurricane Labs*


## Code of conduct

###Contributor License Agreement

At the moment, we can only accept pull requests submitted from either:
* Splunk employees or
* Individuals that have signed our contribution agreement

If you wish to be a contributing member of our community, please see the agreement [for individuals](https://www.splunk.com/goto/individualcontributions) or [for organizations](https://www.splunk.com/goto/contributions).

### Our Pledge

In the interest of fostering an open and welcoming environment, we as
contributors and maintainers pledge to making participation in our project and
our community a harassment-free experience for everyone, regardless of age, body
size, disability, ethnicity, gender identity and expression, level of experience,
nationality, personal appearance, race, religion, or sexual identity and
orientation.

### Our Standards

Examples of behavior that contributes to creating a positive environment
include:

* Using welcoming and inclusive language
* Being respectful of differing viewpoints and experiences
* Gracefully accepting constructive criticism
* Focusing on what is best for the community
* Showing empathy towards other community members

Examples of unacceptable behavior by participants include:

* The use of sexualized language or imagery and unwelcome sexual attention or
advances
* Trolling, insulting/derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or electronic
address, without explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting


### Our Responsibilities

Project maintainers are responsible for clarifying the standards of acceptable
behavior and are expected to take appropriate and fair corrective action in
response to any instances of unacceptable behavior.

Project maintainers have the right and responsibility to remove, edit, or
reject comments, commits, code, wiki edits, issues, and other contributions
that are not aligned to this Code of Conduct, or to ban temporarily or
permanently any contributor for other behaviors that they deem inappropriate,
threatening, offensive, or harmful.

### Scope

This Code of Conduct applies both within project spaces and in public spaces
when an individual is representing the project or its community. Examples of
representing a project or community include using an official project e-mail
address, posting via an official social media account, or acting as an appointed
representative at an online or offline event. Representation of a project may be
further defined and clarified by project maintainers.

### Enforcement

Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported by contacting the project team at tonyl@splunk.com. All
complaints will be reviewed and investigated and will result in a response that
is deemed necessary and appropriate to the circumstances. The project team is
obligated to maintain confidentiality with regard to the reporter of an incident.
Further details of specific enforcement policies may be posted separately.

Project maintainers who do not follow or enforce the Code of Conduct in good
faith may face temporary or permanent repercussions as determined by other
members of the project's leadership.

### Attribution

This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
available at [http://contributor-covenant.org/version/1/4][version]

[homepage]: http://contributor-covenant.org
[version]: http://contributor-covenant.org/version/1/4/

## Filing issues

Use project issue tracker

## Contributing code

Future Docs
77 changes: 77 additions & 0 deletions docker-compose-perf.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
#Splunk Connect for Syslog (SC4S) by Splunk, Inc.
#
#To the extent possible under law, the person who associated CC0 with
#Splunk Connect for Syslog (SC4S) has waived all copyright and related or neighboring rights
#to Splunk Connect for Syslog (SC4S).
#
#You should have received a copy of the CC0 legalcode along with this
#work. If not, see <http://creativecommons.org/publicdomain/zero/1.0/>.
version: "3"
services:
sc4s:
image: rfaircloth/scs:edge
build:
context: ./package
args:
RH_ORG: ${RH_ORG}
RH_ACTIVATION: ${RH_ACTIVATION}
hostname: sc4s
ports:
- "514"
- "601"
- "514/udp"
- "5514"
- "5514/udp"
links:
- splunk
environment:
- SPLUNK_HEC_URL=${SPLUNK_HEC_URL}
- SPLUNK_HEC_STATSURL=${SPLUNK_HEC_STATSURL}
- SPLUNK_HEC_TOKEN=${SPLUNK_HEC_TOKEN}
- SPLUNK_CONNECT_METHOD=${SPLUNK_CONNECT_METHOD}
- SPLUNK_DEFAULT_INDEX=${SPLUNK_DEFAULT_INDEX}
- SPLUNK_METRICS_INDEX=${SPLUNK_DEFAULT_INDEX}
# logging:
# driver: splunk
# options:
# splunk-token: a778f63a-5dff-4e3c-a72c-a03183659e94
# splunk-url: https://splunk:8088/services/collector/event
# splunk-index: main
# splunk-insecureskipverify: true
# splunk-verify-connection: false
splunk:
image: splunk/splunk:latest
hostname: splunk
ports:
- "8000:8000"
- "8088:8088"
- "8089:8089"
environment:
- SPLUNK_HEC_TOKEN=${SPLUNK_HEC_TOKEN}
- SPLUNK_PASSWORD=${SPLUNK_PASSWORD}
- SPLUNK_START_ARGS=${SPLUNK_START_ARGS}
- SPLUNK_APPS_URL=${SPLUNK_APPS_URL}
- SPLUNKBASE_USERNAME=${SPLUNKBASE_USERNAME}
- SPLUNKBASE_PASSWORD=${SPLUNKBASE_PASSWORD}
# logging:
# driver: splunk
# options:
# splunk-token: a778f63a-5dff-4e3c-a72c-a03183659e94
# splunk-url: https://splunk:8088/services/collector/event
# splunk-index: main
# splunk-insecureskipverify: true
# splunk-verify-connection: false
egbundles:
image: rfaircloth/scs:egb-edge
hostname: egbundles
build:
context: perftests/bundlesrv
eventgen:
image: rfaircloth/eventgen:edge
command: standalone
ports:
- "9500:9500"
links:
- egbundles
- sc4s

Loading

0 comments on commit 19b805c

Please sign in to comment.