Splunk Eventgen Output Queue Full Looping Again

Splunk Event Generator (Eventgen)

Status

Introduction

Splunk Event Generator is a utility that helps users easily build existent-fourth dimension event generators. The current maintainers of this project are Brian Bingham ( [email protected] ), Tony Lee ( [email protected] ), and Jack Meixensperger ( [email protected] ).

The goals of this project:

• Eliminate the demand for hand-coded event generators in Splunk apps
• Let for portability of event generators between applications and allow templates to be speedily adjusted betwixt use cases
• Let every type of events or transactions to exist modeled within Eventgen

Downloading a Splunk Eventgen App

Please go to splunkbase-Eventgen

Documentation

Documentation is hosted at Eventgen Documentation .

Contributing

Please note CONTRIBUTING.physician .

License

Splunk Upshot Generator is licensed under the Apache License 2.0. Details can be found in the LICENSE file.

Support

This software is released every bit-is. Splunk provides no warranty and no back up on this software. If you have any issues with the software, please read over the guidelines and file an result.

Pull Requests - Problems - Releases

Pull Requests

Crash-land lxml from 4.5.2 to 4.6.5

Created 13 Dec, 2021 Pull Request #436 User Dependabot

Bumps lxml from 4.5.ii to iv.half dozen.v.

Changelog

Sourced from lxml's changelog.

4.half dozen.5 (2021-12-12)

Bugs stock-still

• A vulnerability (GHSL-2021-1038) in the HTML cleaner allowed sneaking script content through SVG images.

• A vulnerability (GHSL-2021-1037) in the HTML cleaner immune sneaking script content through CSS imports and other crafted constructs.

four.6.4 (2021-eleven-01)

Features added

• GH#317: A new property system_url was added to DTD entities. Patch by Thirdegree.

• GH#314: The STATIC_* variables in setup.py can at present be passed via env vars. Patch past Isaac Jurado.

4.6.3 (2021-03-21)

Bugs stock-still

• A vulnerability (CVE-2021-28957) was discovered in the HTML Cleaner by Kevin Chung, which allowed JavaScript to pass through. The cleaner now removes the HTML5 formaction attribute.

4.six.2 (2020-11-26)

Bugs fixed

• A vulnerability (CVE-2020-27783) was discovered in the HTML Cleaner by Yaniv Nizry, which allowed JavaScript to pass through. The cleaner now removes more sneaky "fashion" content.

4.6.1 (2020-10-18)

... (truncated)

Commits

• a9611ba Fix a test in Py2.
• a3eacbc Fix release of iv.vi.v.
• b7ea687 Update changelog.
• 69a7473 Cleaner: cover some more cases where scripts could sneak through in peculiarly...
• 54d2985 Fix status in test decorator.
• 4b220b5 Utilise the non-depcrecated TextTestResult instead of _TextTestResult (GH-333)
• d85c6de Exclude a test when using the macOS arrangement libraries because it fails with li...
• cd4bec9 Add macOS-M1 as cycle build platform.
• fd0d471 Install automake and libtool in macOS build to be able to install the latest ...
• f233023 Cleaner: Remove SVG paradigm data URLs since they can embed script content.
• Additional commits viewable in compare view

Dependabot will resolve whatsoever conflicts with this PR as long as you don't change information technology yourself. You tin too trigger a rebase manually past commenting @dependabot rebase.

---

Dependabot commands and options

Y'all tin can trigger Dependabot actions past commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting whatever edits that take been made to it
• @dependabot merge will merge this PR after your CI passes on it
• @dependabot squash and merge will squash and merge this PR subsequently your CI passes on it
• @dependabot abolish merge will abolish a previously requested merge and block automerging
• @dependabot reopen will reopen this PR if it is closed
• @dependabot close volition close this PR and cease Dependabot recreating it. Y'all tin achieve the same result past closing it manually
• @dependabot ignore this major version volition close this PR and end Dependabot creating whatsoever more than for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this pocket-sized version volition shut this PR and stop Dependabot creating whatever more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating whatsoever more for this dependency (unless you lot reopen the PR or upgrade to information technology yourself)
• @dependabot use these labels volition set up the current labels as the default for future PRs for this repo and language
• @dependabot utilize these reviewers will set the current reviewers every bit the default for future PRs for this repo and language
• @dependabot utilize these assignees will set the current assignees as the default for future PRs for this repo and linguistic communication
• @dependabot use this milestone will set the current milestone every bit the default for futurity PRs for this repo and language

Yous can disable automated security fix PRs for this repo from the Security Alerts page.

Bump nokogiri from 1.11.0 to 1.12.5 in /docs

Created 27 Sep, 2021 Pull Asking #435 User Dependabot

Bumps nokogiri from 1.xi.0 to 1.12.v.

Release notes

Sourced from nokogiri's releases.

1.12.5 / 2021-09-27

Security

[JRuby] Address CVE-2021-41098 (GHSA-2rr5-8q37-2w7h).

In Nokogiri v1.12.iv and earlier, on JRuby only, the SAX parsers resolve external entities (XXE) by default. This gear up turns off entity-resolution-by-default in the JRuby SAX parsers to match the CRuby SAX parsers' behavior.

CRuby users are not afflicted by this CVE.

Fixed

• [CRuby] Document#to_xhtml properly serializes cocky-endmost tags in libxml > 2.9.10. A beliefs change introduced in libxml 2.9.11 resulted in emitting start and and tags (e.g., <br></br>) instead of a self-closing tag (e.g., <br/>) in previous Nokogiri versions. [#2324]

---

SHA256 checksums:

                          36bfa3a07aced069b3f3c9b39d9fb62cb0728d284d02b079404cd55780beaeff  nokogiri-one.12.5-arm64-darwin.gem 16b1a9ddbb70a9c998462912a5972097cbc79c3e01eb373906886ef8a469f589  nokogiri-one.12.5-java.jewel 218dcc6edd1b49cc6244b5f88afb978739bb2f3f166c271557fe5f51e4bc713c  nokogiri-1.12.v-x64-mingw32.gem e33bb919d64c16d931a5f26dc880969e587d225cfa97e6b56e790fb52179f527  nokogiri-i.12.five-x86-linux.gem e13c2ed011b8346fbd589e96fe3542d763158bc2c7ad0f4f55f6d801afd1d9ff  nokogiri-1.12.5-x86-mingw32.gem 1ed64f7db7c1414b87fce28029f2a10128611d2037e0871ba298d00f9a00edd6  nokogiri-1.12.v-x86_64-darwin.gem 0868c8d0a147904d4dedaaa05af5f06656f2d3c67e4432601718559bf69d6cea  nokogiri-i.12.v-x86_64-linux.gem 2b20905942acc580697c8c496d0d1672ab617facb9d30d156b3c7676e67902ec  nokogiri-ane.12.5.gem                                                  

one.12.iv / 2021-08-29

Notable fix: Namespace inheritance

Namespace behavior when reparenting nodes has historically been poorly specified and the behavior diverged between CRuby and JRuby. As a outcome, making this behavior consistent in v1.12.0 introduced a breaking change.

This patch release reverts the Builder behavior present in v1.12.0..v1.12.3 merely keeps the Document behavior. This release also introduces a Document attribute to allow affected users to easily change this behavior for their legacy lawmaking without invasive changes.

Compensating Feature in XML::Document

This release of Nokogiri introduces a new Document boolean attribute, namespace_inheritance, which controls whether children should inherit a namespace when they are reparented. Nokogiri::XML:Certificate defaults this attribute to false pregnant "do not inherit," thereby making explicit the behavior change introduced in v1.12.0.

CRuby users who desire the pre-v1.12.0 beliefs may ready document.namespace_inheritance = true before reparenting nodes.

See https://nokogiri.org/rdoc/Nokogiri/XML/Document.html#namespace_inheritance-instance_method for example usage.

Fix for XML::Builder

... (truncated)

Changelog

Sourced from nokogiri'south changelog.

1.12.five / 2021-09-27

Security

[JRuby] Address CVE-2021-41098 (GHSA-2rr5-8q37-2w7h).

In Nokogiri v1.12.iv and earlier, on JRuby only, the SAX parsers resolve external entities (XXE) by default. This prepare turns off entity-resolution-past-default in the JRuby SAX parsers to friction match the CRuby SAX parsers' behavior.

CRuby users are not afflicted by this CVE.

Fixed

• [CRuby] Document#to_xhtml properly serializes cocky-closing tags in libxml > two.9.ten. A behavior change introduced in libxml two.9.eleven resulted in emitting start and and tags (due east.g., <br></br>) instead of a self-endmost tag (due east.g., <br/>) in previous Nokogiri versions. [#2324]

ane.12.4 / 2021-08-29

Notable gear up: Namespace inheritance

Namespace behavior when reparenting nodes has historically been poorly specified and the behavior diverged between CRuby and JRuby. Every bit a result, making this beliefs consequent in v1.12.0 introduced a breaking change.

This patch release reverts the Builder behavior present in v1.12.0..v1.12.iii but keeps the Document behavior. This release too introduces a Certificate aspect to allow afflicted users to easily alter this behavior for their legacy lawmaking without invasive changes.

Compensating Feature in XML::Document

This release of Nokogiri introduces a new Document boolean aspect, namespace_inheritance, which controls whether children should inherit a namespace when they are reparented. Nokogiri::XML:Document defaults this attribute to false meaning "exercise not inherit," thereby making explicit the beliefs modify introduced in v1.12.0.

CRuby users who desire the pre-v1.12.0 behavior may set certificate.namespace_inheritance = true before reparenting nodes.

See https://nokogiri.org/rdoc/Nokogiri/XML/Certificate.html#namespace_inheritance-instance_method for case usage.

Fix for XML::Architect

However, recognizing that we want Builder-created children to inherit namespaces, Builder now will set namespace_inheritance=true on the underlying document for both JRuby and CRuby. This means that, on CRuby, the pre-v1.12.0 behavior is restored.

Users who desire to turn this behavior off may pass a keyword argument to the Builder constructor like so:

                          Nokogiri::XML::Builder                          .                          new                          (                          namespace_inheritance:                          false                          )                        

Meet https://nokogiri.org/rdoc/Nokogiri/XML/Builder.html#label-Namespace+inheritance for instance usage.

Downstream gem maintainers

Note that whatever downstream gems may want to specifically omit Nokogiri v1.12.0--v1.12.iii from their dependency specification if they rely on child namespace inheritance:

... (truncated)

Commits

• 47f6a46 version bump to v1.12.five
• 2a0ac88 update CHANGELOG
• 6b60637 Merge pull request #2329 from sparklemotion/flavorjones-GHSA-2rr5-8q37-2w7h_1...
• 4bd943c fix(jruby): SAX parser uses an entity resolver
• f943ee4 refactor(jruby): handle errors more than consistently
• 2790122 format: examination files
• 01e1618 Merge pull request #2327 from sparklemotion/2324-xhtml-self-closing-tags_v1.12.x
• a0180c7 fix: HTML4::Certificate.to_xhtml self-closing tags
• 564ac17 release v1.12.iv
• 4d5754b backport #2320
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR every bit long as you lot don't alter it yourself. You can also trigger a rebase manually past commenting @dependabot rebase.

---

Dependabot commands and options

Y'all tin trigger Dependabot deportment by commenting on this PR:

• @dependabot rebase volition rebase this PR
• @dependabot recreate volition recreate this PR, overwriting any edits that have been made to it
• @dependabot merge volition merge this PR after your CI passes on information technology
• @dependabot squash and merge volition squash and merge this PR after your CI passes on it
• @dependabot cancel merge will cancel a previously requested merge and block automerging
• @dependabot reopen will reopen this PR if it is closed
• @dependabot close will close this PR and cease Dependabot recreating it. You lot tin can achieve the same result by closing information technology manually
• @dependabot ignore this major version will close this PR and cease Dependabot creating any more for this major version (unless y'all reopen the PR or upgrade to it yourself)
• @dependabot ignore this small-scale version will shut this PR and end Dependabot creating whatever more for this minor version (unless you reopen the PR or upgrade to information technology yourself)
• @dependabot ignore this dependency will close this PR and cease Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to information technology yourself)
• @dependabot use these labels volition set up the current labels equally the default for future PRs for this repo and linguistic communication
• @dependabot use these reviewers will set the current reviewers equally the default for future PRs for this repo and language
• @dependabot employ these assignees volition set up the current assignees every bit the default for future PRs for this repo and language
• @dependabot use this milestone will gear up the electric current milestone as the default for hereafter PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

Bump addressable from two.seven.0 to 2.8.0 in /docs

Created 13 Jul, 2021 Pull Request #434 User Dependabot

Bumps addressable from 2.7.0 to 2.8.0.

Changelog

Sourced from addressable'south changelog.

Addressable 2.8.0

• fixes ReDoS vulnerability in Addressable::Template#match
• no longer replaces + with spaces in queries for non-http(s) schemes
• fixed encoding ipv6 literals
• the :compacted flag for normalized_query now dedupes parameters
• fix broken escape_component allonym
• dropping support for Ruby-red two.0 and 2.1
• adding Scarlet three.0 compatibility for development tasks
• driblet back up for rack-mountain and remove Addressable::Template#generate
• performance improvements
• switch CI/CD to GitHub Actions

Commits

• 6469a23 Updating gemspec again
• 2433638 Merge branch 'primary' of github.com:sporkmonger/addressable into chief
• e9c76b8 Merge pull request #378 from ashmaroli/flat-map
• 56c5cf7 Update the gemspec
• c1fed1c Crave a not-vulnerable rake
• 0d8a312 Adding note about ReDoS vulnerability
• 89c7613 Merge branch 'template-regexp' into primary
• cf8884f Annotation about alias fix
• bb03f71 Merge pull request #371 from charleystran/add_missing_encode_component_doc_entry
• 6d1d809 Adding note virtually :compacted normalization
• Boosted commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. Y'all can likewise trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

Y'all tin trigger Dependabot actions by commenting on this PR:

• @dependabot rebase volition rebase this PR
• @dependabot recreate volition recreate this PR, overwriting whatsoever edits that take been made to information technology
• @dependabot merge will merge this PR after your CI passes on information technology
• @dependabot squash and merge will squash and merge this PR afterward your CI passes on information technology
• @dependabot cancel merge will abolish a previously requested merge and block automerging
• @dependabot reopen will reopen this PR if information technology is closed
• @dependabot close will close this PR and stop Dependabot recreating it. Y'all can attain the same effect by closing it manually
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this small-scale version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will shut this PR and terminate Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
• @dependabot employ these labels will set up the electric current labels every bit the default for future PRs for this repo and linguistic communication
• @dependabot use these reviewers will set up the current reviewers as the default for future PRs for this repo and linguistic communication
• @dependabot use these assignees will set the current assignees equally the default for future PRs for this repo and linguistic communication
• @dependabot use this milestone volition prepare the current milestone as the default for hereafter PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

Bump urllib3 from 1.24.2 to one.26.5 in /splunk_eventgen/lib

Created 02 Jun, 2021 Pull Request #432 User Dependabot

Bumps urllib3 from 1.24.2 to 1.26.v.

Release notes

Sourced from urllib3'due south releases.

one.26.5

⚠️ IMPORTANT: urllib3 v2.0 will drop back up for Python two: Read more than in the v2.0 Roadmap

• Fixed deprecation warnings emitted in Python 3.10.
• Updated vendored vi library to ane.16.0.
• Improved performance of URL parser when splitting the potency component.

If you or your organization rely on urllib3 consider supporting us via GitHub Sponsors

1.26.iv

⚠️ Important: urllib3 v2.0 volition drop support for Python ii: Read more in the v2.0 Roadmap

• Changed behavior of the default SSLContext when connecting to HTTPS proxy during HTTPS requests. The default SSLContext now sets check_hostname=True.

If you or your system rely on urllib3 consider supporting united states via GitHub Sponsors

1.26.three

⚠️ IMPORTANT: urllib3 v2.0 will driblet support for Python 2: Read more in the v2.0 Roadmap

• Fixed bytes and string comparison issue with headers (Pull #2141)

• Changed ProxySchemeUnknown error message to exist more actionable if the user supplies a proxy URL without a scheme (Pull #2107)

If you or your organization rely on urllib3 consider supporting us via GitHub Sponsors

1.26.2

⚠️ IMPORTANT: urllib3 v2.0 will drop support for Python two: Read more in the v2.0 Roadmap

• Stock-still an issue where wrap_socket and CERT_REQUIRED wouldn't be imported properly on Python two.vii.8 and earlier (Pull #2052)

i.26.one

⚠️ Important: urllib3 v2.0 will drop support for Python ii: Read more in the v2.0 Roadmap

• Stock-still an outcome where 2 User-Amanuensis headers would be sent if a User-Amanuensis header fundamental is passed as bytes (Pull #2047)

one.26.0

⚠️ Of import: urllib3 v2.0 will drop support for Python ii: Read more in the v2.0 Roadmap

• Added support for HTTPS proxies contacting HTTPS servers (Pull #1923, Pull #1806)

• Deprecated negotiating TLSv1 and TLSv1.1 by default. Users that still wish to use TLS before than 1.2 without a deprecation alarm should opt-in explicitly by setting ssl_version=ssl.PROTOCOL_TLSv1_1 (Pull #2002) Starting in urllib3 v2.0: Connections that receive a DeprecationWarning volition neglect

• Deprecated Retry options Retry.DEFAULT_METHOD_WHITELIST, Retry.DEFAULT_REDIRECT_HEADERS_BLACKLIST and Retry(method_whitelist=...) in favor of Retry.DEFAULT_ALLOWED_METHODS, Retry.DEFAULT_REMOVE_HEADERS_ON_REDIRECT, and Retry(allowed_methods=...) (Pull #2000) Starting in urllib3 v2.0: Deprecated options will be removed

... (truncated)

Changelog

Sourced from urllib3'south changelog.

1.26.5 (2021-05-26)

• Fixed deprecation warnings emitted in Python three.ten.
• Updated vendored vi library to 1.16.0.
• Improved performance of URL parser when splitting the dominance component.

1.26.4 (2021-03-15)

• Changed behavior of the default SSLContext when connecting to HTTPS proxy during HTTPS requests. The default SSLContext now sets check_hostname=True.

i.26.iii (2021-01-26)

• Fixed bytes and cord comparison issue with headers (Pull #2141)

• Inverse ProxySchemeUnknown error bulletin to be more actionable if the user supplies a proxy URL without a scheme. (Pull #2107)

1.26.2 (2020-11-12)

• Fixed an issue where wrap_socket and CERT_REQUIRED wouldn't be imported properly on Python 2.7.8 and before (Pull #2052)

1.26.1 (2020-11-xi)

• Fixed an upshot where two User-Agent headers would be sent if a User-Amanuensis header key is passed as bytes (Pull #2047)

i.26.0 (2020-11-10)

• Notation: urllib3 v2.0 volition drop support for Python two. Read more in the v2.0 Roadmap <https://urllib3.readthedocs.io/en/latest/v2-roadmap.html>_.

• Added support for HTTPS proxies contacting HTTPS servers (Pull #1923, Pull #1806)

• Deprecated negotiating TLSv1 and TLSv1.1 by default. Users that still wish to employ TLS earlier than 1.ii without a deprecation warning

... (truncated)

Commits

• d161647 Release 1.26.5
• 2d4a3fe Improve operation of sub-authorisation splitting in URL
• 2698537 Update vendored 6 to 1.16.0
• 07bed79 Fix deprecation warnings for Python 3.10 ssl module
• d725a9b Add Python iii.ten to GitHub Deportment
• 339ad34 Use pytest==vi.2.4 on Python three.10+
• f271c9c Apply latest Black formatting
• 1884878 [1.26] Properly proxy EOF on the SSLTransport examination suite
• a891304 Release 1.26.4
• 8d65ea1 Merge pull request from GHSA-5phf-pp7p-vc2r
• Additional commits viewable in compare view

Dependabot will resolve whatever conflicts with this PR as long as you don't alter it yourself. Y'all can also trigger a rebase manually past commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot merge will merge this PR later on your CI passes on it
• @dependabot squash and merge volition squash and merge this PR after your CI passes on it
• @dependabot abolish merge will cancel a previously requested merge and block automerging
• @dependabot reopen volition reopen this PR if it is closed
• @dependabot shut volition close this PR and stop Dependabot recreating it. You tin achieve the same upshot by closing it manually
• @dependabot ignore this major version volition close this PR and stop Dependabot creating whatsoever more for this major version (unless y'all reopen the PR or upgrade to information technology yourself)
• @dependabot ignore this minor version volition close this PR and finish Dependabot creating any more for this minor version (unless you lot reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless yous reopen the PR or upgrade to it yourself)
• @dependabot use these labels will set the current labels as the default for hereafter PRs for this repo and language
• @dependabot use these reviewers will set up the current reviewers as the default for future PRs for this repo and language
• @dependabot employ these assignees will set the current assignees as the default for future PRs for this repo and language
• @dependabot employ this milestone will gear up the current milestone as the default for hereafter PRs for this repo and language

You tin can disable automated security set PRs for this repo from the Security Alerts page.

Bump rexml from three.2.4 to 3.two.5 in /docs

Created xxx Apr, 2021 Pull Request #430 User Dependabot

Bumps rexml from 3.two.four to 3.2.5.

Changelog

Sourced from rexml'southward changelog.

3.2.5 - 2021-04-05 {#version-3-2-5}

Improvements

• Add more validations to XPath parser.

• crave "rexml/document" past default. [GitHub#36][Patch by Koichi ITO]

• Don't add #dcloe method to core classes globally. [GitHub#37][Patch by Akira Matsuda]

• Add together more documentations. [Patch by Burdette Lamar]

• Added REXML::Elements#parent. [GitHub#52][Patch by Burdette Lamar]

Fixes

• Fixed a issues that REXML::DocType#clone doesn't copy external ID information.

• Fixed round-trip vulnerability bugs. Encounter also: https://www.ruby-red-lang.org/en/news/2021/04/05/xml-round-trip-vulnerability-in-rexml-cve-2021-28965/ [HackerOne#1104077][CVE-2021-28965][Reported by Juho Nurminen]

Thanks

• Koichi ITO

• Akira Matsuda

• Burdette Lamar

• Juho Nurminen

Commits

• a622645 Add 3.2.v entry
• 3c137eb Prepare a parser problems that some data may be ignored before DOCTYPE
• 9b311e5 Set up a problems that invalid document announcement may be accustomed
• f9d88e4 Fix a issues that invalid document proclamation may be generated
• f7bab89 Fix a problems that invalid element end may exist accepted
• 6a250d2 Fix a bug that invalid element kickoff may be accepted
• 2fe62e2 Gear up a bug that invalid notation declaration may exist accustomed
• a659c63 Fix a issues that invalid notation declaration may exist generated
• 790dd11 Utilise carmine/setup-blood-red (#66)
• eda1b20 Clean up and enhance high-level RDoc (#65)
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long every bit yous don't alter information technology yourself. You can too trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You lot can trigger Dependabot deportment past commenting on this PR:

• @dependabot rebase volition rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot merge volition merge this PR afterward your CI passes on it
• @dependabot squash and merge volition squash and merge this PR afterward your CI passes on it
• @dependabot cancel merge will cancel a previously requested merge and block automerging
• @dependabot reopen will reopen this PR if it is closed
• @dependabot close volition close this PR and stop Dependabot recreating it. You tin accomplish the same result by closing it manually
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless yous reopen the PR or upgrade to it yourself)
• @dependabot ignore this modest version will close this PR and end Dependabot creating any more for this small version (unless y'all reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating whatever more for this dependency (unless you lot reopen the PR or upgrade to it yourself)
• @dependabot use these labels will set the electric current labels as the default for future PRs for this repo and language
• @dependabot use these reviewers will fix the current reviewers as the default for future PRs for this repo and linguistic communication
• @dependabot use these assignees volition ready the current assignees as the default for time to come PRs for this repo and linguistic communication
• @dependabot utilize this milestone will ready the current milestone as the default for future PRs for this repo and linguistic communication

You tin can disable automated security fix PRs for this repo from the Security Alerts page.

Crash-land kramdown from 2.three.0 to 2.3.1 in /docs

Created 29 Mar, 2021 Pull Asking #429 User Dependabot

Bumps kramdown from 2.iii.0 to two.3.1.

Commits

• See full diff in compare view

Dependabot will resolve any conflicts with this PR as long every bit you lot don't change it yourself. You can likewise trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been fabricated to it
• @dependabot merge will merge this PR later your CI passes on it
• @dependabot squash and merge will squash and merge this PR after your CI passes on it
• @dependabot cancel merge volition abolish a previously requested merge and block automerging
• @dependabot reopen volition reopen this PR if information technology is airtight
• @dependabot shut volition close this PR and stop Dependabot recreating it. You can attain the aforementioned upshot past closing information technology manually
• @dependabot ignore this major version volition close this PR and cease Dependabot creating any more than for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will shut this PR and stop Dependabot creating any more than for this modest version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency volition close this PR and terminate Dependabot creating whatever more than for this dependency (unless you reopen the PR or upgrade to information technology yourself)
• @dependabot use these labels will set the current labels as the default for hereafter PRs for this repo and linguistic communication
• @dependabot use these reviewers volition set the current reviewers every bit the default for time to come PRs for this repo and language
• @dependabot employ these assignees will set the current assignees as the default for futurity PRs for this repo and language
• @dependabot use this milestone will prepare the current milestone as the default for futurity PRs for this repo and language

You can disable automated security gear up PRs for this repo from the Security Alerts page.

Bump jinja2 from 2.10.3 to two.xi.3 in /splunk_eventgen/lib

Created 20 Mar, 2021 Pull Request #428 User Dependabot

Bumps jinja2 from two.10.3 to ii.xi.iii.

Release notes

Sourced from jinja2's releases.

two.eleven.three

This contains a fix for a speed issue with the urlize filter. urlize is probable to be called on untrusted user input. For certain inputs some of the regular expressions used to parse the text could accept a very long time due to backtracking. As part of the fix, the email matching became slightly stricter. The various speedups apply to urlize in general, not just the specific input cases.

• PyPI: https://pypi.org/project/Jinja2/two.11.3/
• Changes: https://jinja.palletsprojects.com/en/2.11.x/changelog/#version-2-11-3

two.xi.2

• Changelog: https://jinja.palletsprojects.com/en/2.eleven.10/changelog/#version-2-11-ii

2.11.1

This fixes an issue in async environment when indexing the event of an attribute lookup, like {{ information.items[1:] }}.

• Changes: https://jinja.palletsprojects.com/en/2.eleven.x/changelog/#version-ii-11-1

2.11.0

• Changes: https://jinja.palletsprojects.com/en/2.11.10/changelog/#version-ii-xi-0
• Weblog: https://palletsprojects.com/weblog/jinja-ii-11-0-released/
• Twitter: https://twitter.com/PalletsTeam/status/1221883554537230336

This is the last version to back up Python 2.seven and 3.5. The next version volition exist Jinja 3.0 and will support Python 3.vi and newer.

Changelog

Sourced from jinja2's changelog.

Version 2.xi.three

Released 2021-01-31

• Improve the speed of the urlize filter by reducing regex backtracking. E-mail matching requires a give-and-take grapheme at the start of the domain role, and only discussion characters in the TLD. :pr:1343

Version 2.11.2

Released 2020-04-xiii

• Set up a bug that caused callable objects with __getattr__, like :class:~unittest.mock.Mock to be treated as a :func:contextfunction. :issue:1145
• Update wordcount filter to trigger :class:Undefined methods by wrapping the input in :func:soft_str. :pr:1160
• Fix a hang when displaying tracebacks on Python 32-bit. :issue:1162
• Showing an undefined fault for an object that raises AttributeError on access doesn't crusade a recursion error. :issue:1177
• Revert changes to :grade:~loaders.PackageLoader from ii.10 which removed the dependency on setuptools and pkg_resources, and added limited back up for namespace packages. The changes caused issues when using Pytest. Due to the difficulty in supporting Python 2 and :pep:451 simultaneously, the changes are reverted until 3.0. :pr:1182
• Gear up line numbers in error messages when newlines are stripped. :pr:1178
• The special namespace() consignment object in templates works in async environments. :issue:1180
• Gear up whitespace being removed earlier tags in the centre of lines when lstrip_blocks is enabled. :issue:1138
• :form:~nativetypes.NativeEnvironment doesn't evaluate intermediate strings during rendering. This prevents early on evaluation which could change the value of an expression. :issue:1186

Version 2.11.1

Released 2020-01-30

• Fix a bug that prevented looking up a key subsequently an attribute ({{ data.items[1:] }}) in an async template. :outcome:1141

... (truncated)

Commits

• cf21539 release version ii.11.3
• 15ef8f0 Merge pull asking #1343 from pallets/urlize-speedup
• ef658dc speed up urlize matching
• eeca0fe Merge pull request #1207 from mhansen/patch-1
• 2dd7691 Merge pull asking #1209 from mhansen/patch-3
• 4892940 do_dictsort: update example prepare to copy/paste
• 7db7d33 api.rst: bugfix in docs, import PackageLoader
• 9ec465b fix changelog header
• 737a4cd release version 2.11.2
• 179df6b Merge pull request #1190 from pallets/native-eval
• Boosted commits viewable in compare view

Dependabot volition resolve whatsoever conflicts with this PR as long as you don't modify information technology yourself. You tin can too trigger a rebase manually past commenting @dependabot rebase.

---

Dependabot commands and options

Yous can trigger Dependabot deportment past commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting whatsoever edits that have been made to it
• @dependabot merge will merge this PR after your CI passes on it
• @dependabot squash and merge volition squash and merge this PR afterwards your CI passes on information technology
• @dependabot cancel merge will abolish a previously requested merge and cake automerging
• @dependabot reopen will reopen this PR if information technology is airtight
• @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the aforementioned result by endmost it manually
• @dependabot ignore this major version will close this PR and stop Dependabot creating whatever more than for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this small-scale version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency volition close this PR and stop Dependabot creating any more than for this dependency (unless you lot reopen the PR or upgrade to it yourself)
• @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
• @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
• @dependabot utilize these assignees will set the electric current assignees equally the default for hereafter PRs for this repo and linguistic communication
• @dependabot employ this milestone will ready the current milestone as the default for future PRs for this repo and language

Y'all tin disable automatic security fix PRs for this repo from the Security Alerts page.

Add throughput summary and set timmer blocked by end=3 count=0 settings

Created 08 Nov, 2019 Pull Request #346 User Yangxulight

Here is what I did in this PR:

1. Add Throughput summary when we call "/status" from a controller. (eventgen_controller_api.py)
2. When we have a sample with settings of "count=0, end=three", the timer for that sample volition blocked and only log "There is no data to be generated in worker...". because the execution is always 0. We should increase the execution so that this sample timer can finished the job after 3 execution.

Issues

New token replacement type: weight replacement[feature/comeback]

Created 04 Jul, 2019 Issue #261 User Yangxulight

Is your feature request related to a problem? Please describe.
About time when I want to replace my token in proportion, I have to generate a file with the replacement text in proportion. Now I desire an easy way to ascertain this kind of token replacement. Let'south say I desire to replace "#http_status_code#" one of this values:"200", "404", "503", "400". And I want ninety% events has http_status_code "200", and others events with "404", "503", "400". Is there an good way to exercise this?

Describe the solution y'all'd similar
Maybe we can have a new token.replacementType , and nosotros can supersede the token by reading a a csv file, which the fist cavalcade is the value, and the 2nd column is the weight.

Describe alternatives yous've considered
A clear and concise description of any culling solutions or features you've considered.

Additional context
Nosotros may take to implement a specific random function for it.

Utilize variables within csv files to reuse substituted timestamps in other events [feature/improvement]

Created 31 Oct, 2019 Result #339 User Dieterschmitz

Is your characteristic asking related to a trouble? Delight depict.
No

Describe the solution you'd like
To clarify what we need it is the best to give you lot an case:
We (and some of our customers) use SA-EventGen and import CSV files we created on our own.
The content of the CSV file looks similar this (simplified):
2019-10-29 xv:14:04.403,UserA,"2019-10-29TT14:14:04.403+0100",IndexName,Source,MetricA
2019-10-29 fifteen:14:05.216,UserA,"2019-10-29TT14:14:04.403+0100",IndexName,Source,MetricB

The beginning line contains two timestamps. The first is the timestamp for the event, and the second timestamp is (in our case) the user's logon time. Both timestamps were replaced by SA-Eventgen which is fine.
At the moment SA-Eventgen does the aforementioned with the second line. And then the generated events looks similar this:
"2019-10-29 15:14:04.403,UserA,"2019-ten-29TT14:xiv:04.403+0100",IndexName,Source,MetricA
"2019-10-29 15:xiv:05.216,UserA,"2019-10-29TT14:xiv:05.216+0100",IndexName,Source,MetricB
The difference is that all timestamps in a single row have the same value.

We want to reuse the second timestamp of the commencement event (2019-10-29TT14:xiv:04.403+0100) in the 2d event again. So SA-Eventgen should not replace the 2nd timestamp of the second line with the current appointment/time but with the already replaced of the starting time event.

[feature/comeback] enabling jinja template with splunk eventgen app (eventgen as splunk app)

Created 06 Feb, 2020 Consequence #357 User Siddharthajuprod07

Is your feature request related to a problem? Please describe.
Yes. Splunk eventgen app is non able to initialize jinja template considering the code is resolving to a path "$SPLUNK_HOME\etc\apps\SA-Eventgen\lib\plugins\generator" , which doesn't exists.
Below is my environment details,
OS : Windows
Splunk Version : 8.0.ane
Eventgen Version : half-dozen.5.ii (as eventgen 7 withal non work with splunk eight)
python version : both python2 and python3 having aforementioned result.

Describe the solution you lot'd like
Splunkbase eventgen should work with jinja template.

Describe alternatives you've considered
I did some findings from my end to fix this simply no luck. I followed the beneath steps afterwards seeing the lawmaking of eventgen , roughly in the below order
modinput_eventgen.py >> eventgen_core.py >> eventgentimer.py >> eventgenconfig.py >> eventgenexceptions.py >> eventgen_core.py (equally the exception PluginNotLoaded handled here)
&& jinja.py

The steps I followed,

1. Nether SA-Eventgen\lib I created the binder structure plugins\generator (as the code is looking for this path).
2. Nether generator folder I copied the jinja2 folder from SA-Eventgen\lib folder as jinja.py is looking for that.
3. I also copied jinja.py from $SPLUNK_HOME\etc\apps\SA-Eventgen\lib\plugins\generator folder to SA-Eventgen\lib\plugins\generator folder.
4. I likewise created an empty init.py file in SA-Eventgen\lib\plugins\generator folder.
5. Restarted splunk.

Now I am receiving the below error.

02-06-2020 18:02:32.343 +0530 Error ExecProcessor - message from ""C:\Program Files\Splunk\bin\Python2.exe" "C:\Program Files\Splunk\etc\apps\SA-Eventgen\bin\modinput_eventgen.py"" 2020-02-06 18:02:31 eventgen Mistake MainProcess {'exception': 'Traceback (most contempo phone call last):\n File "C:\Programme Files\Splunk\etc\apps\SA-Eventgen\lib\splunk_eventgen\eventgen_core.py", line 336, in _initializePlugins\n module = imp.load_module(base, mod_name, mod_path, mod_desc)\n File "C:\Program Files\Splunk\etc\apps\SA-Eventgen\lib\plugins\generator\jinja.py", line 8, in \northward from jinja2 import nodes\n File "C:\Program Files\Splunk\etc\apps\SA-Eventgen\lib\jinja2\init.py", line 33, in \northward from jinja2.environs import Surroundings, Template\n File "C:\Plan Files\Splunk\etc\apps\SA-Eventgen\lib\jinja2\surroundings.py", line 15, in \n from jinja2 import nodes\n File "C:\Program Files\Splunk\etc\apps\SA-Eventgen\lib\jinja2\nodes.py", line 19, in \n from jinja2.utils import Markup\n File "C:\Program Files\Splunk\etc\apps\SA-Eventgen\lib\jinja2\utils.py", line 647, in \n from markupsafe import Markup, escape, soft_unicode\nImportError: No module named markupsafe', 'effect': ImportError('No module named markupsafe',)}

The error is coming from utils.py under jinja2 in last line,

from markupsafe import Markup, escape, soft_unicode

Initially I was thinking this is related to python result but when I inverse the python version in server.conf its not able to initialize the modular input.

Additional context
None.

[bug] multiple strptime formats in token replacement

Created 02 Apr, 2020 Issue #369 User Jmeixensperger

Draw the problems
Events are non generated when I use multiple strptime expressions for a single token.

To Reproduce
Generate with included files

Expected behavior
Both strptime expressions are evaluated/written with the correct replaytimestamp

Actual behavior
Nothing is generated

Sample files and eventgen.conf file
eventgen.conf:

                      [sample.mobilemusic.csv] sampletype = csv outputMode = stdout end = 1 mode = replay  token.0.token = ((\westward+\due south+\d+\s+\d{ii}:\d{2}:\d{2}:\d{3})|(\d{4}-\d{ii}-\d{2} \d{2}:\d{2}:\d{two}:\d{iii})) token.0.replacementType = replaytimestamp token.0.replacement = ["%b %d %H:%M:%S:%f", "%Y-%one thousand-%d %H:%Grand:%S:%f"]                                          

sample:

                      index,host,source,sourcetype,_raw eventgenTest,splunk,/var/log/radius.log,radius,May 27 18:28:11:000 aaa2 radiusd[12676]:[ID 959576 local1.info] INFO RADOP(thirteen) acct start for [email protected] 10.94.63.34 from 130.253.37.97 recorded OK. eventgenTest,splunk,/var/log/httpd/access_log,access_custom,"2012-05-27 18:28:eleven:112 x.2.1.35 Post /playhistory/uploadhistory - lxxx - x.94.63.34 ""Mozilla/5.0 (iPhone; CPU iPhone OS 5_0_1 like Mac Bone X) AppleWebKit/534.46 (KHTML, similar Gecko) Version/v.one Mobile/9A405 Safari/7534.48.3"" 503 0 0 468 1488" eventgenTest,splunk,/var/log/httpd/access_log,access_custom,"2012-05-27 eighteen:28:xi:125 ten.2.i.35 GET /sync/addtolibrary/01011207201000005652000000000047 - 80 - 10.94.63.34 ""Mozilla/5.0 (iPhone; CPU iPhone OS 5_0_1 like Mac Bone X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9A405 Safari/7534.48.3"" 200 0 0 468 1488" eventgenTest,splunk,/var/log/httpd/access_log,access_custom,"2012-05-27 18:28:xi:137 10.ii.1.35 GET /sync/addtolibrary/01011207201000005652000000000047 - fourscore - 10.94.63.34 ""Mozilla/5.0 (iPhone; CPU iPhone OS 5_0_1 like Mac OS X) AppleWebKit/534.46 (KHTML, similar Gecko) Version/5.1 Mobile/9A405 Safari/7534.48.three"" 503 0 0 468 1488" eventgenTest,splunk,/var/log/radius.log,radius,May 27 18:28:11:199 aaa2 radiusd[12676]:[ID 959576 local1.info] INFO RADOP(thirteen) acct stop for [e-mail protected] 10.94.63.34 from 130.253.37.97 recorde                                          

Do y'all run eventgen with SA-eventgen?
No

If you are using eventgen with pip module way (please consummate the following data):

• python version: 3.6
• Bone: MacOS
• Virtual Env is used: Yep
• Eventgen Version: 7.0.0

Additional context
Relevant token.X.replacement docs on passing a list of strptime expressions (seems supported): "For ["listing","of","strptime"], but used with replaytimestamp, a JSON formatted list of strptime formats to try."

Stanzas from separate sources with duplicate names do not generate [bug]

Created 15 Apr, 2020 Issue #378 User Jmeixensperger

Describe the issues
When generating data with multiple eventgen.conf files, if the conf files contain the same stanza name, only i stanza gets used. This tin can occur easily if nosotros re-utilize bundles across sources/configs.

To Reproduce

1. Copy bundles to dissever TAs:
   stanzaBug2.tar.gz
   stanzaBug1.tar.gz
2. Add 'myIndex1' and 'myIndex2' indices
3. Enable eventgen and expect at data ingested in both indices

Expected behavior
Both conf files go picked upward and data is ingested in both indices

Actual behavior
Simply 1 stanza is used and information is ingested in 1 alphabetize

Do you run eventgen with SA-eventgen?
Yes

[bug] event with s2s output mode and lines over 120 char

Created 11 May, 2020 Issue #392 User Themrkeys

Describe the problems
Exception in thread OutputThread0:
Traceback (most recent call concluding):
File "/usr/local/lib/python3.vii/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.vii/threading.py", line 870, in run
self._target(*self._args, **cocky._kwargs)
File "/usr/local/lib/python3.vii/site-packages/splunk_eventgen/eventgen_core.py", line 318, in _worker_do_work
raise e
File "/usr/local/lib/python3.7/site-packages/splunk_eventgen/eventgen_core.py", line 304, in _worker_do_work
detail.run()
File "/usr/local/lib/python3.vii/site-packages/splunk_eventgen/lib/outputplugin.py", line 39, in run
self.flush(self.events)
File "/usr/local/lib/python3.7/site-packages/splunk_eventgen/lib/plugins/output/s2s.py", line 204, in flush
1000["_time"],
File "/usr/local/lib/python3.7/site-packages/splunk_eventgen/lib/plugins/output/s2s.py", line 173, in send_event
e = cocky._encode_event(index, host, source, sourcetype, _raw, _time)
File "/usr/local/lib/python3.7/site-packages/splunk_eventgen/lib/plugins/output/s2s.py", line 124, in _encode_event
encoded_raw = self._encode_key_value("_raw", _raw)
File "/usr/local/lib/python3.vii/site-packages/splunk_eventgen/lib/plugins/output/s2s.py", line 78, in _encode_key_value
render "%s%south" % (cocky._encode_string(key), cocky._encode_string(value))
File "/usr/local/lib/python3.seven/site-packages/splunk_eventgen/lib/plugins/output/s2s.py", line 69, in _encode_string
"utf-viii"
UnicodeDecodeError: 'utf-viii' codec can't decode byte 0xd1 in position 3: invalid continuation byte

To Reproduce
Steps to reproduce the behavior:
use outputmode=s2s

Expected beliefs
no exception

Actual behavior
exception above

Exercise y'all run eventgen with SA-eventgen?
No

If you are using SA-Eventgen with Splunk (please complete the following data):
linux python iii.7
eventgen version git master

[bug] global end does not work

Created 12 May, 2020 Issue #393 User Rfaircloth-splunk

Describe the bug
eventgen does not stop later "cease" every bit occurred when end is int

To Reproduce
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Curl down to '....'
4. See error

Expected behavior
A clear and curtailed description of what you expected to happen.

Actual behavior
A clear and concise clarification of what happens subsequently doing the reproduce steps.

Screenshots
If applicable, add screenshots to help explicate your trouble.

Sample files and eventgen.conf file
Please attach your sample files and eventgen conf file

Do you run eventgen with SA-eventgen?
Yes/No(No means you lot run eventgen with pip module style)

If you are using SA-Eventgen with Splunk (delight complete the following information):

• Os: [e.thousand. Windows/Mac OS 10.fourteen]
• Browser [e.m. chrome, safari]
• Eventgen Version [east.1000. 22]
• Splunk Version[e.g. 7.ane.iii]
• What other apps you lot have installed in Splunk etc/apps?

If you are using eventgen with pip module mode (delight complete the post-obit information):

• python version: [due east.k. ii.half-dozen]
• Bone: [e.thousand. Windows10]
• Virtual Env is used: Yes/No
• Eventgen Version [e.g. 6.iii.two]

Boosted context
Add any other context about the problem hither.

[bug] unable to use existing eventgen.conf from v.x with half dozen.ten or seven.10

Created 26 May, 2020 Upshot #398 User Rfaircloth-splunk

Describe the issues
Regex syntax of stanzas are no longer supported

To Reproduce
See Splunk-TA-juniper

Expected behavior
stanzas should exist practical based on regex match to sample name as with EG5.x

Actual behavior
No events are generated

Screenshots
If applicable, add screenshots to help explain your problem.

Sample files and eventgen.conf file
Delight attach your sample files and eventgen conf file

Exercise yous run eventgen with SA-eventgen?
Yes/No(No means you lot run eventgen with pip module manner)

If you are using SA-Eventgen with Splunk (please complete the following data):

• OS: [eastward.g. Windows/Mac OS 10.14]
• Browser [e.1000. chrome, safari]
• Eventgen Version [eastward.chiliad. 22]
• Splunk Version[e.thou. 7.1.3]
• What other apps you have installed in Splunk etc/apps?

If yous are using eventgen with pip module mode (please complete the following information):

• python version: [east.chiliad. 2.vi]
• OS: [e.g. Windows10]
• Virtual Env is used: Yeah/No
• Eventgen Version [due east.g. 6.3.2]

Boosted context
Add together any other context about the problem hither.

[issues] unable to run test cases

Created 28 May, 2020 Issue #399 User Rfaircloth-splunk

Describe the bug
Unable to use pytest

To Reproduce
setup venv and run pytest

Expected behavior
Setup of develop dependencies should exist adequate to test.

Actual behavior

pytest failed for missing dependency immediately in addition I suspect there surface area requirements for access to Splunk instances not defined

Screenshots

`================================================================================================================== ERRORS ===================================================================================================================
____________________________________________________________________________________________ ERROR collecting tests/large/test_output_plugin.py _____________________________________________________________________________________________
ImportError while importing test module '/Users/rfaircloth/Documents/GitHub/eventgen/tests/large/test_output_plugin.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/large/test_output_plugin.py:1: in
from tests.large.utils.splunk_search_util import (
E ModuleNotFoundError: No module named 'tests'
============================================================================================================= warnings summary ==============================================================================================================
venv/lib/python3.7/site-packages/_pytest/mark/structures.py:334
/Users/rfaircloth/Documents/GitHub/eventgen/venv/lib/python3.7/site-packages/_pytest/mark/structures.py:334: PytestUnknownMarkWarning: Unknown pytest.mark.large - is this a typo? Yous tin can register custom marks to avoid this warning - for details, encounter https://docs.pytest.org/en/latest/mark.html
PytestUnknownMarkWarning,

-- Docs: https://docs.pytest.org/en/latest/warnings.html
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
==================================================================================================== 1 warnings, 1 error in 0.99 seconds ====================================================================================================

Boosted context
Add any other context near the problem here.

Source product version tracking

Created 22 Jun, 2020 Issue #401 User Mkarlstrand-splunk

Today we separately maintain a repository of source product data samples in XML format with notations including product, version and notes about how the sample was collected (lab, documentation, internet, etc.). There is no linkage between this critical information and the eventgen templates and config we maintain for the same source products in another repo. This is problematic given that when eventgen is used for QA of an add together-on in that location is no power to determine what source product or version/versions are being simulated. This ways that if a bug/issue is found, transmission investigation must happen to track down the versions in question, if that is fifty-fifty possible with the bachelor information. Additionally, since at that place is no real visibility into the data used to create the templates the quality/trustworthiness of the resulting constructed data is finer unknown.
For example, events provided without information about the source product configuration and/or environment would exist college risk than events from a lab with well documented configuration.

The requested enhancement will provide a machinery to centrally manage and store notated source data samples, eventgen templates and config for a source product. Additionally, the solution will have tracking for source products and versions.

As a Programmer/Researcher/QA I may generate information needed to exam a technical add-on against specific products/components and versions then that I may ensure the compatibility/support that is intended.
For instance, I tin generate events for Cisco ASA firewall events for version 9.xiii.

As a Developer/Researcher/QA I can hands run into how the source events for specific products and versions were captured and which eventgen templates are based on these samples and so that I may gauge how trustworthy the resulting synthetic data is.

Eventgen not starting in standalone mode

Created 21 Aug, 2020 Issue #410 User Asmithhpe

Upshot:
I am invoking eventgen past running "splunk_eventgen -v generate path/to/eventgen.conf". I have attached the current configuration file that nosotros are attempting to use. The sample file is sitting in the "samples" folder where it should be. When I endeavour to start eventgen up there are no errors, no log files generated, nothing – it only just goes to the side by side line on the command line ready to receive the next command. Nothing shows upwards when I try to brandish what is actively running on the organization. Then, I am at a loss for what I need to do to get this functional.

OS:
CentOS seven.8
Eventgen:
7.1.1
eventgen.conf:
[sample_seed]

mode = sample
interval = 5
primeval = 5s
latest = now
generator = default
count = -one
hourOfDayRate = {"0": 0.8, "one": one.0, "2": 0.9, "three": 0.7, "4": 0.iv, "5":0.2, "half-dozen": 0.9, "vii": 0.five, "8": 0.half dozen, "ix": 2.0, "10": 1.0, "11": 0.iv, "12": 0.three, "xiii": 0.v, "14": 0.vi, "15": 0.seven, "xvi": 0.8, "17": 0.9, "18": 0.three, "nineteen": i.0, "xx": 0.4, "21": 0.5, "22": 0.6, "23": 0.8}
dayOfWeekRate = {"0": 0.8, "i": 1.0, "two": 0.9, "three": 0.seven, "4": 0.4, "5":0.two, "6": 0.nine}
perDayVolume = twoscore
randomizeCount = .four
randomizeEvents = true
outputMode = httpevent
httpeventServers = {"servers":[{ "protocol":"http", "accost":"192.168.3.101", "port":"8088", "key":"13109a1a-5576-4ff6-b5c6-a23af993a596"}]}
sourcetype = eventgen_test

[feature/comeback] amend retentiveness usage in perdayvolume generator

Created 28 October, 2020 Issue #414 User Jmeixensperger

Is your feature request related to a trouble? Delight draw.
The perdayvolume generator creates + stores multiple event lists/dicts while populating the output queue.

Draw the solution you lot'd similar
Nosotros should look at using a buffered io stream to mitigate the retentiveness usage.

Describe alternatives you lot've considered
Re-architect perdayvolume generator? Whatever alternative would be circuitous and require more than try

Additional context
This is not currently blocking customers from generating their desired data (hence the feature/comeback tag), merely this could drastically reduce the maximum chapters of perdayvolume's generation. We should also measure out + any report any performance gain associated with this improvement.

[bug] eventgentoken.py - rated integer token

Created thirty October, 2020 Effect #415 User Mickotronic

Describe the bug
eventgentoken.py has a bug where, if you use replacementtype = rated and replacement = integer in the config, information technology incorrectly uses s.now(), instead of the hour value.
Line 282: rateFactor *= s.hourOfDayRate[str(southward.now())]
It should use what float uses:
rateFactor *= s.hourOfDayRate[str(now.hour)]

I made 2 fixes to the file and now it works, to go far more in line with how the float handles information technology:
Before:
if endInt >= startInt:
replacementInt = random.randint(startInt, endInt)
if cocky.replacementType == "rated":
rateFactor = 1.0
if type(s.hourOfDayRate) == dict:
try:
rateFactor *= s.hourOfDayRate[str(southward.now())]

Later:
if endInt >= startInt:
replacementInt = random.randint(startInt, endInt)
if cocky.replacementType == "rated":
rateFactor = 1.0
now = south.now()
if type(s.hourOfDayRate) == dict:
try:
rateFactor *= due south.hourOfDayRate[str(at present.60 minutes)]

To Reproduce
Steps to reproduce the behavior:

1. Add together an eventgen input that uses token replacementtype = rated, replacement = integer, and hourOfDayRate.
2. Also, for that same eventgen input, utilise a token replacement where replacementtype = rated and replacement = bladder.
3. Restart eventgen.
4. Check eventgen logs/errors, and if data is beingness generated.
5. Change the offset token replacement to be a float besides.
6. Restart eventgen.
7. Check eventgen logs/errors, and if data is being generated.

Expected behavior
Step 4: Events are generated where the tokens are replaced randomly in the int/float range, rated past the hourOfDayRate.
Step 7: Aforementioned as higher up.

Bodily behavior
Step 4: Events are not generated, fault in the logs, the value can't exist institute in the hourOfDay dict with the key: "'2020-10-30 17:19:54.819649'". That is because information technology should be giving information technology the hr, not the whole engagement.
Footstep 7. Events are generated and rated as expected.

Screenshots

Sample files and eventgen.conf file
Fastened files

Exercise you run eventgen with SA-eventgen?
Yes

If you are using SA-Eventgen with Splunk (delight complete the following information):

• Bone: Ruddy Hat Linux
• Browser Firefox
• Eventgen Version 7.2
• Splunk Version 8.0.5
• What other apps you have installed in Splunk etc/apps?

Additional context
Python error when using integer replacement for rated replacementtype:
KeyError: '2020-10-30 17:19:54.819649'
During treatment of the above exception, another exception occurred:
Traceback (nearly recent call last):
File "/opt/splunk/etc/apps/SA-Eventgen/lib/splunk_eventgen/eventgen_core.py", line 350, in _generator_do_work
detail.run(output_counter=output_counter)
File "/opt/splunk/etc/apps/SA-Eventgen/lib/splunk_eventgen/lib/generatorplugin.py", line 225, in run
samplename=cocky._sample.name,
File "/opt/splunk/etc/apps/SA-Eventgen/lib/splunk_eventgen/lib/plugins/generator/default.py", line 76, in gen
GeneratorPlugin.build_events(self, eventsDict, startTime, earliest, latest)
File "/opt/splunk/etc/apps/SA-Eventgen/lib/splunk_eventgen/lib/generatorplugin.py", line 42, in build_events
eventsDict, earliest, latest, ignore_tokens=ignore_tokens
File "/opt/splunk/etc/apps/SA-Eventgen/lib/splunk_eventgen/lib/generatorplugin.py", line 272, in replace_tokens
pivot_timestamp=pivot_timestamp,
File "/opt/splunk/etc/apps/SA-Eventgen/lib/splunk_eventgen/lib/eventgentoken.py", line 85, in replace
pivot_timestamp=pivot_timestamp,
File "/opt/splunk/etc/apps/SA-Eventgen/lib/splunk_eventgen/lib/eventgentoken.py", line 289, in _getReplacement
% stack
TypeError: not plenty arguments for format string

[bug] verbosity and debug logging not working in versions vii.ten

Created 01 Nov, 2020 Issue #416 User Mickotronic

Describe the bug
If setting verbosity for eventgen in inputs.conf in versions 7.10, eventgen-main.log does non incorporate annihilation.
This aforementioned configuration would result in debug log output to eventgen-main.log in versions 6.ten.
There is no documentation that I tin can find that describes how to reach the same level of debug logging in versions 7.x, then I am making this a bug as it's unexpected behaviour and an undocumented change.

To Reproduce
Steps to reproduce the beliefs:

1. In eventgen v6.5.three, set verbosity to x or xl in local inputs.conf. Restart splunk.
2. Confirm eventgen-main.log contains debug log entries.
3. Remove eventgen v6.5.3 and install version 7.0.
4. Employ same configuration for verbosity.

Expected behavior
eventgen-main.log contains debug information.

Bodily beliefs
On versions 7.x, eventgen-main.log is empty. In some cases it logs errors.

Screenshots

Sample files and eventgen.conf file

Do yous run eventgen with SA-eventgen?
Yes

If y'all are using SA-Eventgen with Splunk (please consummate the following data):

• Bone: Tested on Centos 7 and Solus Linux
• Browser Firefox
• Eventgen Version 7.x (tested all versions in 7.10 branch).
• Splunk Version 8.0.5
• What other apps y'all accept installed in Splunk etc/apps?

If you are using eventgen with pip module mode (please complete the following information):

• python version: [e.g. 2.six]
• Bone: [eastward.grand. Windows10]
• Virtual Env is used: Yes/No
• Eventgen Version [e.1000. 6.three.2]

Boosted context
Add any other context about the problem here.

[feature/improvement] persists incremental value between splunk restart

Created 03 Nov, 2020 Event #417 User Jalkar

I'm using eventgen on a active evolution platform which needs to be restarted often.
We have data simulated from a BDD with incremental Ids.
It would be slap-up to have a "persistance" of the offset between each eventgen run
Currently when splunk restarts, eventgen increments restart at the "replacement" value set in the conf.
something similar a "persistance" option would be great :

                      [my_bdd] index = idx_bdd count = 10 mode = sample interval= thirty autotimestamp = true sourcetype = st_my_bdd source = my_bdd token.0.token = "ID":(\d+) token.0.replacementType = integerid token.0.replacement = 0 token.0.persistance = true                                          

Boosted debug to identify base of operations directory for relative samples directory search

Created 26 November, 2020 Issue #418 User Marking-sivill-splunk

I added the following debug to help determine the base of operations directory from where the relative search for the samples directory happens.

I'm running eventgen as a standalone program and debugging the files that were existence searched for in the directory structure took me some time. Perhaps the following might assist somebody else.

Change equally sed command ....

sed -i 's/ s.sampleDir = os.path.bring together(base_path, south.sampleDir)/ logger.debug("Using base directory %s" % (base_path) )\due north s.sampleDir = bone.path.join(base_path, due south.sampleDir)/1000' ./eventgen/splunk_eventgen/lib/eventgenconfig.py

"earliest = -1mon" in eventgen.conf not working in january month [eventgen 7.0.0]

Created 07 Jan, 2021 Consequence #426 User Mzadafiya-splunk

Depict the bug
Below stanza in eventgen.conf is stopped working since Jan 2021 in Eventgen v7.0.0. Before January 2021 information technology was working fine.

                      [aws_billing_detailed_planner.sample] outputMode = splunkstream count = -1 end = 1 earliest = -1mon latest = -1mon index = main sourcetype = aws:billing source = s3://aws-billing-detailed-line-items-with-resources-and-tags-2017-xi.csv.zip                                          

It gives an error in splunkd.log. sharing piece of error here

                      01-06-2021 07:48:34.401 +0000 ERROR ExecProcessor - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/SA-Eventgen/bin/modinput_eventgen.py" 2021-01-06 07:48:34 eventgen        Mistake    MainProcess Cannot parse relative time string 01-06-2021 07:48:34.402 +0000 Mistake ExecProcessor - message from "/opt/splunk/bin/python3.seven /opt/splunk/etc/apps/SA-Eventgen/bin/modinput_eventgen.py" 2021-01-06 07:48:34 eventgen        Fault    MainProcess unsupported operand type(s) for -: 'datetime.datetime' and 'bool' 01-06-2021 07:48:34.402 +0000 Fault ExecProcessor - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/SA-Eventgen/bin/modinput_eventgen.py" Traceback (well-nigh recent call last): 01-06-2021 07:48:34.402 +0000 ERROR ExecProcessor - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/SA-Eventgen/bin/modinput_eventgen.py"   File "/opt/splunk/etc/apps/SA-Eventgen/lib/splunk_eventgen/eventgen_core.py", line 259, in _worker_do_work 01-06-2021 07:48:34.402 +0000 ERROR ExecProcessor - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/SA-Eventgen/bin/modinput_eventgen.py"     item.run() 01-06-2021 07:48:34.402 +0000 ERROR ExecProcessor - bulletin from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/SA-Eventgen/bin/modinput_eventgen.py"   File "/opt/splunk/etc/apps/SA-Eventgen/lib/splunk_eventgen/lib/eventgentimer.py", line 84, in run 01-06-2021 07:48:34.402 +0000 Fault ExecProcessor - bulletin from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/SA-Eventgen/bin/modinput_eventgen.py"     cocky.real_run() 01-06-2021 07:48:34.402 +0000 Fault ExecProcessor - bulletin from "/opt/splunk/bin/python3.seven /opt/splunk/etc/apps/SA-Eventgen/bin/modinput_eventgen.py"   File "/opt/splunk/etc/apps/SA-Eventgen/lib/splunk_eventgen/lib/eventgentimer.py", line 170, in real_run 01-06-2021 07:48:34.402 +0000 ERROR ExecProcessor - bulletin from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/SA-Eventgen/bin/modinput_eventgen.py"     et = self.sample.earliestTime() 01-06-2021 07:48:34.402 +0000 Fault ExecProcessor - bulletin from "/opt/splunk/bin/python3.seven /opt/splunk/etc/apps/SA-Eventgen/bin/modinput_eventgen.py"   File "/opt/splunk/etc/apps/SA-Eventgen/lib/splunk_eventgen/lib/eventgensamples.py", line 270, in earliestTime 01-06-2021 07:48:34.402 +0000 Error ExecProcessor - bulletin from "/opt/splunk/bin/python3.seven /opt/splunk/etc/apps/SA-Eventgen/bin/modinput_eventgen.py"     temptd = self.now(realnow=True) - tempearliest 01-06-2021 07:48:34.402 +0000 ERROR ExecProcessor - message from "/opt/splunk/bin/python3.vii /opt/splunk/etc/apps/SA-Eventgen/bin/modinput_eventgen.py" TypeError: unsupported operand type(s) for -: 'datetime.datetime' and 'bool'                                          

Nosotros have tested diverse scenarios by irresolute machine DateTime to February and March 2021 and earliest/latest fourth dimension to -31d and -2mon.
Only this error being generated for the scenario: Motorcar time in Jan 2021 and earliest\latest "-1mon".

To Reproduce
Steps to reproduce the behavior:

1. Install Splunk viii.1.1
2. Install Eventgen vii.0.0
3. Identify eventgen.conf(Content added below) under $SPLUNK_HOME$/etc/apps/search/default/
4. Identify issue-reproduce.sample(Content added below) under $SPLUNK_HOME$/etc/apps/search/samples/
5. Restart Splunk
6. Enable Eventgen modular Input
7. Open search and search "alphabetize=main" string in time range "All time"

Expected behavior
The event placed under issue-reproduce.sample file should be indexed in the main alphabetize.

Actual behavior
The result is not indexed. Error is being reported in the splunkd.log(log snippet mentioned above)

Screenshots
If applicative, add screenshots to assist explain your problem.

Sample files and eventgen.conf file

Please adhere your sample files and eventgen conf file
eventgen.conf

                      [global] debug = false verbose = simulated outputMode = splunkstream splunkHost = localhost splunkUser = admin splunkPass = admin host = eventgen-hod randomizeEvents = fake index = main maxIntervalsBeforeFlush = i  [issue-reproduce.sample] outputMode = splunkstream primeval = -1mon latest = -1mon index = primary sourcetype = test source = test                                          

issue-reproduce.sample

Exercise you run eventgen with SA-eventgen?
Yes

If you are using SA-Eventgen with Splunk (please complete the following information):

• Os: Centos 7.6
• Browser: chrome
• Eventgen Version: 7.0.0
• Splunk Version: 8.ane.1
• What other apps y'all have installed in Splunk etc/apps? It reproducible without any other app installed in Splunk

Additional context
We accept tried to reproduce the aforementioned scenario in Eventgen 7.2.0. We got the aforementioned error in eventgen.log

[bug] eventgen does non work on windows ten 64 bit

Created 08 Feb, 2021 Issue #427 User Dieterschmitz

We're using Splunk eight.i.2 and SA-Eventgen version 7.ii.ane and want to replay CSV files. Everything is installed on a Windows x 64-bit machine.
When restarting Splunk with our eventgen app, the following mistake is shown in eventgen-error.log

2021-02-08 15:36:28 eventgen ERROR MainProcess Python int too large to convert to C long
Traceback (most contempo phone call concluding):
File "C:\Program Files\Splunk\etc\apps\SA-Eventgen\lib\splunk_eventgen\eventgen_core.py", line 325, in _worker_do_work
particular.run()
File "C:\Plan Files\Splunk\etc\apps\SA-Eventgen\lib\splunk_eventgen\lib\eventgentimer.py", line 117, in run
self.real_run()
File "C:\Programme Files\Splunk\etc\apps\SA-Eventgen\lib\splunk_eventgen\lib\eventgentimer.py", line 131, in real_run
raw_event_size = cocky.predict_event_size()
File "C:\Program Files\Splunk\etc\apps\SA-Eventgen\lib\splunk_eventgen\lib\eventgentimer.py", line xc, in predict_event_size
self.sample.loadSample()
File "C:\Program Files\Splunk\etc\apps\SA-Eventgen\lib\splunk_eventgen\lib\eventgensamples.py", line 464, in loadSample
csv.field_size_limit(sys.maxsize)
OverflowError: Python int as well big to convert to C long

The responsible line is found in eventgensamples.py in line 464:
csv.field_size_limit(sys.maxsize)

After changing the line above to (but an example)

                                              csv.field_size_limit(100000000)                                          

eventgen starts sending events to Splunk.

Information technology seems to exist, that this mistake occurs on Windows (64 bit???) systems simply. We tested information technology on a Linux automobile and it worked equally expected.

Releases

release 7.2.0

Created 09 Oct, 2020 Release release 7.2.0 User Jmeixensperger

• added support for "interval" option in replay style
• added new "splitSample" option
• added new "counter" generator
• added metrics logging
• bugfix for logging verbosity
• bugfix for replay mode threading and backfill generation
• bugfix for csv file handling
• bugfix for timeMultiple handling
• bugfix for server model configurations

release 7.one.one

Created 21 Jul, 2020 Release release 7.1.1 User Jmeixensperger

• stock-still cleaved "fileRotator" config option
• gear up for docker prototype build process, adjusted ujson version requirements
• update to default "source" beliefs: unless specified, the default source is set to the sample proper name
• poetry package management integration (see "setup" docs more info)
• improved documentation surrounding upgrade process

release 7.one.0

Created 06 Apr, 2020 Release release 7.one.0 User Jmeixensperger

• stock-still oom error caused by ujson
• added scp output plugin - use outputMode = scsout
• added --multithread support for server/controller compages
• fix CI failures due to jinja in log config

eventgen 7.0.0

Created 23 October, 2019 Release eventgen 7.0.0 User Li-wu

• Drift to Python3 and 7.0.0 merely supports Python3
• Fix random token replacement issues
• Enhance tutorial documentation
• Add syslogAddHeader selection
• Fix timezone setting issues
• Fix out of memory upshot when using multiprocess style
• Remove some stale third party libraries

eventgen 6.5.two

Created 08 Oct, 2019 Release eventgen half dozen.five.2 User Li-wu

• Set zipfile bug
• Fix random token replacement problems
• Set security vulnerability result
• Ready custom plugin stale docs
• Set timezone setting bug
• Fix multiprocess OOM issue
• Add together syslogAddHeader config

eventgen 6.5.0

Created 30 Jul, 2019 Release eventgen 6.5.0 User Arctan5x

• Added metrics output fashion
• Fixed regex token replacement issue
• Added test coverage information
• Increased functional test coverage
• Eventgen server consummate revamp and standalone mode back up
• Added contributor license
• Updated Dockerfile
• Added documentation
• Fixed bugs / stability / optimized speed

eventgen half dozen.4.0

Created 05 Jun, 2019 Release eventgen 6.iv.0 User Li-wu

• Fix exception log error
• Set CircleCI status badage error
• Prepare navigation mistake for app if installed with Splunk Stream
• Prepare generatorWorkers not working error
• Fix interval fault when end = 1
• Fix fileName in global stanza fault
• Add third political party libs in SA-Eventgen App
• Add together httpeventAllowFailureCount for httpevent
• Add together 3rd party libs in license credit
• Disable logging queue in multiprocess mode
• Change implementation of extendIndex for better performance

eventgen 6.3.half dozen

Created 08 May, 2019 Release eventgen six.iii.six User Li-wu

• Add functional tests for jinja template and modular input feature
• Fix default jinja template directory is non correctly resolved when sampleDir is ready outcome
• Fix verbose flag not working in splunk_eventgen command line issue
• Fix index, source, sourcetype are not correct when using splunkstream mode issue
• Fix ssh to container issue
• Prepare perdayvolume without end setting error
• Update documentation for ameliorate reading and remove unrelated role

eventgen half dozen.iii.5

Created xviii Apr, 2019 Release eventgen vi.3.5 User Li-wu

• Added extendIndexes feature to support a listing of indexes
• Fixed timer and token logic
• Changed terminate=-1 to continuously iterate without stopping
• Changed stop=0 to not execute
• Added a linter for code quality
• Updated docs / docs format
• Added a suite of functional tests

eventgen six.3.iv

Created xv Mar, 2019 Release eventgen half-dozen.iii.4 User Arctan5x

• Cleaned up documentation
• Jinja template bugfix in SA-Eventgen app
• Implementation of 'timeMultiple' option
• Templates for bugs/feature requests
• Fixed Jinja test configuration stanzas
• Fix for default behavior for 'count' edge cases

eventgen vi.3.3

Created 05 Mar, 2019 Release eventgen half dozen.3.3 User Jmeixensperger

• Added performance metrics compared to Eventgen v.ten
• New config option for generation-fourth dimension metrics: outputCounter
• Jinja template fixes
• Timestamp parsing fix
• Output queueing fix for outputMode splunkstream
• Count rater fixes, now supports indefinite generation

eventgen 6.three.ane

Created 16 Nov, 2018 Release eventgen 6.three.1 User Arctan5x

• Fixed Eventgen Book APIs
• Improved Eventgen Server Logging
• Corrected Eventgen Server and Controller conf syncing issue
• Adding verbosity options (Mistake, INFO, DEBUG) to Eventgen modinput
• Implemented future event generation support in replay mode
• Stock-still Jinja template'south missing default values
• Adapted logging bulletin levels for less verbosity
• Fixed event count off past 1 issue
• Fixed unnecessary empty data generators being created
• Updated dependency list

eventgen 6.3.0

Created 26 October, 2018 Release eventgen 6.three.0 User Arctan5x

• Bug fixes for the customer issues
• Documentation upgrade
• Code refactoring for version unification
• Logging improvements

eventgen six.2.1

Created 01 Jun, 2018 Release eventgen 6.2.1 User Arctan5x

• Fixing SA-Eventgen Dashboard and log searching
• Improving internal logging and fixing splunkd logging issue
• Fixing timestamping in default generator
• Fixing custom plugin integration
• Fixing SA-Eventgen app settings
• Supporting Eventgen five backward compatibility with additional features
• Ameliorate modinput process management
• Minor Bugfixes with diverse customer cases

fenwicktheized.blogspot.com

Source: https://pythonlang.dev/repo/splunk-eventgen/

Share this post