logstash-plugins/logstash-patterns-core

This system is the core pattern library for the Logstash Grok filter, providing reusable regular expression definitions for parsing diverse log sources. It maintains a dual structure of legacy patterns for backward compatibility and a comprehensive set of ECS-compliant patterns that standardize field names across various services like AWS, firewalls, and databases. The repository focuses on defining these parsing rules and validating their correctness through a dedicated test suite, rather than implementing the filtering logic itself.

This entry marks the initial commit of the repository, establishing the baseline for the logstash-patterns-core plugin at version 4.3.4. The release includes the core pattern definitions compliant with the Elastic Common Schema (ECS), a comprehensive CHANGELOG documenting fixes for patterns like CISCOFW, CLOUDFRONT\_ACCESS\_LOG, and BIN9\_QUERYLOG, and standard project infrastructure files such as the Apache 2.0 LICENSE, NOTICE, and CONTRIBUTORS list. It also updates the CI badge to GitHub Actions and adds a Rakefile requirement for logstash devutils.

(repo-wide) · high confidence

This release introduces a comprehensive set of new Grok patterns under the \patterns/ecs-v1\ directory, enabling structured parsing of logs from a wide variety of sources into the Elastic Common Schema (ECS) v1 format. The new patterns cover AWS services (S3, ELB, CloudFront), backup software (Bacula), DNS servers (BIND), network security tools (Bro/Zeek, Cisco ASA, Juniper SRX, NetScreen, iptables, Shorewall, SuSE Firewall 2), mail servers (Exim), load balancers (HAProxy), web servers (Apache/HTTPD), application logs (Java/Tomcat), system utilities (Linux Syslog, Cron, PAM), build tools (Maven), distributed computing (Mcollective), databases (MongoDB), and monitoring systems (Nagios). Users can now ingest and correlate these diverse log types with consistent field names.

patterns/ecs-v1 · high confidence

The Java and Junos pattern files have been deleted from the repository. This removes the previously defined patterns for Java class names, file names, methods (including special methods like \<init\> and \<clinit\>), and stack trace parts, as well as the Junos RT\_FLOW session creation, closure, and denial patterns. Users relying on these specific pattern definitions for parsing Java logs or Junos flow logs will no longer have these patterns available.

patterns · high confidence

The \rakelib/publish.rake\ and \rakelib/vendor.rake\ files have been deleted, removing the \publish\_gem\ task used to publish gems to RubyGems.org and the \vendor\ task along with its associated helper methods for downloading and extracting third-party dependencies. Users can no longer use these Rake tasks to manage gem releases or vendor external libraries within the project.

rakelib · high confidence

Log parsing patterns for AWS, Bacula, BIND, Bro, Exim, HTTPD, Java, Junos, Maven, Rails, Squid, and various firewalls (Cisco ASA, Shorewall, SuSE) have been moved into a new \patterns/legacy\ directory. The core \grok-patterns\ file has been updated to support broader character sets in email addresses, URIs, and months, while removing duplicate Apache log definitions now handled in the HTTPD file. Additionally, new patterns have been added for MongoDB 3.x logs, Nagios notification controls, Redis monitoring logs, and Syslog 5424 base formats, enhancing the ability to parse these specific log sources.

patterns/legacy · high confidence

The core patterns module has been refactored to support multiple pattern sets, specifically introducing an 'ecs-v1' path alongside the existing 'legacy' path. Users can now access ECS-compliant pattern definitions via the new path configuration, while the legacy patterns remain available for backward compatibility. This change enables Logstash to align with Elastic Common Schema standards for log parsing.

lib/logstash · high confidence

Added test coverage for AWS, Bacula, BIND, Bro/Zeek, Exim, and Firewall log patterns; Added test infrastructure for Grok filter validation.

The gemspec has been updated to depend on the new logstash-core-plugin-api (versions 1.60 to 2.99) instead of the legacy logstash gem, aligning with the Logstash 2.0 plugin architecture. The project also switched to using a version file for dynamic versioning, replaced git ls-files with explicit directory globs for file listing, and added development dependencies for logstash-devutils and logstash-filter-grok to support testing and local development workflows.

(dependencies) · high confidence

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

• First survey — no prior run to compare against. CAI 62.

• Code Health 98
• Architecture 69
• Maturity 55
• Readiness 62
• Security 74

• 185 commits — 159 feature/other, 26 fixes

• (root) — 81 commits
• spec/patterns — 32 commits
• patterns/grok-patterns — 24 commits
• patterns/firewalls — 9 commits
• (repo) — 4 commits
• patterns/java — 4 commits
• patterns/linux-syslog — 4 commits
• patterns/aws — 3 commits
• patterns/haproxy — 3 commits
• patterns/nagios — 3 commits
• patterns/s3 — 3 commits
• patterns/exim — 2 commits
• spec/spec_helper.rb — 2 commits
• patterns/bacula — 1 commit
• patterns/bind — 1 commit
• patterns/bro — 1 commit
• patterns/ecs-v1 — 1 commit
• patterns/httpd — 1 commit
• patterns/junos — 1 commit
• patterns/legacy — 1 commit

• fix: Added regression test to make sure that TIMEPERIOD TRANSITION doesn't end with a semi-colon (;)
• fix: Build: revert the need to use a git source (#302)
• fix: Chore: Fix broken link on readme. (#305)
• fix: Fix #93, add test that checks for matched values
• fix: Fix error with commit
• fix: Fix error with commit
• fix: Fix grokparsefailure due to truncated http_request.
• fix: Fix: HTTPD access log parse failure on missing response (#282)
• fix: Fix: Java stack trace's JAVAFILE to better match generated names (#272)
• fix: Fix: NAGIOS TIMEPERIOD unknown (from/to) field matching (#275)
• fix: Fix: UNIXPATH to avoid DoS on long paths with unmatching chars (#292)
• fix: Fix: Use greedy matching for UNIXPATH pattern
• fix: Fix: incorrect syslog (priority) field name (#303)
• fix: Fix: match Information/INFORMATION in LOGLEVEL (#274)
• fix: Fix: parsing x-edge-location in CLOUDFRONT_ACCESS_LOG (#311)
• fix: Fixed nagios timetransition pattern
• fix: Test: fix and add ECS compatibility specs (#310)
• fix: Version 0.1.10 bump, including a fix for the logstash_home issue
• fix: fix CISCOFW302013_302014_302015_302016 grok pattern (#313)
• fix: fix COMMONAPACHELOG specs
• …and 165 more

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

logstash-plugins/logstash-patterns-core was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.