LogstashIntegration » History » Version 3
Foreman has many compoments involved and when Katello plugin is installed with it's orchestration capabilities as well as other plugins, possibility to effectively collect logs in a single place with flexible metadata and interface is a big plus.
Logstash open-source project was picked up as the target to integrate with. It is a Ruby modular framework (jRuby actually) with modular backends and inputs.
- Lukáš Zapletal (Foreman) - `lzap_at_redhat_dot_com`
- Design done
- Not yet implemented
Foreman uses logging gem which is modular log4j-like logging framework with great extensibility. Although direct integration via logging-logstash should be possible (https://rubygems.org/gems/logging-logstash/versions/0.0.2 tries to implement logstash API), integration via journald would provide better flexibility of logging from all other components (foreman-proxy, backend systems) or even scripts (seed, migrate, foreman_hooks) with support of all metadata. There is a POC journald reader for logstash available: https://github.com/stuart-warren/logstash-input-journald.
Using journald as a mediator avoids working with logstash completely and opens other possibilities with integration to other (even proprietary) logging solutions for those who need it.
This solution has limitation as it can only be used on systems with journald. For the initial version, this is not a problem and if this feature proves to be useful, we can add a fallback mode via syslog. This will not support custom metadata (see below) tho.
1. Extend Foreman logging framework with custom metadata (below)
1. Send session for each message
1. Send corelation ids for Host orchestration
1. Implement logging-journald plugin with support of custom metadata (using https://github.com/sandfox-im/journald-logger gem)
1. Document how to configure journald with logstash for remote collection
1. Hand over correlation id to smart proxy
1. Implement logging gem on smart proxy as well
1. Configure journald on smart-proxy machines via Puppet to send logs on Foreman node
Foreman would benefit from the following extra logging fields:
- Session ID: User session - usually a browser session if applicable.
- Transaction/Correlation ID - Usually UUID of the transaction that the logs belong to. This way we can correlate logs from orchestration runs for all services that are under our control: Foreman, Foreman-Proxy, Plugins, Candlepin, Pulp