Project

General

Profile

LogstashIntegration » History » Version 5

Lukas Zapletal, 11/16/2015 10:19 AM

1 5 Lukas Zapletal
h1. BitScout Integration
2 1 Lukas Zapletal
3 1 Lukas Zapletal
h2. Summary
4 1 Lukas Zapletal
5 5 Lukas Zapletal
Foreman orchestrates several other compoments (when Katello and other plugins are installed):
6 1 Lukas Zapletal
7 5 Lukas Zapletal
* DHCP
8 5 Lukas Zapletal
* DNS
9 5 Lukas Zapletal
* TFTP
10 5 Lukas Zapletal
* IPMI
11 5 Lukas Zapletal
* Puppet Master
12 5 Lukas Zapletal
* Puppet CA
13 5 Lukas Zapletal
* Compute Resources: libvirt, oVirt, VmWare, OpenStack, Docker...
14 5 Lukas Zapletal
* Candlepin
15 5 Lukas Zapletal
* Pulp, Crane
16 5 Lukas Zapletal
* Dynflow
17 5 Lukas Zapletal
* Cron jobs
18 5 Lukas Zapletal
* Other configuration management tools: Salt, Chef
19 1 Lukas Zapletal
20 5 Lukas Zapletal
Typical transaction of creating new host in Foreman involves:
21 1 Lukas Zapletal
22 5 Lukas Zapletal
* acquiring next free IP on DHCP server
23 5 Lukas Zapletal
* making DHCP reservation
24 5 Lukas Zapletal
* creating DNS record
25 5 Lukas Zapletal
* deploying PXELinux configuration, kernel and initramdisk on TFTP
26 5 Lukas Zapletal
* signing certs on Puppet CA
27 5 Lukas Zapletal
* contacting compute resrouce (e.g. oVirt) to configure and start the instance
28 5 Lukas Zapletal
* and more (depending on plugins installed)
29 5 Lukas Zapletal
30 5 Lukas Zapletal
All those components generate logs depending on their configuration and they all log to either files or syslog/journald. Centralised logging with metadata brings transparency as we currently struggle to correlate messages coming out of individual systems with orchestration runs, sessions or users. Possibility to pass the metadata IDs into subsystems would be big advantage. Possible correlation identifiers are:
31 5 Lukas Zapletal
32 5 Lukas Zapletal
* Foreman user id
33 5 Lukas Zapletal
* Foreman session id
34 5 Lukas Zapletal
* Foreman transaction id
35 5 Lukas Zapletal
* Foreman host IP address
36 5 Lukas Zapletal
* Foreman host MAC address
37 5 Lukas Zapletal
* Foreman host provisioning token
38 5 Lukas Zapletal
39 5 Lukas Zapletal
h2. Stories
40 5 Lukas Zapletal
41 5 Lukas Zapletal
As a Foreman user, I would like to collect logs from Smart Proxies and managed services at one place
42 5 Lukas Zapletal
As a Foreman user, I would like to extend the logs with metadata information
43 5 Lukas Zapletal
As a Foreman user, I would like to deploy log Dashboard and integrate Foreman with it
44 5 Lukas Zapletal
As a Dashboard user, I would like to browse Foreman logs and correlate them
45 5 Lukas Zapletal
As a Foreman user, I would like to be able to deploy Dashboard
46 5 Lukas Zapletal
47 3 Lukas Zapletal
h2. Owners
48 1 Lukas Zapletal
49 3 Lukas Zapletal
* Lukáš Zapletal (Foreman) - `lzap_at_redhat_dot_com`
50 3 Lukas Zapletal
51 3 Lukas Zapletal
h2. Current status
52 1 Lukas Zapletal
53 5 Lukas Zapletal
* Design WIP
54 1 Lukas Zapletal
* Not yet implemented
55 3 Lukas Zapletal
56 3 Lukas Zapletal
h2. Implementation
57 3 Lukas Zapletal
58 5 Lukas Zapletal
The implementation must build on top of tools which are available in RHEL 6 and 7 today. Looks like rsyslog/systemd would be the correct choice.
59 1 Lukas Zapletal
60 5 Lukas Zapletal
Foreman uses "logging" gem which is modular log4j-like logging framework with extensibility today. It allows integration with syslog at the moment.
61 1 Lukas Zapletal
62 5 Lukas Zapletal
TBD
63 1 Lukas Zapletal
64 1 Lukas Zapletal
h2. Required steps
65 1 Lukas Zapletal
66 5 Lukas Zapletal
TBD