Project

General

Profile

How to Create a Smart-Proxy Plugin » History » Version 19

Dominic Cleal, 05/05/2016 09:54 AM
add 1.12 warnings and improve readability

1 1 Anonymous
h1. How to Create a Smart-Proxy Plugin
2
3 4 Anonymous
This guide outlines main components of a plugin, but assumes some degree of familiarity with ruby gems, bundler, rack, and Sinatra. You'll find links to useful documentation in each of the sections below.
4 1 Anonymous
5 9 Dominic Cleal
{{toc}}
6
7 1 Anonymous
h2. Plugin Organization
8
9 11 Dominic Cleal
Smart-Proxy plugins are normal ruby gems, please follow documentation at http://guides.rubygems.org/make-your-own-gem/ for guidance on gem creation and packaging. It is strongly recommended to follow smart_proxy_<your plugin name here> naming convention for your plugin.
10 1 Anonymous
11 11 Dominic Cleal
We have some templates for creating your plugin:
12
13
* "smart_proxy_example plugin":https://github.com/theforeman/smart_proxy_example is a minimal example plugin that can be used as a skeleton
14
* "smart_proxy_dns_plugin_template":https://github.com/theforeman/smart_proxy_dns_plugin_template is a template for creating new DNS provider plugins
15
16
Also, "smart_proxy_pulp plugin":https://github.com/theforeman/smart-proxy-pulp is an example for a fully functional, yet easy to understand Smart-Proxy plugin.
17
18 6 Dominic Cleal
h2. Making your plugin official
19
20 1 Anonymous
Once you're ready to release the first version, please see [[How_to_Create_a_Plugin#Making-your-plugin-official]] for info on making your plugin part of the Foreman project.
21
22 11 Dominic Cleal
h2. Plugin definition
23 2 Anonymous
24 11 Dominic Cleal
A plugin definition is used to define plugin's name, version, location of rackup configuration, and other parameters. At a minimum, Plugin Descriptor must define plugin name and version. Note the base class of the descriptor is ::Proxy::Plugin:
25 3 Anonymous
26 1 Anonymous
<pre><code class='ruby'>
27 8 Anonymous
module Proxy::Example
28
  class Plugin < ::Proxy::Plugin
29 1 Anonymous
    plugin :example, "0.0.1"
30 8 Anonymous
    http_rackup_path File.expand_path("http_config.ru", File.expand_path("../", __FILE__))
31 1 Anonymous
    https_rackup_path File.expand_path("https_config.ru", File.expand_path("../", __FILE__))
32
    default_settings :hello_greeting => 'Hello there!', :important_path => '/must/exist'
33
    load_classes ::Proxy::Example::ClassLoader
34
    load_programmable_settings "::Proxy::Example::ProgrammableSettings"
35
    load_dependency_injection_wirings "::Proxy::Example::DIConfiguration"
36
    load_validators "Proxy::Example::CustomValidators"
37 8 Anonymous
    validate_readable :optional_path, :important_path
38 1 Anonymous
    validate :a_setting, :my_validator => true, :if => lambda {|settings| !settings[:a_setting].nil?}
39
    validate :another_setting, :my_other_validator => true
40
    start_services :service_a, :service_b
41
  end
42
end
43
</code></pre>
44
45 19 Dominic Cleal
Here we defined a plugin called "example", with version "0.0.1", that is going to listen on both http and https ports.
46 1 Anonymous
47 19 Dominic Cleal
h3. Full list of descriptor parameters
48
49
Following is the full list of parameters that can be defined by the Plugin Descriptor, and the version of the Smart Proxy that they were added in.
50
51
General smart proxy configuration parameters:
52
53 17 Anonymous
 * plugin :example, "1.2.3": *required*. Sets plugin name to "example" and version to "0.0.1".
54 19 Dominic Cleal
 * http_rackup_path "path/to/http_config.ru": _optional_, _1.6+_. Sets path to http rackup configuration. If omitted, the plugin is not going to listen on the http port. Please see below for information on rackup configuration.
55
 * https_rackup_path "path/to/https_config.ru": _optional_, _1.6+_. Sets path to https rackup configuration. If omitted, the plugin is not going to listen on the https port. Please see below for information on rackup configuration.
56
57
Loading and dependencies:
58
59
 * requires :another_plugin, '~> 1.2.0': _optional_, _1.6+_. Specifies plugin dependencies, where ":another_plugin" is another plugin name, and '~> 1.2.0' is version specification (pls. see http://guides.rubygems.org/patterns/#pessimistic_version_constraint for details on version specification).
60
 * bundler_group :my_plugin_group: _optional_, _1.6+_.  Sets the name of the bundler group for plugin dependencies. If omitted the plugin name is used.
61
 * after_activation { do_something }: _optional_, _1.6+_. Supplied block is going to be executed after the plugin has been loaded and enabled. Note that the block is going to be executed in the context of the Plugin Descriptor class.
62
 * load_classes: _1.12+_. must be a class or a block. Specified class must implement "load_classes" instance method that loads module's dependencies.
63
 * load_dependency_injection_wirings: can be a class or a class name. _1.12+_. The class must implement load_dependency_injection_wirings(di_container, settings_hash) instance method.
64
 * start_services: _1.12+_. list of dependency injection wiring labels. Services that perform work independently (asynchroniously) of http requests should implement #start method.
65
66
Settings related:
67
68
 * default_settings :first => 'my first setting', :another => 'my other setting': _optional_. _1.6+_. Defines default values for plugin parameters. These parameters can be overridden in plugin settings file. Setting any of the parameters in default_settings to nil will trigger a validation error.
69
 * load_programmable_settings: can be a class or a class name. _1.12+_. Specified class must implement load_programmable_settings(settings_hash) instance method that returns new or updated settings.
70
 * load_validators: can be a class or a class name. _1.12+_. Specified class must implement load_validators instance method that returns a hash of validator name to validator class mappings.
71
 * validate: validate :setting_one, :setting_two, ..., :setting_n, :validator_name => { :validator_param_one => 'value one', ...,}, :if => lambda {|settings| ... }, alternatively use :validator_name => true if validator has no parameters. If predicate is specified, the validator will be called only if the lambda evaluates to true. Predicate's lambda expects module's settings passed as a parameter. (_1.12+_)
72
 * validate_readable :optional_path, :important_path: _optional_, _1.10+_. Verifies that settings listed here contain paths to files that exist and are readable. Optional settings (not listed under default_settings) will be skipped if left uninitialized. 
73
74 1 Anonymous
h3. Provider definition
75
76
Some plugins are *providers* for an existing plugin or module in the Smart Proxy, e.g. a DNS provider.
77
78
These are registered almost identically, but use Proxy::Provider instead of Proxy::Plugin. No rackup_paths are used for providers, since they don't add any new REST API, they only add functionality to an existing module.
79
80
<pre>
81
module Proxy::Dns::PluginTemplate
82
  class Plugin < ::Proxy::Provider
83
    plugin :dns_plugin_template, ::Proxy::Dns::PluginTemplate::VERSION
84
85
    requires :dns, '>= 1.11'
86
87
    after_activation do
88
      require 'smart_proxy_dns_plugin_template/dns_plugin_template_main'
89
      require 'smart_proxy_dns_plugin_template/dns_plugin_template_dependencies'
90
    end
91
  end
92 11 Dominic Cleal
end
93 1 Anonymous
</pre>
94
95 11 Dominic Cleal
Additionally, each provider must specify which class implements interface expected by the main plugin. This is done by declaring an association for module's dependency injection container.
96 13 Anonymous
97 11 Dominic Cleal
<pre>
98
require 'dns_common/dependency_injection/dependencies'
99
100
class Proxy::Dns::DependencyInjection::Dependencies
101 13 Anonymous
  dependency :dns_provider, Proxy::Dns::PluginTemplate::Record
102 1 Anonymous
end
103 15 Dominic Cleal
</pre>
104 11 Dominic Cleal
105 19 Dominic Cleal
h2. Plugin Initialization
106
107
The initialization process can be thought of as consisting of two phases: loading and validation of settings and runtime initialization -- selection of classes, their parameters, and how they will be instantiated.
108
109
During the first phase of the process, all modules are gathered into groups consisting of the main plugin and one or more providers. If any of the members of the group fail at any time during initialization, the rest of the modules in the group will be failed as well. Initialization starts with all loaded and enabled plugin (main modules) classes being collected, then for each:
110
* configuration file is loaded
111
* dependencies are loaded (also see load_classes)
112
* configuration-related code executed, and configuration updated (also see load_runtime_configuration)
113
* validators executed (also see load_validators)
114
* provider names resolved to provider classes
115
116
At this point the steps above are repeated for all providers, one module group at a time. During the second phase, for each of the modules:
117
* dependency injection wirings are resolved (also see load_dependency_injection_wirings)
118
* services started (also see start_services)
119
* module's versions are checked against other modules stated requirements (also see)
120
121 1 Anonymous
h2. How to Load Dependencies
122 17 Anonymous
123 19 Dominic Cleal
_This technique requires Smart Proxy 1.12 or higher._
124
125 17 Anonymous
The class loader must implement load_classes instance method:
126
127
<pre>
128
class ::Proxy::Example::ClassLoader
129
  def load_classes
130
    require 'example/class_a'
131
    require 'example/class_b'
132
  end
133
end
134
</pre>
135 1 Anonymous
136
alternatively, a block can be used to load dependencies:
137
138
<pre>
139
module Proxy::Example
140 18 Anonymous
  class Plugin < ::Proxy::Plugin
141
    ...
142
    load_classes do
143
      require 'example/class_a'
144
      require 'example/class_b'
145
    end
146
    ...
147
  end
148
end
149
</pre>
150
151
h2. How to Programmatically Update Settings
152
153 19 Dominic Cleal
_This technique requires Smart Proxy 1.12 or higher._
154
155 17 Anonymous
The class must implement load_programmable_settings(settings_hash) instance method that returns new or updated settings:
156 1 Anonymous
157
<pre>
158
class ::Proxy::Example::RuntimeConfiguration
159 18 Anonymous
  def load_programmable_settings(settings_hash)
160 17 Anonymous
    settings_hash[:a_setting] = "Hello, world"
161
    settings_hash
162
  end
163 18 Anonymous
end
164 17 Anonymous
</pre>
165
166
h2. How to Define Dependency Injection Wirings
167
168 19 Dominic Cleal
_This technique requires Smart Proxy 1.12 or higher._
169
170 17 Anonymous
The class must implement load_dependency_injection_wirings instance method that has dependency injection container and settings hash as its parameters:
171
172
<pre>
173
class ::Proxy::Example::DIConfiguration
174
  def load_dependency_injection_wirings(container_instance, settings)
175
    container_instance.dependency :depedency_a, ::Proxy::Example::ClassA
176
    container_instance.dependency :dependency_b, ::Proxy::Example::ClassB
177
    container_instance.singleton_dependency :service_a, lambda {|container| ::Proxy::Example::ServiceA.new(settings[:example_setting], container_instance.get_dependency(:dependency_a))}
178
  end
179
end
180
</pre>
181
182
When Proxy::DependencyInjection::Container#dependency is used to define a dependency, a new instance of a class will be returned, or lambda executed every time the dependency is requested.
183
If only a single instance of a class is ever required, use Proxy::DependencyInjection::Container#singleton_dependency: the class will be instantiated first time the dependency is requested, and then reused on all subsequent requests. 
184
185
h2. How to Create and Load Custom Validators
186
187 19 Dominic Cleal
_This technique requires Smart Proxy 1.12 or higher._
188
189 17 Anonymous
The class used for loading of custom validators must implement load_validators that returns a hash of validator name to validator class mappings. A validator must use ::Proxy::PluginValidators::Base as its base class and implement validate!(settings) instance method that accepts a hash containing module settings. validate! should raise an exception if the check it's making fails.
190
191
<pre>
192
class Proxy::Example::CustomValidators
193
  class MyValidator < ::Proxy::PluginValidators::Base
194
    def validate!(settings)
195
      raise ::Proxy::Error::ConfigurationError("Unsupported greeting") if settings[@setting_name] != "Hello, world"
196
    end
197
  end
198
199
  def load_validators
200
    {:my_validator => MyValidator}
201
  end
202
end
203
</pre>
204
205 11 Dominic Cleal
206 4 Anonymous
h2. API
207 1 Anonymous
208 4 Anonymous
Modular Sinatra app is used to define plugin API. Note the base class Sinatra::Base and inclusion of ::Proxy::Helpers:
209 8 Anonymous
<pre><code class='ruby'>
210
module Proxy::Example
211 4 Anonymous
 class Api < Sinatra::Base
212
  helpers ::Proxy::Helpers
213
214 8 Anonymous
  get "/hello" do
215 4 Anonymous
    Proxy::Example::Plugin.settings.hello_greeting
216
  end
217
end
218
</code></pre>
219 1 Anonymous
220 4 Anonymous
Here we return a string defined in 'hello_greeting' parameter (see Plugin Descriptor above and settings file below) when a client performs a GET /hello. Please refer to "Sinatra documentation":http://www.sinatrarb.com/intro.html on details about routing, template rendering, available helpers, etc.
221
222
h2. Rackup Configuration
223
224
During startup Smart-Proxy assembles web applications listening on http and https ports using rackup files of enabled plugins. Plugin rackup files define mounting points of plugin API:
225
<pre><code class="ruby">
226
require 'example_plugin/example_api'
227
228 8 Anonymous
map "/example" do
229 1 Anonymous
  run Proxy::Example::Api
230 4 Anonymous
end
231
</code></pre>
232
233
The example above should be sufficient for the majority of plugins. Please refer to "Sinatra+Rack":http://www.sinatrarb.com/intro.html documentation for additional information. 
234
235
h2. Plugin Settings
236
237
On startup Smart-Proxy will load and parse plugin configuration files located in its settings.d/ directory. Each plugin config file is named after the plugin and is a yaml-encoded collection of key-value pairs and used to override default values of plugin parameters. 
238
<pre>
239
---
240
:enabled: true
241
:hello_greeting: "O hai!"
242
</pre>
243
244
This settings file enables the plugin (by default all plugins are disabled), and overrides :hello_greeting parameter. Plugin settings can be accessed through .settings method of the Plugin class, for example: ExamplePlugin.settings.hello_greeting. Global Smart-Proxy parameters are accessible through Proxy::SETTINGS, for example Proxy::SETTINGS.foreman_url returns Foreman url configured for this Smart-Proxy. 
245
246
h2. Bundler Configuration
247
248
Smart-Proxy relies on bundler to load its dependencies and plugins. We recommend to create a dedicated bundler config file for your plugin, and name it after the plugin. For example:
249
<pre><code class="ruby">
250
  gem 'smart_proxy_example'
251
  group :example do
252
    gem 'json'
253
  end
254 1 Anonymous
</code></pre>
255
 
256
You'll need to create a dedicated bundler group for additional dependencies of your plugin. By default the group shares the name with the plugin, but you can override it using bundler_group parameter in Plugin Descriptor. Please refer to [[How_to_Install_a_Smart-Proxy_Plugin]] for additional details on "from source" plugin installations.
257 11 Dominic Cleal
258
h2. Adding a DNS provider
259 15 Dominic Cleal
260 19 Dominic Cleal
_Requires Smart Proxy 1.11 or higher (1.10 has a slightly different interface.)_
261 16 Anonymous
262 11 Dominic Cleal
When extending the 'dns' smart proxy module, the plugin needs to create a new Proxy::Dns::Record class with @create_a_record@, @create_ptr_record@, @remove_a_record@, and @remove_ptr_record@ methods for adding and removing of DNS records.
263
264
The easiest way to do this is using the "Smart Proxy DNS plugin template":https://github.com/theforeman/smart_proxy_dns_plugin_template which can get you up and running with a new DNS provider plugin in minutes.
265 16 Anonymous
266 11 Dominic Cleal
DNS Provider classes are instantiated by DNS module's dependency injection container.
267
268 14 Anonymous
<pre>
269 11 Dominic Cleal
plugin :dns_plugin_template, ::Proxy::Dns::PluginTemplate::VERSION
270
</pre>
271
272 1 Anonymous
And then in the main file of the plugin:
273
274 14 Anonymous
<pre>
275
require 'dns_common/dns_common'
276 11 Dominic Cleal
277 1 Anonymous
module Proxy::Dns::PluginTemplate
278 11 Dominic Cleal
  class Record < ::Proxy::Dns::Record
279 1 Anonymous
    include Proxy::Log
280 11 Dominic Cleal
    include Proxy::Util
281 1 Anonymous
282 14 Anonymous
    def initialize
283
      super('localhost', ::Proxy::Dns::Plugin.settings.dns_ttl)
284 1 Anonymous
    end
285
286 14 Anonymous
    def create_a_record(fqdn, ip)
287
      # adds a forward 'A' record with fqdn, ip
288 1 Anonymous
    end
289
290 14 Anonymous
    def create_ptr_record(fqdn, ip)
291
      # adds a reverse 'PTR' record with ip, fqdn
292
    end
293
294
    def remove_a_record(fqdn)
295
      # removes the forward 'A' record with fqdn
296
    end
297
298
    def remove_ptr_record(ip)
299
      # removes the reverse 'PTR' record with ip
300 11 Dominic Cleal
    end
301
  end
302 1 Anonymous
end
303
</pre>
304
305
DNS provider support was first added in version 1.10, but the interface was updated between 1.10 and 1.11.  Please see the history of this page for 1.10-compatible recommendations and the 1.10-stable branch of the example DNS plugin instead of master.
306 16 Anonymous
307
h2. Adding a DHCP provider
308
309 19 Dominic Cleal
_Requires Smart Proxy 1.11 or higher._
310 16 Anonymous
311
When creating a new 'dhcp' provider smart proxy module, the plugin needs to create a new Proxy::DHCP::Server class that implements @load_subnets@, @load_subnet_data@, @find_subnet@, @subnets@, @all_hosts@, @unused_ip@, @find_record@, @add_record@, and @del_record@ methods.
312
313
Provider classes are instantiated by DHCP module's dependency injection container.
314
315
<pre>
316
plugin :example_dhcp_provider, ::ExampleDhcpPlugin::Provider::VERSION
317
</pre>
318
319
And then in the main file of the plugin:
320
321
<pre>
322
require 'dhcp_common/server'
323
324
module ::ExampleDhcpPlugin
325
  class Provider < ::Proxy::DHCP::Server
326
    include Proxy::Log
327
    include Proxy::Util
328
329
    def initialize
330
      super('localhost')
331
    end
332
333
    def load_subnets
334
      # loads subnet data into memory
335
    end
336
337
    def find_subnet(network_address)
338
      # returns Proxy::DHCP::Subnet that has network_address or nil if none was found
339
    end
340
341
    def load_subnet_data(a_subnet)
342
      # loads lease- and host-records for a Proxy::DHCP::Subnet
343
    end
344
345
    def subnets
346
      # returns all available subnets (instances of Proxy::DHCP::Subnet)
347
    end
348
349
    def all_hosts(network_address)
350
      # returns all reservations in a subnet with network_address
351
    end
352
353
    def unused_ip(network_address, mac_address, from_ip_address, to_ip_address)
354
      # returns first available ip address in a subnet with network_address, for a host with mac_address, in the range of ip addresses: from_ip_address, to_ip_address
355
    end
356
357
    def find_record(network_address, ip_or_mac_address)
358
      # returns a Proxy::DHCP::Record from a subnet with network_address that has ip- or mac-address specified in ip_or_mac_address, or nil of none was found 
359
    end
360
361
    def add_record(params)
362
      # creates a record
363
    end
364
365
    def del_record(network_address,a_record)
366
      # removes a Proxy::DHCP::Record from a subnet with network_address
367
    end
368
  end
369
end
370
</pre>
371
372
DHCP provider support was first added in version 1.11.
373 5 Anonymous
374
h2. Testing
375 1 Anonymous
376 9 Dominic Cleal
Make sure that Gemfile includes "smart-proxy" gem as a development dependency:
377 7 Anonymous
378
<pre><code class="ruby">
379
group :development do
380
  gem 'smart_proxy', :git => "https://github.com/theforeman/smart-proxy.git"
381
end
382
</code></pre>
383
384 5 Anonymous
Load 'smart_proxy_for_testing' in your tests:
385 1 Anonymous
386 12 Dominic Cleal
<pre><code>
387 5 Anonymous
$: << File.join(File.dirname(__FILE__), '..', 'lib')
388
389
require 'smart_proxy_for_testing'
390
require 'test/unit'
391
require 'webmock/test_unit'
392
require 'mocha/test_unit'
393
require "rack/test"
394
395
require 'smart_proxy_pulp_plugin/pulp_plugin'
396
require 'smart_proxy_pulp_plugin/pulp_api'
397
398
class PulpApiTest < Test::Unit::TestCase
399
  include Rack::Test::Methods
400
401
  def app
402
    PulpProxy::Api.new
403
  end
404
405
  def test_returns_pulp_status_on_200
406
    stub_request(:get, "#{::PulpProxy::Plugin.settings.pulp_url.to_s}/api/v2/status/").to_return(:body => "{\"api_version\":\"2\"}")
407
    get '/status'
408
409
    assert last_response.ok?, "Last response was not ok: #{last_response.body}"
410
    assert_equal("{\"api_version\":\"2\"}", last_response.body)
411
  end
412
end
413
</code></pre>
414
415 10 Anonymous
To execute all tests <code><pre>bundle exec rake test</code></pre>.  To save time during development it is possible to execute tests in a single file: <code><pre>bundle exec rake test TEST=path/to/test/file</pre></code>
416
417 1 Anonymous
Please refer to "Sinatra documention":http://www.sinatrarb.com/testing.html for detailed information on testing of Sinatra applications.
418 9 Dominic Cleal
419
Once you have tests, see "Jenkins":http://projects.theforeman.org/projects/foreman/wiki/Jenkins#Smart-proxy-plugin-testing for info on setting up tests under Jenkins.