Matt Moran wrote:

It would be beneficial to be able to clean up hostnames via a http call. We use EC2 quite a bit and being able to remove old hostnames from the Foreman when they are no longer needed/running automatically is a good benefit and a necessity.

1. security!
2. deletion might trigger additional actions - e.g. removal of dns records (at least when Foreman will manage it).

The reason behind this is that EC2 re-uses dns names, which can easily cause a vanilla server to pick up an old manifest when they are left lying around.

Is there a way we could integrate your EC2 initialization process with Foreman?
we would need to do it in any case if you want it to manage the certificates for you.

If this is something that you don't think belongs in the Foreman, is it possible to recommend the rows we need to remove from the DB to delete a host?

The Goal of Foreman is to do everything that we normally do manually so... ;)

1 - Yes security is an issue, but it is for most of Foreman's features, I guess Foreman needs a whole security/user management/roles thing - though to us this isn't an issue currently.
2 - DNS/DHCP etc. isn't a problem with EC2 nodes, and I think if you're deleting an EC2 node then you want to remove all reference to it. I accept with other use cases it may be different.

EC2 capabilities could certainly be integrated in to the Foreman, using it to initialise and manage EC2 nodes. This is a fair bit of work though, like all EC2 libraries the Foreman would need to support quite a few EC2 specific facilities. The certificates is an issue, at the moment we do something similar to what the Foreman does but with a Ruby script.

I've temporary hacked the Foreman code to not import EC2 nodes from facts. It would be great to remove this and manage the EC2 node with the Foreman, but I need away to clean up. The easiest way would be to poll for our EC2 instances and then remove any that exist in the Foreman with a HTTP call. This can then run as a cron job.

We can have 10s of EC2 nodes going up/down per day, which can equate to 100s of DNS names.

Do you think the Foreman should have an API? where we can remove/add nodes with the information required for the Foreman to serve them?

Matt Moran wrote:

1 - Yes security is an issue, but it is for most of Foreman's features, I guess Foreman needs a whole security/user management/roles thing - though to us this isn't an issue currently.

Yes,I'm working on it now

2 - DNS/DHCP etc. isn't a problem with EC2 nodes, and I think if you're deleting an EC2 node then you want to remove all reference to it. I accept with other use cases it may be different.

I guess we need to enable/disable certain functionality based on the host type. (e.g. make no sense to create dhcp entires for ec2)

EC2 capabilities could certainly be integrated in to the Foreman, using it to initialise and manage EC2 nodes. This is a fair bit of work though, like all EC2 libraries the Foreman would need to support quite a few EC2 specific facilities. The certificates is an issue, at the moment we do something similar to what the Foreman does but with a Ruby script.

whats missing on the certificate integration?
I would love to create some plugin architecture, were we could have a ec2 plugin... need to think about this a bit more :)

I've temporary hacked the Foreman code to not import EC2 nodes from facts. It would be great to remove this and manage the EC2 node with the Foreman, but I need away to clean up. The easiest way would be to poll for our EC2 instances and then remove any that exist in the Foreman with a HTTP call. This can then run as a cron job.

I'm still not sure why you had to hack the code? could you elaborate?

We can have 10s of EC2 nodes going up/down per day, which can equate to 100s of DNS names.

Do you think the Foreman should have an API? where we can remove/add nodes with the information required for the Foreman to serve them?

as Foreman is restful, you can already do it if you know the host id.
for workaround you can use the console (or a runner script) to destory the host.
I can extend the destroy method to accept hostnames instead of id's.

Ohad Levy wrote:

whats missing on the certificate integration?

For the EC2 nodes we just modify the autosign.conf like the Foreman does, until the first poll then remove. This could obviously handled by the Foreman if it had EC2 integration.

I would love to create some plugin architecture, were we could have a ec2 plugin... need to think about this a bit more :)

Yep - that would be cool.

I've temporary hacked the Foreman code to not import EC2 nodes from facts. It would be great to remove this and manage the EC2 node with the Foreman, but I need away to clean up. The easiest way would be to poll for our EC2 instances and then remove any that exist in the Foreman with a HTTP call. This can then run as a cron job.

I'm still not sure why you had to hack the code? could you elaborate?

We have the 'rake puppet:import:hosts_and_facts RAILS_ENV=production' running as a cron job, IIRC this imported all nodes that the puppetmaster knows about - so would import every EC2 node that was polling that puppetmaster. This created the EC2 nodes in the Foreman, these nodes were being managed by flat file manifests.

as Foreman is restful, you can already do it if you know the host id.
for workaround you can use the console (or a runner script) to destory the host.
I can extend the destroy method to accept hostnames instead of id's.

That would be great, adding of nodes would be pretty cool too.

Matt Moran wrote:

Ohad Levy wrote:

whats missing on the certificate integration?

For the EC2 nodes we just modify the autosign.conf like the Foreman does, until the first poll then remove. This could obviously handled by the Foreman if it had EC2 integration.

what does that mean? at the end, foreman just adds a entry in the autosign, and there is a url (check the kickstart for example) which tells foreman that the host has been installed, which removes the autosign entry.
so in this case you would just need to call that url when your EC2 servers are up.

We have the 'rake puppet:import:hosts_and_facts RAILS_ENV=production' running as a cron job, IIRC this imported all nodes that the puppetmaster knows about - so would import every EC2 node that was polling that puppetmaster. This created the EC2 nodes in the Foreman, these nodes were being managed by flat file manifests.

So really, your problem is that Foreman imports old node data.

I would suggest not to use the rack task (slow) and switch to the http post type, this allows both to collect data from multiple puppetmasters and will not import old data - see source:extras/puppet/foreman/files/push_facts.rb

you could probably just use: source:extras/puppet/foreman/manifests/import_facts.pp

After thinking this through, I guess the conclusion is that the Foreman at the moment is a node -> purpose specific tool.

If we start 20 EC2 nodes we then have to manage them individually in the Foreman. Maybe a multiple assign/selection might help, but we still have to distinguish them nodes from the other 100+ dynamic nodes we have running. To get around that we could import the EC2 security group or other data in the Foreman too to try and distinguish between what we want nodes to be. But this all feels a bit of a hack.

So the key is obviously that you know what purpose the nodes are going to be before you start them, you don't know mac or IP addresses, but you don't worry about that as that's what the cloud provider is for - you just want a bunch of DNS names, which are already in a pool that the cloud provider manages.

So I guess there is two ways. Either externally add remove the nodes and their purpose/details. Or as you said have a EC2/cloud/Eucalyptus plug in. The later sounds great, being able to say I want 10 machines that do this this and this and use this cloud provider.

Matt Moran wrote:

After thinking this through, I guess the conclusion is that the Foreman at the moment is a node -> purpose specific tool.

If we start 20 EC2 nodes we then have to manage them individually in the Foreman. Maybe a multiple assign/selection might help, but we still have to distinguish them nodes from the other 100+ dynamic nodes we have running. To get around that we could import the EC2 security group or other data in the Foreman too to try and distinguish between what we want nodes to be. But this all feels a bit of a hack.

So the key is obviously that you know what purpose the nodes are going to be before you start them, you don't know mac or IP addresses, but you don't worry about that as that's what the cloud provider is for - you just want a bunch of DNS names, which are already in a pool that the cloud provider manages.

So I guess there is two ways. Either externally add remove the nodes and their purpose/details. Or as you said have a EC2/cloud/Eucalyptus plug in. The later sounds great, being able to say I want 10 machines that do this this and this and use this cloud provider.

From my point of view, Foreman should always be the starting point when you want to deploy new hosts.
given, for EC2 management there is some more work to be done ;)

If you know how to generate the EC2 instances via some API, I would be happy to play with it, honestly, I had very little time with EC2.

I would assume that it would require some changes, like not relaying on the hostname as a key and using some UUID etc...

in anycase, does using the source:extras/puppet/foreman/files/push_facts.rb solves your current problem?

testing now

This seems to work. Though i've still hacked the puppet.rake file to not import any of the EC2 servers, until we can externally remove them when shutting them down.

Maybe it could be a feature to list host names we want to exclude from importing in to the foreman?

Matt Moran wrote:

This seems to work. Though i've still hacked the puppet.rake file to not import any of the EC2 servers, until we can externally remove them when shutting them down.

maybe I'm missing the point, but the fact update script (not the rake job) is uploading only new hosts to the database, doesn't it solve your problem?

(e.g. it will not upload old EC2 hosts).

Maybe it could be a feature to list host names we want to exclude from importing in to the foreman?

I guess one thing we could consider is a retention period, like in reports, where you can automatically purge old reports based on some time stamps.

I considering adding a disabled functionally to hosts (e.g. a host is a virtual machine that comes up only when required etc) - once a host is disabled, it will not show up in the default lists, and if new facts / reports arrive for this host, it will automatically be enabled again - would that work for you?

It solves the problem of deleting the node from the Foreman and not having it get imported again. But starting a 100 temporary instances a day that re-use DNSs - I don't really want to see any of them in the foreman, unless I can modify and remove the host via an API/REST call instead of using flat files.

I think that feature would be very beneficial, i'd also like to see the ability to not import DNSs via wildcard/regex would be great.

At the moment I just hack puppet.rake to not import host if it matches a EC2 DNS. I'd take this out if I could modify my EC2 scripts to add, remove, or expire the nodes in the Foreman.

Does this make sense?