Project

General

Custom queries

Profile

Actions

Feature #22666

open

Global Compute Profiles

Added by Shawn Q about 7 years ago. Updated about 7 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Compute resources
Target version:
-
Difficulty:
Triaged:
Fixed in Releases:
Found in Releases:

Description

Have a bunch of KVMs I'd like to set up as compute resources. However, all my Compute Profiles on one KVM will not propagate to the others. It's a manual process presently.

I've searched around a bit and only found the following relevant post:

https://community.theforeman.org/t/compute-profile/5192

Would rather give users the option to make Global Compute Profiles, that are inherited like the default compute profiles when a new KVM is added.
These global profiles would set a series of defaults for these compute profiles, such as CPU, RAM, Disk, and even interface bridge, etc.

Actions #1

Updated by Shawn Q about 7 years ago

Just something to add, I find this peculiar, as it seems Foreman API already has something like this in the forefront, without specifying a kvm node.

Any way we can get this to be the default in the UI? Foreman adds the Compute Profile name to any new KVM, but doesn't seem to copy the vm_attrs associated, see the following Ruby code for an example of the keys involved:

#!/usr/bin/env ruby

require 'rest-client'
require 'uri'
require 'json'

# User information:
user='admin'
pass='changeme'

hostgroup='5' # Change to your /api/hostgroups ID associated

# Get and encode the url variable for infoblox REST API call:
url = 'https://'+user+':'+pass+'@foreman-front/api/hostgroups/'+hostgroup+'?per_page=1000' # this really should be fixed for all pages
encoded_url = URI.encode(url)

# Get the REST response (header is necessary, verify_ssl is an issue has to be set to false in my case)
response = RestClient::Request.execute(:url => encoded_url, :method => :get, :verify_ssl => false, :headers => {accept: 'version=2,application/json'})

# Pull just the body into results:
results = response.body

# Make a list/array of the JSON parsing of results:
rlist = JSON.parse(results)['compute_profile_id'] # This is important, the compute_profile_id key

#puts rlist

# Get and encode the compute profiles from REST API:
url = 'https://'+user+':'+pass+'@foreman-front/api/compute_profiles/'+rlist.to_s+'?per_page=1000'
encoded_url = URI.encode(url)

# Get REST response:
response = RestClient::Request.execute(:url => encoded_url, :method => :get, :verify_ssl => false, :headers => {accept: 'version=2,application/json'})

# Pull body into results:
results = response.body

#puts results

# Make a list/array of the JSON parsing of results:
slist = JSON.parse(results)['compute_attributes'] # This appears to be the global profile key

#puts slist

slist.each do |row|
  #puts row['id']
  if row['id'] == rlist
    puts "CPUs  : "+row['vm_attrs']['cpus']
    puts "RAM   : "+row['vm_attrs']['memory']
    puts "Bridge: "+row['vm_attrs']['nics_attributes']['0']['bridge']
    puts "Disk  : "+row['vm_attrs']['volumes_attributes']['0']['capacity']
  end
end

# Example global profile vm_attrs results:
#vm_attrs"=>{"cpus"=>"2", "memory"=>"2147483648", "nics_attributes"=>{"0"=>{"type"=>"bridge", "bridge"=>"br10", "model"=>"virtio"}}, "volumes_attributes"=>{"0"=>{"pool_name"=>"default", "capacity"=>"12G", "allocation"=>"12G", "format_type"=>"raw"}}}

Actions #2

Updated by Justin Sherrill about 7 years ago

  • Project changed from Katello to Foreman
Actions #3

Updated by Ondřej Pražák about 7 years ago

  • Category changed from Web UI to Compute resources

Moving to compute resources category, this will need more than just changes in UI.

Actions #4

Updated by Shawn Q about 7 years ago

Ondřej Pražák wrote:

Moving to compute resources category, this will need more than just changes in UI.

Here's some sloppy code to help perform the actual global update, it's based on compute resource name 'my-kvmXXX' right now:

#!/usr/bin/env ruby

=begin
What are we doing?
We're taking a compute profile KVM hostname, getting the compute_profile_id for it
Then parsing compute_profile information for CPU DISK RAM / etc.
We're gathering a list of kvms
We're running PUT/POST REST calls to the API, to update all KVMs with those compute attributes
=end

## TODO: There's a problem handling exceptions in timeouts, see the following for bug:
=begin
Executing creation of compute_attribute on compute_profile 14 for compute_resource_id 23

./.gem/ruby/gems/rest-client-2.0.2/lib/restclient/request.rb:733:in `rescue in transmit': Timed out reading data from server (RestClient::Exceptions::ReadTimeout)
        from /home/ouroboros/.gem/ruby/gems/rest-client-2.0.2/lib/restclient/request.rb:642:in `transmit'
        from /home/ouroboros/.gem/ruby/gems/rest-client-2.0.2/lib/restclient/request.rb:145:in `execute'
        from /home/ouroboros/.gem/ruby/gems/rest-client-2.0.2/lib/restclient/request.rb:52:in `execute'
        from cprun.rb:151:in `block in <main>'
        from cprun.rb:128:in `each'
        from cprun.rb:128:in `<main>'
=end
## Suggested to put in a multi-try, else catch exception and just report it to the user continuing script

require 'rest-client' # gem install rest-client
require 'uri'
require 'json' # gem install json

# User information:
user='admin'
pass='changeme'
foreman='foreman.example.com'
ssl=false # we're setting this to false because we have internal issues with ssl

# Make sure user is passing proper parameters to this program:
if ARGV[0].nil? or ARGV[1].nil?
  puts "Usage: cprun.rb [compute_profile] [kvm_host]" 
  puts "Error: You need to specify the exact name of the Compute Profile you'd like to copy," 
  puts "       and which KVM host to copy it from (full FQDN)." 
  exit(1)
else
  profilename = ARGV[0]
  kvmhost = ARGV[1]
end

# Get and encode the url variable for REST API call for all compute_profiles
url = 'https://'+user+':'+pass+'@'+foreman+'/api/compute_profiles'
encoded_url = URI.encode(url)
# Get the REST response (header is necessary):
response = RestClient::Request.execute(:url => encoded_url, :method => :get, :verify_ssl => ssl, :headers => {accept: 'version=2,application/json'})
# Pull just the body into results:
results = response.body
# Make a list/array of the JSON parsing of results:
ylist = JSON.parse(results)['results']

# We're looking to see if that profile name actually exists, and if so, get its ID:
profile=nil
ylist.each do |row|
  if row['name'] == profilename
    profile = row['id'].to_s
  end
end

# If no profile was found, exit out:
if profile.nil?
  puts "Error: Profile name not found." 
  exit(1)
end

# Get and encode the url variable for REST API call to pull the compute_profile by ID:
url = 'https://'+user+':'+pass+'@'+foreman+'/api/compute_profiles/'+profile+'?per_page=1000'
encoded_url = URI.encode(url)
response = RestClient::Request.execute(:url => encoded_url, :method => :get, :verify_ssl => ssl, :headers => {accept: 'version=2,application/json'})
results = response.body

# Make a list/array of the JSON parsing of compute_attributes:
slist = JSON.parse(results)['compute_attributes']

# Global variable which will ultimately set our compute_attributes on all the kvms:
vmattrs = "" 

# Go through the compute_profile compute_attributes and get vm_attrs:
slist.each do |row|  
  if row['compute_resource_name'] == kvmhost
    vmattrs=row['vm_attrs']
    cpus=row['vm_attrs']['cpus']
    mem=row['vm_attrs']['memory']
    nics=row['vm_attrs']['nics_attributes']
    disk=row['vm_attrs']['volumes_attributes']
    puts "Found the following resources for "+kvmhost+" profile "+profilename+":" 
    puts "CPUs  : "+cpus
    puts "RAM   : "+mem
    puts "NICS  : "+nics.to_s
    puts "Disk  : "+disk.to_s  
  end
end

# Example we expect from vm_attrs:
#vm_attrs"=>{"cpus"=>"2", "memory"=>"2147483648", "nics_attributes"=>{"0"=>{"type"=>"bridge", "bridge"=>"br10", "model"=>"virtio"}}, "volumes_attributes"=>{"0"=>{"pool_name"=>"default", "capacity"=>"12G", "allocation"=>"12G", "format_type"=>"raw"}}}

# Get and encode the url variable for infoblox REST API call to retrieve all compute_resources:
url = 'https://'+user+':'+pass+'@'+foreman+'/api/compute_resources'
encoded_url = URI.encode(url)
response = RestClient::Request.execute(:url => encoded_url, :method => :get, :verify_ssl => false, :headers => {accept: 'version=2,application/json'})
results = response.body
slist = JSON.parse(results)['results']

servers = Array.new
# specifically parse for our kvms here:
slist.each do |row|
  #puts row
  ## ToDo: This is expecting a libvirt system, what if it's using VMWare or something else?
  s = row['url'].gsub('qemu+tcp://','').gsub('/system','')
  ## ToDo: This should be an option, it conflicts with publicizing this source code:
  if s.match("^my-kvm") ## ToDo: This is a direct compute resource name reference, needs to account for all KVMs, not just by name
    servers.push(row['id'].to_s)
  end
end

# Now we get our specific compute profile:
url = 'https://'+user+':'+pass+'@'+foreman+'/api/compute_profiles/'+profile
encoded_url = URI.encode(url)
# Get the REST response (header is necessary):
response = RestClient::Request.execute(:url => encoded_url, :method => :get, :verify_ssl => ssl, :headers => {accept: 'version=2,application/json'})
# Pull just the body into results:
results = response.body
#puts results
# Make a list/array of the JSON parsing of results:
clist = JSON.parse(results)
#clist['compute_attributes'].each do |row|
#  puts row
#end

# This is where we take the original compute profile vm_attrs and create a parameter hash to pass to REST during pushes
params = {
  compute_attribute: {
    :vm_attrs => vmattrs
  }
}

# For each compute_resource_id we know of from the earlier parsing:
servers.each do |server|  
  # The attrexists variable determines whether we are performing PUT or POST:
  attrexists = false
  compattrid = "" 
  # For each part of the compute_attributes in the source compute_profile, look to see if compute_resource_id already exists as a row:
  clist['compute_attributes'].each do |row|
    #puts row['compute_resource_id']
    if row['compute_resource_id'] == server.to_i
      compattrid = row['id'].to_s
      attrexists = true      
    end
  end
  if attrexists == true # if we found an existing compute_attributes_id for this compute_resource_id, we need to use PUT to update that compute_attribute_id
    url='https://'+user+':'+pass+'@'+foreman+'/api/compute_resources/'+server+'/compute_profiles/'+profile+'/compute_attributes/'+compattrid
    encoded_url = URI.encode(url)
    puts "\nExecuting update for compute_attribute_id "+compattrid+" on compute_profile "+profile+" for compute_resource_id "+server
    response = RestClient::Request.execute(:url => encoded_url, :method => :put, :payload => params.to_json, :verify_ssl => false, :headers => { :accept => :json, :content_type => :json })
    results = response.body    
    puts results
  else # In the event the compute_attributes_id doesn't exist for this compute_resource_id, we'll need to Post to create new compute_attribute
    url='https://'+user+':'+pass+'@'+foreman+'/api/compute_resources/'+server+'/compute_profiles/'+profile+'/compute_attributes'
    encoded_url = URI.encode(url)
    puts "\nExecuting creation of compute_attribute on compute_profile "+profile+" for compute_resource_id "+server
    response = RestClient::Request.execute(:url => encoded_url, :method => :post, :payload => params.to_json, :verify_ssl => false, :headers => { :accept => :json, :content_type => :json })
    results = response.body    
    puts results
  end
end
Actions

Also available in: Atom PDF