Bug #8535
open
Libvirt over SSH instability
Description
Hi,
I've recently set up a Libvirt compute instance using qemu+ssh. This seemed to work fine for the most part but I'm now running into frequent issues like the following:
- VM power state on the host view page hangs on 'Loading power state...'
- Virtual Machine tab on the New Host page displays an error when the compute resource is selected
- Deleting a libvirt host leads to the host being deleted out of Foreman, but the VM itself isn't deleted
This is an intermittent issue, and the only way I've found of mitigating it at the moment is to either restart Foreman or kill any open SSH connections to the libvirt host.
Updated by Andy Taylor about 9 years ago
A quick update on this - after further investigation I found that the connection tracking feature of our office firewall was closing the SSH connections after 30 minutes of inactivity. I've switched our libvirt setup over to qemu+tls instead of qemu+ssh to get around this.
So this doesn't look to be a bug, but I wouldn't be surprised if other users ran into similar issues. Perhaps the SSH connection should be closed after a configurable time limit? Or perhaps just disconnect after commands have been executed and start new SSH connections for each command?
Updated by Lukas Zapletal about 9 years ago
Can't you just configure your ssh client in /usr/share/foreman-proxy/.ssh/config to ping every 10 seconds? I believe there is a feature for that in OpenSSH...
Updated by Andy Taylor about 9 years ago
Good point - I've had a look at the OpenSSH documentation and I think this is the setting you are referring to:
ServerAliveInterval
So that would probably sort the issue. I'm still not a big fan of the side effects of the SSH connection getting killed though (Foreman reporting a VM had been deleted when in fact it hadn't deleted it in libvirt).
Updated by Lukas Zapletal about 9 years ago
- Priority changed from Normal to Low
I understand, but since libvirt is mainly used for non-production setups and unless you intend to fix this for us, I am guessing this ticket will take some time to resolve. ;-)
At least the workaround is simple.
Updated by Dominic Cleal about 9 years ago
Lukas Zapletal wrote:
I understand, but since libvirt is mainly used for non-production setups
It's our second most popular CR, so I think it's widely used in production :-)