Plugins » Katello

Can you try refreshing your subscription manifest and see if that has any effect on the results in the list? Thanks for the report!

I have the same issue and have refreshed the manifest a couple times. Tried both with simple content access enabled and without.

refresh of the manifest unfortunately brings no improvement. From time to time other release versions are shown but never completely

This is broken for the Katello 3.16.0. I think it should be at least cherry-picked back to 3.16 as its a very important issue.

In the meantime I have noticed that only redhat7 repositories are affected. redhat8 works.

Justin Sherrill wrote:

This is caused by an issue in dynflow, and fixed here: https://github.com/Dynflow/dynflow/pull/362

the fix isn't released yet though. You can try to apply the patch locally and restart all services, or enable the repo using hammer cli.

Applying the patch slightly improved things:
Now, on a foreman server that accesses the Internet through an HTTP proxy, for the first product selected all versions are shown, but trying to list versions for another product after enabling one of them leads to "No repositories available"
After reloading the page the versions for that product can be listed completely, so you now have to reload the page every time you extended a products version list and want to list another

Also, after reproducing the issue, can you go to Monitor > tasks and look for the most recent 'scan CDN' task. Are there any errors in it?

1) Enabling a repository seems to increase the frequency at which it happens, but after expanding multiple repos it eventually happens without too

2) When I wrote the first comment it happened every time, now its more sporadic

3) I'll have to try that still

4) the scan CDN tasks do not show any error

As Richard noted further up, this seems to only affect RHEL 7 repositories

I can now say it only happens with HTTP Proxy.

For completeness sake I also tried disabling Auth in squid, with the same result:
after a random number of expanded repos I get No repositories available.
The squid access log shows no connection errors either.

My reproducer is simply putting "rhel-7" in the search box and then expand a number of repos until the failure to load more occurs

Specifically /var/log/foreman/production.log

With the patch I get this in production.log:

2020-09-17T15:48:49 [I|app|] Completed 500 Internal Server Error in 7526ms (Views: 0.6ms | ActiveRecord: 15.5ms | Allocations: 41764)
2020-09-17T15:48:49 [E|app|] Failed at scanning for repository: could not obtain a connection from the pool within 5.000 seconds (waited 5.000 seconds); all pooled connections were in use

I also now see the scan CDN tasks ending up in warning state with

Exception:
ActiveRecord::ConnectionTimeoutError: could not obtain a connection from the pool within 5.000 seconds (waited 5.000 seconds); all pooled connections were in use

Would you be able to try this patch? https://github.com/Katello/katello/pull/8963

this limits the number of concurrent fetches to 5. If you still hit the issue, you may want to try lowering it. Putting it at 1 would basically make it sequential.

You'll need to restart the dynflow workers between each try.

After some testing I found that to reliably not run into the error even when extending multiple repos at the same time I had to decrease the group size down to 2.

Hello!, I updated the pr with another change. This change should cause the http proxy information being loaded prior to spawning threads. This should reduce the amount of db access that is needed within the threads. I also bumped up the thread count to 8 by default, but i'm curious again with this change what seems to work for you.

Thank you Justin, with the latest version of the patch it works reliable with your set group size

Awesome to hear! Thank you for the testing!