The new naming convention implemented in #19389 is not good, while it workarounds concurrency problems it creates unecessary amount of files. Here is how we could implement this:

• we can keep the current unique id approach in core (in Katello lets change from DBID to URL hash like in core)
• new code in proxy

• before anything is downloaded, do HTTP HEAD and acquire last-modified/etag sha1 hash
• then download the content to a filename SHA1 (or skip if the file already exists)
• then create a relative symlink FilenameFromForemanCore-XYZ -> SHA
• extend TFTP API with "expiration" flag - every created symlink would have expiration time, this would be stored as FILENAME.expire_at timestamp or text file

* expiration is set by Foreman core according to Token expiration (when it expires TFTP files are actually also invalid) - when set to zero no expiration is performed * change back naming convention sent from foreman to Operating Sytem, Architecture and Version (https://community.theforeman.org/t/pxe-files-kernel-initrd/15354) * there will be a cronjob running every day cleaning up expired symlinks * the cronjob can optionally delete SHA1 named (content) files which are "orphans" (have zero symlinks) and are older than 1 year

We can also use hardlinks instead of symlinks.

Problems:

• Foreman can download from any mirror, not all mirrors provide etag/last-modified. In that case, we'd need to redownload every time.