Page MenuHomePhabricator

Speed up CMake fixup bundle
Closed, ResolvedPublic


Current installer build times can reach staggering proportions as CMakes fixup step is anything but fast and will be done again and again for the same libraries (as noted e.g. in T22794). @maleike was kind enough to provide a custom BundleUtilities.cmake and GetPrerequisites.cmake which implement a caching functionality which speeds this process up significantly. We should consider integrating them.

Event Timeline

goch triaged this task as Wishlist priority.Mar 26 2018, 2:55 PM
goch created this task.

The provided files significantly speed up the analysis for each individual executable. However, they do need some adaption to our use case, as previously we started a new analysis for each executable and they automatically do the analysis for each executable found in the same folder. No using this automatic feature still results in a long (if now somewhat shorter) package build.

Good to have your feedback and I am glad that this fix could be helpful for your build. However, I do not fully understand what kind of adaption you would need. Just in case that your problem is that fixup_bundle() clears its cache on each call: I think it should be safe to remove the call to get_prerequisites_clear_cache() in fixup_bundle(). In the latest version the cached result is stored under a name using all parameters passed to get_prerequisites. That should be stable even withour clearing the cache. If this is not sufficient, I am curious to see your use case

Basically what you said. We have a number of executables in the /bin/ folder (however many BlueBerryApps and command line applications have been switched on). As we might have either one, several or none of each we need to trigger prerequisite collection for each of them. However for many applications (such as the twenty-odd diffusion command line app activated by a single CMake switch) that resulted in unbearably slow installer builds.
This has been an issue for example in T22794 where the solution for now was to just not add the prerequisite trigger for command line apps.

Just not clearing the cache would mean we still could have upward of thirty calls to GetPrerequisites, where each call would scan the same directory for all executables. This would probably still qualify as fast enough, so that might work for us. Alternatively we could add an additional cache for scanned directories, so the same scan is not triggered multiple times.

As far as I am aware we do not have any executables in any other directories (unless someone wants something arcane like a combined Release/Debug installer), so for us having a fix which just issues a single call for the bin directory irrespective of how many applications are built or not built would work. That workaround however would offend my sense of elegance.

@maleike Are you using the script on Linux as well? It seems to work fine on Windows, but on Linux it fails to copy relevant dependencies. Seems to be true for command line apps (which did not work previously, so no regression there) as well as e.g. the Workbench (which did work before).

@goch No, I did not test the script on other platforms than Windows yet. I am surprised to see differences here because I am not aware of modifications other than that caching mechanism that should be working independent of the operation system. If you go through the changes, you might look out to any regex expressions that might be Windows specific..

Hm, after a rebuild it seems to work.