[JENKINS-67829] compute the checksum per JVM #811
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
b415c7c to
c16b704
Compare
14 tasks
a78d8a4 to
8c94636
Compare
as we do not support dynamic class unloading a URL to a checksum can never change for the lifetime of the JVM. That is in Jenkins we can only install new plugins, upgrades always require a restart. To that end we the Checksum cache in JarLoaderImpl always is the checksum of the file on the local side so the only way it can change is if the JVM is restarted. Therefor we do not need to caclulate it for every channel, rather we can create it once for all Channels on a given JVM.
As access to CHECKSUMS and the computation of the Checksum was non-atomic we could still have lots of threads attempting to checksum the same jar file at startup. Whilst the jar would likely end up in the disk cache and IO would not be too high, the digest calculation would still occur causing wasted CPU that could be used for other things. Make the computation atomic wrt checking that we have the checksum
8c94636 to
84ce6e5
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
https://issues.jenkins.io/browse/JENKINS-67829
As we do not support dynamic class unloading the mappting between a
URLand aChecksumcan never change for the lifetime of the JVM. That is in Jenkins we can only install new plugins, upgrades always require a restart.To that end the
Checksumcache inJarLoaderImplalways is the checksum of the file on the local side so the only way it can change is if the JVM is restarted.Therefor we do not need to calculate it for every Channel, rather we can create it once for all Channels on a given JVM.
Testing done
Both of the above had some extra logging in
hudson.remoting.Checksum.forUrl(URL)to show when a checksum was created. Observed that without the each new agent / reconnection of the agent the method is called and with the path the method is called once per URL for the first connection since startup.ToDo: run the ATH suite, benchmark(?), stress tests(?)
Submitter checklist