[rubygems/rubygems] Reduce allocations when parsing compact index

This still allocates a ton (a string for each line, plus a bunch of
splits into arrays), but it helps a bit when Bundler has to go through
dependency resolution.

```
==> memprof.after.txt <==
Total allocated: 194.14 MB (2317172 objects)
Total retained:  60.81 MB (593164 objects)

==> memprof.before.txt <==
Total allocated: 211.97 MB (2404890 objects)
Total retained:  62.85 MB (640342 objects)
```

https://github.com/rubygems/rubygems/commit/c68b41b0e5

1 files changed, 6 insertions, 1 deletions

diff --git a/lib/bundler/compact_index_client/updater.rb b/lib/bundler/compact_index_client/updater.rb
index 0f7bf9bb50..3b75d5c129 100644
--- a/lib/bundler/compact_index_client/updater.rb
+++ b/lib/bundler/compact_index_client/updater.rb
@@ -95,7 +95,12 @@ module Bundler
         # because we need to preserve \n line endings on windows when calculating
         # the checksum
         SharedHelpers.filesystem_access(path, :read) do
-          SharedHelpers.digest(:MD5).hexdigest(File.read(path))
+          File.open(path, "rb") do |f|
+            digest = SharedHelpers.digest(:MD5).new
+            buf = String.new(:capacity => 16_384, :encoding => Encoding::BINARY)
+            digest << buf while f.read(16_384, buf)
+            digest.hexdigest
+          end
         end
       end