Fix bundling logic on download page

[allejo]

What the fudge is going on with the download page?

Alright, until I get around to completing #2, this PR will serve as the documentation of the expected behavior of bundles.

The Before: Django website downloads (aka API v1)

A POST /download request was treated as a trigger to download a bundle with the specified languages. This endpoint accepted Content-Type of type application/x-www-form-urlencoded where the keys were languages to be packaged (e.g. bash=1&cpp=1&javascript=1).

The Now: Next.js website downloads (aka API v2)

A POST /api/download request triggers a bundle download. This endpoint accepts a Content-Type of application/json where languages are listed in the languages string array.

{
  "api": 2,
  "languages": []
}

Were there any BC breaks with the new website?

No. The current /download page rewrites any requests matching API v1 expectations to be v2 compatible using the /api/download endpoint.

Nothing should have broken with the release of the new website. There have been no complaints since that haven't been addressed, so either no one cares, or it works as expected.

Now, what's in the download bundle?

Here's where things get a bit tricky. The overall structure of the downloaded ZIP between API v1 and v2 is almost identical, or at least shouldn't have any breaking changes.

Here is a non-exhaustive overview of a downloaded bundle's structure,

highlight.zip/
├─ es/
│  ├─ languages/
│  │  ├─ bash.js
│  │  ├─ bash.min.js
│  │  ├─ powershell.js
│  │  ├─ powershell.min.js
│  ├─ core.js
│  ├─ core.min.js
│  ├─ highlight.js
│  ├─ highlight.min.js
├─ languages/
│  ├─ bash.js
│  ├─ bash.min.js
│  ├─ powershell.js
│  ├─ powershell.min.js
├─ styles/
│  ├─ a11y-dark.css
│  ├─ a11y-dark.min.css
│  ├─ ...
├─ DIGESTS.md
├─ highlight.js
├─ highlight.min.js

• DIGESTS.md: contains the SHA384 hashes of each of the generated files that are included in the bundle
• es/
  • languages/: contains minified and non-minified versions of grammars in ESM format
  • highlight.js: the pure library in an ESM format; non-minified with NO grammars concatenated
  • highlight.min.js: a minified version of the highlight.js in this same folder
• languages/: contains both minified and non-minified variants of the grammars in CJS format
• highlight.js: the pure library WITH the selected grammars concatenated at the end in CJS format; this can be dropped on a web server and it'll work in the browser
• highlight.min.js: a minified version of the CJS version of highlight.js above

API v1 vs v2 differences

API v1 appears to have a long-standing bug of bundling up a pure version of highlight.js and ONLY concatenating grammars in the highlight.min.js. If a bundle is requested using V1 of the API, then the highlight.js will remain without the concatenated languages at the end for... legacy reasons.

API v2 is new and was just introduced with this Next.js website. I consider it a bug that the unminified highlight.js file does not have grammars concatenated at the end. Therefore, this has been changed in the V2 bundles.

---

Fixes #5

[netlify]

✅ Deploy Preview for hljs ready!

Name	Link
🔨 Latest commit	39be5e2
🔍 Latest deploy log	https://app.netlify.com/sites/hljs/deploys/655563a16f4243000877d535
😎 Deploy Preview	https://deploy-preview-10--hljs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[joshgoebel]

Oh that deploy preview stuff seems kinda nice.

[allejo]

@joshgoebel I've updated the PR with a wall of documentation of expected behavior. If you could download a bundle from the deploy preview and see if it all makes sense, that would be great! ❤️