Feature request: Could framing return properties in the order listed by the frame?

[cboettig]

Currently it appears that properties are sorted into alphabetical order after any JSON-LD operation (compaction, framing).

In the context of framing, this is sometimes a nice feature, since it means that after framing multiple input JSON files, the JSON data is at least in a consistent order.

I understand that ordering is semantically meaningless, but as framing exists to turn the graph (which could correspond to multiple different trees) into a predictable JSON tree as a convenience for developers, it seems natural to me that if an explicit ordering is given in the frame, that the algorithm would respect that order rather than alphabetize. For example, if my data is:

{
  "@context": "http://schema.org/",
  "@id": "document",
  "b": "text",
  "a": "more text"
 
}

Under the frame:

{
  "@context": "http://schema.org/",
  "@id":  "document",
  "b": {},
  "a": {}
}

(example in playground)

the returned document reverses the order of b and a (to be alphabetical), and not the order given in the frame. Framing is a really elegant way to specify the nesting order, but it would be nice for framing to also be able to dictate the ordering, so that the output data file really follows the exact structure of the given frame.

Related issue: there is no way to indicate that referenced nodes should occur before they are references (excluding circular references), which can be useful in streaming files. Having control of the node order via the frame would also give a mechanism to address that.

Hope this makes sense and apologies if I'm missing something fundamental here that makes alphabetizing the node order the only logical thing to do; or if I've misunderstood the expected behavior.

[gkellogg]

Framing involves expanding both the input document and the frame, and then re-compacting the document using the content included within the original frame document. Thus, ordering is alphabetical, as described in the compact algorithm. I'm afraid there's no option for preserving order (and that likely would not be portable in any case).

Sorry, same on the referenced nodes, ordering is determined by the compact algorithm.

[cboettig]

Thanks, yes, I realize this is already well documented as the current behavior of the compaction algorithm. Could you shed some light on why the algorithm uses an alphabetical sort though?

The docs have a comment about sort being a potential bottleneck in the speed as well.

[gkellogg]

Indeed, it does affect speed; I think it would be a reasonable enhancement to add an option to not order result keys when compacting.

As to why, IIRC, it was to ensure that results were repeatable across implementations. It may have also been to ensure that the result did not contain repeated keys, although there are other ways to ensure this. Perhaps @msporny, @dlongley, @lanthaler, or @niklasl can remember the original reasoning. In any case, it's proven a weakness for serializing larger datasets, due to the need to keep so much in memory.

I don't think we can preserve frame order universally, due to the fact that for many implementations, turning the JSON Object into an internal representation may not preserve this order. However, we could potentially have a feature that does not try to re-order properties when compacting. (Note that the expansion algorithm performs ordering as well, which is certainly not necessary when the results are going to be compacted or turned into Quads, which my implementation takes advantage of.

Reopening to consider adding a orderKeys option, defaulting to true, in the API; not sure how this would be tested, though.

[azaroth42]

In most JSON implementations, the keys in an object are an unordered set. There would need to be some framing property that gave the order explicitly, I believe.

[gkellogg]

Suggested resolution:

• Add an ordered option to Compaction, and other algorithms, defaulting to true, that would make the "ordered lexicographically" language depend on this option. (Note that, when compacting, expansion will already work properly is ordering is skipped).
• The Framing algorithm can then natively process options in the order specified in the frame, which would only have effect for systems which preserve order within objects. Interoperability suffers, but it is potentially useful for systems that don't preserve order for performance purposes, and for systems which do preserve order, adds an element of control for the look of the output.

[azaroth42]

Given that it's not possible to preserve the order using a large number of libraries from common languages, and previously design decisions have been made on whether there have been /any/ languages that could not implement (e.g. the empty keys in PHP before v7), I think this beyond the scope of JSON-LD. There's nothing wrong with custom key ordering, but we shouldn't require it of every implementation.

Secondly, I don't think that the proposal solves the requested functionality, where the ordering is not alphanumeric.

[gkellogg]

Ignoring the expectations, there is a value to making ordering optional, but perhaps not sufficient for the specification to define it. My implementation does, and I believe @dlongley's does as well, but that's as a practical issue for users of our respective libraries.

[gkellogg]

Transfered to the WG: w3c/json-ld-api#8.

[millercl]

json-stable-stringify has an opts.cmp for a custom sort order.

function frame_sort( frame ) {
  var properties = Object.keys( frame )
  return function ( a, b ) {
    var ia = properties.indexOf( a )
    var ib = properties.indexOf( b )
    return ia < ib ? 1 : -1
  }
}

var f = {
  "@context": "http://schema.org/",
  "@id": "document",
  "b": "text",
  "a": "more text"
}
var o = {
  "@context": "http://schema.org/",
  "@id":  "document",
  "b": {},
  "a": {}
}

var stringify = require( 'json-stable-stringify' )
stringify( o )
stringify( o, frame_sort( f ) )

But the browser debugger always displays lexicographic order. As it also does for CSS. Why hast thou turned from Unicode so soon?