Entry compression in 3D Tiles packages

[javagl]

There is a proposal for a specification of the .3dtiles format at #727. This proposal currently says that the entries may be compressed. It would really be good to clarify what this means, and get this right, on the level of the specifcation and implementations.

---

One question is: Should all entries of .3dtiles be compressed?

There seem to be places where the entries are always compressed, and other places that involve assumptions about whether or not they are compressed (and how). And obviously, there are trade-offs. In many cases, compression can reduce the size considerably, which is important for efficient streaming. On the other hand, compressing specific kinds of data, like JPG or PNG (or Draco-compressed GLB), will hardly reduce the file size.

If entries should always be compressed:

• This may cause unnecessary processing overhead
• Should it always be GZIP? Or could it be DEFLATE, or ZSTD...?

If the entries should not always be compressed:

• How to decide which entries will be processed by producers?
  • Should it be based solely on the file type (i.e. file extension?)
  • Should they check if a GLB declares the KHR_draco_mesh_compression extension, and omit the compression in this case?
• How does the consumer detect whether an entry is compressed?
  • GZIP can be detected from "magic bytes". Other compression methods can not be detected from magic bytes...

---

Some of these questions will have to be aligned with the 3TZ format. This currently says

For optimal read performance, files in the archive should be stored without compression, however, for a good trade of read performance and file size, use the Zstandard compression method. For best compatibility with legacy software, choose the standard deflate compression method, noting that this is the slowest of the three methods.

For an implementation that is supposed to handle 3DTILES/3TZ/files transparently, a seemingly shallow (but important) question for the implementation is: What does the client receive?

// Read the "tileset.json":
const buffer = source.read("tileset.json");

// Write the "tileset.json"
target.write("tileset.json", buffer);

Does the buffer contain compressed or uncompressed data?

(Each answer will raise further questions for the read and write case, but I'll stop here for now...)

---

One aspect that may be relevant for the compression method (somewhat independent of whether (any) compression is used at all) is that of browser support.

One use-case for 3D Tiles packages will likely be that they are stored on a server that directly serves the content from these packages. In this case, the data from the package should be sent over the network. And one could make a strong case from that to prefer compression methods that can be used as the Content-Encoding. This currently includes gzip, compress, deflate and br (for Brotli).

^{(An aside: deflate does not really mean the DEFLATE compression method. See here for details...)}

One should be aware of the implications that possible decisions here may have on such a server. Specifically: When the server is extracting one "entry" from a package, will it

• be able to see that the data is compressed with the method that was requested, and send out the data directly?
• or have to to uncompress it (e.g. from ZSTD), and re-compress it (e.g. to GZIP) before sending it out?