Refactor encoders and decoders modules, add support for JPEG-LS encoding (pydicom#2015)

Author: scaramallion

README.md

@@ -10,18 +10,18 @@
 
 *pydicom* is a pure Python package for working with [DICOM](https://www.dicomstandard.org/) files.
 It lets you read, modify and write DICOM data in an easy "pythonic" way. As a pure Python package,
-*pydicom* can run anywhere Python runs without any other requirements, although if you're working 
+*pydicom* can run anywhere Python runs without any other requirements, although if you're working
 with *Pixel Data* then we recommend you also install [NumPy](https://numpy.org).
 
-Note that *pydicom* is a general-purpose DICOM framework concerned with 
-reading and writing DICOM datasets. In order to keep the 
+Note that *pydicom* is a general-purpose DICOM framework concerned with
+reading and writing DICOM datasets. In order to keep the
 project manageable, it does not handle the specifics of individual SOP classes
 or other aspects of DICOM. Other libraries both inside and outside the
-[pydicom organization](https://github.com/pydicom) are based on *pydicom* 
-and provide support for other aspects of DICOM, and for more 
+[pydicom organization](https://github.com/pydicom) are based on *pydicom*
+and provide support for other aspects of DICOM, and for more
 specific applications.
 
-Examples are [pynetdicom](https://github.com/pydicom/pynetdicom), which 
+Examples are [pynetdicom](https://github.com/pydicom/pynetdicom), which
 is a Python library for DICOM networking, and [deid](https://github.com/pydicom/deid),
 which supports the anonymization of DICOM files.
 
@@ -81,7 +81,7 @@ array([[175, 180, 166, ..., 203, 207, 216],
 #### JPEG, JPEG-LS and JPEG 2000
 Converting JPEG compressed *Pixel Data* to an ``ndarray`` requires installing one or more additional Python libraries. For information on which libraries are required, see the [pixel data handler documentation](https://pydicom.github.io/pydicom/stable/old/image_data_handlers.html#guide-compressed).
 
-Compressing data into one of the JPEG formats is not currently supported.
+Currently only JPEG-LS (with [pyjpegls](https://github.com/pydicom/pyjpegls)) is supported for compressing data.
 
 #### RLE
 Encoding and decoding RLE *Pixel Data* only requires NumPy, however it can
@@ -121,12 +121,12 @@ plt.show()
 
 ## Contributing
 
-We are all volunteers working on *pydicom* in our free time. As our 
-resources are limited, we very much value your contributions, be it bug fixes, new 
+We are all volunteers working on *pydicom* in our free time. As our
+resources are limited, we very much value your contributions, be it bug fixes, new
 core features, or documentation improvements. For more information, please
 read our [contribution guide](https://github.com/pydicom/pydicom/blob/main/CONTRIBUTING.md).
 
-If you have examples or extensions of *pydicom* that don't belong with the 
-core software, but that you deem useful to others, you can add them to our 
+If you have examples or extensions of *pydicom* that don't belong with the
+core software, but that you deem useful to others, you can add them to our
 contribution repository:
 [contrib-pydicom](https://www.github.com/pydicom/contrib-pydicom).

doc/conf.py

@@ -74,7 +74,7 @@
 
 autodoc_default_options = {
     "members": None,
-    "no-inherited-members": None,
+    "inherited-members": True,
 }
 
 # copybutton conf

doc/guides/encoding/encoder_plugin_options.rst

@@ -39,11 +39,26 @@ gdcm
 pylibjpeg
 =========
 
-+--------------------------+-----------------+------------------+-----------------------------+
-| Encoder                  | Options                                                          |
-+                          +-----------------+------------------+-----------------------------+
-|                          | Key             | Value            | Description                 |
-+==========================+=================+==================+=============================+
-|:attr:`RLELosslessEncoder`| ``'byteorder'`` | ``'<'``, ``'>'`` | The byte order of `src` may |
-|                          |                 |                  | be little- or big-endian    |
-+--------------------------+-----------------+------------------+-----------------------------+
++--------------------------------+-----------------+------------------+-----------------------------+
+| Encoder                        | Options                                                          |
++                                +-----------------+------------------+-----------------------------+
+|                                | Key             | Value            | Description                 |
++================================+=================+==================+=============================+
+|:attr:`RLELosslessEncoder`      | ``'byteorder'`` | ``'<'``, ``'>'`` | The byte order of `src` may |
+|                                |                 |                  | be little- or big-endian    |
++--------------------------------+-----------------+------------------+-----------------------------+
+
+
+.. _encoder_plugin_pyjpegls:
+
+pyjpegls
+========
+
++---------------------------------+-----------------+----------------------------+-----------------------------+
+| Encoder                         | Options                                                                    |
++                                 +-----------------+----------------------------+-----------------------------+
+|                                 | Key             | Value                      | Description                 |
++=================================+=================+============================+=============================+
+|:attr:`JPEGLSLosslessEncoder`    | (none available)                                                           |
+|:attr:`JPEGLSNearLosslessEncoder`|                                                                            |
++---------------------------------+-----------------+----------------------------+-----------------------------+

doc/guides/encoding/encoder_plugins.rst

@@ -4,6 +4,11 @@
 Pixel Data Encoder Plugins
 ==========================
 
+.. note::
+
+    This guide is intended for advanced users who need support for something
+    not provided by the :doc:`existing encoder plugins </reference/pixels.encoders>`.
+
 *Pixel Data* encoding in *pydicom* uses an :class:`~pydicom.pixels.encoders.base.Encoder`
 instance for the specific *Transfer Syntax* as a manager for plugins that
 perform the encoding work. This guide covers the requirements for those plugins
@@ -25,17 +30,26 @@ An encoding plugin must implement three objects within the same module:
 
   .. code-block:: python
 
-      def foo(src: bytes, **kwargs: Any) -> bytes:
+      def foo(src: bytes, runner: EncodeRunner) -> bytes | bytearray:
 
-  Where
+  Where:
 
   * `src` is the raw uncompressed data to be encoded as :class:`bytes`. When
-    the data in `src` represents multi-byte values
-    (such as 16-bit pixels), then `src` will use little-endian byte
-    ordering by default. Support for big-endian byte ordering by the encoding
-    function is completely optional.
-  * `kwargs` is a :class:`dict` which at a minimum contains the following
-    required keys:
+    the data in `src` represents multi-byte values (such as 16-bit pixels), then
+    `src` will use little-endian byte ordering by default. Support for big-endian
+    byte ordering by the encoding function is completely optional.
+
+    The data in `src` will be sized as:
+
+    * 1 byte per sample for 0 < *Bits Stored* <= 8
+    * 2 bytes per sample for 8 < *Bits Stored* <= 16
+    * 4 bytes per sample for 16 < *Bits Stored* <= 32
+    * 8 bytes per sample for 32 < *Bits Stored* <= 64
+
+  * `runner` is an :class:`~pydicom.pixels.encoders.base.EncodeRunner` instance
+    that manages the encoding process and has access to the encoding options,
+    either directly through the class properties or indirectly with the
+    :meth:`~pydicom.pixels.encoders.base.EncodeRunner.get_option` method.
 
     * ``'transfer_syntax_uid'``: :class:`~pydicom.uid.UID` - the intended
       *Transfer Syntax UID* of the encoded data.
@@ -59,15 +73,19 @@ An encoding plugin must implement three objects within the same module:
     * ``'photometric_interpretation'``: :class:`str` - the intended color space
       of the encoded data, such as ``'YBR_FULL'``
 
-    `kwargs` may also contain optional parameters intended to be used
-    with the encoder function to allow customization of the encoding process
-    or to provide additional functionality. Support for these optional
-    parameters is not required, however.
+    If your encoder needs to signal that one of the encoding option values needs
+    to be modified then this can be done with the
+    :meth:`~pydicom.pixels.encoders.base.EncodeRunner.set_option` method. This
+    should only be done after successfully encoding the frame, as if the
+    encoding fails changing the option value may cause issues with
+    other encoding plugins that may also attempt to encode the same frame. It's also
+    important to be aware that any changes you make will also affect following frames
+    (if any).
 
   At a minimum the encoding function must support the encoding of
   little-endian byte ordered data and should return the encoded
   data in a format meeting the requirements of the corresponding *Transfer
-  Syntax UID* as :class:`bytes`.
+  Syntax UID* as :class:`bytes` or :class:`bytearray`.
 
 * A function named ``is_available`` with the following signature:
 

doc/guides/encoding/index.rst

@@ -8,10 +8,11 @@ Pixel Data Encoding
 .. toctree::
    :maxdepth: 1
 
+   jpeg_ls
    rle_lossless
 
 
-Encoding plugin information: 
+Encoding plugin information:
 
 .. toctree::
    :maxdepth: 1