Drivers:video:GC2145 - Wish there was more control over image resolution and windowing.

[KurtE]

Is your feature request related to a problem? Please describe.

Suppose, I am using a display that is not one of the three defined resolutions: 320x240, 640x480 or 1600x1200
Such as the Arduino GIGA display 480x800 or ILI9488/ST7796: 320x480 and I would like the image to fill the full screen, what should I do?

For example with the GIGA display shield, my current only choice is to read it in as 1600x1200 and crop the image myself.
However this approach requires a lot of resources, including memory and DCMI band width. That is you throw away about 80% of the image returned.

Note: This is not totally specific to the GC2145

Describe the solution you'd like

I wish that the zephyr code had some of the functionality that is contained in the Arduino library
https://github.com/ArduCAM/Arducam_dvp
Or better yet from my updated version of this library, that @mjs513 an myself extended in particular for the Teensy boards.
https://github.com/mjs513/Teensy_Camera

In particular: extend the table of resolutions: We currently have:

const int resolution[][2] = {
    {640, 480},   /* VGA       */
    {160, 120},   /* QQVGA     */
    {320, 240},   /* QVGA      */
    {480, 320},   /* ILI9488   */
    {320, 320},   /* 320x320   */
    {320, 240},   /* QVGA      */
    {176, 144},   /* QCIF      */
    {352, 288},   /* CIF       */
    {800, 600},   /* SVGA      */
    {1600, 1200}, /* UXGA      */
    {0, 0},
};

Also implement some form of zoom/clipping: The Arducam library has: GC2145::setResolutionWithZoom(int32_t resolution, int32_t zoom_resolution, uint32_t zoom_x, uint32_t zoom_y)

https://github.com/ArduCAM/Arducam_dvp/blob/master/src/GC2145/gc2145.cpp#L795-L891

Our version of it we have: GC2145::setWindow(uint16_t reg, uint16_t x, uint16_t y, uint16_t w,
uint16_t h) {
https://github.com/mjs513/Teensy_Camera/blob/main/src/Teensy_GC2145/GC2145.cpp#L1103-L1125

It appears like the current file zephyr\drivers\video\gc2145.c has most of the capabilities built in to it like the function:
gc2145_set_window, however I don't see any way to easily access it.

Potentially, maybe one or more new methods should be added the the Camera class?
That would then probably require changes to the video object, for each of the cameras.

Describe alternatives you've considered

No response

[josuah]

I would like the image to fill the full screen, what should I do?

Sensors sometimes have the ability to arbitrarily scale an image, however rare (OV5640 mostly).

1. Most sensors allow multiple factors of the full frame size to be output: 2x2 binning, 3x3 binning, 4x4 binning, etc. This is controlled by the sensor "native resolution".

2. On top of this, "windowing" (or "cropping", most sensors allow it) as well as "composing" (or "scaling", more rare like OV5640) can be performed as you observed.

gc2145_set_window(), however I don't see any way to easily access it.

This looks like what #91878 tries to acheive.

It introduces Linux-style "video selection" API to Zephyr: https://docs.kernel.org/userspace-api/media/v4l/selection-api.html

=> It seems like the way to implement this would be to first map all the native resolutions of the sensor in the driver (1.), then on top of this, implement the selection API on the GC2145 (2.).

More comments about it on #91975

Thank you for your interest and the effort of documenting and building these Arduino drivers!

[avolmat-st]

Indeed, that's the purpose of the video_set_selection API. This could be done at various locations within the pipeline.
Either at the sensor level, by defining a different window straight from the sensor, meaning only part of the image would be streamed to the receiver and displayed.
Either at the receiver side. In such case, depending on the receiver, if only "crop" is supported, then result would be quite similar to the case done at the sensor level, but if "compose" is also supported, something like downscale would be possible, hence not losing part of the image.

This really depends also on the sensor, sensor with integrated ISP might would themself be able to perform crop / compose. I did some work on the GC2145 (PR #91991) and crop/compose would most probably work quite easily by setting registers WINDOWS and OUT_WINDOWS, which I kind of understand as cropped area from the sensor, and generated frame resolution (I may misunderstanding this for the GC2145 since I haven't yet checked with lot of details those aspects).