Skip to content

set-exif timeout with mass edit of multi-gb files #1461

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
deanjarnold opened this issue Jun 8, 2025 · 1 comment
Open

set-exif timeout with mass edit of multi-gb files #1461

deanjarnold opened this issue Jun 8, 2025 · 1 comment
Labels
needs triage To be triaged

Comments

@deanjarnold
Copy link

deanjarnold commented Jun 8, 2025

Describe the bug

When mass editing EXIF information for large multi-gb files the process times out, the EXIF values are not updated and temporary files (suffix exiftool_tmp) remain in the folder.

Steps To Reproduce

  1. Have multiple (10+) large Multi GB (2Gb+) MOV files in a folder within NextCloud/Memories Folder view.
  2. Select all of the files.
  3. From ... select "Edit metadata".
  4. Change a value e.g. year
  5. Select Save
  6. Process bar will move.
  7. Timeout notifications will appear in top right of nextcloud/memories UI.
  8. Process completes, but some files exif information is unchanged.
  9. Files with an extension suffix of "exiftool_tmp" will exist in the folder the files are located.
  10. Logs will containe timout errors.

Platform

- Server OS: Ubuntu 24.04.2 LTS
- Client OS: MacOS 15.5
- Browser: Chrome 136.0.7103.114
- Memories Version: 7.5.2
- Nextcloud Version: 30.0.11 (AIO 10.15.0)
- PHP Version: 8.3.21

Screenshots

Temp Files with exiftool_tmp siffix
Image

Timeout Errors in UI
Image

Additional context

Nextcloud Timeout Error Message (1 for every file that timesout)
{"reqId":"MPR0wHA3bvMzU7KbBQiH","level":3,"time":"2025-06-08T22:27:45+00:00","remoteAddr":"98.210.201.152","user":"darnold","app":"no app in context","method":"PATCH","url":"/apps/memories/api/image/set-exif/77029223","message":"Timeout","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/136.0.0.0 Safari/537.36","version":"30.0.11.1","data":{"trace":"[{\"file\":\"/var/www/html/custom_apps/memories/lib/Util.php\",\"line\":505,\"function\":\"readOrTimeout\",\"class\":\"OCA\\Memories\\Util\",\"type\":\"::\",\"args\":[\"[unknown(resource (closed))]\",30000]},{\"file\":\"/var/www/html/custom_apps/memories/lib/Util.php\",\"line\":474,\"function\":\"execSafe2\",\"class\":\"OCA\\Memories\\Util\",\"type\":\"::\",\"args\":[[\"/tmp/exiftool-oc0zd2uegi9v-12.70-c435c63a\",\"-overwrite_original_in_place\",\"-n\",\"-api\",\"LargeFileSupport=1\",\"-json=-\",\"/extmnt/BraveCreative/x_New_Folder_Structure/Video/2025/UPDATED Campus B-roll/Dublin/NINJAV_S001_S001_T362.MOV\"],30000,\"[{\\\"AllDates\\\":\\\"2025:02:01 03:13:24\\\",\\\"SourceFile\\\":\\\"\\/extmnt\\/BraveCreative\\/x_New_Folder_Structure\\/Video\\/2025\\/UPDATED Campus B-roll\\/Dublin\\/NINJAV_S001_S001_T362.MOV\\\"}]\",true,false]},{\"file\":\"/var/www/html/custom_apps/memories/lib/Exif.php\",\"line\":351,\"function\":\"execSafe\",\"class\":\"OCA\\Memories\\Util\",\"type\":\"::\",\"args\":[[\"/tmp/exiftool-oc0zd2uegi9v-12.70-c435c63a\",\"-overwrite_original_in_place\",\"-n\",\"-api\",\"LargeFileSupport=1\",\"-json=-\",\"/extmnt/BraveCreative/x_New_Folder_Structure/Video/2025/UPDATED Campus B-roll/Dublin/NINJAV_S001_S001_T362.MOV\"],30000,\"[{\\\"AllDates\\\":\\\"2025:02:01 03:13:24\\\",\\\"SourceFile\\\":\\\"\\/extmnt\\/BraveCreative\\/x_New_Folder_Structure\\/Video\\/2025\\/UPDATED Campus B-roll\\/Dublin\\/NINJAV_S001_S001_T362.MOV\\\"}]\"]},{\"file\":\"/var/www/html/custom_apps/memories/lib/Exif.php\",\"line\":379,\"function\":\"setExif\",\"class\":\"OCA\\Memories\\Exif\",\"type\":\"::\",\"args\":[\"/extmnt/BraveCreative/x_New_Folder_Structure/Video/2025/UPDATED Campus B-roll/Dublin/NINJAV_S001_S001_T362.MOV\",{\"AllDates\":\"2025:02:01 03:13:24\",\"SourceFile\":\"/extmnt/BraveCreative/x_New_Folder_Structure/Video/2025/UPDATED Campus B-roll/Dublin/NINJAV_S001_S001_T362.MOV\"}]},{\"file\":\"/var/www/html/custom_apps/memories/lib/Controller/ImageController.php\",\"line\":284,\"function\":\"setFileExif\",\"class\":\"OCA\\Memories\\Exif\",\"type\":\"::\",\"args\":[{\"[object] (OC\\Files\\Node\\File)\":[]},{\"AllDates\":\"2025:02:01 03:13:24\"}]},{\"file\":\"/var/www/html/custom_apps/memories/lib/UtilController.php\",\"line\":20,\"function\":\"OCA\\Memories\\Controller\\{closure}\",\"class\":\"OCA\\Memories\\Controller\\ImageController\",\"type\":\"->\",\"args\":[]},{\"file\":\"/var/www/html/custom_apps/memories/lib/Controller/ImageController.php\",\"line\":267,\"function\":\"guardEx\",\"class\":\"OCA\\Memories\\Util\",\"type\":\"::\",\"args\":[{\"[object] (Closure)\":[]}]},{\"file\":\"/var/www/html/lib/private/AppFramework/Http/Dispatcher.php\",\"line\":208,\"function\":\"setExif\",\"class\":\"OCA\\Memories\\Controller\\ImageController\",\"type\":\"->\",\"args\":[77029223,{\"AllDates\":\"2025:02:01 03:13:24\"}]},{\"file\":\"/var/www/html/lib/private/AppFramework/Http/Dispatcher.php\",\"line\":114,\"function\":\"executeController\",\"class\":\"OC\\AppFramework\\Http\\Dispatcher\",\"type\":\"->\",\"args\":[{\"[object] (OCA\\Memories\\Controller\\ImageController)\":[]},\"setExif\"]},{\"file\":\"/var/www/html/lib/private/AppFramework/App.php\",\"line\":161,\"function\":\"dispatch\",\"class\":\"OC\\AppFramework\\Http\\Dispatcher\",\"type\":\"->\",\"args\":[{\"[object] (OCA\\Memories\\Controller\\ImageController)\":[]},\"setExif\"]},{\"file\":\"/var/www/html/lib/private/Route/Router.php\",\"line\":303,\"function\":\"main\",\"class\":\"OC\\AppFramework\\App\",\"type\":\"::\",\"args\":[\"OCA\\Memories\\Controller\\ImageController\",\"setExif\",{\"[object] (OC\\AppFramework\\DependencyInjection\\DIContainer)\":[]},{\"id\":\"77029223\",\"_route\":\"memories.image.setexif\"}]},{\"file\":\"/var/www/html/lib/base.php\",\"line\":1010,\"function\":\"match\",\"class\":\"OC\\Route\\Router\",\"type\":\"->\",\"args\":[\"/apps/memories/api/image/set-exif/77029223\"]},{\"file\":\"/var/www/html/index.php\",\"line\":24,\"function\":\"handleRequest\",\"class\":\"OC\",\"type\":\"::\",\"args\":[]}]"},"id":"684610cc3889d"}

@deanjarnold deanjarnold added the needs triage To be triaged label Jun 8, 2025
@deanjarnold
Copy link
Author

deanjarnold commented Jun 8, 2025

Appears EXIFTOOL_TIMEOUT timeout is 30sec (30000ms) defined a s constant in Exif.php.

To work around I changed the EXIFTOOL_TIMEOUT to 120sec (120000ms) and the timeout no longer occurs.

The issues appears to a combination of the file size and the number of concurrent and enqueued set-exif processes launched by the mass edit.

@deanjarnold deanjarnold changed the title set-exif timeout with multi-gb files set-exif timeout with mass edit of multi-gb files Jun 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs triage To be triaged
Projects
None yet
Development

No branches or pull requests

1 participant