Troubleshooting Lycophron installation and uploads — celery

[juwingert]

Summary

I've been trying to get Lycophron to work on Windows 11. Upon investigation, I found a solution that involved using 3 terminals (one to run celery worker, one for celery beat and another for lycophron proper).
Once I got through it, I got a .db file with TODOs for the listed files, but it doesn't progress beyond that.

Installing

Here's a rundown of the steps I did to finish my installation of Lycophron:

Steps to run Lycophron by Windows:

*Terminal 1: Start Celery Worker
$env:LYCOPHRON_CONFIG = "C:\Users\wingert_plazi\odonte\.lycophron\config.toml" $env:CELERY_BROKER_URL = "redis://localhost:6379/0" $env:CELERY_RESULT_BACKEND = "redis://localhost:6379/0" & "$env:USERPROFILE\pipx\venvs\lycophron\Scripts\celery.exe" -A lycophron.tasks worker --loglevel=info --pool=solo

*Terminal 2: Start Celery Beat
$env:LYCOPHRON_CONFIG = "C:\Users\wingert_plazi\odonte\.lycophron\config.toml" & "$env:USERPROFILE\pipx\venvs\lycophron\Scripts\celery.exe" -A lycophron.tasks beat --loglevel=info

*Terminal 3: Queue Records for Publishing
cd C:\Users\wingert_plazi\odonte

*I had to create another file to replace lycophron.cfg, it's .toml:
[zenodo]
access_token = "I put my token"
sandbox = true

[celery]
broker_url = "redis://localhost:6379/0"
result_backend = "redis://localhost:6379/0"

*Lycophron load my spreadsheet (in attached)
*I stopped when it generated the status as TODO in the .DB file

Example spreadsheet

This is the spreadsheet I used for my upload attempt
LycophronOdontesthesTemplate2.csv

[flsimoes]

We are now getting the following error:

sqlite3 lycophron.db "SELECT id, status FROM record;" Dyer_2006|DRAFT_FAILED PS C:\Users\wingert_plazi\test15> sqlite3 lycophron.db "SELECT id, status, error FROM record WHERE id='Dyer_2006';" Dyer_2006|DRAFT_FAILED|('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))

apparently it is a connection problem

[slint]

The corrected CSV import file is:

input_fixed.csv

Changes made (some of which Lycophron should validate)

• The community was changed from "BLR" to biosyslit
• access.files should be either empty (for open access) or restricted for closed access
• creators.name should be empty if creators.family_name
  • For institutions/organizations it should have a value and creators.family_name and creators.given_name should be empty
• ORCiD was fixed (but this should be Lycohpron validating)
• contributors.role.id was set to datacurator (full list of valid values at https://sandbox.zenodo.org/api/vocabularies/creatorsroles?size=100)
  • ⚠️ requires a fix in Lycophron code for it work. fix has been pushed
• Communities not included (even though configured correctly)

[juwingert]

Hi @slint , now we find the following error:

export.csv

I have already changed the token, but the error persists.

[flsimoes]

@slint

It seems that, in order to run Lycophron in Windows (PowerShell) you'll need to add/define celery to the base installation of lycophron
Otherwise, any user will have to go through the hassle of running celery and lycophron in a virtual environment, with both beat and worker running on separate instances of powershell

[flsimoes]

I managed to make lycophron run on Linux (Raspbian GNU/Linux 11)

The test uploads are all in sandbox. However, I get this never ending string of records being dispatched. Not sure what's going on, so i killed the process after some time running it. By the way, there's a typo in record-dispathcer

[slint]

It seems that, in order to run Lycophron in Windows (PowerShell) you'll need to add/define celery to the base installation of lycophron Otherwise, any user will have to go through the hassle of running celery and lycophron in a virtual environment, with both beat and worker running on separate instances of powershell

I found a workaround for this actually that will make it possible to run Lycophron on Windows as well. AFAIK, on fresh Windows 11 setup, I only had to install uv and git (which was needed for the uv tool install "..." installation command to work for GitHub).

I'll push this fix (along with the others that were discussed) and update the documenation.