Problem installing polars_1xx branch, possible issue through pybigtools [BUG] #197
Description
I want to install the polars_1xx branch to process a large dataset >100k cells, but am having problems installing pycisTopic. It seems the issue may be from installing pybigtools.
I followed these steps to install the polars_1xx branch in a conda env:
$ conda create --name pycistopicpolars python=3.11
$ conda install -n pycistopicpolars -c conda-forge rust=1.82.0
$ git clone https://github.com/aertslab/pycisTopic.git
$ git checkout polars_1xx
$ pip install .
But got this error:
Installing build dependencies ... done
Checking if build backend supports build_editable ... done
Getting requirements to build editable ... done
Preparing editable metadata (pyproject.toml) ... done
Collecting numpy<2,>=1.20.3 (from pycisTopic==2.0a0)
Using cached numpy-1.26.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (61 kB)
Collecting pandas==1.5 (from pycisTopic==2.0a0)
Using cached pandas-1.5.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (11 kB)
Collecting polars>=1 (from pycisTopic==2.0a0)
Using cached polars-1.18.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (14 kB)
Collecting pyarrow>=8.0.0 (from pycisTopic==2.0a0)
Using cached pyarrow-18.1.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.3 kB)
Collecting pyranges<0.0.128 (from pycisTopic==2.0a0)
Using cached pyranges-0.0.127-py3-none-any.whl.metadata (15 kB)
Collecting numba (from pycisTopic==2.0a0)
Using cached numba-0.60.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (2.7 kB)
Collecting ray (from pycisTopic==2.0a0)
Using cached ray-2.40.0-cp311-cp311-manylinux2014_x86_64.whl.metadata (17 kB)
Collecting scatac_fragment_tools<0.2.0 (from pycisTopic==2.0a0)
Using cached scatac_fragment_tools-0.1.3-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.8 kB)
Collecting scikit-learn (from pycisTopic==2.0a0)
Using cached scikit_learn-1.6.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (18 kB)
Collecting lda (from pycisTopic==2.0a0)
Using cached lda-3.0.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (7.6 kB)
Collecting matplotlib<3.7 (from pycisTopic==2.0a0)
Using cached matplotlib-3.6.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.8 kB)
Collecting tmtoolkit (from pycisTopic==2.0a0)
Using cached tmtoolkit-0.12.0-py3-none-any.whl.metadata (14 kB)
Collecting gensim (from pycisTopic==2.0a0)
Using cached gensim-4.3.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (8.1 kB)
Collecting macs2 (from pycisTopic==2.0a0)
Using cached MACS2-2.2.9.1-cp311-cp311-linux_x86_64.whl
Collecting pybiomart (from pycisTopic==2.0a0)
Using cached pybiomart-0.2.0-py3-none-any.whl.metadata (1.1 kB)
Collecting pyscenic (from pycisTopic==2.0a0)
Using cached pyscenic-0.12.1-py3-none-any.whl.metadata (9.8 kB)
Collecting scrublet (from pycisTopic==2.0a0)
Using cached scrublet-0.2.3-py3-none-any.whl.metadata (3.1 kB)
Collecting harmonypy (from pycisTopic==2.0a0)
Using cached harmonypy-0.0.10-py3-none-any.whl.metadata (3.9 kB)
Collecting igraph (from pycisTopic==2.0a0)
Using cached igraph-0.11.8-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting leidenalg (from pycisTopic==2.0a0)
Using cached leidenalg-0.10.2-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (10 kB)
Collecting seaborn (from pycisTopic==2.0a0)
Using cached seaborn-0.13.2-py3-none-any.whl.metadata (5.4 kB)
Collecting adjustText (from pycisTopic==2.0a0)
Using cached adjustText-1.3.0-py3-none-any.whl.metadata (3.1 kB)
Collecting statsmodels (from pycisTopic==2.0a0)
Using cached statsmodels-0.14.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (9.2 kB)
Collecting scanpy (from pycisTopic==2.0a0)
Using cached scanpy-1.10.4-py3-none-any.whl.metadata (9.3 kB)
Collecting scanorama (from pycisTopic==2.0a0)
Using cached scanorama-1.7.4-py3-none-any.whl.metadata (525 bytes)
Collecting bbknn (from pycisTopic==2.0a0)
Using cached bbknn-1.6.0-py3-none-any.whl.metadata (8.2 kB)
Collecting loomxpy (from pycisTopic==2.0a0)
Using cached loomxpy-0.4.2-py3-none-any.whl.metadata (1.6 kB)
Collecting bs4 (from pycisTopic==2.0a0)
Using cached bs4-0.0.2-py2.py3-none-any.whl.metadata (411 bytes)
Collecting lxml (from pycisTopic==2.0a0)
Using cached lxml-5.3.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting python-dateutil>=2.8.1 (from pandas==1.5->pycisTopic==2.0a0)
Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB)
Collecting pytz>=2020.1 (from pandas==1.5->pycisTopic==2.0a0)
Using cached pytz-2024.2-py2.py3-none-any.whl.metadata (22 kB)
Collecting contourpy>=1.0.1 (from matplotlib<3.7->pycisTopic==2.0a0)
Using cached contourpy-1.3.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.4 kB)
Collecting cycler>=0.10 (from matplotlib<3.7->pycisTopic==2.0a0)
Using cached cycler-0.12.1-py3-none-any.whl.metadata (3.8 kB)
Collecting fonttools>=4.22.0 (from matplotlib<3.7->pycisTopic==2.0a0)
Using cached fonttools-4.55.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (165 kB)
Collecting kiwisolver>=1.0.1 (from matplotlib<3.7->pycisTopic==2.0a0)
Using cached kiwisolver-1.4.8-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.2 kB)
Collecting packaging>=20.0 (from matplotlib<3.7->pycisTopic==2.0a0)
Using cached packaging-24.2-py3-none-any.whl.metadata (3.2 kB)
Collecting pillow>=6.2.0 (from matplotlib<3.7->pycisTopic==2.0a0)
Using cached pillow-11.0.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (9.1 kB)
Collecting pyparsing>=2.2.1 (from matplotlib<3.7->pycisTopic==2.0a0)
Using cached pyparsing-3.2.0-py3-none-any.whl.metadata (5.0 kB)
Collecting ncls>=0.0.63 (from pyranges<0.0.128->pycisTopic==2.0a0)
Using cached ncls-0.0.68-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.6 kB)
Collecting tabulate (from pyranges<0.0.128->pycisTopic==2.0a0)
Using cached tabulate-0.9.0-py3-none-any.whl.metadata (34 kB)
Collecting sorted-nearest>=0.0.33 (from pyranges<0.0.128->pycisTopic==2.0a0)
Using cached sorted_nearest-0.0.39-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (964 bytes)
Collecting natsort (from pyranges<0.0.128->pycisTopic==2.0a0)
Using cached natsort-8.4.0-py3-none-any.whl.metadata (21 kB)
Collecting pybigwig (from scatac_fragment_tools<0.2.0->pycisTopic==2.0a0)
Using cached pyBigWig-0.3.23-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (19 kB)
Collecting pybigtools>=0.2.0 (from scatac_fragment_tools<0.2.0->pycisTopic==2.0a0)
Using cached pybigtools-0.2.3.tar.gz (9.5 MB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... error
error: subprocess-exited-with-error
× Preparing metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [22 lines of output]
Updating crates.io index
Locking 15 packages to latest compatible versions
Adding base64 v0.21.7 (latest: v0.22.1)
Updating bigtools v0.3.1-dev (/work/tmp/pip-install-0jw93rsr/pybigtools_1227f15a78884eeebc41971527c93b01/bigtools) -> v0.5.4
Adding getrandom v0.2.10 (latest: v0.2.15)
Adding index_list v0.2.15
Updating pybigtools v0.1.0 (/work/tmp/pip-install-0jw93rsr/pybigtools_1227f15a78884eeebc41971527c93b01/pybigtools) -> v0.2.3
Adding ring v0.17.3 (latest: v0.17.8)
Adding rustls v0.21.12 (latest: v0.23.20)
Adding rustls-native-certs v0.6.3 (latest: v0.8.1)
Adding rustls-pemfile v1.0.4 (latest: v2.2.0)
Adding rustls-webpki v0.101.7 (latest: v0.102.8)
Adding sct v0.7.1
Adding spin v0.9.8
Adding untrusted v0.9.0
Adding wasi v0.11.0+wasi-snapshot-preview1 (latest: v0.13.3+wasi-0.2.2)
Adding webpki v0.22.4
� maturin failed
Caused by: `project.version` field is required in pyproject.toml unless it is present in the `project.dynamic` list
Error running maturin: Command '['maturin', 'pep517', 'write-dist-info', '--metadata-directory', '/work/tmp/pip-modern-metadata-pnst2ea0', '--interpreter', '/miniconda/envs/pycistopicpolars/bin/python3.11']' returned non-zero exit status 1.
Checking for Rust toolchain....
Running `maturin pep517 write-dist-info --metadata-directory /work/tmp/pip-modern-metadata-pnst2ea0 --interpreter /miniconda/envs/pycistopicpolars/bin/python3.11`
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
Version:
- Python: 3.11.10
- gcc 11.3.0
I think the bug is related to installing scatac_fragment_tools, which in turn is trying to install pybigtools, which is where the installation stalls. Would appreciate any help with this!