COPY format "parquet" not recognized

[dalonso-clarivate]

Hi all
I think I have successfully installed "pg_parquet" following the instructions:

****Installed these dependencies:

sudo yum install -y epel-release
sudo yum install -y cmake boost-devel zlib-devel
sudo yum install -y https://apache.jfrog.io/artifactory/arrow/almalinux/8/apache-arrow-release-latest.rpm
sudo yum install -y arrow-devel parquet-devel
sudo yum install openssl-devel

*** Installed cargo
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
cargo install cargo-pgrx

*** Installed pgrx

export PATH=$PATH:/var/lib/pgsql/.cargo/bin
cargo pgrx install
cargo pgrx init --pg15 /usr/pgsql-15/bin/pg_config

#Created a new Extension project
mkdir extensions
cd extensions
cargo pgrx new pg_parquet
cd pg_parquet

Updated Cargo.toml as "https://github.com/CrunchyData/pg_parquet/blob/main/Cargo.toml"

Run

cargo pgrx run

A test Postgres instance is created. I successfully create the extension:

pg_parquet=# create extension pg_parquet;
CREATE EXTENSION
pg_parquet=# \dx+
Objects in extension "pg_parquet"
Object description

function hello_pg_parquet()
(1 row)

pg_parquet=# SELECT hello_pg_parquet();
hello_pg_parquet

Hello, pg_parquet
(1 row)

##But cannot export the data:
pg_parquet=# INSERT INTO test_table (id, name) VALUES (1, 'Alice'), (2, 'Bob');
INSERT 0 2
pg_parquet=#
pg_parquet=# COPY test_table TO '/tmp/test_table.parquet' (FORMAT 'parquet', COMPRESSION 'gzip');
ERROR: COPY format "parquet" not recognized
LINE 1: COPY test_table TO '/tmp/test_table.parquet' (FORMAT 'parque...

Any clue?

Thanks in advance!

[aykut-bozkurt]

You need to add pg_parquet to shared_preload_libraries and restart postgres instance. (should be somewhere like ~/.pgrx/data-15/postgresql.conf)

[dalonso-clarivate]

Thanks for your answer aykut-bozkurt. I had already added the extension to the shared_preload_libraries parameter. However, I have finally got it running. I paste the steps I have followed in case they can help to anyone else. I am using Oracle Linux 8:

*****Installed these packages

$> sudo yum install -y epel-release
$> sudo yum install -y cmake boost-devel zlib-devel
$> sudo yum install -y https://apache.jfrog.io/artifactory/arrow/almalinux/8/apache-arrow-release-latest.rpm
$> sudo yum install -y arrow-devel parquet-devel
$> sudo yum install openssl-devel
$> sudo yum install git

**** Clone the source and install cargo and pgrx
$> git clone https://github.com/CrunchyData/pg_parquet.git
$> cd pg_parquet
$> cargo install cargo-pgrx
$> cargo pgrx init
$> curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
$> . "$HOME/.cargo/env"
$> cargo install cargo-pgrx
$> cargo pgrx init --pg15 /usr/pgsql-15/bin/pg_config

*** Open Cargo.toml and set up as default the postgres version you are using. In this case it is 15:
[features]
default = ["pg15"]
pg17 = ["pgrx/pg17", "pgrx-tests/pg17"]
pg16 = ["pgrx/pg16", "pgrx-tests/pg16"]
pg15 = ["pgrx/pg15", "pgrx-tests/pg15"]
pg14 = ["pgrx/pg14", "pgrx-tests/pg14"]
pg_test = []

---

$> cargo build --release --features pg15
$> echo "shared_preload_libraries = 'pg_parquet'" >> ~/.pgrx/data-15/postgresql.conf
$> cargo pgrx run

[seunggihong]

In my case, I solved it this way.
OS : Ubuntu 22.04

First Install PostgreSQL 15.

$ sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'
$ wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo tee /etc/apt/trusted.gpg.d/pgdg.asc &>/dev/null
$ sudo apt update

$ sudo apt install -y postgresql-15 postgresql-client-15 postgresql-server-dev-15

Installing the package for pg_parquet.

Apache Arrow install -> ref. Apache Arrow

$ sudo apt update
$ sudo apt install -y -V ca-certificates lsb-release wget
$ wget https://apache.jfrog.io/artifactory/arrow/$(lsb_release --id --short | tr 'A-Z' 'a-z')/apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb
$ sudo apt install -y -V ./apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb
$ sudo apt update
$ sudo apt install -y -V libarrow-dev # For C++
$ sudo apt install -y -V libarrow-glib-dev # For GLib (C)
$ sudo apt install -y -V libarrow-dataset-dev # For Apache Arrow Dataset C++
$ sudo apt install -y -V libarrow-dataset-glib-dev # For Apache Arrow Dataset GLib (C)
$ sudo apt install -y -V libarrow-acero-dev # For Apache Arrow Acero
$ sudo apt install -y -V libarrow-flight-dev # For Apache Arrow Flight C++
$ sudo apt install -y -V libarrow-flight-glib-dev # For Apache Arrow Flight GLib (C)
$ sudo apt install -y -V libarrow-flight-sql-dev # For Apache Arrow Flight SQL C++
$ sudo apt install -y -V libarrow-flight-sql-glib-dev # For Apache Arrow Flight SQL GLib (C)
$ sudo apt install -y -V libgandiva-dev # For Gandiva C++
$ sudo apt install -y -V libgandiva-glib-dev # For Gandiva GLib (C)
$ sudo apt install -y -V libparquet-dev # For Apache Parquet C++
$ sudo apt install -y -V libparquet-glib-dev # For Apache Parquet GLib (C)

$ sudo apt update && sudo apt install -y \
      clang \
      llvm \
      llvm-dev \
      libclang-dev \
      build-essential \
      pkg-config \
      cmake \
      libarrow-dev \
      libssl-dev

rustc, cargo install

$ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
$ sudo vi ~/.bashrc

## bashrc
...
export PATH="$HOME/.cargo/bin:$PATH"

$ source ~/.bashrc

pg_parquet install

$ git clone https://github.com/CrunchyData/pg_parquet.git
$ cd pg_parquet
$ cargo install cargo-pgrx
$ cargo pgrx init --pg15 /usr/lib/postgresql/15/bin/pg_config
$ echo “shared_preload_libraries = 'pg_parquet'” >> ~/.pgrx/data-15/postgresql.conf
$ cargo pgrx run

[beydogan]

I'm having similar problem on a fresh install on pg16/ubuntu. i have to do DROP EXTENSION pg_parquet; CREATE EXTENSION pg_parquet; on each session.

I already have:

$ cat ~/.pgrx/data-16/postgresql.conf
shared_preload_libraries = 'pg_parquet'