ERROR: index row size 1648 exceeds maximum 1336 for index "..."

[To4e]

New problem... Trying to create the rum index:

create index some_index on some_table using rum (some_column rum_tsvector_ops);

Here is the text of the error:

SQL Error [54000]: ERROR: index row size 1648 exceeds maximum 1336 for index "some_index"
org.postgresql.util.PSQLException: ERROR: index row size 1648 exceeds maximum 1336 for index "some_index"

[za-arthur]

Index key can't have size more than 1336 bytes. It is a limitation of the current version. It is true also for GIN (it could has another limit).

It seems there is some big string in your table. You can check it with the query:

select word, char_length(word)
from ts_stat('select some_column from some_table')
order by char_length(word) desc
limit 10;

[To4e]

Yes, i got HUGE strings in this table, not the only one, a column with ts_vector of it and a gin index on it.

[za-arthur]

Closed

[za-arthur]

We are working on this issue. And as I know there are huge urls in your strings.

Are you do search by these urls? Are they important to you? If not, then it is easy to fix it. Just don't store it in index (I can explain how).

If they important we can fix RUM and cut off urls. But I think we need it anyway, other users may have similar issue.

[To4e]

Glad to hear that you are working on this issue.

We are not using these urls at the moment, but they can be used in the near future. So, it would be great, if you had the RUM fixed according to this fact.

[za-arthur]

We find a solution. We can add new OPERATOR CLASS which will store hash of lexems, then you can store huge strings. But in this OPERATOR CLASS we cant use prefix (or partial) matching.

Do you use prefix matching?

[To4e]

Can you trim the url and raise some notice like:

NOTICE: Index key can't have size more than 1336 bytes.
HINT: key has been trimmed.

[za-arthur]

Can you trim the url and raise some notice like:

NOTICE: Index key can't have size more than 1336 bytes.
HINT: key has been trimmed.

I think we can trim the url also. We will fix limits for posting trees, and they will become as in GIN.

[To4e]

I think we can trim the url also. We will fix limits for posting trees, and they will become as in GIN.

Sounds great, thank you.

[za-arthur]

@To4e , please can you check the issue_9_max_item_size branch?
https://github.com/postgrespro/rum/tree/issue_9_max_item_size

[To4e]

@select-artur, still having the same problem:

ERROR: index row size 1544 exceeds maximum 1352 for index "some_index"

[akorotkov]

Please, try commit 58fee28.