Re: [PROPOSAL] Improvements of Hunspell dictionaries support

08.11.2015 14:23, Artur Zakirov пишет:
> Thank you for reply.
>
> This was because of the flag field size of the SPELL struct. And long
> flags were being trancated in the .dict file.
>
> I attached new patch. It is temporary patch, not final. It can be done
> better.
>

I have updated the patch and attached it. Now dynamic memory allocation
is used to the flag field of the SPELL struct.

I have valued time of a dictionary loading and memory using by a
dictionary in the new patch. Dictionary is loaded at the first reference
to it. For example, if we execute ts_lexize function. And first
ts_lexize executing takes more time than second.

The following table shows performance of some dictionaries before patch
and after in my computer.

-------------------------------------------------
|      |  loading time, ms  |      memory, MB   |
|      |  before  |  after  |  before |  after  |
-------------------------------------------------
|ar    | 700      | 300     | 23,7    | 15,7    |
|br_fr | 410      | 450     | 27,4    | 27,5    |
|ca    | 248      | 245     | 14,7    | 15,4    |
|en_us | 100      | 100     | 5,4     | 6,2     |
|fr    | 160      | 178     | 13,7    | 14,1    |
|gl_es | 160      | 150     | 9       | 9,4     |
|is    | 260      | 202     | 16,1    | 16,3    |
-------------------------------------------------

As you can see, substantially loading time and memory using before and
after the patch are same.

Link to patch in commitfest:
https://commitfest.postgresql.org/8/420/

Link to regression tests:
https://dl.dropboxusercontent.com/u/15423817/HunspellDictTest.tar.gz

--
Artur Zakirov
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company