Re: [GENERAL] Performance while loading data and indexing

Hi Shridhar,

Shridhar Daithankar wrote:
<snip>
> 3) Any suggsestions for runtime as data load and query will be going in
> parallel.

That sounds unusual.  From reading this, it *sounds* like you'll be
running queries against an incomplete dataset, or maybe just running the
queries that affect the tables loaded thus far (during the initial
load).

<snip>
> fsync=true (Sad but true. Left untouched.. Will that make a difference on
> SCSI?)

Definitely.  Have directly measured a ~ 2x tps throughput increase on
FreeBSD when leaving fsync off whilst performance measuring stuff
recently (PG 7.2.2).  Like anything it'll depend on workload, phase of
moon, etc, but it's a decent indicator.

<snip>
> Now questions.
>
> 1)  Instead of copying from a single 12GB data file,  will a parallel copy from
> say 5 files will speed up the things?

Not sure yet.  Haven't get done enough performance testing (on the cards
very soon though).

> Couple MB of data per sec. to disk is just not saturating it. It's a RAID 5
> setup..

fsync = off would help during the data load, but not a good idea if
you're going to be running queries against it at the same time.

Am still getting the hang of performance tuning stuff.  Have a bunch of
Ultra160 hardware for the Intel platform, and am testing against it as
time permits.

Not as high end as I'd like, but it's a start.

:-)

Regards and best wishes,

Justin Clift

<snip>
> Bye
>  Shridhar

--
"My grandfather once told me that there are two kinds of people: those
who work and those who take the credit. He told me to try to be in the
first group; there was less competition there."
   - Indira Gandhi