path: root/doc/TODO.detail/replication

From goran@kirra.net Mon Dec 20 14:30:54 1999
Received: from villa.bildbasen.se (villa.bildbasen.se [193.45.225.97])
	by candle.pha.pa.us (8.9.0/8.9.0) with SMTP id PAA29058
	for <pgman@candle.pha.pa.us>; Mon, 20 Dec 1999 15:30:17 -0500 (EST)
Received: (qmail 2485 invoked from network); 20 Dec 1999 20:29:53 -0000
Received: from a112.dial.kiruna.se (HELO kirra.net) (193.45.238.12)
  by villa.bildbasen.se with SMTP; 20 Dec 1999 20:29:53 -0000
Sender: goran
Message-ID: <385E9192.226CC37D@kirra.net>
Date: Mon, 20 Dec 1999 21:29:06 +0100
From: Goran Thyni <goran@kirra.net>
Organization: kirra.net
X-Mailer: Mozilla 4.6 [en] (X11; U; Linux 2.2.13 i586)
X-Accept-Language: sv, en
MIME-Version: 1.0
To: Bruce Momjian <pgman@candle.pha.pa.us>
CC: "neil d. quiogue" <nquiogue@ieee.org>,
        PostgreSQL-development <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] Re: QUESTION: Replication
References: <199912201508.KAA20572@candle.pha.pa.us>
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: 8bit
Status: OR

Bruce Momjian wrote:
> We need major work in this area, or at least a plan and an FAQ item.
> We are getting major questions on this, and I don't know enough even to
> make an FAQ item telling people their options.

My 2 cents, or 2 �ren since I'm a Swede, on this:

It is pretty simple to build a replication with pg_dump, transfer,
empty replic and reload.
But if we want "live replicas" we better base our efforts on a
mechanism using WAL-logs to rollforward the replicas.

regards, 
-----------------
G�ran Thyni
On quiet nights you can hear Windows NT reboot!

From owner-pgsql-hackers@hub.org Fri Dec 24 10:01:18 1999
Received: from renoir.op.net (root@renoir.op.net [207.29.195.4])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id LAA11295
	for <pgman@candle.pha.pa.us>; Fri, 24 Dec 1999 11:01:17 -0500 (EST)
Received: from hub.org (hub.org [216.126.84.1]) by renoir.op.net (o1/$Revision: 1.10 $) with ESMTP id KAA20310 for <pgman@candle.pha.pa.us>; Fri, 24 Dec 1999 10:39:18 -0500 (EST)
Received: from localhost (majordom@localhost)
	by hub.org (8.9.3/8.9.3) with SMTP id KAA61760;
	Fri, 24 Dec 1999 10:31:13 -0500 (EST)
	(envelope-from owner-pgsql-hackers)
Received: by hub.org (bulk_mailer v1.5); Fri, 24 Dec 1999 10:30:48 -0500
Received: (from majordom@localhost)
	by hub.org (8.9.3/8.9.3) id KAA58879
	for pgsql-hackers-outgoing; Fri, 24 Dec 1999 10:29:51 -0500 (EST)
	(envelope-from owner-pgsql-hackers@postgreSQL.org)
Received: from bocs170n.black-oak.COM ([38.149.137.131])
	by hub.org (8.9.3/8.9.3) with ESMTP id KAA58795
	for <pgsql-hackers@postgreSQL.org>; Fri, 24 Dec 1999 10:29:00 -0500 (EST)
	(envelope-from DWalker@black-oak.com)
From: DWalker@black-oak.com
To: pgsql-hackers@postgreSQL.org
Subject: [HACKERS] database replication
Date: Fri, 24 Dec 1999 10:27:59 -0500
Message-ID: <OFD38C9424.B391F434-ON85256851.0054F41A@black-oak.COM>
X-Priority: 3 (Normal)
X-MIMETrack: Serialize by Router on notes01n/BOCS(Release 5.0.1|July 16, 1999) at 12/24/99
	10:28:01 AM
MIME-Version: 1.0
MIME-Version: 1.0
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Sender: owner-pgsql-hackers@postgreSQL.org
Status: OR

<P>I've been toying with the idea of implementing database replication for =
the last few days. &nbsp;The system I'm proposing will be a seperate progra=
m which can be run on any machine and will most likely be implemented in Py=
thon. &nbsp;What I'm looking for at this point are gaping holes in my think=
ing/logic/etc. &nbsp;Here's what I'm thinking...</P><P>&nbsp;</P><P>1) I wa=
nt to make this program an additional layer over PostgreSQL. &nbsp;I really=
 don't want to hack server code if I can get away with it. &nbsp;At this po=
int I don't feel I need to.</P><P>2) The replication system will need to ad=
d at least one field to each table in each database that needs to be replic=
ated. &nbsp;This field will be a date/time stamp which identifies the &quot=
;last update&quot; of the record. &nbsp;This field will be called PGR=5FTIM=
E for lack of a better name. &nbsp;Because this field will be used from wit=
hin programs and triggers it can be longer so as to not mistake it for a us=
er field.</P><P>3) For each table to be replicated the replication system w=
ill programatically add one plpgsql function and trigger to modify the PGR=
=5FTIME field on both UPDATEs and INSERTs. &nbsp;The name of this function =
and trigger will be along the lines of &lt;table=5Fname&gt;=5Freplication=
=5Fupdate=5Ftrigger and &lt;table=5Fname&gt;=5Freplication=5Fupdate=5Ffunct=
ion. &nbsp;The function is a simple two-line chunk of code to set the field=
 PGR=5FTIME equal to NOW. &nbsp;The trigger is called before each insert/up=
date. &nbsp;When looking at the Docs I see that times are stored in Zulu (G=
T) time. &nbsp;Because of this I don't have to worry about time zones and t=
he like. &nbsp;I need direction on this part (such as &quot;hey dummy, look=
 at page N of file X.&quot;).</P><P>4) At this point we have tables which c=
an, at a basic level, tell the replication system when they were last updat=
ed.</P><P>5) The replication system will have a database of its own to reco=
rd the last replication event, hold configuration, logs, etc. &nbsp;I'd pre=
fer to store the configuration in a PostgreSQL table but it could just as e=
asily be stored in a text file on the filesystem somewhere.</P><P>6) To han=
dle replication I basically check the local &quot;last replication time&quo=
t; and compare it against the remote PGR=5FTIME fields. &nbsp;If the remote=
 PGR=5FTIME is greater than the last replication time then change the local=
 copy of the database, otherwise, change the remote end of the database. &n=
bsp;At this point I don't have a way to know WHICH field changed between th=
e two replicas so either I do ROW level replication or I check each field. =
&nbsp;I check PGR=5FTIME to determine which field is the most current. &nbs=
p;Some fine tuning of this process will have to occur no doubt.</P><P>7) Th=
e commandline utility, fired off by something like cron, could run several =
times during the day -- command line parameters can be implemented to say P=
USH ALL CHANGES TO SERVER A, or PULL ALL CHANGES FROM SERVER B.</P><P>&nbsp=
;</P><P>Questions/Concerns:</P><P>1) How far do I go with this? &nbsp;Do I =
start manhandling the system catalogs (pg=5F* tables)?</P><P>2) As to #2 an=
d #3 above, I really don't like tools automagically changing my tables but =
at this point I don't see a way around it. &nbsp;I guess this is where the =
testing comes into play.</P><P>3) Security: the replication app will have t=
o have pretty good rights to the database so it can add the nessecary funct=
ions and triggers, modify table schema, etc. &nbsp;</P><P>&nbsp;</P><P>&nbs=
p; So, any &quot;you're insane and should run home to momma&quot; comments?=
</P><P>&nbsp;</P><P>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Damond=
</P><P></P>=

************

From owner-pgsql-hackers@hub.org Fri Dec 24 18:31:03 1999
Received: from renoir.op.net (root@renoir.op.net [207.29.195.4])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id TAA26244
	for <pgman@candle.pha.pa.us>; Fri, 24 Dec 1999 19:31:02 -0500 (EST)
Received: from hub.org (hub.org [216.126.84.1]) by renoir.op.net (o1/$Revision: 1.10 $) with ESMTP id TAA12730 for <pgman@candle.pha.pa.us>; Fri, 24 Dec 1999 19:30:05 -0500 (EST)
Received: from localhost (majordom@localhost)
	by hub.org (8.9.3/8.9.3) with SMTP id TAA57851;
	Fri, 24 Dec 1999 19:23:31 -0500 (EST)
	(envelope-from owner-pgsql-hackers)
Received: by hub.org (bulk_mailer v1.5); Fri, 24 Dec 1999 19:22:54 -0500
Received: (from majordom@localhost)
	by hub.org (8.9.3/8.9.3) id TAA57710
	for pgsql-hackers-outgoing; Fri, 24 Dec 1999 19:21:56 -0500 (EST)
	(envelope-from owner-pgsql-hackers@postgreSQL.org)
Received: from Mail.austin.rr.com (sm2.texas.rr.com [24.93.35.55])
	by hub.org (8.9.3/8.9.3) with ESMTP id TAA57680
	for <pgsql-hackers@postgresql.org>; Fri, 24 Dec 1999 19:21:25 -0500 (EST)
	(envelope-from ELOEHR@austin.rr.com)
Received: from austin.rr.com ([24.93.40.248]) by Mail.austin.rr.com  with Microsoft SMTPSVC(5.5.1877.197.19);
  Fri, 24 Dec 1999 18:12:50 -0600
Message-ID: <38640E2D.75136600@austin.rr.com>
Date: Fri, 24 Dec 1999 18:22:05 -0600
From: Ed Loehr <ELOEHR@austin.rr.com>
X-Mailer: Mozilla 4.7 [en] (X11; U; Linux 2.2.12-20smp i686)
X-Accept-Language: en
MIME-Version: 1.0
To: DWalker@black-oak.com
CC: pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] database replication
References: <OFD38C9424.B391F434-ON85256851.0054F41A@black-oak.COM>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Sender: owner-pgsql-hackers@postgreSQL.org
Status: OR

DWalker@black-oak.com wrote:

> 6) To handle replication I basically check the local "last
> replication time" and compare it against the remote PGR_TIME
> fields.  If the remote PGR_TIME is greater than the last replication
> time then change the local copy of the database, otherwise, change
> the remote end of the database.  At this point I don't have a way to
> know WHICH field changed between the two replicas so either I do ROW
> level replication or I check each field.  I check PGR_TIME to
> determine which field is the most current.  Some fine tuning of this
> process will have to occur no doubt.

Interesting idea.  I can see how this might sync up two databases
somehow.  For true replication, however, I would always want every
replicated database to be, at the very least, internally consistent
(i.e., referential integrity), even if it was a little behind on
processing transactions.  In this method, its not clear how
consistency is every achieved/guaranteed at any point in time if the
input stream of changes is continuous.  If the input stream ceased,
then I can see how this approach might eventually catch up and totally
resync everything, but it looks *very* computationally  expensive.

But I might have missed something.  How would internal consistency be
maintained?


> 7) The commandline utility, fired off by something like cron, could
> run several times during the day -- command line parameters can be
> implemented to say PUSH ALL CHANGES TO SERVER A, or PULL ALL CHANGES
> FROM SERVER B.

My two cents is that, while I can see this kind of database syncing as
valuable, this is not the kind of "replication" I had in mind.  This
may already possible by simply copying the database.  What replication
means to me is a live, continuously streaming sequence of updates from
one database to another where the replicated database is always
internally consistent, available for read-only queries, and never "too
far" out of sync with the source/primary database.

What does replication mean to others?

Cheers,
Ed Loehr



************

From owner-pgsql-hackers@hub.org Fri Dec 24 21:31:10 1999
Received: from renoir.op.net (root@renoir.op.net [207.29.195.4])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id WAA02578
	for <pgman@candle.pha.pa.us>; Fri, 24 Dec 1999 22:31:09 -0500 (EST)
Received: from hub.org (hub.org [216.126.84.1]) by renoir.op.net (o1/$Revision: 1.10 $) with ESMTP id WAA16641 for <pgman@candle.pha.pa.us>; Fri, 24 Dec 1999 22:18:56 -0500 (EST)
Received: from localhost (majordom@localhost)
	by hub.org (8.9.3/8.9.3) with SMTP id WAA89135;
	Fri, 24 Dec 1999 22:11:12 -0500 (EST)
	(envelope-from owner-pgsql-hackers)
Received: by hub.org (bulk_mailer v1.5); Fri, 24 Dec 1999 22:10:56 -0500
Received: (from majordom@localhost)
	by hub.org (8.9.3/8.9.3) id WAA89019
	for pgsql-hackers-outgoing; Fri, 24 Dec 1999 22:09:59 -0500 (EST)
	(envelope-from owner-pgsql-hackers@postgreSQL.org)
Received: from bocs170n.black-oak.COM ([38.149.137.131])
	by hub.org (8.9.3/8.9.3) with ESMTP id WAA88957;
	Fri, 24 Dec 1999 22:09:11 -0500 (EST)
	(envelope-from dwalker@black-oak.com)
Received: from gcx80 ([151.196.99.113])
          by bocs170n.black-oak.COM (Lotus Domino Release 5.0.1)
          with SMTP id 1999122422080835:6 ;
          Fri, 24 Dec 1999 22:08:08 -0500 
Message-ID: <001b01bf4e9e$647287d0$af63a8c0@walkers.org>
From: "Damond Walker" <dwalker@black-oak.com>
To: <owner-pgsql-hackers@postgreSQL.org>
Cc: <pgsql-hackers@postgreSQL.org>
References: <OFD38C9424.B391F434-ON85256851.0054F41A@black-oak.COM> <38640E2D.75136600@austin.rr.com>
Subject: Re: [HACKERS] database replication
Date: Fri, 24 Dec 1999 22:07:55 -0800
MIME-Version: 1.0
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 5.00.2314.1300
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300
X-MIMETrack: Itemize by SMTP Server on notes01n/BOCS(Release 5.0.1|July 16, 1999) at 12/24/99
	10:08:09 PM,
	Serialize by Router on notes01n/BOCS(Release 5.0.1|July 16, 1999) at 12/24/99
	10:08:11 PM,
	Serialize complete at 12/24/99 10:08:11 PM
Content-Transfer-Encoding: 7bit
Content-Type: text/plain;
	charset="iso-8859-1"
Sender: owner-pgsql-hackers@postgreSQL.org
Status: OR

>
> Interesting idea.  I can see how this might sync up two databases
> somehow.  For true replication, however, I would always want every
> replicated database to be, at the very least, internally consistent
> (i.e., referential integrity), even if it was a little behind on
> processing transactions.  In this method, its not clear how
> consistency is every achieved/guaranteed at any point in time if the
> input stream of changes is continuous.  If the input stream ceased,
> then I can see how this approach might eventually catch up and totally
> resync everything, but it looks *very* computationally  expensive.
>

    What's the typical unit of work for the database?  Are we talking about
update transactions which span the entire DB?  Or are we talking about
updating maybe 1% or less of the database everyday?  I'd think it would be
more towards the latter than the former.  So, yes, this process would be
computationally expensive but how many records would actually have to be
sent back and forth?

> But I might have missed something.  How would internal consistency be
> maintained?
>

    Updates that occur at site A will be moved to site B and vice versa.
Consistency would be maintained.  The only problem that I can see right off
the bat would be what if site A and site B made changes to a row and then
site C was brought into the picture?  Which one wins?

    Someone *has* to win when it comes to this type of thing.  You really
DON'T want to start merging row changes...

>
> My two cents is that, while I can see this kind of database syncing as
> valuable, this is not the kind of "replication" I had in mind.  This
> may already possible by simply copying the database.  What replication
> means to me is a live, continuously streaming sequence of updates from
> one database to another where the replicated database is always
> internally consistent, available for read-only queries, and never "too
> far" out of sync with the source/primary database.
>

    Sounds like you're talking about distributed transactions to me.  That's
an entirely different subject all-together.  What you describe can be done
by copying a database...but as you say, this would only work in a read-only
situation.


                Damond


************

From owner-pgsql-hackers@hub.org Sat Dec 25 16:35:07 1999
Received: from hub.org (hub.org [216.126.84.1])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id RAA28890
	for <pgman@candle.pha.pa.us>; Sat, 25 Dec 1999 17:35:05 -0500 (EST)
Received: from localhost (majordom@localhost)
	by hub.org (8.9.3/8.9.3) with SMTP id RAA86997;
	Sat, 25 Dec 1999 17:29:10 -0500 (EST)
	(envelope-from owner-pgsql-hackers)
Received: by hub.org (bulk_mailer v1.5); Sat, 25 Dec 1999 17:28:09 -0500
Received: (from majordom@localhost)
	by hub.org (8.9.3/8.9.3) id RAA86863
	for pgsql-hackers-outgoing; Sat, 25 Dec 1999 17:27:11 -0500 (EST)
	(envelope-from owner-pgsql-hackers@postgreSQL.org)
Received: from mtiwmhc08.worldnet.att.net (mtiwmhc08.worldnet.att.net [204.127.131.19])
	by hub.org (8.9.3/8.9.3) with ESMTP id RAA86798
	for <pgsql-hackers@postgreSQL.org>; Sat, 25 Dec 1999 17:26:34 -0500 (EST)
	(envelope-from pgsql@rkirkpat.net)
Received: from [192.168.3.100] ([12.74.72.219])
          by mtiwmhc08.worldnet.att.net (InterMail v03.02.07.07 118-134)
          with ESMTP id <19991225222554.VIOL28505@[12.74.72.219]>;
          Sat, 25 Dec 1999 22:25:54 +0000
Date: Sat, 25 Dec 1999 15:25:47 -0700 (MST)
From: Ryan Kirkpatrick <pgsql@rkirkpat.net>
X-Sender: rkirkpat@excelsior.rkirkpat.net
To: DWalker@black-oak.com
cc: pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] database replication
In-Reply-To: <OFD38C9424.B391F434-ON85256851.0054F41A@black-oak.COM>
Message-ID: <Pine.LNX.4.10.9912251433310.1551-100000@excelsior.rkirkpat.net>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: owner-pgsql-hackers@postgreSQL.org
Status: OR

On Fri, 24 Dec 1999 DWalker@black-oak.com wrote:

> I've been toying with the idea of implementing database replication
> for the last few days.

	I too have been thinking about this some over the last year or
two, just trying to find a quick and easy way to do it. I am not so
interested in replication, as in synchronization, as in between a desktop
machine and a laptop, so I can keep the databases on each in sync with
each other. For this sort of purpose, both the local and remote databases
would be "idle" at the time of syncing.

> 2) The replication system will need to add at least one field to each
> table in each database that needs to be replicated. This field will be
> a date/time stamp which identifies the "last update" of the record.  
> This field will be called PGR_TIME for lack of a better name.  
> Because this field will be used from within programs and triggers it
> can be longer so as to not mistake it for a user field.

	How about a single, seperate table with the fields of 'database',
'tablename', 'oid', 'last_changed', that would store the same data as your
PGR_TIME field. It would be seperated from the actually data tables, and
therefore would be totally transparent to any database interface
applications. The 'oid' field would hold each row's OID, a nice, unique
identification number for the row, while the other fields would tell which
table and database the oid is in. Then this table can be compared with the
this table on a remote machine to quickly find updates and changes, then
each differences can be dealt with in turn.

> 3) For each table to be replicated the replication system will
> programatically add one plpgsql function and trigger to modify the
> PGR_TIME field on both UPDATEs and INSERTs.  The name of this function
> and trigger will be along the lines of
> <table_name>_replication_update_trigger and
> <table_name>_replication_update_function.  The function is a simple
> two-line chunk of code to set the field PGR_TIME equal to NOW.  The
> trigger is called before each insert/update.  When looking at the Docs
> I see that times are stored in Zulu (GT) time.  Because of this I
> don't have to worry about time zones and the like.  I need direction
> on this part (such as "hey dummy, look at page N of file X.").

	I like this idea, better than any I have come up with yet. Though,
how are you going to handle DELETEs? 

> 6) To handle replication I basically check the local "last replication
> time" and compare it against the remote PGR_TIME fields.  If the
> remote PGR_TIME is greater than the last replication time then change
> the local copy of the database, otherwise, change the remote end of
> the database.  At this point I don't have a way to know WHICH field
> changed between the two replicas so either I do ROW level replication
> or I check each field.  I check PGR_TIME to determine which field is
> the most current.  Some fine tuning of this process will have to occur
> no doubt.

	Yea, this is indeed the sticky part, and would indeed require some
fine-tunning. Basically, the way I see it, is if the two timestamps for a
single row do not match (or even if the row and therefore timestamp is
missing on one side or the other altogether):
	local ts > remote ts => Local row is exported to remote.
	remote ts > local ts => Remote row is exported to local.
	local ts > last sync time && no remote ts => 
		Local row is inserted on remote.
	local ts < last sync time && no remote ts =>
		Local row is deleted.
	remote ts > last sync time && no local ts =>
		Remote row is inserted on local.
	remote ts < last sync time && no local ts =>
		Remote row is deleted.
where the synchronization process is running on the local machine. By
exported, I mean the local values are sent to the remote machine, and the
row on that remote machine is updated to the local values. How does this
sound?

> 7) The commandline utility, fired off by something like cron, could
> run several times during the day -- command line parameters can be
> implemented to say PUSH ALL CHANGES TO SERVER A, or PULL ALL CHANGES
> FROM SERVER B.

	Or run manually for my purposes. Also, maybe follow it
with a vacuum run on both sides for all databases, as this is going to
potenitally cause lots of table changes that could stand with a cleanup. 

> 1) How far do I go with this?  Do I start manhandling the system catalogs (pg_* tables)?

	Initially, I would just stick to user table data... If you have
changes in triggers and other meta-data/executable code, you are going to
want to make syncs of that stuff manually anyway. At least I would want
to.

> 2) As to #2 and #3 above, I really don't like tools automagically
> changing my tables but at this point I don't see a way around it.  I
> guess this is where the testing comes into play.

	Hence the reason for the seperate table with just a row's
identification and last update time. Only modifications to the synced
database is the update trigger, which should be pretty harmless.

> 3) Security: the replication app will have to have pretty good rights
> to the database so it can add the nessecary functions and triggers,
> modify table schema, etc.

	Just run the sync program as the postgres super user, and there
are no problems. :)

>   So, any "you're insane and should run home to momma" comments?

	No, not at all. Though it probably should be remaned from
replication to synchronization. The former is usually associated with a
continuous stream of updates between the local and remote databases, so
they are almost always in sync, and have a queuing ability if their
connection is loss for span of time as well. Very complex and difficult to
implement, and would require hacking server code. :( Something only Sybase
and Oracle have (as far as I know), and from what I have seen of Sybase's
replication server support (dated by 5yrs) it was a pain to setup and get
running correctly.
	The latter, synchronization, is much more managable, and can still
be useful, especially when you have a large database you want in two
places, mainly for read only purposes at one end or the other, but don't
want to waste the time/bandwidth to move and load the entire database each
time it changes on one end or the other. Same idea as mirroring software
for FTP sites, just transfers the changes, and nothing more.
	I also like the idea of using Python. I have been using it
recently for some database interfaces (to PostgreSQL of course :), and it
is a very nice language to work with. Some worries about performance of
the program though, as python is only an interpreted lanuage, and I have
yet to really be impressed with the speed of execution of my database
interfaces yet.
	Anyway, it sound like a good project, and finally one where I
actually have a clue of what is going on, and the skills to help. So, if
you are interested in pursing this project, I would be more than glad to
help. TTYL.

---------------------------------------------------------------------------
|   "For to me to live is Christ, and to die is gain."                    |
|                                            --- Philippians 1:21 (KJV)   |
---------------------------------------------------------------------------
|   Ryan Kirkpatrick  |  Boulder, Colorado  |  http://www.rkirkpat.net/   |
---------------------------------------------------------------------------



************

From owner-pgsql-hackers@hub.org Sun Dec 26 08:31:09 1999
Received: from renoir.op.net (root@renoir.op.net [207.29.195.4])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id JAA17976
	for <pgman@candle.pha.pa.us>; Sun, 26 Dec 1999 09:31:07 -0500 (EST)
Received: from hub.org (hub.org [216.126.84.1]) by renoir.op.net (o1/$Revision: 1.10 $) with ESMTP id JAA23337 for <pgman@candle.pha.pa.us>; Sun, 26 Dec 1999 09:28:36 -0500 (EST)
Received: from localhost (majordom@localhost)
	by hub.org (8.9.3/8.9.3) with SMTP id JAA90738;
	Sun, 26 Dec 1999 09:21:58 -0500 (EST)
	(envelope-from owner-pgsql-hackers)
Received: by hub.org (bulk_mailer v1.5); Sun, 26 Dec 1999 09:19:19 -0500
Received: (from majordom@localhost)
	by hub.org (8.9.3/8.9.3) id JAA90498
	for pgsql-hackers-outgoing; Sun, 26 Dec 1999 09:18:21 -0500 (EST)
	(envelope-from owner-pgsql-hackers@postgreSQL.org)
Received: from bocs170n.black-oak.COM ([38.149.137.131])
	by hub.org (8.9.3/8.9.3) with ESMTP id JAA90452
	for <pgsql-hackers@postgreSQL.org>; Sun, 26 Dec 1999 09:17:54 -0500 (EST)
	(envelope-from dwalker@black-oak.com)
Received: from vmware98 ([151.196.99.113])
          by bocs170n.black-oak.COM (Lotus Domino Release 5.0.1)
          with SMTP id 1999122609164808:7 ;
          Sun, 26 Dec 1999 09:16:48 -0500 
Message-ID: <002201bf4fb3$623f0220$b263a8c0@vmware98.walkers.org>
From: "Damond Walker" <dwalker@black-oak.com>
To: "Ryan Kirkpatrick" <pgsql@rkirkpat.net>
Cc: <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] database replication
Date: Sun, 26 Dec 1999 10:10:41 -0500
MIME-Version: 1.0
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 4.72.3110.1
X-MimeOLE: Produced By Microsoft MimeOLE V4.72.3110.3
X-MIMETrack: Itemize by SMTP Server on notes01n/BOCS(Release 5.0.1|July 16, 1999) at 12/26/99
	09:16:51 AM,
	Serialize by Router on notes01n/BOCS(Release 5.0.1|July 16, 1999) at 12/26/99
	09:16:54 AM,
	Serialize complete at 12/26/99 09:16:54 AM
Content-Transfer-Encoding: 7bit
Content-Type: text/plain;
	charset="iso-8859-1"
Sender: owner-pgsql-hackers@postgreSQL.org
Status: OR

>
>     I too have been thinking about this some over the last year or
>two, just trying to find a quick and easy way to do it. I am not so
>interested in replication, as in synchronization, as in between a desktop
>machine and a laptop, so I can keep the databases on each in sync with
>each other. For this sort of purpose, both the local and remote databases
>would be "idle" at the time of syncing.
>

    I don't think it would matter if the databases are idle or not to be
honest with you.  At any single point in time when you replicate I'd figure
that the database would be in a consistent state.  So, you should be able to
replicate (or sync) a remote database that is in use.  After all, you're
getting a snapshot of the database as it stands at 8:45 PM.  At 8:46 PM it
may be totally different...but the next time syncing takes place those
changes would appear in your local copy.

    The one problem you may run into is if the remote host is running a
large batch process.  It's very likely that you will get 50% of their
changes when you replicate...but then again, that's why you can schedule the
event to work around such things.

>     How about a single, seperate table with the fields of 'database',
>'tablename', 'oid', 'last_changed', that would store the same data as your
>PGR_TIME field. It would be seperated from the actually data tables, and
>therefore would be totally transparent to any database interface
>applications. The 'oid' field would hold each row's OID, a nice, unique
>identification number for the row, while the other fields would tell which
>table and database the oid is in. Then this table can be compared with the
>this table on a remote machine to quickly find updates and changes, then
>each differences can be dealt with in turn.
>

    The problem with OID's is that they are unique at the local level but if
you try and use them between servers you can run into overlap.  Also, if a
database is under heavy use this table could quickly become VERY large.  Add
indexes to this table to help performance and you're taking up even more
disk space.

    Using the PGR_TIME field with an index will allow us to find rows which
have changed VERY quickly.  All we need to do now is somehow programatically
find the primary key for a table so the person setting up replication (or
syncing) doesn't have to have an indepth knowledge of the schema in order to
setup a syncing schedule.

>
>     I like this idea, better than any I have come up with yet. Though,
>how are you going to handle DELETEs?
>

    Oops...how about defining a trigger for this?  With deletion I guess we
would have to move a flag into another table saying we deleted record 'X'
with this primary key from this table.

>
>     Yea, this is indeed the sticky part, and would indeed require some
>fine-tunning. Basically, the way I see it, is if the two timestamps for a
>single row do not match (or even if the row and therefore timestamp is
>missing on one side or the other altogether):
>     local ts > remote ts => Local row is exported to remote.
>     remote ts > local ts => Remote row is exported to local.
>     local ts > last sync time && no remote ts =>
>          Local row is inserted on remote.
>     local ts < last sync time && no remote ts =>
>          Local row is deleted.
>     remote ts > last sync time && no local ts =>
>          Remote row is inserted on local.
>     remote ts < last sync time && no local ts =>
>          Remote row is deleted.
>where the synchronization process is running on the local machine. By
>exported, I mean the local values are sent to the remote machine, and the
>row on that remote machine is updated to the local values. How does this
>sound?
>

    The replication part will be the most complex...that much is for
certain...

    I've been writing systems in Lotus Notes/Domino for the last year or so
and I've grown quite spoiled with what it can do in regards to replication.
It's not real-time but you have to gear your applications to this type of
thing (it's possible to create documents, fire off email to notify people of
changes and have the email arrive before the replicated documents do).
Replicating large Notes/Domino databases takes quite a while....I don't see
any kind of replication or syncing running in a blink of an eye.

    Having said that, a good algo will have to be written to cut down on
network traffic and to keep database conversations down to a minimum.  This
will be appreciated by people with low bandwidth connections I'm sure
(dial-ups, fractional T1's, etc).

>     Or run manually for my purposes. Also, maybe follow it
>with a vacuum run on both sides for all databases, as this is going to
>potenitally cause lots of table changes that could stand with a cleanup.
>

    What would a vacuum do to a system being used by many people?

>     No, not at all. Though it probably should be remaned from
>replication to synchronization. The former is usually associated with a
>continuous stream of updates between the local and remote databases, so
>they are almost always in sync, and have a queuing ability if their
>connection is loss for span of time as well. Very complex and difficult to
>implement, and would require hacking server code. :( Something only Sybase
>and Oracle have (as far as I know), and from what I have seen of Sybase's
>replication server support (dated by 5yrs) it was a pain to setup and get
>running correctly.

    It could probably be named either way...but the one thing I really don't
want to do is start hacking server code.  The PostgreSQL people have enough
to do without worrying about trying to meld anything I've done to their
server.   :)

    Besides, I like the idea of having it operate as a stand-alone product.
The only PostgreSQL feature we would require would be triggers and
plpgsql...what was the earliest version of PostgreSQL that supported
plpgsql?  Even then I don't see the triggers being that complex to boot.

>     I also like the idea of using Python. I have been using it
>recently for some database interfaces (to PostgreSQL of course :), and it
>is a very nice language to work with. Some worries about performance of
>the program though, as python is only an interpreted lanuage, and I have
>yet to really be impressed with the speed of execution of my database
>interfaces yet.

    The only thing we'd need for Python is the Python extensions for
PostgreSQL...which in turn requires libpq and that's about it.  So, it
should be able to run on any platform supported by Python and libpq.  Using
TK for the interface components will require NT people to get additional
software from the 'net.  At least it did with older version of Windows
Python.  Unix folks should be happy....assuming they have X running on the
machine doing the replication or syncing.  Even then I wrote a curses based
Python interface awhile back which allows buttons, progress bars, input
fields, etc (I called it tinter and it's available at
http://iximd.com/~dwalker).  It's a simple interface and could probably be
cleaned up a bit but it works.  :)

>     Anyway, it sound like a good project, and finally one where I
>actually have a clue of what is going on, and the skills to help. So, if
>you are interested in pursing this project, I would be more than glad to
>help. TTYL.
>


    That would be a Good Thing.  Have webspace somewhere?  If I can get
permission from the "powers that be" at the office I could host a website on
our (Domino) webserver.

                Damond


************

From owner-pgsql-hackers@hub.org Sun Dec 26 19:11:48 1999
Received: from hub.org (hub.org [216.126.84.1])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id UAA26661
	for <pgman@candle.pha.pa.us>; Sun, 26 Dec 1999 20:11:46 -0500 (EST)
Received: from localhost (majordom@localhost)
	by hub.org (8.9.3/8.9.3) with SMTP id UAA14959;
	Sun, 26 Dec 1999 20:08:15 -0500 (EST)
	(envelope-from owner-pgsql-hackers)
Received: by hub.org (bulk_mailer v1.5); Sun, 26 Dec 1999 20:07:27 -0500
Received: (from majordom@localhost)
	by hub.org (8.9.3/8.9.3) id UAA14820
	for pgsql-hackers-outgoing; Sun, 26 Dec 1999 20:06:28 -0500 (EST)
	(envelope-from owner-pgsql-hackers@postgreSQL.org)
Received: from mtiwmhc02.worldnet.att.net (mtiwmhc02.worldnet.att.net [204.127.131.37])
	by hub.org (8.9.3/8.9.3) with ESMTP id UAA14749
	for <pgsql-hackers@postgreSQL.org>; Sun, 26 Dec 1999 20:05:39 -0500 (EST)
	(envelope-from rkirkpat@rkirkpat.net)
Received: from [192.168.3.100] ([12.74.72.56])
          by mtiwmhc02.worldnet.att.net (InterMail v03.02.07.07 118-134)
          with ESMTP id <19991227010506.WJVW1914@[12.74.72.56]>;
          Mon, 27 Dec 1999 01:05:06 +0000
Date: Sun, 26 Dec 1999 18:05:02 -0700 (MST)
From: Ryan Kirkpatrick <pgsql@rkirkpat.net>
X-Sender: rkirkpat@excelsior.rkirkpat.net
To: Damond Walker <dwalker@black-oak.com>
cc: pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] database replication
In-Reply-To: <002201bf4fb3$623f0220$b263a8c0@vmware98.walkers.org>
Message-ID: <Pine.LNX.4.10.9912261742550.7666-100000@excelsior.rkirkpat.net>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: owner-pgsql-hackers@postgreSQL.org
Status: OR

On Sun, 26 Dec 1999, Damond Walker wrote:

> >     How about a single, seperate table with the fields of 'database',
> >'tablename', 'oid', 'last_changed', that would store the same data as your
> >PGR_TIME field. It would be seperated from the actually data tables, and
...
>     The problem with OID's is that they are unique at the local level but if
> you try and use them between servers you can run into overlap.  

	Yea, forgot about that point, but became dead obvious once you
mentioned it. Boy, I feel stupid now. :)

>     Using the PGR_TIME field with an index will allow us to find rows which
> have changed VERY quickly.  All we need to do now is somehow programatically
> find the primary key for a table so the person setting up replication (or
> syncing) doesn't have to have an indepth knowledge of the schema in order to
> setup a syncing schedule.

	Hmm... Yea, maybe look to see which field(s) has a primary, unique
index on it? Then use those field(s) as a primary key. Just require that
any table to be synchronized to have some set of fields that uniquely
identify each row. Either that, or add another field to each table with
our own, cross system consistent, identification system. Don't know which
would be more efficient and easier to work with.
	The former could potentially get sticky if it takes a lots of
fields to generate a unique key value, but has the smallest effect on the
table to be synced. The latter could be difficult to keep straight between
systems (local vs. remote), and would require a trigger on inserts to
generate a new, unique id number, that does not exist locally or
remotely (nasty issue there), but would remove the uniqueness
requirement.

>     Oops...how about defining a trigger for this?  With deletion I guess we
> would have to move a flag into another table saying we deleted record 'X'
> with this primary key from this table.

	Or, according to my logic below, if a row is missing on one side
or the other, then just compare the remaining row's timestamp to the last
synchronization time (stored in a seperate table/db elsewhere). The
results of the comparsion and the state of row existences tell one if the
row was inserted or deleted since the last sync, and what should be done
to perform the sync.

> >     Yea, this is indeed the sticky part, and would indeed require some
> >fine-tunning. Basically, the way I see it, is if the two timestamps for a
> >single row do not match (or even if the row and therefore timestamp is
> >missing on one side or the other altogether):
> >     local ts > remote ts => Local row is exported to remote.
> >     remote ts > local ts => Remote row is exported to local.
> >     local ts > last sync time && no remote ts =>
> >          Local row is inserted on remote.
> >     local ts < last sync time && no remote ts =>
> >          Local row is deleted.
> >     remote ts > last sync time && no local ts =>
> >          Remote row is inserted on local.
> >     remote ts < last sync time && no local ts =>
> >          Remote row is deleted.
> >where the synchronization process is running on the local machine. By
> >exported, I mean the local values are sent to the remote machine, and the
> >row on that remote machine is updated to the local values. How does this
> >sound?

>     Having said that, a good algo will have to be written to cut down on
> network traffic and to keep database conversations down to a minimum.  This
> will be appreciated by people with low bandwidth connections I'm sure
> (dial-ups, fractional T1's, etc).

	Of course! In reflection, the assigned identification number I
mentioned above might be the best then, instead of having to transfer the
entire set of key fields back and forth.

>     What would a vacuum do to a system being used by many people?

	Probably lock them out of tables while they are vacuumed... Maybe
not really required in the end, possibly optional?

>     It could probably be named either way...but the one thing I really don't
> want to do is start hacking server code.  The PostgreSQL people have enough
> to do without worrying about trying to meld anything I've done to their
> server.   :)

	Yea, they probably would appreciate that. They already have enough
on thier plate for 7.x as it is! :)

>     Besides, I like the idea of having it operate as a stand-alone product.
> The only PostgreSQL feature we would require would be triggers and
> plpgsql...what was the earliest version of PostgreSQL that supported
> plpgsql?  Even then I don't see the triggers being that complex to boot.

	No, provided that we don't do the identification number idea
(which the more I think about it, probably will not work). As for what
version support plpgsql, I don't know, one of the more hard-core pgsql
hackers can probably tell us that.

>     The only thing we'd need for Python is the Python extensions for
> PostgreSQL...which in turn requires libpq and that's about it.  So, it
> should be able to run on any platform supported by Python and libpq.  

	Of course. If it ran on NT as well as Linux/Unix, that would be
even better. :)

> Unix folks should be happy....assuming they have X running on the
> machine doing the replication or syncing.  Even then I wrote a curses
> based Python interface awhile back which allows buttons, progress
> bars, input fields, etc (I called it tinter and it's available at
> http://iximd.com/~dwalker).  It's a simple interface and could
> probably be cleaned up a bit but it works.  :)

	Why would we want any type of GUI (X11 or curses) for this sync
program. I imagine just a command line program with a few options (local
machine, remote machine, db name, etc...), and nothing else.
	Though I will take a look at your curses interface, as I have been
wanting to make a curses interface to a few db interfaces I have, in a
simple as manner as possible.

>     That would be a Good Thing.  Have webspace somewhere?  If I can get
> permission from the "powers that be" at the office I could host a website on
> our (Domino) webserver.

	Yea, I got my own web server (www.rkirkpat.net) with 1GB+ of disk
space available, sitting on a decent speed DSL. Even can setup of a
virtual server if we want (i.e. pgsync.rkirkpat.net :). CVS repository,
email lists, etc... possible with some effort (and time). 
	So, where should we start? TTYL.

	PS. The current pages on my web site are very out of date at the
moment (save for the pgsql information). I hope to have updated ones up
within the week. 

---------------------------------------------------------------------------
|   "For to me to live is Christ, and to die is gain."                    |
|                                            --- Philippians 1:21 (KJV)   |
---------------------------------------------------------------------------
|   Ryan Kirkpatrick  |  Boulder, Colorado  |  http://www.rkirkpat.net/   |
---------------------------------------------------------------------------


************

From owner-pgsql-hackers@hub.org Mon Dec 27 12:33:32 1999
Received: from hub.org (hub.org [216.126.84.1])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id NAA24817
	for <pgman@candle.pha.pa.us>; Mon, 27 Dec 1999 13:33:29 -0500 (EST)
Received: from localhost (majordom@localhost)
	by hub.org (8.9.3/8.9.3) with SMTP id NAA53391;
	Mon, 27 Dec 1999 13:29:02 -0500 (EST)
	(envelope-from owner-pgsql-hackers)
Received: by hub.org (bulk_mailer v1.5); Mon, 27 Dec 1999 13:28:38 -0500
Received: (from majordom@localhost)
	by hub.org (8.9.3/8.9.3) id NAA53248
	for pgsql-hackers-outgoing; Mon, 27 Dec 1999 13:27:40 -0500 (EST)
	(envelope-from owner-pgsql-hackers@postgreSQL.org)
Received: from gtv.ca (h139-142-238-17.cg.fiberone.net [139.142.238.17])
	by hub.org (8.9.3/8.9.3) with ESMTP id NAA53170
	for <pgsql-hackers@hub.org>; Mon, 27 Dec 1999 13:26:40 -0500 (EST)
	(envelope-from aaron@genisys.ca)
Received: from stilborne (24.67.90.252.ab.wave.home.com [24.67.90.252])
	by gtv.ca (8.9.3/8.8.7) with SMTP id MAA01200
	for <pgsql-hackers@hub.org>; Mon, 27 Dec 1999 12:36:39 -0700
From: "Aaron J. Seigo" <aaron@gtv.ca>
To: pgsql-hackers@hub.org
Subject: Re: [HACKERS] database replication
Date: Mon, 27 Dec 1999 11:23:19 -0700
X-Mailer: KMail [version 1.0.28]
Content-Type: text/plain
References: <199912271135.TAA10184@netrinsics.com>
In-Reply-To: <199912271135.TAA10184@netrinsics.com>
MIME-Version: 1.0
Message-Id: <99122711245600.07929@stilborne>
Content-Transfer-Encoding: 8bit
Sender: owner-pgsql-hackers@postgreSQL.org
Status: OR

hi..

> Before anyone starts implementing any database replication, I'd strongly
> suggest doing some research, first:
> 
> http://sybooks.sybase.com:80/onlinebooks/group-rs/rsg1150e/rs_admin/@Generic__BookView;cs=default;ts=default

good idea, but perhaps sybase isn't the best study case.. here's some extremely
detailed online coverage of Oracle 8i's replication, from the oracle online
library:

http://bach.towson.edu/oracledocs/DOC/server803/A54651_01/toc.htm

-- 
Aaron J. Seigo
Sys Admin

************

From owner-pgsql-hackers@hub.org Thu Dec 30 08:01:09 1999
Received: from renoir.op.net (root@renoir.op.net [207.29.195.4])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id JAA10317
	for <pgman@candle.pha.pa.us>; Thu, 30 Dec 1999 09:01:08 -0500 (EST)
Received: from hub.org (hub.org [216.126.84.1]) by renoir.op.net (o1/$Revision: 1.10 $) with ESMTP id IAA02365 for <pgman@candle.pha.pa.us>; Thu, 30 Dec 1999 08:37:10 -0500 (EST)
Received: from localhost (majordom@localhost)
	by hub.org (8.9.3/8.9.3) with SMTP id IAA87902;
	Thu, 30 Dec 1999 08:34:22 -0500 (EST)
	(envelope-from owner-pgsql-hackers)
Received: by hub.org (bulk_mailer v1.5); Thu, 30 Dec 1999 08:32:24 -0500
Received: (from majordom@localhost)
	by hub.org (8.9.3/8.9.3) id IAA85771
	for pgsql-hackers-outgoing; Thu, 30 Dec 1999 08:31:27 -0500 (EST)
	(envelope-from owner-pgsql-hackers@postgreSQL.org)
Received: from sandman.acadiau.ca (dcurrie@sandman.acadiau.ca [131.162.129.111])
	by hub.org (8.9.3/8.9.3) with ESMTP id IAA85234
	for <pgsql-hackers@postgresql.org>; Thu, 30 Dec 1999 08:31:10 -0500 (EST)
	(envelope-from dcurrie@sandman.acadiau.ca)
Received: (from dcurrie@localhost)
	by sandman.acadiau.ca (8.8.8/8.8.8/Debian/GNU) id GAA18698;
	Thu, 30 Dec 1999 06:30:58 -0400
From: Duane Currie <dcurrie@sandman.acadiau.ca>
Message-Id: <199912301030.GAA18698@sandman.acadiau.ca>
Subject: Re: [HACKERS] database replication
In-Reply-To: <OFD38C9424.B391F434-ON85256851.0054F41A@black-oak.COM> from "DWalker@black-oak.com" at "Dec 24, 99 10:27:59 am"
To: DWalker@black-oak.com
Date: Thu, 30 Dec 1999 10:30:58 +0000 (AST)
Cc: pgsql-hackers@postgresql.org
X-Mailer: ELM [version 2.4ME+ PL39 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Sender: owner-pgsql-hackers@postgresql.org
Status: OR

Hi Guys,

Now for one of my REALLY rare posts.
Having done a little bit of distributed data systems, I figured I'd
pitch in a couple cents worth.

> 2) The replication system will need to add at least one field to each 
>    table in each database that needs to be re plicated. &nbsp;This 
>    field will be a date/time stamp which identifies the &quot; last 
>    update&quot; of the record. &nbsp;This field will be called PGR_TIME 
>    for la ck of a better name. &nbsp;Because this field will be used 
>    from within programs and triggers it can be longer so as to not 
>    mistake it for a user field.

I just started reading this thread, but I figured I'd throw in a couple
suggestions for distributed data control  (a few idioms I've had to
deal with b4):
	- Never use time (not reliable from system to system).  Use
	  a version number of some sort that can stay consistent across
	  all replicas

	  This way, if a system's time is or goes out of wack, it doesn't
	  cause your database to disintegrate, and it's easier to track
	  conflicts (see below.  If using time, the algorithm gets
	  nightmarish)

	- On an insert, set to version 1

	- On an update, version++

	- On a delete, mark deleted, and add a delete stub somewhere for the
	  replicator process to deal with in sync'ing the databases.

	- If two records have the same version but different data, there's
	  a conflict.  A few choices:
	  	1.  Pick one as the correct one (yuck!! invisible data loss)
		2.  Store both copies, pick one as current, and alert 
		    database owner of the conflict, so they can deal with
		    it "manually."
		3.  If possible, some conflicts can be merged.  If a disjoint
		    set of fields were changed in each instance, these changes
		    may both be applied and the record merged.  (Problem:
		    takes a lot more space.  Requires a version number for
		    every field, or persistent storage of some old records.
		    However, this might help the "which fields changed" issue
		    you were talking about in #6)

	- A unique id across all systems should exist (or something that
	  effectively simulates a unique id.  Maybe a composition of the
	  originating oid (from the insert) and the originating database
	  (oid of the database's record?) might do it.  Store this as
	  an extra field in every record.  
	  
	  (Two extra fieldss so far: 'unique id' and 'version')

I do like your approach:  triggers and a separate process. (Maintainable!! :)

Anyway, just figured I'd throw in a few suggestions,
Duane

************

From owner-pgsql-patches@hub.org Sun Jan  2 23:01:38 2000
Received: from renoir.op.net (root@renoir.op.net [207.29.195.4])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id AAA16274
	for <pgman@candle.pha.pa.us>; Mon, 3 Jan 2000 00:01:28 -0500 (EST)
Received: from hub.org (hub.org [216.126.84.1]) by renoir.op.net (o1/$Revision: 1.10 $) with ESMTP id XAA02655 for <pgman@candle.pha.pa.us>; Sun, 2 Jan 2000 23:45:55 -0500 (EST)
Received: from hub.org (hub.org [216.126.84.1])
	by hub.org (8.9.3/8.9.3) with ESMTP id XAA13828;
	Sun, 2 Jan 2000 23:40:47 -0500 (EST)
	(envelope-from owner-pgsql-patches@hub.org)
Received: by hub.org (TLB v0.10a (1.23 tibbs 1997/01/09 00:29:32)); Sun, 02 Jan 2000 23:38:34 +0000 (EST)
Received: (from majordom@localhost)
	by hub.org (8.9.3/8.9.3) id XAA13624
	for pgsql-patches-outgoing; Sun, 2 Jan 2000 23:37:36 -0500 (EST)
	(envelope-from owner-pgsql-patches@postgreSQL.org)
Received: from falla.videotron.net (falla.videotron.net [205.151.222.106])
	by hub.org (8.9.3/8.9.3) with ESMTP id XAA13560
	for <pgsql-patches@postgresql.org>; Sun, 2 Jan 2000 23:37:02 -0500 (EST)
	(envelope-from P.Marchesso@Videotron.ca)
Received: from Videotron.ca ([207.253.210.234])
	by falla.videotron.net (Sun Internet Mail Server sims.3.5.1999.07.30.00.05.p8)
	with ESMTP id <0FNQ000TEST8VI@falla.videotron.net> for pgsql-patches@postgresql.org; Sun,
	2 Jan 2000 23:37:01 -0500 (EST)
Date: Sun, 02 Jan 2000 23:39:23 -0500
From: Philippe Marchesseault <P.Marchesso@Videotron.ca>
Subject: [PATCHES] Distributed PostgreSQL!
To: pgsql-patches@postgreSQL.org
Message-id: <387027FB.EB88D757@Videotron.ca>
MIME-version: 1.0
X-Mailer: Mozilla 4.51 [en] (X11; I; Linux 2.2.11 i586)
Content-type: MULTIPART/MIXED; BOUNDARY="Boundary_(ID_GeYGc69fE1/bkYLTPwOGFg)"
X-Accept-Language: en
Sender: owner-pgsql-patches@postgreSQL.org
Precedence: bulk
Status: ORr

This is a multi-part message in MIME format.

--Boundary_(ID_GeYGc69fE1/bkYLTPwOGFg)
Content-type: text/plain; charset=us-ascii
Content-transfer-encoding: 7bit

Hi all!

Here is a small patch to make postgres a distributed database. By
distributed I mean that you can have the same copy of the database on N
different machines and keep them all in sync.
It does not improve performances unless you distribute your clients in a
sensible manner. It does not allow you to do parallel selects.

The support page is : pages.infinit.net/daemon  and soon to be in
english.

The patch was tested with RedHat Linux 6.0 on Intel with kernel 2.2.11.
Only two machines where used so i'm not competely sure that it works
with more than two. -But it should-

I would like to know if somebody else is interested in this otherwise
i'm probably not gonna keep it growing. So please reply me to my e-mail
(P.Marchesso@videotron.ca) to give me an idea of the amount of people
interested in this.

Thanks all.

Philippe Marchesseault

--
It's not the size of the dog in the fight,
but the size of the fight in the dog.
                        -Archie Griffen



--Boundary_(ID_GeYGc69fE1/bkYLTPwOGFg)
Content-type: application/octet-stream; name=replicator-0.1.tgz
Content-disposition: attachment; filename=replicator-0.1.tgz
Content-transfer-encoding: base64

H4sIAOeZbzgAA+w8a3PayLL7Nara/9Dx3iRAME/brO11tjDGDjc2OIBvTm5OihLSAFoLiUgi
LGfX//12z0MaAX7UOU72nLpW7cagmenp6enp9xCwmetYZuQHxR++1QM7pVq5DD8AQG2vxv+W
d3b4X/mUsKVUrdbKO7XKLrZWdkqVH2D3m2GkPfMwMgOAH2Z+GI0DFt7ejwW3Nv7nPkGy/xfm
NRs5Lnv0Ocql0l6pdOv+l8u1XbH/e7W90m6NWsu7pR+g9OiYbHj+n+9/q904vzppwhFst6A4
D4NiGFjF2Tj84vJPjme5c5vFra5vma5sl21FwzBdFw7AGwb+NQtgKv8mvAXYZWw6HnayXGZ6
d3cVXQ7AeBZMYXukOquv0/TXZKhhGLJrjEvBAvlhYjwbW5b2/r8ycu1Z2B6P7SFs+8lMYRQ4
s/irodA8UJOvwZ3eCXeahqu+GtqyD7SFrEFPNW2aQAO07c6+wPb52m65ztB49pNAQKfZzIys
CR7DA+OZ7YxGm5hgaFrXzLOLkeXPiuqgFHzXfnhnC96A+rJX2C3wWf9q5n96dPn/zbaH5L+U
95vkf6VUS/T/Xq2Crbt7ldqT/P8eD+6NWS6V8+VSzXgDxdzlBfTnDE6YBZWfcSMOqtWDnX0o
7+/v54rY4yelELaKE3/KYqYpaoykZNeW3v+XcBkWo+UMZcHkzVqDM7M2vZ6GY/G6mHuOmHEM
jL3arrn3cxU/PnO8SJmQan72fs6C5SH22sNeu/m9/T3qWcwBAuiZEfw36hYoQ7l6UKrg6pD9
SiXqATlojSCaMECOiNiUeVFIHwP8s3CiCZjw6vWrPPXw0t3AQkqEMAr8qQREzZpIngW+xcKw
IFppEepxRpD5QggPSDB740+lz3B0RBNl/+C9Ya0VXsGrQ9GWXjM2ndbPe03eeEP/MDdkf9zS
td+90npy+iJme7V9a29/3/gFnskH4CVHIQoYG7hOGOXBZvSvabmdrywIHJtlD43t7W0x0V3P
gyDVSjsmegH5WnWfID7j/6wxZvWgXDso7SSMSX0APuC2oPKRG4DThRHR2IleheD5Ee5hyFxm
Rcnm8aEgQcTbQThuv8F9nZqe3UeuhedH0Lg4GfSa581GH16+XKGo2C5cf69/0R9c9M4AefdQ
viM2nhLvI93L6mU4wy2NRhnRyH5HQmy9fhFu5VNbnlXd5R+HDwg9m/6MWZRpd06ag3fNj3lo
XTYGjW6z3oc/obS3t5fNw0vslAfEZtBr/W8zD6Us/FKKUQWYIdX9ILN1EY57SLVTE81/+wC2
4lk5cwgWURshGOWvlltPz+M8mtiOTdjHnuNu/69cre0I/69Sq5XLpSq1Vit7T/r/ezzFnP4Y
/YkTQivk8nOGhrw5ZvgZtSZ6eP4ijIWe43vKnO+9PwfbjMyhGTJAF880bIdk13AeMRuY99UJ
fI8kbcEwGv5sGTjjSQSZRpbLboDLieM6M5SNF2ZgTVBNMnPuRoZABTXnODCn4JB6ZSjc/VG0
MAN2CEt/Dhaq8oAl06GcB5TXRXRrpj46M0t6Mfds9NxoQRELpiH4Qsmfta/gjHksMF24nA9x
VXDuWMwLmWHivPQmnOAChkve/ZRm78nZ4dRHqJwKh8DQNsAJUIGFRJWKISeQ0PKABkAGKYgI
B+DPaFAWsVyCi7ojHldYX7BORkeYHBN/JvcDF7Zw0OseMkDOHM3dvIE94UOr/7Zz1Yd6+yN8
qHe79Xb/4yE3X3xsZV+l6eJMcRsRLC4mML1oiUQxLprdxlvsXz9unbf6Hwnt01a/3ez14LTT
hTpc1rv9VuPqvN6Fy6vuZafXLKC6Y4QQM24nJ4wQ0tRHqtksQgWDZpDxETcvRJTQg5yYX0lV
W8z5igiZaEvNlg/ZI9f3xsIuizS6HZK2R0Wfh0XgIENE/vruGcnuodL0rEIedvehz5AmDC5d
02KwDb05Da9WS3k4Rj6nrhd1gFKlXC5vl6ulWh6uevWCkTo+RcNAIyQJBRxAf0J8HdJGyxCC
Iw+XMAlxBeh8R7gJ4RS3lO+NhwSbiiNGJiXSxPNtZqBZqT8CXKhW6HMmpI6qpUCTI9Vxwvbr
8goWwdzz0LoA31uFS8Cmoi9KPOROvn9tDlrABIQmV4FNBI0vwrQsNkNb2fI9Dy0sRB93GnIE
/wOZzAsmrGkSJfTF9gnKwsRl0wwEThuKJ5060QmnSQiKJBla2pmLeq/f7MJxt/MO/1x2Ow1k
02Yva5Ahp7kQke345ECkXrnOcPUdWVrpd2gYeatD0SEJfeuaRen3Hosc/L/oeOv9aXn01rjF
D9roBW30gZK3W4l/ZRhffQeF7O9ONPDwZLjLDLlEhOWpjXYeWnjCyNxquH5IG56QmIw8AAvf
s4wcQC8IVqaEH2+ImXsT349WvB3JcZp/E02CuWAc3B9SGWjBzhluBceuPpu5y54anrEmZpBD
cFM8pN3mWavX79b7rU4bN3gQsDHHmhbxxbHzgIZul4WoDwi1tGmNL4q5fhoRjzGbH4lrz1+I
07NANzWF2wLFO/oIEaKXtsz59NtvWic0F/KENVvqxjlhzClEExdPjq/OFG0z8sRnIV7lAbxA
Nn1hZ//uoUGfghK4zEvgZhVIXC7iII16wd+3mPWHdyDw/qp1cnAEL2w+L8JU4BO/gdP1NsdA
dwq0ZZFLNeLOgeAaDpID5eeEXNZm5bh1lo35jX9VnVP96g08rFpH/r23setp/eq8r3Xl3zf2
bJ10L7SO9HVzv3a/q/fDr7f0+5/6eaojft/Ys91BKmo9+XfVkx+mMv9yIw6UMLNQmnFtfzr3
+FnkTqsQqa0TMmHAM6dMqUEu1ddOHXq05ERzkcfPWZeNHZLbbeyeuedkrR8nvqX6cUAIg/7H
yyY1Tdl05TRwiMjOzj+YP0rN9k9y9L/Ko2L1gdCbTwz7CAyLNOMsC2/JjOD6GY1G5B+p/Ikt
j4UqSpiwIfRL32/HrasqKeY7cjAYl+w6/yCLJzt5KFnWm0+PlxELlfzNdYXFmOrMJTquNaN6
I/uhZflVqjfkKL33ZuYlNruTzxrcaqVYUrABhY0HH9JHM4WF4v0FekEMR4hp9RWmI5J8jXJP
b1+noGy8QkXx1dU9cIVqUSsLo+fGkCeLMG6gsYD2/oL8ExRXaOBJ+5zJrY6RD7HBmmTEWyFx
NJQs8iTPO413g8t6412zf4BWKzOvD1c7nMXtsgPi0PFc9Kz8cXrWZ3wEJ4MaYjyDVcskRodL
uNVtegYb0Wj+rZUA1Q0xZVMhUk18jQRFq95dHnC6MImjE/4aEwXQQRqR73uQkGKzpJPO0tC0
5RIpQqnvEmzEJG6+0XYP/yczihzsWBkJz8xjX3lmNpoHdLQS9bX5KRp0UKem42X4Xl6z5QDP
PVp7zH7HlofSWsPXSpeJNlIufrBEFTieCrNMnXm2uERNBZufYu4qFO45+QeZrDj8ZJ6awwYe
Jjtg3qdy6fOhWp6UWqQzSbGSN27JfvFYLx6KR6q0OnUaEB6+IdIHQXEwtKfclw1n5sJjNoeJ
7DO3hPAzbTsYoBvPU7J1/KJEGzWOUCELg7aH5J6tOEO4yejaFnJFgYYUcmIc4il8kkz9dNBq
N/t56NHJQYHWrF/I8w7Jgb/ruFvI3hGT8BJ2Sh35G6VM43UU0K8YjMyp41JaQWJxuNZj5geU
qzmCCTrTYeYcRW6zPbjsdPvZ9c4m/8D/xCNa7frJSXdQb3/kZ+qcBqApwek8/AcL/MzLTBoK
vUSR93NiASDdhg7aF4J2ecisbFAuCy9jGLH4XOn0UP1AM+n6QD+OI1vqBoEUMSV6BQqtYxQn
A5Rv+tbdNZMY/pC5ijkZBzj3/VlIR4cOOadhWgNt4Fyh/RXrxme0x6VL8gop30O6EWNuJB+C
ynLvjbD5oNx/buNqnlzC/VI+cn2nZkPgItxw504mCKM0l2il1d9dRBUTrDnMd1CY20xSHxZz
ZzzdlAx/LheiVi6iJxTdYYs4HCS7o2CTMjIxstJ0uOTGtRB9Dz/h1P8hy9ARbY1ItJnKBOSy
TqKboKQwOtIQ4fLeU0uiBZHM1UJU4HsrsatCoRCrw02WZMwA2cM1raWrq8RM2kSNNbixxuM2
DwF4/vxuMpGkNkcsWuK0UYK1IJxIuyoifCCdx2YU5LOuVzVPgqiuthI19Po1ZXylJkzvzQ0Z
FujmIMQPdHQ5pI1benNH/H9D2v7Rcwz31P9VK3s7qv6vWqtWeP3fTvUp//M9nqf8z1P+5yn/
83j5H+MnZ4QMN5K5icHbH42f8KvjMe1N/Oqi/rdBo9NuNxsUfOhRLVLcplnIXEzu7lSrCSxp
ISrXpFJKA20fJ63VSnm/koBFt/ld86MaCLs/7yRDVTGJatwraQglITw5Eo9v0ijDdGpkuVRB
uCLWf3Lcrl80j/7Ymi7t4dbNITmSypMKtYyCTDvRN4asTSzOQwkUjyS9ibpcSB6HDKUppTMc
Mg+pA5GcoUsGf2iBg7wWJMjr7n9ed9vzcajzBujfQeeyKaJBiKmCLczLP7ingvPykAXPn+AK
gUcMPikSfBbRMxV0wW+x4xwHfvG/eZhIl8TK48HfdaLIxsiX0SYWGLLMjKxHbjrBBZ0xPXtI
4hHlA5rDOJOQNsrz5quZB4xH7mDDKosS0dYJODai44wcFgIzrQmX+3KrkjQintwpNhIrZBx+
egOmEJiitPs1cdBFgoUTboKg2uaUfdpFR924ScUBkfh4dpiHchyxUYcnV/zxqfboER/N/vP+
mvof2N2t7Kj6H5Rxu7z+p7z7ZP99j+fJ/nuy/57sv0et/7mj2sXxnMgxXQqP3RoA/5E0O1kB
vDRGi55oJpKQ1NQT1eG/Y71Lqre9Mn/kTNm3LIrR8hA/bg7CCzKqUKbsQMYIGV453Lcx43aJ
nibPqTz5vclKFZN0WfxKptyFYXVpBiFTMf6RM55Ls5YuX8a1UEJSiPiQIeNIVN4iA4ZyY2Tk
6RHyAd6/kg8oFvudkw78Znrz0BDRLEVCRAg/EW2HSypqyGzxXgpcshO35hHWOt2TSljrz9MI
1D+XUWFi+TaX1TDdfjPhLyUQlVNYAbaeVNictFH7It/cGahOprgr57CeCti0Z4m8EFBvyUhL
vCl+OA/Ws+hyy/iG6Y0FZbFz7halSDleFIiOgvI3vCWKZJTOc3QbuPAzw+vVkjFicx+dBREV
D1EL2f40Q5Ih0746PxdFJKmZcYIjkP14KxKXymZiyj4wu/9AQvJrJA9I7ce05NKeRtHSabVk
FKXlNl9rqizmjtsbh2s5mdWCnY2lFEmCQaVWzLj0TZaTKlP3C5oP4rJmEsEWNTiB9fWWGpx0
Gc4mEqq6BVF+A4nc0LLnWoJBrwFAumhO+qFKRFHpUTo3n5TPpYt70vxwVxXEg4UggbxD9tH/
SQ3lveH2f7tH8//0S8aPOsd9/l9V+n90/3Nnl+5/7JbK5Sf/73s838gES16i9Tn7sj1i/HUx
d+b6Q9Mls+byjHRVDsYDSn3hx08UM/x8aKRLF3mzXtynuol6DUrihTyiCOl0oFSEFGcjISys
JinuVOIxQMM7EtG3cK2A7Vj0lfXKswGOp8m40ODBKzlDT1zN3C1XPitrz7nXTDTimsoVrRXP
c0dtJVU5kd9IMTkqtUZVnJoAvFd073GGfqzNfjPR0QRunl3URbUEKoWMc1RCX+mXmILw+rUj
5WEx1/ahzUQOWoUbxbVXzxc2KDbI1cdqAzLrO+V8pnJqygqvaPKsFKcy26rp0nUgEkPKg6ar
WrOoVV3XtzK3EkqS+HaYDzIaOKjXr2NekwpH3d9MsUEetshCOnqBHilaqUe7O9UK2EOyo474
3c5bjCkRIt9Q2v0i5JXcqUl4t+TYOESay/eyiz3MrHc2xA5dvqdCwnmY0QdnaX+SJMTguH4i
GUHEtsPxp3Ll589SGyeXVnGpf/cyidKIM+vEgcgg8vzAi/AAqSGLcR9CgFgf4yR31lHH0fT4
0JszimuE6xcWeOBann4aFqLVazHun6hyLq3gWomJxcQXsfj5Csik3pWXEMbCIswk3uJsQPgT
JS/PAn6HIYeLD1My4iGH8dQJ8DBbE2Zdg7NyQR1N7qk063gUJEacAg7eqwgWplg/t2adSNWg
FIsOdzGs6Uz5Y2SBEjccHaFxhByEPv6cxeXDfDHbb6SVdrThmIqzvjpUzpbiaHE3QYcodzng
layX79nvzEqxqNY7sfmIoZ/TkD//xDGCwj3B3vglSxe2L8+6zd6g0bm4qLdPBp13scEXc3al
tKNY+x7mjstEgY6kNG7JvCYeBnFMdSzzt9BIsXxsUq4xO9BhdpkZ8HXoBrNeWLK5zw0SWxaF
IG4rNZv8qlcH3b3f/CFnexLvps1DVNoNmVAxVBxu4gqSx6QUK/EYFQ7z6W69cnKUCaXYMOR5
IQk8uT8m7qT5VKiLHiBBcjzkaM+S50+Lgmr5pQQdylmlq01/TEdZ5DECPFXGP3ET4T6n6y/x
r1a48R4nS3xedbUc9WMD8hCrtKdWsfVBFrGmNT/uiG49pau/YN1kWnXPbimLEjPp8qwg5Knk
KV1oJ3VhqxL3ZXJyhEP2H+V//dWP5v/hOTi5aH6DOe75/Z+9cq0W+3/lCv3+G358uv//XZ5u
EpL7WiqUDcM4b120+tzy7cmUlMwD0sVJLrUj9C4oeUTJFxd1/e+Abp/HXKgUKoVymaR1l9lv
zUi27hVKBaPuhn4qrUjgdEgC9II8i+lsHtHdZin2PYavg+sC/T5P6E8ZaY0JjqYK3ZCrMZ5e
kvUHiDAqxlD+EAyaRzzxRj/lQvdnCpChlFX9qv+208Uu3G40hsz1F1nDaIEZhvMpaSuyRma+
F5pDx3UiMiqTZJPKQhbgCueJc3sLDwInvMY5WrCgAJLhOtdSlgpx7Xi4skS9cp5aVXp5uvqs
sptkTZkWKjpmmOIermkHjJSx+MaErufjbd+aCyHa15EEDpptm+5sYj7nqTeOHIVqIz8ykeoX
9ROO5dwTkCiLS+jYc2FW60lcIABccg8Z8wycwWN2gX7/Rz2G8eEtatRWD1r9X9Ms5JAVQQUp
28JHSFLISLM4yUz1IiKlzHeRLqGHwEOlvayBo4gZcPR4wsgaoIEhMnEkbPTIxB0Ac8qrdAiQ
MF5wmDPFJSHeqFJ5jQwaHGEhjZ/tM/E7QUuGQOcznmmg2xeui+zNf5cHKM3KE3NEQwph2+YS
9TtZNgbn4YW5jPdvw5x5osJwmaR3ycARPMrr1YntDfolPjSUKMQh7Ka6NKPk7w9pP1FFW6yu
xseRDX6JQZ2IgsFvRdH7dqe9fSuImTk2tdLzpCybUrbG284HOOk0aVfhQ6f77leDJzrp1wm8
NZi0RpXUzSeXWMmhinO9aY+KZ7P5kU3ngFNuHP8dJ7E2vY48idvym+ho1dG+slQfHr9Irtmu
XWfXTF2+l15ee21wSPEvS8WG8vpVeFXIv2K5huSnye0wMglgDWPQadx/2wTyW67arYaIINUv
Ou0zXqDXk+Tvo9xzPGeqsslobwUmUsfkXMCNJ48Hktbse9pvrcbMSGWV11eP7hJd7OPL5dfp
nWidFgU4UTxrqHKQUJ6SuUx3ciqIWfJJMQJ/q1yAAjQV4kAyPvKN2BDFI87cUXKpMLVlHQHM
JikcOrTsib+gKwF55TXrW2mjQBdLxAW5hB1tVhj5MzprkAkZo/pCd5mFse/bVANnamKTV2CM
jERLMJ3hN/pPYqkyPSQYk4nfjdM8mwIdNC491Ulrd+izsU1JYyGFxA+YhbLjqx78X3vX2tzG
dWQ/79T+iAlSFYlVEEyKlmRbqexCJCQhoUiZDzv6tBkSQ3BsAIOdAUhjf/32Od195w4ethMn
dtUuUXFsAjP32befp/ueDN5eppdn8th/yHOXEZAy+z4v71WDn43lt2+9dJ09IGP+Xo4d8CUL
2Rd5qNYFDKadHLeQqiAN9MGg00tznd5nfEoZrBH67WTVCw8Ko8dshd6XdbqcA3MxG+9ZQZOe
DTa7Lu+1WFsB2xECXv6vYpLepByPeYoc+omXPoihXzyTcwPblSJxHkBJpFFKJ7cvXazizeEm
N3Y2s6VaX/omv8mWTGjNZ2sPweAUTQFnqc1KMjtzsqBTUS3b9U1aT97K2a3vxJhNP+TZjEZn
d53v8qSOSiUZ7qIJ9xsnxbqc3CtKSWaNXU7Xp8OF2CiKF6UXyUHIyHbrnMvcFKdBdZiGkRj9
Z7TQ1sw2DmqKsr3ZTMfvW+ioQll+1CCxc0BIAaZRyaFZgGaS5M3VuwsnUlPzfJE7VonvtS2O
/9lBa3Xuf+t6jblewjhSLBCcSYRQLR4KsJchZVPGCiEzEeZjTn74ZEq5/9+E/dTLClirB92m
H7CZohD04hMU0RxfUm6yAs6HYXifalH5ZHvpRankkKXXxUJdstATqijR2z0JHSQPutM0dIyn
R6CxDoBEo+sOv6HkbKz2abnWFouxQBdayrFe9Ri+gDNR8Qq2cJ1iflN31MeDrvB3Ne3YIa00
AQzrIntPtYobUf8OUkuV6WQnlE9+6Pk35X/eCy8tF6JO9m6yRyjvb/2J7P/l/F/Uh9j/r37E
/k+fvzww+3//4OWrA9j/+4fPH+3/X+Pz+999Jqzms/ouSX5vduNNVcyDOrCcwxosq1FefSVP
HOzFCoar355bKQ881wdOt2nn8vPhnmkPQTzFbs2k50Uo/yD/OWv+s3nmD7/1gv0f+0TnH5rf
v6SPnzr/6SvD/7/Yf7XP+t+Hh4eP+P9f5dOcf1gC0Pv9cgL/e7b2d0Mxv/XYHz+//BOd/6Oz
j5+Gp+/++X3I+f989/n//HD/1cs0ffn54YsXL/afHwL/dfji5WP9/1/lo3XAmWgxOB2c90/S
j1dvToZHqfwzOL0YJP/mNd2/8QSXbvrnpZhmB19+eZAk6XpKzxdfdvnTzowZzblI0p2fl69e
oFppnfbvxSI+yoQhFaNxzgSM/ecHh18y9SJJB/d5tVIjERb9tFgsHJo0X9GSiTKD5Nlr6X6K
H4u8ToLXfGJZKu4979LBe3OXzeh5KGgush4IXBpwdSf/pmvyscplbJMctUwuWVmILdWW8FIv
Ghc8zfi8LsYzc69m38uX5iiuEqQ2jeAyKjUlhIPnEJDS1EvTNytGAqoM5fK3Z8Yknm5D//1C
7H3tarzMkOKTWxzkx7rCb4mP+dkzQsi/N2tYrfYmosBsLEa/malbayyjl1KHTHZkAgV4Tqnr
Y76XXdk5T6IghoEPguUZIAIPd7C0s+XirqyYFsw6umWyrHX7ZEhPL+BT0td2UWVrcjfwmtGr
kvhinxTXVVatduU4wX2ZZ6PeXsr4CFz/WRPJTrj0NmJ4CsqyB6oJmULzPGNpkFaOW9frrlT5
LW5HoFPDN7ALmkzmFX0aBHVsH1m9QXvxnmroKgmB+Ig6orOjR2ZjfOlTo51qTFJINDaWV/fS
tTs3Hor6bq8bugrODgVgSdNiJ6DcjyzYOEdgJfEXgVwqFtGrjIcppbaoUV6Hl07GeKOjRCMz
IhY4Xl/31+bdsOZYItbbHZXmSaKTrebuXJZ4dYFsCu4fuVzNXXEQJNdShIaslJUtZvOyGNfF
KBFiBXvCYuYzjQlpJ9oSBg6Srr/Xn0rsSpWHdEV9ipGMer0XFIxG/iDZXV4tcLeWuaKLELDE
8UTLuqLJ1h2NV5Kpirb8IX+SS/FWfsh/yJAh1/UntjZXL1EJugllPtzlOHbJGO5bzlgRM7e5
NMR+4CQdu/tLqKOYaxDMgxa2VlhXHCM6unp6yvjuGjnDCccD1g2kFpEXgh0R5Uk7fSGJMI6a
Pr+7fOrEwIRRjVuvlGDo0kt8axLeOLKFSixI8SB7usjn9Vfp04M9yiUVle1VF7JMnorhXCJ+
YmQSSaaHO5TXxhrV/HGSj+WYU+LVtYEt0XQ33mFNg/VtjPvjqBGN73Iv6N5V9vmk9qkQS6m5
QUrwjiPkahvBJVzw3KUw82wRxBnVYSuUnc7KJtFUXeEmQJImkB7u3onYMAdfqHfbHcIAEXFo
88zinRhfYtyijinIy0BzMA9OHOopNZlOd7tsSTHLJoi765QgZGQhRLRPKUsZCNdhqNdTXa5a
LeqW18l4JStrKzF59ATW0nKRWTV0nCT8PFl12UnMnhSHyow7MGoR91jLhYgQzt6E4xw/w5cM
ugNvJQchElVTjoQ7VjrjKDG6VJRBposeJCcmUcxGxX0xoms4La/JSLSToM904QHKhTZveNos
vSk0g8BxVeSiRa96xjQR2ltwm7uhLOI0GzE/m6jF1NfZJqTH7zroUIqsdNJ6YuoGuDxLxS2a
5zQ1vec62Bz7H04u5VMpM1Suecu6cvMVcdpN+F1pXTPG3aV/W0Lb6yX/nvykgiy/Xg7OP1yk
/dNj4KiPh1rLBUnTZlJ102PAyYdvrjSUKw9+ODsevrXYLga/bzGULaqSkSMXG/Ec6jHEQChn
IEZEFJAEsacFZO8cScwhdb1hO3flBMKlzlam2k5FA73O40zzZGu6vAxsu3rR02XvfNTxdUR7
BpSmm1BnCcOnWIjmgNF71KnDqSBcHNAL3lqCiFLtGfbRL2iDIJG8Ku5lx+5zXRAdfDPhSfbw
lZ5phbvKzKVbfdaWzbP14paJ3QcZUJnoJk2Kv9oQmAH4e0wytbPcIJuRtc/5c8eSiZzNZTbG
kj1FRWVhBLcLhKf9BatsY5kzo5C1PylEpbWfZ4nvTNqJe+9A82Tk3E6GYm1Go0phIVmddkR2
dOSg9IW936uCUNq6Emi041y0JkllEopnoyErdRg5vFYWS61suUAYnky5ltadVIBmKm8Tx0fE
S29M2TUdIEcsHoXWWsgpeyWJlHXGOXnXF8sokk8qGy0WlIjpBqEl3vNTYYP5HKrXjFaJxlKJ
cFLGJfPcMuK9HjAwvohKZNUS6vbc4c8ud8IkATEiuzpQKBWO4M8wWF1Xs2ae1LEeg+2NlWuo
zcBwyQmZihRYiiKGwHdBk9CVfizNvLhZlst6or0LzyEvF9qVb6wERsCS2CDjp5LmpBnnsUnc
TLJiqsVxXfK/1iqIheLZTLtL9LXaJdatlyiKOeEsIJIACLOYPeYWmk7wDJXIxj6MFIH20rXw
RU0/7YoS4WkrVsFdUkuHymsI6c/vVnUBPJLStR5mN9e0J1XwVtZKuxKI6XxBPYr0LwjdH9wy
d6WZlPO8oRzT7wziiFlV2wnGOaZxtkQ5W0pkBrAQOtydrLhrslTpNFY0ydrbjNAY/LbKKxc2
uYOEsI8tdCmkIQr3NM8XjkLxEL/L8a+0eGy21xgBitNg0Nt1RhRTIM+/kbXlwsocFVBHkrPk
YkJSIhuT6608R1twDjSCtWWEp0/1dBzXG+MgbRKj6M1G6wWAiZ4sM22LGZtRHBIMMNZ7XgSx
zu9qFXUxtqi9sZZLXZnaXd7CCGppVADMWi8ZVsHpGSKKp7GoRqEVENAuTcBFv07/Zs9V97D0
Lui14rmwEiB16gZrAPdUlUEMCZ+xyQujFQYb2YS6lKBR/shiKtK2c2GcCJCeJjY0DVJJtGo5
6mOqRiJpK8IyuSZM8sGqyaaIogSCVnqazcqlcBeFL1II81C0OF66leNlbMC+2G37PIVOO0FJ
INPAAn3YKdBxhBf2GoeFIoVx4lv1jsiBbLW5XWxh/cCYGM0nE5dfaM4BLvdF/rDGE9lKo+E9
HfyAUsjS1FeOpQsiWyF5RWtT4UFlEwFOGyhBF1+9BLPWkneVibU4kM9mU0MgWGzUbOdaY709
0dzdb8JnDeiq2KWyTa/sszkeNEYTr3qXiRnoOHE7AzAn+YoqQzuPZpdyyapiyBmoy5m0Rlcu
VKOKGmKjd2jlP6CNF6rO1qbvTWWN7w2vOGudQd1ZQp9xRLsOP4rmWYpoC8N/CCiiFg1NU96J
FHcNp/NyEV5I1oiOmM3QLApmZQr3chajpgmRVs2WJutChYw1VjhNaGkbbhTaW86FkvYKqAO4
cYeonadKgCvDNcCxeVAkEmxtZd24kql54uoOkS9ofOq0qnycVaMJEXG3xDA/QEyrc+xSXuxG
YQKMlP73RWCYdZMUS8Uo8v9RUa0XSew6coQ7KlhJO6kOVh0B8tzrVHbpjoZD0xXNmyT/Ia/U
/HXHmWVqL6pysnWxIwOqrESdm8Cb4eZUvVUVkDkPieEvNJozJTpsPMYqebMOAuQ8WAt/S0PJ
uq5FBmlXLe/URPa02OR9OVkyYTcBALOsAHRTnt7MT3XfhgtdV87/otEp2yRNw0rZKuUOf1xV
X5/C+uhhQqowdfXn+R5RNdffwafiPnBFKpLfQCPbIn+TCz9xBxzDcwPP7lCihBnAZWZnSl0a
sgKN/tS/Qa4N1BWWk7PdwHeTnLKuUp8yBaGh159BmGOQqkA1RkjXzryf2rh83W5NUGVNezpa
TVQ370ZaK6dZhapQS3cMNU5CCB3Vxl7LEnaDRrY5syycJ6rc3fQ+mxTaHBKVhDsv6H/Tea3y
rGKgpjErqCCRIay6ppCbBjWzBCQa0gzoUTGyCJdbCJB+ijsm1FwXLqbXLqWwrj1bWF/xVl5E
e3Na+0DFTwXwz9uD3euvM/kH9uBmF3XFBWojm5X6qSeeYINU9q/FoXZMGToKvWfZRMYyU35m
aoyFbdU9oDdNzJgMJpxSzLYNd4e7ESD08H4rXSX4an/y8HK+QUHNAtXBLJd1qSzp7WJ57dLh
WldfVBfmrsTm/W3DVNQjpmNhWFC3YxokJx5CMM48tW3LjCk0PY3nZLN40OqRC0dfe0/Yu3bp
8ZiNcQEyVI6WsJWKxmoRy26yrGmZZHVd3hTuEJMjgAqLLBtdhLJ3/rzyYQAQrUCySDWXXxhc
YX4yqj3wkE8mWaw4NDOSWb73BBDodkk9z7njuSuz3Y35xMeFIT5IDfPHJaGsmbt6glIbv/YU
Zru6C61lWaNrWiAJ9mkvTl/5jhrAVCia2ulTnSFGrAmVqprUYON7NsPEUiFwAFb1QlQ3Opl4
i2xr/rCUZFWXM+otHHPoKjG1PbMT6iUzotUTIX+7oS1ErUPFik4AojXmJyOhIxOhSc8k37Lk
eA1Fkxropjat1t9Koa4La8Yo1xrYoD5Xt6mMsjHkRVDPr5NtamWLS1r+Ubkc30W8vbCIuTo5
p/Oc6TFbhrDmLooWA1GDNP280Rm0Lh8cQequEfuPTvRQHna7LpEopYJ68x/mcOTSgDJR7+w8
UlUQzYSDSahivkio4zxQGyx3dr+7d/BPxJWUBhkrypYQAwsTZpAiBTayFffcMqwknENfYKjQ
7bqv6rPiYniYndsLCeEaWuQTDPE3Ry4UVQO/CQPj0eE2wbwBL/YBiD2IQJf873Y5Uc4yKZh+
Bun1QrfOzbvY2rRLgdoWSF3AKenBaZKOwS3IbMP0LYVHY5hjmPjqtm2Hcs2lJyx8x8aUzIRb
j30o9gYWb+ZWWcUg3V1xXSzUVT/JHkL0vowvOVojo0t4L3A3VBcQGB0Q07CqdtbqhuLrjq9t
TvY9de4g4HgTqEb7b9X+8D3WGwQRpobH0WFGf09gT0cchp+sLeKaiWNQh5eWgYhSlKag/Jiq
/xMzXsSghrUDZMQPE9lPo7O0xAPJ9osiRfQQt32JUYDfxyWn+zstAZosIhhDe80cQmHsqRDJ
YJ7L22XFeFULcBIKprlT/UkajE3PoFMGQLqWpbhjiKuXtE+SIVSszm0OXfBGr6X2E2ghpYgd
cx5rFtkrFANQwU53CrLaPDIAISBW+3fLEespp6qkRNapxpwT0UQhcXJ/6Nb20+MH8NekTzXa
PC0MW2jxas1L2+smERVSGeY6khBAO08N/4JJ6aiYSSEDF3PZO2449Z7LaUD95JgsTNMPXayd
kVbOGcQFnJ/oN4jG3e8q5MLwT3g99umXpo3XQO0IedXFdDmRY5prsEgDGCJDxqZXNlw/icM2
EVovrxbqfo9eM9G/sYlQvZ0wd5w9C/tvIpMy392AngmVPBUjmlblSsyE1TNCCqLDHekJ3osw
P1V79VrTpo6DhVhGhWYvqts+/CVmJLUKmYdOkZynXRuAo/LlvZZFstr1bR4YFWpARL2C0Aru
IG7yjwxfdbgo6LPhkJL/vMsn0KTVGAaSbqaHMqeWp6KXTeAw3iwnqHtYVDfLqaZrK4e7ziZx
Lm3UfIRETdQp6fEUfygKS6whVw1AOVMSSuJuEUEdtlxu82VFDrbF5yY7szT5zL/01Efok7qB
VcDRL6S6Mu8Z3XUO1DNfnToOChbNZyP0ZuuTr9udM4GdKuOkNUKP8hmSBpMeV9biwmCYjYHd
2mJV+rvBv5qglD85iYr4ucIznPrndMljwdL0A/cxL1HmPkBykjFwHXKsletYN8EUf0AIv2IM
Eui+jSHlo8SpnazLbBKiEY2fay3OutDi6nHerDnTE33ptTlRUUnbwr0EUX02Kme6ASORPiMi
Swm1Sus70gyUQYr3lrMgjNXH1zAjG6TCTwJewtigSUJlxHdlQZ3wcu3UxGRKSBwGil7g3SfA
6cGMxGtZhvxeDwBqY69LK5Wq9WKDPdOI+KLnwbV1P8Vnhnpd41isQuDwCYQPHBxKw6jiHT6F
S6SI+q9XTWQrttOVRzfqyAaWCFyRplfdGsemGUCOno1G6ncAEch2j3M8Pr9jBL01xQj0InJN
Y3GJMuIwla5CM7NF+9VWOoC6c2ZUAlATJmkWQlnHsrYO8hFE4kyDU7jaudvmxaLkl3KCESKp
ydCjIco5F6p0B6OFH6/L0QbKgMrLlyyxsRuKjpVy9EWV3xeM3uqWA9RsN4bUftHIrms3qANA
i8VxwpULaXqBucVt8PCAMEXCF2DuMvZ6XlRRUT+hJxxce0PTIzBCLVGDF/ReD7J4BRz5laaK
oNQwhxAiIZBUrv2yFFkY+Ffhb8QWyh4vZdLgi/6E3rLc4EPdNqY3h3dSZWvPbhgSyiojQJ1J
2g6Yd+sSlk63seIosh2j0TjPIwdqW6F2kJhHCH1QZeWogVZX22+YSbaQw8bcm4CGLsJq2xKs
BclWAcNSup7vr/Da1B+/7ybKyVDo0n7PlUfHoEang7rCBv6EWDjlvzEKtbb4XesErynVoRjS
2q0uKh8Sw9BDfW8saVMNgxQI8ciYzf3Eyv/YJTKtK4CQwlFOcxyyOqE8CE7GOiCeLU0DQozr
7tf3CMmPmrEAMj4uswlPN89ede9kp2oBS5ySpuT9xglQ+9WybeUBtrNqHNMy2OzI/FFsA644
MDESXhkrP2FRDkt1Oj0Ltwlx/w966ZvBUf/qYsBKRR/Pz96d9z+g5JehYo/Tt+eDQXr2Nj16
3z9/N+jiufMBnojbAkY2aqCLMjb4e/DXy8HpZfpxcP5heHkprb35lPY/fpTG+29OBulJ/1tZ
zcFfjwYfL9Nv3w9OkzM0/+1QxnNx2ccLw9P02/Ph5fD0ndVS+vjpfPju/WX6/uzkeHBOtO5n
0jtf1KuNBheJjOOb4XF7Up3+hQy7E65W8sFjcrhm6S/D0+NuOhiyocFfP54PLmT+ibQ9/CAj
HsiPw9Ojk6tjAoHfXGlNH1bZk3FennFp/FlvXQYj7W/cyQTk8M+4lIlLKI3Igp8PL/6S9i8S
W9ivr/qhIVldaeND//SIG7W2kZhu+unsClJD5n1yjAcSfwALNUiPB29RNPob2V55Urq5uPow
sPW+uOQCnZykp4MjGW///FN6MTj/ZniEdUjOBx/7Q1l+YKTPz7X0tPKW5z1snlDJ4BvQwNXp
CWZ7Pvj6SuazhRLQRv+dUBsWM9r35NuhdI4dWt/8Ll+RH5rN/yRkhALpnxSY/cnIQ4YZkNtt
qhCiaKiz/+YMa/BGxjPksGQgWBBs0XH/Q//d4KKbBCJg1wYm76YXHwdHQ/yH/C6kJ3t9oqsi
p+jrK+yifGGNpH3ZTkwNdGhbhjMIWjt1GpG+18/l06bvNfoDXZycXYDYpJPLfsoRy7/fDPD0
+eBU1ovHqX90dHUuRwtP4A0ZzcWVHLbhKTclwXx5mofnx36euM7p2/7w5Op8g8akZ71oc6C0
FjbEiexir0saSIdvpauj97Z7aevUfkrfy1a8Gchj/eNvhuA82k8iZ+FiaGtyZi3YOpKxMddU
5sfntwD4gf3vzwHOKX74Ck5cyAEtT6t+1ktqAfLlJ7DdU1F5TNbVoGOTjyMRr5Ny3lzz3qAp
oyw3w+qZyBwzC6ReJGKJqLNsWQcppAae2d05K0yt1DN9B0NDVR9Fu1MSFYukLRFUEoa0nY07
9KKE0BAydiei58W5Y3axyCzw1ChIAdLr+qM6I1KrvFRnt5gaRhzenjb3khqMiDAci7TwWixP
GdU8FEUOippwn68sciUqfG3KWgM5JpAHTbGN+MY5j/lTk+8EpaCDkqXmvErnJe0gLXuXWxIs
AwYG9UMaE9QAg0L+EevJ9x03EC3Ak1pLzGvT12KB3GpVOUKK9EY/YsP/xLbW06pXK2nfa9Sr
4vMn7fUX3ZNo/mNFUf4DdyWma3clBsTez78v0TuJ7ktkKz/rzsRtC/D33ptIyMgvuzsRTfyy
+xNDktHPvkMRb/zyexQVIvGP36WI9zfvU/x5KfxIRgFOCV6BGBYCz5myWy9/CyIWBZmVD6ty
JuPXFEDR91EDfqKuzhZCo4VI7Tov9ESSDMtWBRCv1umF+5qAR9S3XNCIYRZFC9sqByY3BNW7
mSjV96rNOzm/lKltObvtk7vxthaC5A7031ycnYi2cfIp1pRfkwJs8/VG7L8xW/XhSa85BOun
v5EzZPz5BP1olbgWM2ALljsV/EVugr2Ou7t5Eg+kp1CVu9Uchh3jWg3K28fHMYS3jVo907aV
TdKyG3fmm53dMpRi0Y+mP4aKa3g1V3BoIMbGCLDYZfQoRMlOW4dmuUvqmedpv86TaSlNPruR
EXxPR8Y0ny1lwfJp/ewZuDaN5xr1/9I4x799xSnBeEg/5iO4Z7RcoS6WZ7oH+LG9PUVdWM3d
rpIaJvtEYxszRbAjuIzEucYZ16TcdJrMFNc1UKgUqfG1Zmi+N2R6BtzEfCIigqgpvgMy1fyK
T+WqHK1muZ9oyD8tWmw+8WwSewN5QqCNGMO1zqWhv0V0/gQBMWIE5TTWmsLLUtYOfKn3ghNN
OvszRpO+R6H/igzvjwodQbK3UMnlSk5aOftTNz0QvawqJqw+AgVFf+iiQkddeE7XN0JB5snd
wWSDX8UiRY1PI5RonkfejCTKfA1FBkJYrYpZUYagbFUiJg1mw1ISwSmTOB6cGZlg8iqZGG7U
kYhSQTRX3GOr6qnjUBJr3J1GyhQeHBbqadwjUd48Y2ZLdYtke3WLTWfmb1295vHzSz9R/afh
qZhzJyf//D5+4v6H588P9v3+h8NXuAvw4PDF4eP9f7/KZ+P+h6+vhkd/SY0WNi+AiG9siC6A
D+pP7wWFqvy7dwhFbiiCYWLXQSThOohu+8YIYe4He2l6NUNPUYG5ZzKi3mL8P+o8Vqba9Gnm
4aJiWuRzaeBDA4ahwed7itHMs8XNXagOrY9UdYB3qrEXCrpDqBmCGkH9EhyVuup0rYvm0kNZ
kygZn0/opWahloNDmsF5P51dnadeJ51yt5ce57eolwuO3ZmuRtedXnoUKvV+EJGJHnvJ4Z5d
gqOS1ufF5bHQZpSu3NEHnn3cT/+4uSCd5HNp7Tw3ARwe6KVPj3mBmGgTLJxSsjg664QTGbaX
vJAXj5pyPPPxf91dZz3caG0phaJVBS+Dgrn4p5WxfvpJhO3QK3QYWLteip6117NyWvLeUPP5
6jUfgzT5FV/g5Vv+gUoizONLIamXXwjl7IO5vHjR83/2+dSikiOfvOS01d8Q7q1SRO+qQXU+
KwDoBaY8X3gd5730dpKNheJeSRNUjTs0LqT3jtosupSRGlQnX+BZSw1O14C6pKNZKJbaVE7d
CwlJ/NIFuJORQjIKRHFd7fKbUpIvMb2lftlZzjsGqpZzts+rzDDrCZMQUXn70nZX94couspf
RlVQf113revbxvLwSxTJlzZ21H7eWfn5QlSSP4sunh6mz/f3fxVW//h5/Dx+Hj+Pn8fP4+fx
8/h5/Dx+Hj+Pn8fP/9fP/wKykq3cAMgAAA==

--Boundary_(ID_GeYGc69fE1/bkYLTPwOGFg)--

************


From owner-pgsql-hackers@hub.org Mon Jan  3 13:47:07 2000
Received: from hub.org (hub.org [216.126.84.1])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id OAA23987
	for <pgman@candle.pha.pa.us>; Mon, 3 Jan 2000 14:47:06 -0500 (EST)
Received: from localhost (majordom@localhost)
	by hub.org (8.9.3/8.9.3) with SMTP id OAA03234;
	Mon, 3 Jan 2000 14:39:56 -0500 (EST)
	(envelope-from owner-pgsql-hackers)
Received: by hub.org (bulk_mailer v1.5); Mon, 3 Jan 2000 14:39:49 -0500
Received: (from majordom@localhost)
	by hub.org (8.9.3/8.9.3) id OAA03050
	for pgsql-hackers-outgoing; Mon, 3 Jan 2000 14:38:50 -0500 (EST)
	(envelope-from owner-pgsql-hackers@postgreSQL.org)
Received: from ara.zf.jcu.cz (zakkr@ara.zf.jcu.cz [160.217.161.4])
	by hub.org (8.9.3/8.9.3) with ESMTP id OAA02975
	for <pgsql-hackers@postgreSQL.org>; Mon, 3 Jan 2000 14:38:05 -0500 (EST)
	(envelope-from zakkr@zf.jcu.cz)
Received: from localhost (zakkr@localhost)
	by ara.zf.jcu.cz (8.9.3/8.9.3/Debian/GNU) with SMTP id UAA19297;
	Mon, 3 Jan 2000 20:23:35 +0100
Date: Mon, 3 Jan 2000 20:23:35 +0100 (CET)
From: Karel Zak - Zakkr <zakkr@zf.jcu.cz>
To: P.Marchesso@videotron.ca
cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: [HACKERS] replicator
Message-ID: <Pine.LNX.3.96.1000103194931.19115A-100000@ara.zf.jcu.cz>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: owner-pgsql-hackers@postgresql.org
Status: OR


Hi,

I look at your (Philippe's) replicator, but I don't good understand
your replication concept.


    node1:  SQL --IPC--> node-broker
                       |
                      TCP/IP
                       |
                    master-node --IPC--> replikator
                                         |   |   |
                                           libpq
                                         |   |   |
                                       node2 node..n     

(Is it right picture?)

If I good understand, all nodes make connection to master node and data
replicate "replicator" on this master node. But it (master node) is very
critical space in this concept - If master node not work replication for 
*all* nodes is lost. Hmm.. but I want use replication for high available
applications...

IMHO is problem with node registration / authentification on master node.
Why concept is not more upright? As:

	SQL --IPC--> node-replicator
			|  |  | 
		     via libpq send data to all nodes with
                     current client/backend auth.

	(not exist any master node, all nodes have connection to all nodes)	


Use replicator as external proces and copy data from SQL to this replicator
via IPC is (your) very good idea. 

							Karel


----------------------------------------------------------------------
Karel Zak <zakkr@zf.jcu.cz>              http://home.zf.jcu.cz/~zakkr/

Docs:        http://docs.linux.cz                    (big docs archive)	
Kim Project: http://home.zf.jcu.cz/~zakkr/kim/        (process manager)
FTP:         ftp://ftp2.zf.jcu.cz/users/zakkr/        (C/ncurses/PgSQL)
-----------------------------------------------------------------------


************

From owner-pgsql-hackers@hub.org Tue Jan  4 10:31:01 2000
Received: from renoir.op.net (root@renoir.op.net [207.29.195.4])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id LAA17522
	for <pgman@candle.pha.pa.us>; Tue, 4 Jan 2000 11:31:00 -0500 (EST)
Received: from hub.org (hub.org [216.126.84.1]) by renoir.op.net (o1/$Revision: 1.10 $) with ESMTP id LAA01541 for <pgman@candle.pha.pa.us>; Tue, 4 Jan 2000 11:27:30 -0500 (EST)
Received: from localhost (majordom@localhost)
	by hub.org (8.9.3/8.9.3) with SMTP id LAA09992;
	Tue, 4 Jan 2000 11:18:07 -0500 (EST)
	(envelope-from owner-pgsql-hackers)
Received: by hub.org (bulk_mailer v1.5); Tue, 4 Jan 2000 11:17:58 -0500
Received: (from majordom@localhost)
	by hub.org (8.9.3/8.9.3) id LAA09856
	for pgsql-hackers-outgoing; Tue, 4 Jan 2000 11:17:17 -0500 (EST)
	(envelope-from owner-pgsql-hackers@postgreSQL.org)
Received: from ara.zf.jcu.cz (zakkr@ara.zf.jcu.cz [160.217.161.4])
	by hub.org (8.9.3/8.9.3) with ESMTP id LAA09763
	for <pgsql-hackers@postgreSQL.org>; Tue, 4 Jan 2000 11:16:43 -0500 (EST)
	(envelope-from zakkr@zf.jcu.cz)
Received: from localhost (zakkr@localhost)
	by ara.zf.jcu.cz (8.9.3/8.9.3/Debian/GNU) with SMTP id RAA31673;
	Tue, 4 Jan 2000 17:02:06 +0100
Date: Tue, 4 Jan 2000 17:02:06 +0100 (CET)
From: Karel Zak - Zakkr <zakkr@zf.jcu.cz>
To: Philippe Marchesseault <P.Marchesso@Videotron.ca>
cc: pgsql-hackers <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] replicator
In-Reply-To: <38714B6F.2DECAEC0@Videotron.ca>
Message-ID: <Pine.LNX.3.96.1000104162226.27234D-100000@ara.zf.jcu.cz>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: owner-pgsql-hackers@postgreSQL.org
Status: OR


On Mon, 3 Jan 2000, Philippe Marchesseault wrote:

> So it could become:
> 
> SQL --IPC--> node-replicator
>                            |   |   |
>       via TCP send statements to each node
>                       replicator (on local node)
>                            |
>          via libpq send data to
>         current (local) backend.
> 
> >  (not exist any master node, all nodes have connection to all nodes)
> 
> Exactly, if the replicator dies only the node dies, everything else keeps
> working.


 Hi,

 I a little explore replication conception on Oracle and Sybase (in manuals).
(Know anyone some interesting links or publication about it?)

 Firstly, I sure, untimely is write replication to PgSQL now, if we
haven't exactly conception for it. It need more suggestion from more
developers. We need firstly answers for next qestion:

	1/ How replication concept choose for PG?
	2/ How manage transaction for nodes? (and we need define any 
           replication protocol for this)
	3/ How involve replication in current PG transaction code?

My idea (dream:-) is replication that allow you use full read-write on all
nodes and replication which use current transaction method in PG - not is
difference between more backends on one host or more backend on more hosts
- it makes "global transaction consistency".

Now is transaction manage via ICP (one host), my dream is alike manage 
this transaction, but between more host via TCP. (And make optimalization 
for this - transfer commited data/commands only.)


Any suggestion?


-------------------
Note:
 
(transaction oriented replication)

 Sybase - I. model (only one node is read-write) 

	 primary SQL data (READ-WRITE)
                |
	 replication agent (transaction log monitoring)
		|
	 primary distribution server (one or more repl. servers)
	        |               /  |  \
                |            nodes (READ-ONLY)
                |
         secondary dist. server
                          /  |  \
                       nodes (READ-ONLY)


       If primary SQL is read-write and the other nodes *read-only* 
       => system good work if connection is disable (data are save to
          replication-log and if connection is available log is write 
	  to node).   


 Sybase - II. model (all nodes read-write)

     	    SQL data 1 --->--+                        NODE I.
                |            |
                ^            |
	        |     replication agent 1 (transaction log monitoring)
                V        |
		|        V
                |        |
         replication server 1
                |
		^
                V
                |
         replication server 2                        NODE II.
                |         |
                ^         +-<-->--- SQL data 2
                |                    |                      
               replcation agent 2 -<--



Sorry, I not sure if I re-draw previous picture total good..

								Karel   


	
    


************

From pgsql-hackers-owner+M3133@hub.org Fri Jun  9 15:02:25 2000
Received: from hub.org (root@hub.org [216.126.84.1])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id PAA22319
	for <pgman@candle.pha.pa.us>; Fri, 9 Jun 2000 15:02:24 -0400 (EDT)
Received: from hub.org (majordom@localhost [127.0.0.1])
	by hub.org (8.10.1/8.10.1) with SMTP id e59IsET81137;
	Fri, 9 Jun 2000 14:54:14 -0400 (EDT)
Received: from ultra2.quiknet.com (ultra2.quiknet.com [207.183.249.4])
	by hub.org (8.10.1/8.10.1) with SMTP id e59IrQT80458
	for <pgsql-hackers@postgresql.org>; Fri, 9 Jun 2000 14:53:26 -0400 (EDT)
Received: (qmail 13302 invoked from network); 9 Jun 2000 18:53:21 -0000
Received: from 18.67.tc1.oro.pmpool.quiknet.com (HELO quiknet.com) (pecondon@207.231.67.18)
  by ultra2.quiknet.com with SMTP; 9 Jun 2000 18:53:21 -0000
Message-ID: <39413D08.A6BDC664@quiknet.com>
Date: Fri, 09 Jun 2000 11:52:57 -0700
From: Paul Condon <pecondon@quiknet.com>
X-Mailer: Mozilla 4.73 [en] (X11; U; Linux 2.2.14-5.0 i686)
X-Accept-Language: en
MIME-Version: 1.0
To: ohp@pyrenet.fr, pgsql-hackers@postgresql.org
Subject: [HACKERS] Re: Big project, please help
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
X-Mailing-List: pgsql-hackers@postgresql.org
Precedence: bulk
Sender: pgsql-hackers-owner@hub.org
Status: OR

Two way replication on a single "table" is availabe in Lotus Notes. In
Notes, every record has a time-stamp, which contains the time of the
last update. (It also has a creation timestamp.) During replication,
timestamps are compared at the row/record level, and compared with the
timestamp of the last replication. If, for corresponding rows in two
replicas, the timestamp of one row is newer than the last replication,
the contents of this newer row is copied to the other replica. But if
both of the corresponding rows have newer timestamps, there is a
problem. The Lotus Notes solution is to:
  1. send a replication conflict message to the Notes Administrator,
which message contains full copies of both rows.
  2. copy the newest row over the less new row in the replicas.
  3. there is a mechanism for the Administrator to reverse the default
decision in 2, if the semantics of the message history, or off-line
investigation indicates that the wrong decision was made.

In practice, the Administrator is not overwhelmed with replication
conflict messages because updates usually only originate at the site
that originally created the row. Or updates fill only fields that were
originally 'TBD'. The full logic is perhaps more complicated than I have
described here, but it is already complicated enough to give you an idea
of what you're really being asked to do. I am not aware of a supplier of
relational database who really supports two way replication at the level
that Notes supports it, but Notes isn't a relational database.

The difficulty of the position that you appear to be in is that
management might believe that the full problem is solved in brand X
RDBMS, and you will have trouble convincing management that this is not
really true.


From pgsql-hackers-owner+M2401@hub.org Tue May 23 12:19:54 2000
Received: from news.tht.net (news.hub.org [216.126.91.242])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id MAA28410
	for <pgman@candle.pha.pa.us>; Tue, 23 May 2000 12:19:53 -0400 (EDT)
Received: from hub.org (majordom@hub.org [216.126.84.1])
	by news.tht.net (8.9.3/8.9.3) with ESMTP id MAB53304;
	Tue, 23 May 2000 12:00:08 -0400 (EDT)
	(envelope-from pgsql-hackers-owner+M2401@hub.org)
Received: from gwineta.repas.de (gwineta.repas.de [193.101.49.1])
	by hub.org (8.9.3/8.9.3) with ESMTP id LAA39896
	for <pgsql-hackers@postgresql.org>; Tue, 23 May 2000 11:57:31 -0400 (EDT)
	(envelope-from kardos@repas-aeg.de)
Received: (from smap@localhost)
	by gwineta.repas.de (8.8.8/8.8.8) id RAA27154
	for <pgsql-hackers@postgresql.org>; Tue, 23 May 2000 17:57:23 +0200
Received: from dragon.dr.repas.de(172.30.48.206) by gwineta.repas.de via smap (V2.1)
	id xma027101; Tue, 23 May 00 17:56:20 +0200
Received: from kardos.dr.repas.de ([172.30.48.153])
  by dragon.dr.repas.de (UCX V4.2-21C, OpenVMS V6.2 Alpha);
	Tue, 23 May 2000 17:57:24 +0200
Message-ID: <010201bfc4cf$7334d5a0$99301eac@Dr.repas.de>
From: "Kardos, Dr. Andreas" <kardos@repas-aeg.de>
To: "Todd M. Shrider" <tshrider@varesearch.com>,
        <pgsql-hackers@postgresql.org>
References: <Pine.LNX.4.04.10005180846290.15739-100000@silicon.su.valinux.com>
Subject: Re: [HACKERS] failing over with postgresql
Date: Tue, 23 May 2000 17:56:20 +0200
Organization: repas AEG Automation GmbH
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 8bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 5.00.2314.1300
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300
X-Mailing-List: pgsql-hackers@postgresql.org
Precedence: bulk
Sender: pgsql-hackers-owner@hub.org
Status: OR

For a SCADA system (Supervisory Control and Data Akquisition) which consists
of one  master and one hot-standby server I have implemented such a
solution. To these UNIX servers client workstations are connected (NT and/or
UNIX). The database client programms run on client and server side.

When developing this approach I had to goals in mind:
1) Not to get dependend on the PostgreSQL sources since they change very
dynamically.
2) Not to get dependend on the fe/be protocol  since there are discussions
around to change it.

So the approach is quite simple: Forward all database requests to the
standby server on TCP/IP level.

On both servers the postmaster listens on port 5433 and not on 5432. On
standard port 5432 my program listens instead. This program forks twice for
every incomming connection. The first instance forwards all packets from the
frontend to both backends. The second instance receives the packets from all
backends and forwards the packets from the master backend to the frontend.
So a frontend running on a server machine connects to port 5432 of
localhost.

On the client machine runs another program (on NT as a service). This
program forks for every incomming connections twice. The first instance
forwards all packets to port 5432 of the current master server and the
second instance forwards the packets from the master server to the frontend.

During standby computer startup the database of the master computer is
dumped, zipped, copied to the standby computer, unzipped and loaded into
that database.
If a standby startup took place, all client connections are aborted to allow
a login into the standby database. The frontends need to reconnect in this
case. So the database of the standby computer is always in sync.

The disadvantage of this method is that a query cannot be canceled in the
standby server since the request key of this connections gets lost. But we
can live with that.

Both programms are able to run on Unix and on (native!) NT. On NT threads
are created instead of forked processes.

This approach is simple, but it is effective and it works.

We hope to survive this way until real replication will be implemented in
PostgreSQL.

Andreas Kardos

-----Urspr�ngliche Nachricht-----
Von: Todd M. Shrider <tshrider@varesearch.com>
An: <pgsql-hackers@postgresql.org>
Gesendet: Donnerstag, 18. Mai 2000 17:48
Betreff: [HACKERS] failing over with postgresql


>
> is anyone working on or have working a fail-over implentation for the
> postgresql stuff. i'd be interested in seeing if and how any might be
> dealing with just general issues as well as the database syncing issues.
>
> we are looking to do this with heartbeat and lvs in mind. also if anyone
> is load ballancing their databases that would be cool to talk about to.
>
> ---
> Todd M. Shrider VA Linux Systems
> Systems Engineer
> tshrider@valinux.com www.valinux.com
>


From pgsql-hackers-owner+M3662@postgresql.org Tue Jan 23 16:23:34 2001
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id QAA04456
	for <pgman@candle.pha.pa.us>; Tue, 23 Jan 2001 16:23:34 -0500 (EST)
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by mail.postgresql.org (8.11.1/8.11.1) with SMTP id f0NLKf004705;
	Tue, 23 Jan 2001 16:20:41 -0500 (EST)
	(envelope-from pgsql-hackers-owner+M3662@postgresql.org)
Received: from sectorbase2.sectorbase.com ([208.48.122.131])
	by mail.postgresql.org (8.11.1/8.11.1) with SMTP id f0NLAe003753
	for <pgsql-hackers@postgresql.org>; Tue, 23 Jan 2001 16:10:40 -0500 (EST)
	(envelope-from vmikheev@SECTORBASE.COM)
Received: by sectorbase2.sectorbase.com with Internet Mail Service (5.5.2653.19)
	id <DG1W4Q8F>; Tue, 23 Jan 2001 12:49:07 -0800
Message-ID: <8F4C99C66D04D4118F580090272A7A234D32AF@sectorbase1.sectorbase.com>
From: "Mikheev, Vadim" <vmikheev@SECTORBASE.COM>
To: "'dom@idealx.com'" <dom@idealx.com>, pgsql-hackers@postgresql.org
Subject: RE: [HACKERS] Re: AW: Re: MySQL and BerkleyDB (fwd)
Date: Tue, 23 Jan 2001 13:10:34 -0800
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: ORr

>   I had thought that the pre-commit information could be stored in an
> auxiliary table by the middleware program ; we would then have
> to re-implement some sort of higher-level WAL (I thought of the list
> of the commands performed in the current transaction, with a sequence
> number for each of them that would guarantee correct ordering between
> concurrent transactions in case of a REDO). But I fear I am missing

This wouldn't work for READ COMMITTED isolation level.
But why do you want to log commands into WAL where each modification
is already logged in, hm, correct order?
Well, it has sense if you're looking for async replication but
you need not in two-phase commit for this and should aware about
problems with READ COMMITTED isolevel.

Back to two-phase commit - it's easiest part of work required for
distributed transaction processing.
Currently we place single commit record to log and transaction is
committed when this record (and so all other transaction records)
is on disk.
Two-phase commit:

1. For 1st phase we'll place into log "prepared-to-commit" record
   and this phase will be accomplished after record is flushed on disk.
   At this point transaction may be committed at any time because of
   all its modifications are logged. But it still may be rolled back
   if this phase failed on other sites of distributed system.

2. When all sites are prepared to commit we'll place "committed"
   record into log. No need to flush it because of in the event of
   crash for all "prepared" transactions recoverer will have to
   communicate other sites to know their statuses anyway.

That's all! It is really hard to implement distributed lock- and
communication- managers but there is no problem with logging two
records instead of one. Period.

Vadim

From pgsql-hackers-owner+M3665@postgresql.org Tue Jan 23 17:05:26 2001
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id RAA05972
	for <pgman@candle.pha.pa.us>; Tue, 23 Jan 2001 17:05:24 -0500 (EST)
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by mail.postgresql.org (8.11.1/8.11.1) with SMTP id f0NM31008120;
	Tue, 23 Jan 2001 17:03:01 -0500 (EST)
	(envelope-from pgsql-hackers-owner+M3665@postgresql.org)
Received: from candle.pha.pa.us (candle.navpoint.com [162.33.245.46])
	by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id f0NLsU007188
	for <pgsql-hackers@postgresql.org>; Tue, 23 Jan 2001 16:54:30 -0500 (EST)
	(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost)
	by candle.pha.pa.us (8.9.0/8.9.0) id QAA05300;
	Tue, 23 Jan 2001 16:53:53 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <200101232153.QAA05300@candle.pha.pa.us>
Subject: Re: [HACKERS] Re: AW: Re: MySQL and BerkleyDB (fwd)
In-Reply-To: <8F4C99C66D04D4118F580090272A7A234D32AF@sectorbase1.sectorbase.com>
	"from Mikheev, Vadim at Jan 23, 2001 01:10:34 pm"
To: "Mikheev, Vadim" <vmikheev@SECTORBASE.COM>
Date: Tue, 23 Jan 2001 16:53:53 -0500 (EST)
CC: "'dom@idealx.com'" <dom@idealx.com>, pgsql-hackers@postgresql.org
X-Mailer: ELM [version 2.4ME+ PL77 (25)]
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=US-ASCII
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR

[ Charset ISO-8859-1 unsupported, converting... ]
> >   I had thought that the pre-commit information could be stored in an
> > auxiliary table by the middleware program ; we would then have
> > to re-implement some sort of higher-level WAL (I thought of the list
> > of the commands performed in the current transaction, with a sequence
> > number for each of them that would guarantee correct ordering between
> > concurrent transactions in case of a REDO). But I fear I am missing
> 
> This wouldn't work for READ COMMITTED isolation level.
> But why do you want to log commands into WAL where each modification
> is already logged in, hm, correct order?
> Well, it has sense if you're looking for async replication but
> you need not in two-phase commit for this and should aware about
> problems with READ COMMITTED isolevel.
> 

I believe the issue here is that while SERIALIZABLE ISOLATION means all
queries can be run serially, our default is READ COMMITTED, meaning that
open transactions see committed transactions, even if the transaction
committed after our transaction started.  (FYI, see my chapter on
transactions for help,  http://www.postgresql.org/docs/awbook.html.)

To do higher-level WAL, you would have to record not only the queries,
but the other queries that were committed at the start of each command
in your transaction.

Ideally, you could number every commit by its XID your log, and then
when processing the query, pass the "committed" transaction ids that
were visible at the time each command began.

In other words, you can replay the queries in transaction commit order,
except that you have to have some transactions committed at specific
points while other transactions are open, i.e.:

XID	Open XIDS	Query
500			UPDATE t SET col = 3;
501	500		BEGIN;
501	500		UPDATE t SET col = 4;
501			UPDATE t SET col = 5;
501			COMMIT;

This is a silly example, but it shows that 500 must commit after the
first command in transaction 501, but before the second command in the
transaction.  This is because UPDATE t SET col = 5 actually sees the
changes made by transaction 500 in READ COMMITTED isolation level.

I am not advocating this.  I think WAL is a better choice.  I just
wanted to outline how replaying the queries in commit order is 
insufficient.

> Back to two-phase commit - it's easiest part of work required for
> distributed transaction processing.
> Currently we place single commit record to log and transaction is
> committed when this record (and so all other transaction records)
> is on disk.
> Two-phase commit:
> 
> 1. For 1st phase we'll place into log "prepared-to-commit" record
>    and this phase will be accomplished after record is flushed on disk.
>    At this point transaction may be committed at any time because of
>    all its modifications are logged. But it still may be rolled back
>    if this phase failed on other sites of distributed system.
> 
> 2. When all sites are prepared to commit we'll place "committed"
>    record into log. No need to flush it because of in the event of
>    crash for all "prepared" transactions recoverer will have to
>    communicate other sites to know their statuses anyway.
> 
> That's all! It is really hard to implement distributed lock- and
> communication- managers but there is no problem with logging two
> records instead of one. Period.

Great.


-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From pgsql-general-owner+M805@postgresql.org Tue Nov 21 23:53:04 2000
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id AAA19262
	for <pgman@candle.pha.pa.us>; Wed, 22 Nov 2000 00:53:03 -0500 (EST)
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by mail.postgresql.org (8.11.1/8.11.1) with SMTP id eAM5qYs47249;
	Wed, 22 Nov 2000 00:52:34 -0500 (EST)
	(envelope-from pgsql-general-owner+M805@postgresql.org)
Received: from racerx.cabrion.com (racerx.cabrion.com [166.82.231.4])
	by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id eAM5lJs46653
	for <pgsql-general@postgresql.org>; Wed, 22 Nov 2000 00:47:19 -0500 (EST)
	(envelope-from rob@cabrion.com)
Received: from cabrionhome (gso163-25-211.triad.rr.com [24.163.25.211])
	by racerx.cabrion.com (8.8.7/8.8.7) with SMTP id AAA13731
	for <pgsql-general@postgresql.org>; Wed, 22 Nov 2000 00:45:20 -0500
Message-ID: <006501c05447$fb9aa0c0$4100fd0a@cabrion.org>
From: "rob" <rob@cabrion.com>
To: <pgsql-general@postgresql.org>
Subject: [GENERAL] Synchronization Toolkit
Date: Wed, 22 Nov 2000 00:49:29 -0500
MIME-Version: 1.0
Content-Type: multipart/mixed;
	boundary="----=_NextPart_000_0062_01C0541E.125CAF30"
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 5.50.4133.2400
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400
Precedence: bulk
Sender: pgsql-general-owner@postgresql.org
Status: OR

This is a multi-part message in MIME format.

------=_NextPart_000_0062_01C0541E.125CAF30
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: 7bit

Not to be confused with replication, my concept of synchronization is to
manage changes between a server table (or tables) and one or more mobile,
disconnected databases (i.e. PalmPilot, laptop, etc.).

I read through the notes in the TODO for this topic and devised a tool kit
for doing synchronization.  I hope that the Postgresql development community
will find this useful and will help me refine this concept by offering
insight, experience and some good old fashion hacking if you are so
inclined.

The bottom of this message describes how to use the attached files.

I look forward to your feedback.

--rob


Methodology:

I devised a concept that I call "session versioning".  This means that every
time a row changes it does NOT get a new version.  Rather it gets stamped
with the current session version common to all published tables.  Clients,
when they connect for synchronization, will immediately increment this
common version number reserve the result as a "post version" and then
increment the session version again.  This version number, implemented as a
sequence, is common to all synchronized tables and rows.

Any time the server makes changes to the row gets stamped with the current
session version, when the client posts its changes it uses the reserved
"post version".  The client then makes all it's changes stamping the changed
rows with it's reserved "post version" rather than the current version.  The
reason why is explained later.  It is important that the client post all its
own changes first so that it does not end up receiving records which changed
since it's last session that it is about to update anyway.

Reserving the post version is a two step process.  First, the number is
simply stored in a variable for later use.  Second, the value is added to a
lock table (last_stable) to indicate to any concurrent sessions that rows
with higher version numbers are to be considered "unstable" at the moment
and they should not attempt to retrieve them at this time.  Each client,
upon connection, will use the lowest value in this lock table (max_version)
to determine the upper boundary for versions it should retrieve.  The lower
boundary is simply the  previous session's "max_version" plus one.  Thus
when the client retrieves changes is uses the following SQL "where"
expression:

WHERE row_version >= max_version and row_version <= last_stable_version and
version <> this_post_version

The point of reserving and locking a post version is important in that it
allows concurrent synchronization by multiple clients.  The first, of many,
clients to connect basically dictates to all future clients that they must
not take any rows equal to or greater than the one which it just reserved
and locked.  The reason the session version is incremented a second time is
so that the server may continue to post changes concurrent with any client
changes and be certain that these concurrent server changes will not taint
rows the client is about to retrieve. Once the client is finished with it's
session it removes the lock on it's post version.

Partitioning data for use by each node is the next challenge we face.  How
can we control which "slice" of data each client receives?  A slice can be
horizontal or vertical within a table.  Horizontal slices are easy,  it's
just the where clause of an SQL statement that says "give me the rows that
match X criteria".  We handle this by storing and appending a where clause
to each client's retrieval statement  in addition to where clause described
above.  Actually, two where clauses are stored and appended.  One is per
client and one is per publication (table).

We defined horizontal slices by filtering rows.  Vertical slices are limits
by column.  The tool kit does provide a mechanism for pseudo vertical
partitioning.  When a client is "subscribed" to a publication, the toolkit
stores what columns that node is to receive during a session.  These are
stored in the subscribed_cols table.  While this does limit the number
columns transmitted, the insert/update/delete triggers do not recognize
changes based on columns.   The "pseudo" nature of our vertical partitioning
is evident by example:

Say you have a table with name, address and phone number as columns.  You
restrict a client to see only name and address.  This means that phone
number information will not be sent to the client during synchronization,
and the client can't attempt to alter the phone number of a given entry.
Great, but . . . if, on the server, the phone number (but not the name or
address) is changed, the entire row gets marked with a new version.  This
means that the name and address will get sent to the client even though they
didn't change.

Well, there's the flaw in vertical partitioning.  Other than wasting
bandwidth, the extra row does no harm to the process.  The workaround for
this is to highly normalize your schema when possible.

Collisions are the next crux one encounters with synchronization.  When two
clients retrieve the same row and both make (different)changes, which one is
correct?  So far the system operates totally independent of time.  This is
good because it doesn't rely on the server or client to keep accurate time.
We can just ignore time all together, but then we force our clients to
synchronize on a strict schedule in order to avoid (or reduce) collisions.
If every node synchronized immediately after making changes we could just
stop here.  Unfortunately this isn't reality.  Reality dictates that of two
clients: Client A & B will each pick up the same record on Monday.  A will
make changes on Monday, then leave for vacation.  B will make changes on
Wednesday because new information was gathered in A's absence.  Client B
posts those changes Wednesday.  Meanwhile, client A returns from vacation on
Friday and synchronizes his changes.  A over writes B's changes even though
A made changes before the most recent information was posted by B.

It is clear that we need some form of time stamp to cope with the above
example.  While clocks aren't the most reliable, they are the only common
version control available to solve this problem.  The system is set up to
accept (but not require) timestamps from clients and changes on the server
are time stamped.  The system, when presented a time stamp with a row, will
compare them to figure out who wins in a tie.   The system makes certain
"sanity" checks with regard to these time stamps.  A client may not attempt
to post a change with a timestamp that is more than one hour in the future
(according to what the server thinks "now" is) nor one hour before it's last
synchronization date/time.  The client row will be immediately placed into
the collision table if the timestamp is that far out of whack.
Implementations of the tool kit should take care to ensure that client &
server agree on what "now" is before attempting to submit changes with
timestamps.

Time stamps are not required.  Should a client be incapable of tracking
timestamps, etc.  The system will assume that any server row which has been
changed since the client's last session will win a tie.  This is quite error
prone, so timestamps are encouraged where possible.

Inserts pose an interesting challenge.  Since multiple clients cannot share
a sequence (often used as a primary key) while disconnected.  They will be
responsible for their own unique "row_id" when inserting records.   Inserts
accept any arbitrary key, and write back to the client a special kind of
update that gives the server's row_id.  The client is responsible for making
sure that this update takes place locally.

Deletes are the last portion of the process.  When deletes occur, the
row_id, version, etc. are stored in a "deleted" table.  These entries are
retrieved by the client using the same version filter as described above.
The table is pruned at the end of each session by deleting all records with
versions that are less than the lowest 'last_version' stored for each
client.

Having wrapped up the synchronization process, I'll move on to describe some
points about managing clients, publications and the like.

The tool kit is split into two objects: SyncManagement and Synchronization.
The Synchronization object exposes an API that client implementations use to
communicate and receive changes.  The management functions handle system
install and uninstall in addition to publication of tables and client
subscriptions.

Installation and uninstallation are handled by their corresponding functions
in the API.  All system tables are prefixed and suffixed with four
underscores, in hopes that this avoids conflict with an existing tables.
Calling the install function more than once will generate an error message.
Uninstall will remove all related tables, sequences,  functions and triggers
from the system.

The first step, after installing the system, is to publish a table.  A table
can be published more than once under different names.  Simply provide a
unique name as the second argument to the publish function.  Since object
names are restricted to 32 characters in Postgres, each table is given a
unique id and this id is used to create the trigger and sequence names.
Since one table can be published multiple times, but only needs one set of
triggers and one sequence for change management a reference count is kept so
that we know when to add/drop triggers and functions.  By default, all
columns are published, but the third argument to the publish function
accepts an array reference of column names that allows you to specify a
limited set.  Information about the table is stored in the "tables" table,
info about the publication is in the "publications" table and column names
are stored in "subscribed_cols" table.

The next step is to subscribe a client to a table.  A client is identified
by a user name and a node name.  The subscribe function takes three
arguments: user, node & publication.  The subscription process writes an
entry into the "subscribed" table with default values.  Of note, the
"RefreshOnce" attribute is set to true whenever a table is published.  This
indicates to the system that a full table refresh should be sent the next
time the client connects even if the client requests synchronization rather
than refresh.

The toolkit does not, yet, provide a way to manage the whereclause stored at
either the publication or client level.  To use or test this feature, you
will need to set the whereclause attributes manually.

Tables and users can be unpublished and unsubscribed using the corresponding
functions within the tool kit's management interface.  Because postgres
lacks an "ALTER TABLE DROP COLUMN" function, the unpublish function only
removes default values and indexes for those columns.

The API isn't the most robust thing in the world right now.  All functions
return undef on success and an error string otherwise (like DBD).  I hope to
clean up the API considerably over the next month.  The code has not been
field tested at this time.


The files attached are:

1) SynKit.pm (A perl module that contains install/uninstall functions and a
simple api for synchronization & management)

2) sync_install.pl (Sample code to demonstrate the installation, publishing
and subscribe process)

3) sync_uninstall.pl (Sample code to demonstrate the uninstallation,
unpublishing and unsubscribe process)


To use them on Linux (don't know about Win32 but should work fine):

 - set up a test database and make SURE plpgsql is installed

 - install perl 5.05 along with Date::Parse(TimeDate-1.1) , DBI and DBD::Pg
modules [www.cpan.org]

 - copy all three attached files to a test directory

 - cd to your test directory

 - edit all three files and change the three DBI variables to suit your
system (they are clearly marked)

 - % perl sync_install.pl

 - check out the tables, functions & triggers installed

 - % perl sync.pl

 - check out the 'sync_test' table, do some updates/inserts/deletes and run
sync.pl again
        NOTE: Sanity checks default to allow no more than 50% of the table
to be changed by the client in a single session.
        If you delete all (or most of) the rows  you will get errors when
you run sync.pl again! (by design)

 - % perl sync_uninstall.pl  (when you are done)

 - check out  the sample scripts and the perl module code (commented, but
not documented)






------=_NextPart_000_0062_01C0541E.125CAF30
Content-Type: application/octet-stream; name="sync.pl"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment; filename="sync.pl"



# This script depicts the syncronization process for two users.


##  CHANGE THESE THREE VARIABLE TO MATCH YOUR SYSTEM  ###########
my $dbi_connect_string =3D 'dbi:Pg:dbname=3Dtest;host=3Dsnoopy';	#
my $db_user =3D 'test';						#
my $db_pass =3D 'test';						#
#################################################################

my $ret; #holds return value

use SynKit;

#create a synchronization object (pass dbi connection info)
my $s =3D Synchronize->new($dbi_connect_string,$db_user,$db_pass);

#start a session by passing a user name, "node" identifier and a collision =
queue name (client or server)
$ret =3D $s->start_session('JOE','REMOTE_NODE_NAME','server');
print "Handle this error: $ret\n\n" if $ret;

#call this once before attempting to apply individual changes
$ret =3D $s->start_changes('sync_test',['name']);
print "Handle this error: $ret\n\n" if $ret;

#call this for each change the client wants to make to the database
$ret =3D  $s->apply_change(CLIENTROWID,'insert',undef,['ted']);
print "Handle this error: $ret\n\n" if $ret;

#call this for each change the client wants to make to the database
$ret =3D  $s->apply_change(CLIENTROWID,'insert','1973-11-10 11:25:00 AM -05=
',['tim']);
print "Handle this error: $ret\n\n" if $ret;

#call this for each change the client wants to make to the database
$ret =3D  $s->apply_change(999,'update',undef,['tom']);
print "Handle this error: $ret\n\n" if $ret;

#call this for each change the client wants to make to the database
$ret =3D  $s->apply_change(1,'update',undef,['tom']);
print "Handle this error: $ret\n\n" if $ret;

#call this once after all changes have been submitted
$ret =3D $s->end_changes();
print "Handle this error: $ret\n\n" if $ret;

#call this to get updates from all subscribed tables
$ret =3D $s->get_all_updates();
print "Handle this error: $ret\n\n" if $ret;

print "\n\nSyncronization session is complete. (JOE) \n\n";


# make some changes to the database (server perspective)

print "\n\nMaking changes to the the database. (server side) \n\n";

use DBI;
my $dbh =3D DBI->connect($dbi_connect_string,$db_user,$db_pass);

$dbh->do("insert into sync_test values ('roger')");
$dbh->do("insert into sync_test values ('john')");
$dbh->do("insert into sync_test values ('harry')");
$dbh->do("delete from sync_test where name =3D 'roger'");
$dbh->do("update sync_test set name =3D 'tom' where name =3D 'harry'");

$dbh->disconnect;


#now do another session for a different user

#start a session by passing a user name, "node" identifier and a collision =
queue name (client or server)
$ret =3D $s->start_session('KEN','ANOTHER_REMOTE_NODE_NAME','server');
print "Handle this error: $ret\n\n" if $ret;

#call this to get updates from all subscribed tables
$ret =3D $s->get_all_updates();
print "Handle this error: $ret\n\n" if $ret;

print "\n\nSynchronization session is complete. (KEN)\n\n";

print "Now look at your database and see what happend, make changes to the =
test table, etc. and run this again.\n\n";

------=_NextPart_000_0062_01C0541E.125CAF30
Content-Type: application/octet-stream; name="sync_uninstall.pl"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment; filename="sync_uninstall.pl"


# this script uninstalls the synchronization system using the SyncManager o=
bject;

use SynKit;

###  CHANGE THESE TO MATCH YOUR SYSTEM   ########################
my $dbi_connect_string =3D 'dbi:Pg:dbname=3Dtest;host=3Dsnoopy';	#
my $db_user =3D 'test';						#
my $db_pass =3D 'test';						#
#################################################################


my $ret; #holds return value

#create an instance of the SyncManager object
my $m =3D SyncManager->new($dbi_connect_string,$db_user,$db_pass);

# call this to unsubscribe a user/node (not necessary if you are uninstalli=
ng)
print $m->unsubscribe('KEN','ANOTHER_REMOTE_NODE_NAME','sync_test');

#call this to unpublish a table (not necessary if you are uninstalling)
print $m->unpublish('sync_test');

#call this to uninstall the syncronization system
#  NOTE: this will automatically unpublish & unsubscribe all users
print $m->UNINSTALL;

# now let's drop our little test table
use DBI;
my $dbh =3D DBI->connect($dbi_connect_string,$db_user,$db_pass);
$dbh->do("drop table sync_test");
$dbh->disconnect;

print "\n\nI hope you enjoyed this little demonstration\n\n";



------=_NextPart_000_0062_01C0541E.125CAF30
Content-Type: application/octet-stream; name="sync_install.pl"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment; filename="sync_install.pl"


# This script shows how to install the synchronization system=20
# using the SyncManager object

use SynKit;

### CHANGE THESE TO MATCH YOUR SYSTEM  ##########################
my $dbi_connect_string =3D 'dbi:Pg:dbname=3Dtest;host=3Dsnoopy';	#
my $db_user =3D 'test';						#
my $db_pass =3D 'test';						#
#################################################################
my $ret; #holds return value


#create an instance of the sync manager object
my $m =3D SyncManager->new($dbi_connect_string,$db_user,$db_pass);

#Call this to install the syncronization management tables, etc.
$ret =3D $m->INSTALL;
die "Handle this error: $ret\n\n" if $ret;



#create a test table for us to demonstrate with
use DBI;
my $dbh =3D DBI->connect($dbi_connect_string,$db_user,$db_pass);
$dbh->do("create table sync_test (name text)");
$dbh->do("insert into sync_test values ('rob')");
$dbh->do("insert into sync_test values ('rob')");
$dbh->do("insert into sync_test values ('rob')");
$dbh->do("insert into sync_test values ('ted')");
$dbh->do("insert into sync_test values ('ted')");
$dbh->do("insert into sync_test values ('ted')");
$dbh->disconnect;




#call this to "publish" a table
$ret =3D $m->publish('sync_test');
print "Handle this error: $ret\n\n" if $ret;

#call this to "subscribe" a user/node to a publication (table)
$ret =3D $m->subscribe('JOE','REMOTE_NODE_NAME','sync_test');
print "Handle this error: $ret\n\n" if $ret;

#call this to "subscribe" a user/node to a publication (table)
$ret =3D $m->subscribe('KEN','ANOTHER_REMOTE_NODE_NAME','sync_test');
print "Handle this error: $ret\n\n" if $ret;


print "Now you can do: 'perl sync.pl' a few times to play\n\n";
print "Do 'perl sync_uninstall.pl' to uninstall the system\n";


------=_NextPart_000_0062_01C0541E.125CAF30
Content-Type: application/octet-stream; name="SynKit.pm"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment; filename="SynKit.pm"

# Perl DB synchronization toolkit

#created for postgres 7.0.2 +
use strict;

BEGIN {
        use vars       qw($VERSION);
        # set the version for version checking
        $VERSION     =3D 1.00;
}


package Synchronize;

use DBI;

use Date::Parse;

# new requires 3 arguments: dbi connection string, plus the corresponding u=
sername and password to get connected to the database
sub new {
	my $proto =3D shift;
	my $class =3D ref($proto) || $proto;
	my $self =3D {};

	my $dbi =3D shift;
	my $user =3D shift;
	my $pass =3D shift;

	$self->{DBH} =3D DBI->connect($dbi,$user,$pass) || die "Failed to connect =
to database: ".DBI->errstr();

	$self->{user} =3D undef;
	$self->{node} =3D undef;
	$self->{status} =3D undef; # holds status of table update portion of sessi=
on
	$self->{pubs} =3D {}; #holds hash of pubs available to sessiom with val =
=3D 1 if ok to request sync
	$self->{orderpubs} =3D undef; #holds array ref of subscribed pubs ordered =
by sync_order
	$self->{this_post_ver} =3D undef; #holds the version number under which th=
is session will post changes
	$self->{max_ver} =3D undef; #holds the maximum safe version for getting up=
dates
	$self->{current} =3D {}; #holds the current publication info to which chan=
ges are being applied
	$self->{queue} =3D 'server'; # tells collide function what to do with coll=
isions. (default is to hold on server)

	$self->{DBLOG}=3D DBI->connect($dbi,$user,$pass) || die "cannot log to DB:=
 ".DBI->errstr();=20


	return bless ($self, $class);
}

sub dblog {=20
	my $self =3D shift;
	my $msg =3D $self->{DBLOG}->quote($_[0]);
	my $quser =3D $self->{DBH}->quote($self->{user});
	my $qnode =3D $self->{DBH}->quote($self->{node});
	$self->{DBLOG}->do("insert into ____sync_log____ (username, nodename,stamp=
, message) values($quser, $qnode, now(), $msg)");
}


#start_session establishes session wide information and other housekeeping =
chores
	# Accepts username, nodename and queue (client or server) as arguments;

sub start_session {
	my $self =3D shift;
	$self->{user} =3D shift || die 'Username is required';
	$self->{node} =3D shift || die 'Nodename is required';
	$self->{queue} =3D shift;


	if ($self->{queue} ne 'server' && $self->{queue} ne 'client') {
		die "You must provide a queue argument of either 'server' or 'client'";
	}

	my $quser =3D $self->{DBH}->quote($self->{user});
	my $qnode =3D $self->{DBH}->quote($self->{node});

	my $sql =3D "select pubname from ____subscribed____ where username =3D $qu=
ser and nodename =3D $qnode";
	my @pubs =3D $self->GetColList($sql);

	return 'User/Node has no subscriptions!' if !defined(@pubs);

	# go though the list and check permissions and rules for each
	foreach my $pub (@pubs) {
		my $qpub =3D $self->{DBH}->quote($pub);
		my $sql =3D "select disabled, pubname, fullrefreshonly, refreshonce,post_=
ver from ____subscribed____ where username =3D $quser and pubname =3D $qpub=
 and nodename =3D $qnode";
		my $sth =3D $self->{DBH}->prepare($sql) || die $self->{DBH}->errstr;
		$sth->execute || die $self->{DBH}->errstr;
		my @row;
		while (@row =3D $sth->fetchrow_array) {
			next if $row[0]; #publication is disabled
			next if !defined($row[1]); #publication does not exist (should never occ=
ur)
			if ($row[2] || $row[3]) { #refresh of refresh once flag is set
				$self->{pubs}->{$pub} =3D 0; #refresh only
				next;
			}
			if (!defined($row[4])) { #no previous session exists, must refresh
				$self->{pubs}->{$pub} =3D 0; #refresh only
				next;
			}
			$self->{pubs}->{$pub} =3D 1; #OK for sync
		}
		$sth->finish;
	}


	$sql =3D "select pubname from ____publications____ order by sync_order";
	my @op =3D $self->GetColList($sql);
	my @orderpubs;

	#loop through ordered pubs and remove non subscribed publications
	foreach my $pub (@op) {
		push @orderpubs, $pub if defined($self->{pubs}->{$pub});
	}
=09
	$self->{orderpubs} =3D \@orderpubs;

# Now we obtain a session version number, etc.

	$self->{DBH}->{AutoCommit} =3D 0; #allows "transactions"
	$self->{DBH}->{RaiseError} =3D 1; #script [or eval] will automatically die=
 on errors

	eval { #start DB transaction

	#lock the version sequence until we determin that we have gotten
	#a good  value.  Lock will be released on commit.
		$self->{DBH}->do('lock ____version_seq____ in access exclusive mode');

	# remove stale locks if they exist
		my $sql =3D "delete from ____last_stable____ where username =3D $quser an=
d nodename =3D $qnode";
		$self->{DBH}->do($sql);

	# increment version sequence & grab the next val as post_ver
		my $sql =3D "select nextval('____version_seq____')";
		my $sth =3D $self->{DBH}->prepare($sql);
		$sth->execute;
		($self->{this_post_ver}) =3D $sth->fetchrow_array();
		$sth->finish;
	# grab max_ver from last_stable

		$sql =3D "select min(version) from ____last_stable____";=20
		$sth =3D $self->{DBH}->prepare($sql);
		$sth->execute;
		($self->{max_ver}) =3D $sth->fetchrow_array();
		$sth->finish;

	# if there was no version in lock table, then take the ID that was in use
	# when we started the session ($max_ver -1)

		$self->{max_ver} =3D $self->{this_post_ver} -1 if (!defined($self->{max_v=
er}));

	# lock post_ver by placing it in last_stable
		$self->{DBH}->do("insert into ____last_stable____ (version, username, nod=
ename) values ($self->{this_post_ver}, $quser,$qnode)");

	# increment version sequence again (discard result)
		$sql =3D "select nextval('____version_seq____')";
		$sth =3D $self->{DBH}->prepare($sql);
		$sth->execute;
		$sth->fetchrow_array();
		$sth->finish;

	}; #end eval/transaction

	if ($@) { # part of transaction failed
		return 'Start session failed';
		$self->{DBH}->rollback;
	} else { # all's well commit block
		$self->{DBH}->commit;
	}
	$self->{DBH}->{AutoCommit} =3D 1;
	$self->{DBH}->{RaiseError} =3D 0;

	return undef;

}

#start changes should be called once before applying individual change requ=
ests
	# Requires publication and ref to columns that will be updated as arguments
sub start_changes {
	my $self =3D shift;
	my $pub =3D shift || die 'Publication is required';
	my $colref =3D shift || die 'Reference to column array is required';

	$self->{status} =3D 'starting';

	my $qpub =3D $self->{DBH}->quote($pub);
	my $quser =3D $self->{DBH}->quote($self->{user});
	my $qnode =3D $self->{DBH}->quote($self->{node});

	my @cols =3D @{$colref};
	my @subcols =3D $self->GetColList("select col_name from ____subscribed_col=
s____ where username =3D $quser and nodename =3D $qnode and pubname =3D $qp=
ub");
	my %subcols;
	foreach my $col (@subcols) {
		$subcols{$col} =3D 1;
	}
	foreach my $col (@cols) {=09
		return "User/node is not subscribed to column '$col'" if !$subcols{$col};
	}

	my $sql =3D "select pubname, readonly, last_session, post_ver, last_ver, w=
hereclause, sanity_limit,=20
sanity_delete, sanity_update, sanity_insert from ____subscribed____ where u=
sername =3D $quser and pubname =3D $qpub and nodename =3D $qnode";
	my ($junk, $readonly, $last_session, $post_ver, $last_ver, $whereclause, $=
sanity_limit,=20
$sanity_delete, $sanity_update, $sanity_insert) =3D $self->GetOneRow($sql);
=09
	return 'Publication is read only' if $readonly;

	$sql =3D "select whereclause from ____publications____ where pubname =3D $=
qpub";
	my ($wc) =3D $self->GetOneRow($sql);
	$whereclause =3D '('.$whereclause.')' if $whereclause;
	$whereclause =3D $whereclause.' and ('.$wc.')' if $wc;

	my ($table) =3D $self->GetOneRow("select tablename from ____publications__=
__ where pubname =3D $qpub");

	return 'Publication is not registered correctly' if !defined($table);

	my %info;
	$info{pub} =3D $pub;
	$info{whereclause} =3D $whereclause;
	$info{post_ver} =3D $post_ver;
	$last_session =3D~ s/([+|-]\d\d?)$/ $1/;	#put a space before timezone=09
	$last_session =3D str2time ($last_session); #convert to perltime (seconds =
since 1970)
	$info{last_session} =3D $last_session;
	$info{last_ver} =3D $last_ver;
	$info{table}  =3D $table;
	$info{cols} =3D \@cols;

	my $sql =3D "select count(oid) from $table";
	$sql =3D $sql .' '.$whereclause if $whereclause;
	my ($rowcount) =3D $self->GetOneRow($sql);

	#calculate sanity levels (convert from % to number of rows)
	# limits defined as less than 1 mean no limit
	$info{sanitylimit} =3D $rowcount * ($sanity_limit / 100) if $sanity_limit =
> 0;
	$info{insertlimit} =3D $rowcount * ($sanity_insert / 100) if $sanity_inser=
t > 0;
	$info{updatelimit} =3D $rowcount * ($sanity_update / 100) if $sanity_updat=
e > 0;
	$info{deletelimit} =3D $rowcount * ($sanity_delete / 100) if $sanity_delet=
e > 0;

	$self->{sanitycount} =3D 0;
	$self->{updatecount} =3D 0;
	$self->{insertcount} =3D 0;
	$self->{deletecount} =3D 0;

	$self->{current} =3D \%info;

	$self->{DBH}->{AutoCommit} =3D 0; #turn on transaction behavior so we can =
roll back on sanity limits, etc.

	$self->{status} =3D 'ready';

	return undef;
}

#call this once all changes are submitted to commit them;
sub end_changes {
	my $self =3D shift;
	return undef if $self->{status} ne 'ready';
	$self->{DBH}->commit;
	$self->{DBH}->{AutoCommit} =3D 1;
	$self->{status} =3D 'success';
	return undef;
}

#call apply_change once for each row level client update
	# Accepts 4 params: rowid, action, timestamp and reference to data array
	#	Note: timestamp can be undef, data can be undef
	#		timestamp MUST be in perl time (secs since 1970)

#this routine checks basic timestamp info and sanity limits, then passes th=
e info along to do_action() for processing
sub apply_change {
	my $self =3D shift;
	my $rowid =3D shift || return 'Row ID is required'; #don't die just for on=
e bad row
	my $action =3D shift || return 'Action is required'; #don't die just for o=
ne bad row
	my $timestamp =3D shift;
	my $dataref =3D shift;
	$action =3D lc($action);

	$timestamp =3D str2time($timestamp) if $timestamp;

	return 'Status failure, cannot accept changes: '.$self->{status} if $self-=
>{status} ne 'ready';

	my %info =3D %{$self->{current}};

	$self->{sanitycount}++;
	if ($info{sanitylimit} && $self->{sanitycount} > $info{sanitylimit}) {
		# too many changes from client
		my $ret =3D $self->sanity('limit');
		return $ret if $ret;
	}

=09
	if ($timestamp && $timestamp > time() + 3600) { # current time + one hour
		#client's clock is way off, cannot submit changes in future
		my $ret =3D $self->collide('future', $info{table}, $rowid, $action, undef=
, $timestamp, $dataref, $self->{queue});
		return $ret if $ret;
	}

	if ($timestamp && $timestamp < $info{last_session} - 3600) { # last sessio=
n time less one hour
		#client's clock is way off, cannot submit changes that occured before las=
t sync date
		my $ret =3D $self->collide('past', $info{table}, $rowid, $action, undef, =
$timestamp, $dataref , $self->{queue});
		return $ret if $ret;
	}

	my ($crow, $cver, $ctime); #current row,ver,time
	if ($action ne 'insert') {
		my $sql =3D "select ____rowid____, ____rowver____, ____stamp____ from $in=
fo{table} where ____rowid____ =3D $rowid";
		($crow, $cver, $ctime) =3D $self->GetOneRow($sql);
		if (!defined($crow)) {
			my $ret =3D $self->collide('norow', $info{table}, $rowid, $action, undef=
, $timestamp, $dataref , $self->{queue});
			return $ret if $ret;=09=09
		}

		$ctime =3D~ s/([+|-]\d\d?)$/ $1/; #put space between timezone
		$ctime =3D str2time($ctime) if $ctime; #convert to perl time

		if ($timestamp) {
			if ($ctime < $timestamp) {
				my $ret =3D $self->collide('time', $info{table}, $rowid, $action, undef=
, $timestamp, $dataref, $self->{queue} );=09=09
				return $ret if $ret;
			}

		} else {
			if ($cver > $self->{this_post_ver}) {
				my $ret =3D $self->collide('version', $info{table}, $rowid, $action, un=
def, $timestamp, $dataref, $self->{queue} );
				return $ret if $ret;
			}
		}
=09
	}

	if ($action eq 'insert') {
		$self->{insertcount}++;
		if ($info{insertlimit} && $self->{insertcount} > $info{insertlimit}) {
			# too many changes from client
			my $ret =3D $self->sanity('insert');
			return $ret if $ret;
		}

		my $qtable =3D $self->{DBH}->quote($info{table});
		my ($rowidsequence) =3D '_'.$self->GetOneRow("select table_id from ____ta=
bles____ where tablename =3D $qtable").'__rowid_seq';
		return 'Table incorrectly registered, cannot get rowid sequence name: '.$=
self->{DBH}->errstr() if not defined $rowidsequence;

		my @data;
		foreach my $val (@{$dataref}) {
			push @data, $self->{DBH}->quote($val);
		}
		my $sql =3D "insert into $info{table} (";
		if ($timestamp) {
			$sql =3D $sql . join(',',@{$info{cols}}) . ',____rowver____, ____stamp__=
__) values (';
			$sql =3D $sql . join (',',@data) .','.$self->{this_post_ver}.',\''.local=
time($timestamp).'\')';
		} else {
			$sql =3D $sql . join(',',@{$info{cols}}) . ',____rowver____) values (';
			$sql =3D $sql . join (',',@data) .','.$self->{this_post_ver}.')';
		}
		my $ret =3D $self->{DBH}->do($sql);
		if (!$ret) {
			my $ret =3D $self->collide($self->{DBH}->errstr(), $info{table}, $rowid,=
 $action, undef, $timestamp, $dataref , $self->{queue});
			return $ret if $ret;=09=09
		}
		my ($newrowid) =3D $self->GetOneRow("select currval('$rowidsequence')");
		return 'Failed to get current rowid on inserted row'.$self->{DBH}->errstr=
 if not defined $newrowid;
		$self->changerowid($rowid, $newrowid);
	}

	if ($action eq 'update') {
		$self->{updatecount}++;
		if ($info{updatelimit} && $self->{updatecount} > $info{updatelimit}) {
			# too many changes from client
			my $ret =3D $self->sanity('update');
			return $ret if $ret;
		}
		my @data;
		foreach my $val (@{$dataref}) {
			push @data, $self->{DBH}->quote($val);
		}=09

		my $sql =3D "update $info{table} set ";
		my @cols =3D @{$info{cols}};
		foreach my $col (@cols) {
			my $val =3D shift @data;
			$sql =3D $sql . "$col =3D $val,";
		}
		$sql =3D $sql." ____rowver____ =3D $self->{this_post_ver}";
		$sql =3D $sql.", ____stamp____ =3D '".localtime($timestamp)."'" if $times=
tamp;
		$sql =3D $sql." where ____rowid____ =3D $rowid";
		$sql =3D $sql." and $info{whereclause}" if $info{whereclause};
		my $ret =3D $self->{DBH}->do($sql);
		if (!$ret) {
			my $ret =3D $self->collide($self->{DBH}->errstr(), $info{table}, $rowid,=
 $action, undef, $timestamp, $dataref , $self->{queue});
			return $ret if $ret;=09=09
		}

	}

	if ($action eq 'delete') {
		$self->{deletecount}++;
		if ($info{deletelimit} && $self->{deletecount} > $info{deletelimit}) {
			# too many changes from client
			my $ret =3D $self->sanity('delete');
			return $ret if $ret;
		}
		if ($timestamp) {
			my $sql =3D "update $info{table} set ____rowver____ =3D $self->{this_pos=
t_ver}, ____stamp____ =3D '".localtime($timestamp)."'  where ____rowid____ =
=3D $rowid";
			$sql =3D $sql . " where $info{whereclause}" if $info{whereclause};
			$self->{DBH}->do($sql) || return 'Predelete update failed: '.$self->{DBH=
}->errstr;
		} else {
			my $sql =3D "update $info{table} set ____rowver____ =3D $self->{this_pos=
t_ver} where ____rowid____ =3D $rowid";
			$sql =3D $sql . " where $info{whereclause}" if $info{whereclause};
			$self->{DBH}->do($sql) || return 'Predelete update failed: '.$self->{DBH=
}->errstr;
		}
		my $sql =3D "delete from $info{table} where ____rowid____ =3D $rowid";
		$sql =3D $sql . " where $info{whereclause}" if $info{whereclause};
		my $ret =3D $self->{DBH}->do($sql);
		if (!$ret) {
			my $ret =3D $self->collide($self->{DBH}->errstr(), $info{table}, $rowid,=
 $action, undef, $timestamp, $dataref , $self->{queue});
			return $ret if $ret;=09=09
		}
}
=09
=09
	return undef;
}

sub changerowid {
	my $self =3D shift;
	my $oldid =3D shift;
	my $newid =3D shift;
	$self->writeclient('changeid',"$oldid\t$newid");
}

#writes info to client
sub writeclient {
	my $self =3D shift;
	my $type =3D shift;
	my @info =3D @_;
	print "$type: ",join("\t",@info),"\n";
	return undef;
}

# Override this for custom behavior.  Default is to echo back the sanity fa=
ilure reason.=20=20
# If you want to override a collision, you can do so by returning undef.
sub sanity {
	my $self =3D shift;
	my $reason =3D shift;
	$self->{status} =3D 'sanity exceeded';
	$self->{DBH}->rollback;
	return $reason;
}

# Override this for custom behavior.  Default is to echo back the failure r=
eason.=20=20
# If you want to override a collision, you can do so by returning undef.
sub collide {
	my $self =3D shift;
	my ($reason,$table,$rowid,$action,$rowver,$timestamp,$data, $queue) =3D @_;

	my @data;
	foreach my $val (@{$data}) {
		push @data, $self->{DBH}->quote($val);
	}=09

	if ($reason =3D~ /integrity/i || $reason =3D~ /constraint/i) {
		$self->{status} =3D 'intergrity violation';
		$self->{DBH}->rollback;
	}

	my $datastring;
	my @cols =3D @{$self->{current}->{cols}};
	foreach my $col (@cols) {
		my $val =3D shift @data;
		$datastring =3D $datastring . "$col =3D $val,";
	}
	chop $datastring; #remove trailing comma

	if ($queue eq 'server') {
		$timestamp =3D localtime($timestamp) if defined($timestamp);
		$rowid =3D $self->{DBH}->quote($rowid);
		$rowid =3D 'null' if !defined($rowid);
		$rowver =3D 'null' if !defined($rowver);
		$timestamp =3D $self->{DBH}->quote($timestamp);
		$data =3D $self->{DBH}->quote($data);
		my $qtable =3D $self->{DBH}->quote($table);
		my $qreason =3D $self->{DBH}->quote($reason);
		my $qaction =3D $self->{DBH}->quote($action);
		my $quser =3D $self->{DBH}->quote($self->{user});
		my $qnode =3D $self->{DBH}->quote($self->{node});
		$datastring =3D $self->{DBH}->quote($datastring);


		my $sql =3D "insert into ____collision____ (rowid,
tablename, rowver, stamp, data, reason, action, username,
nodename, queue) values($rowid,$qtable, $rowver, $timestamp,$datastring,
$qreason, $qaction,$quser, $qnode)";
		$self->{DBH}->do($sql) || die 'Failed to write to collision table: '.$sel=
f->{DBH}->errstr;

	} else {

		$self->writeclient('collision',$rowid,$table, $rowver, $timestamp,$reason=
, $action,$self->{user}, $self->{node}, $data);

	}
	return $reason;
}

#calls get_updates once for each publication the user/node is subscribed to=
 in correct sync_order
sub get_all_updates {
	my $self =3D shift;
	my $quser =3D $self->{DBH}->quote($self->{user});
	my $qnode =3D $self->{DBH}->quote($self->{node});

	foreach my $pub (@{$self->{orderpubs}}) {
		$self->get_updates($pub, 1); #request update as sync unless overrridden b=
y flags
	}

}

# Call this once for each table the client needs refreshed or sync'ed AFTER=
 all inbound client changes have been posted
#	Accepts publication and sync flag as arguments
sub get_updates {
	my $self =3D shift;
	my $pub =3D shift || die 'Publication is required';
	my $sync =3D shift;

	my $qpub =3D $self->{DBH}->quote($pub);
	my $quser =3D $self->{DBH}->quote($self->{user});
	my $qnode =3D $self->{DBH}->quote($self->{node});

	#enforce refresh and refreshonce flags
	undef $sync if !$self->{pubs}->{$pub};=20


	my %info =3D $self->{current};

	my @cols =3D $self->GetColList("select col_name from ____subscribed_cols__=
__ where username =3D $quser and nodename =3D $qnode and pubname =3D $qpub"=
);;

	my ($table) =3D $self->GetOneRow("select tablename from ____publications__=
__ where pubname =3D $qpub");
	return 'Table incorrectly registered for read' if !defined($table);
	my $qtable =3D $self->{DBH}->quote($table);=09


	my $sql =3D "select pubname, last_session, post_ver, last_ver, whereclause=
 from ____subscribed____ where username =3D $quser and pubname =3D $qpub an=
d nodename =3D $qnode";
	my ($junk, $last_session, $post_ver, $last_ver, $whereclause) =3D $self->G=
etOneRow($sql);

	my ($wc) =3D $self->GetOneRow("select whereclause from ____publications___=
_ where pubname =3D $qpub");

	$whereclause =3D '('.$whereclause.')' if $whereclause;

	$whereclause =3D $whereclause.' and ('.$wc.')' if $wc;


	if ($sync) {
		$self->writeclient('start synchronize', $pub);
	} else {
		$self->writeclient('start refresh', $pub);
		$self->{DBH}->do("update ____subscribed____ set refreshonce =3D false whe=
re pubname =3D $qpub and username =3D $quser and nodename =3D $qnode") || r=
eturn 'Failed to clear RefreshOnce flag: '.$self->{DBH}->errstr;
	}

	$self->writeclient('columns',@cols);



	my $sql =3D "select ____rowid____, ".join(',', @cols)." from $table";
	if ($sync) {
		$sql =3D $sql." where (____rowver____ <=3D $self->{max_ver} and ____rowve=
r____ > $last_ver)";
		if (defined($self->{this_post_ver})) {
			$sql =3D $sql . " and (____rowver____ <> $post_ver)";
		}
	} else {
		$sql =3D $sql." where (____rowver____ <=3D $self->{max_ver})";
	}
	$sql =3D $sql." and $whereclause" if $whereclause;
=09
	my $sth =3D $self->{DBH}->prepare($sql) || return 'Failed to get prepare S=
QL for updates: '.$self->{DBH}->errstr;
	$sth->execute || return 'Failed to execute SQL for updates: '.$self->{DBH}=
->errstr;
	my @row;
	while (@row =3D $sth->fetchrow_array) {
		$self->writeclient('update/insert',@row);
	}

	$sth->finish;

	# now get deleted rows
	if ($sync) {
		$sql =3D "select rowid from ____deleted____ where (tablename =3D $qtable)=
";
		$sql =3D $sql." and (rowver <=3D $self->{max_ver} and rowver > $last_ver)=
";
		if (defined($self->{this_post_ver})) {
			$sql =3D $sql . " and (rowver <> $self->{this_post_ver})";
		}
		$sql =3D $sql." and $whereclause" if $whereclause;

		$sth =3D $self->{DBH}->prepare($sql) || return 'Failed to get prepare SQL=
 for deletes: '.$self->{DBH}->errstr;
		$sth->execute || return 'Failed to execute SQL for deletes: '.$self->{DBH=
}->errstr;
		my @row;
		while (@row =3D $sth->fetchrow_array) {
			$self->writeclient('delete',@row);
		}

		$sth->finish;
	}

	if ($sync) {
		$self->writeclient('end synchronize', $pub);
	} else {
		$self->writeclient('end refresh', $pub);
	}

	my $qpub =3D $self->{DBH}->quote($pub);
	my $quser =3D $self->{DBH}->quote($self->{user});
	my $qnode =3D $self->{DBH}->quote($self->{node});

	$self->{DBH}->do("update ____subscribed____ set last_ver =3D $self->{max_v=
er}, last_session =3D now(), post_ver =3D $self->{this_post_ver} where user=
name =3D $quser and nodename =3D $qnode and pubname =3D $qpub");
	return undef;
}


# Call this once when everything else is done.  Does housekeeping.=20
# (MAKE THIS AN OBJECT DESTRUCTOR?)
sub DESTROY {
	my $self =3D shift;

#release version from lock table (including old ones)
	my $quser =3D $self->{DBH}->quote($self->{user});
	my $qnode =3D $self->{DBH}->quote($self->{node});
	my $sql =3D "delete from ____last_stable____ where username =3D $quser and=
 nodename =3D $qnode";
	$self->{DBH}->do($sql);

#clean up deleted table
	my ($version) =3D $self->GetOneRow("select min(last_ver) from ____subscrib=
ed____");
	return undef if not defined $version;
	$self->{DBH}->do("delete from ____deleted____ where rowver < $version") ||=
 return 'Failed to prune deleted table'.$self->{DBH}->errstr;;


#disconnect from DBD sessions
	$self->{DBH}->disconnect;
	$self->{DBLOG}->disconnect;
	return undef;
}

############# Helper Subs ############
sub GetColList {
	my $self =3D shift;
	my $sql =3D shift || die 'Must provide sql select statement';
	my $sth =3D $self->{DBH}->prepare($sql) || return undef;
	$sth->execute || return undef;
	my $val;
	my @col;
	while (($val) =3D $sth->fetchrow_array) {
		push @col, $val;
	}
	$sth->finish;
	return @col;
}

sub GetOneRow {
	my $self =3D shift;
	my $sql =3D shift || die 'Must provide sql select statement';
	my $sth =3D $self->{DBH}->prepare($sql) || return undef;
	$sth->execute || return undef;
	my @row =3D $sth->fetchrow_array;
	$sth->finish;
	return @row;
}

=20



package SyncManager;

use DBI;
# new requires 3 arguments: dbi connection string, plus the corresponding u=
sername and password

sub new {
	my $proto =3D shift;
	my $class =3D ref($proto) || $proto;
	my $self =3D {};

	my $dbi =3D shift;
	my $user =3D shift;
	my $pass =3D shift;

	$self->{DBH} =3D DBI->connect($dbi,$user,$pass) || die "Failed to connect =
to database: ".DBI->errstr();

	$self->{DBLOG}=3D DBI->connect($dbi,$user,$pass) || die "cannot log to DB:=
 ".DBI->errstr();
=09
	return bless ($self, $class);
}

sub dblog {=20
	my $self =3D shift;
	my $msg =3D $self->{DBLOG}->quote($_[0]);
	my $quser =3D $self->{DBH}->quote($self->{user});
	my $qnode =3D $self->{DBH}->quote($self->{node});
	$self->{DBLOG}->do("insert into ____sync_log____ (username, nodename,stamp=
, message) values($quser, $qnode, now(), $msg)");
}

#this should never need to be called, but it might if a node bails without =
releasing their locks
sub ReleaseAllLocks {
	my $self =3D shift;
	$self->{DBH}->do("delete from ____last_stable____)");
}
# Adds a publication to the system.  Also adds triggers, sequences, etc ass=
ociated with the table if approproate.
	# accepts two argument: the name of a physical table and the name under wh=
ich to publish it=20
	# 	NOTE: the publication name is optional and will default to the table na=
me if not supplied
	# returns undef if ok, else error string;
sub publish {
	my $self =3D shift;
	my $table =3D shift || die 'You must provide a table name (and optionally =
a unique publication name)';
	my $pub =3D shift;
	$pub =3D $table if not defined($pub);

	my $qpub =3D $self->{DBH}->quote($pub);
	my $sql =3D "select tablename from ____publications____ where pubname =3D =
$qpub";
	my ($junk) =3D $self->GetOneRow($sql);
	return 'Publication already exists' if defined($junk);

	my $qtable =3D $self->{DBH}->quote($table);

	$sql =3D "select table_id, refcount from ____tables____ where tablename =
=3D $qtable";
	my ($id, $refcount) =3D $self->GetOneRow($sql);

	if(!defined($id)) {
		$self->{DBH}->do("insert into ____tables____ (tablename, refcount) values=
 ($qtable,1)") || return 'Failed to register table: ' . $self->{DBH}->errst=
r;
		my $sql =3D "select table_id from ____tables____ where tablename =3D $qta=
ble";
		($id) =3D $self->GetOneRow($sql);
	}

	if (defined($refcount)) {
		$self->{DBH}->do("update ____tables____ set refcount =3D refcount+1 where=
 table_id =3D $id") || return 'Failed to update refrence count: ' . $self->=
{DBH}->errstr;
	} else {
=09=09
		$id =3D '_'.$id.'_';=20

		my @cols =3D $self->GetTableCols($table, 1); # 1 =3D get hidden cols too
		my %skip;
		foreach my $col (@cols) {
			$skip{$col} =3D 1;
		}
=09=09
		if (!$skip{____rowver____}) {
			$self->{DBH}->do("alter table $table add column ____rowver____ int4"); #=
don't fail here in case table is being republished, just accept the error s=
ilently
		}
		$self->{DBH}->do("update $table set ____rowver____ =3D ____version_seq___=
_.last_value - 1") || return 'Failed to initialize rowver: ' . $self->{DBH}=
->errstr;

		if (!$skip{____rowid____}) {
			$self->{DBH}->do("alter table $table add column ____rowid____ int4"); #d=
on't fail here in case table is being republished, just accept the error si=
lently
		}

		my $index =3D $id.'____rowid____idx';
		$self->{DBH}->do("create index $index on $table(____rowid____)") || retur=
n 'Failed to create rowid index: ' . $self->{DBH}->errstr;

		my $sequence =3D $id.'_rowid_seq';
		$self->{DBH}->do("create sequence $sequence") || return 'Failed to create=
 rowver sequence: ' . $self->{DBH}->errstr;

		$self->{DBH}->do("alter table $table alter column ____rowid____ set defau=
lt nextval('$sequence')"); #don't fail here in case table is being republis=
hed, just accept the error silently

		$self->{DBH}->do("update $table set ____rowid____ =3D  nextval('$sequence=
')") || return 'Failed to initialize rowid: ' . $self->{DBH}->errstr;

		if (!$skip{____stamp____}) {
			$self->{DBH}->do("alter table $table add column ____stamp____ timestamp"=
); #don't fail here in case table is being republished, just accept the err=
or silently
		}

		$self->{DBH}->do("update $table set ____stamp____ =3D  now()") || return =
'Failed to initialize stamp: ' . $self->{DBH}->errstr;

		my $trigger =3D $id.'_ver_ins';
		$self->{DBH}->do("create trigger $trigger before insert on $table for eac=
h row execute procedure sync_insert_ver()") || return 'Failed to create tri=
gger: ' . $self->{DBH}->errstr;

		my $trigger =3D $id.'_ver_upd';
		$self->{DBH}->do("create trigger $trigger before update on $table for eac=
h row execute procedure sync_update_ver()") || return 'Failed to create tri=
gger: ' . $self->{DBH}->errstr;

		my $trigger =3D $id.'_del_row';
		$self->{DBH}->do("create trigger $trigger after delete on $table for each=
 row execute procedure sync_delete_row()") || return 'Failed to create trig=
ger: ' . $self->{DBH}->errstr;
	}

	$self->{DBH}->do("insert into ____publications____ (pubname, tablename) va=
lues ('$pub','$table')") || return 'Failed to create publication entry: '.$=
self->{DBH}->errstr;

	return undef;
}


# Removes a publication from the system.  Also drops triggers, sequences, e=
tc associated with the table if approproate.
	# accepts one argument: the name of a publication
	# returns undef if ok, else error string;
sub unpublish {
	my $self =3D shift;
	my $pub =3D shift || return 'You must provide a publication name';
	my $qpub =3D $self->{DBH}->quote($pub);
	my $sql =3D "select tablename from ____publications____ where pubname =3D =
$qpub";
	my ($table) =3D $self->GetOneRow($sql);
	return 'Publication does not exist' if !defined($table);

	my $qtable =3D $self->{DBH}->quote($table);

	$sql =3D "select table_id, refcount from ____tables____ where tablename =
=3D $qtable";
	my ($id, $refcount) =3D $self->GetOneRow($sql);
	return 'Table: $table is not correctly registered!' if not defined($id);

	$self->{DBH}->do("update ____tables____ set refcount =3D refcount -1 where=
 tablename =3D $qtable") || return 'Failed to decrement reference count: ' =
. $self->{DBH}->errstr;

	$self->{DBH}->do("delete from ____subscribed____ where pubname =3D $qpub")=
 || return 'Failed to delete user subscriptions: ' . $self->{DBH}->errstr;
	$self->{DBH}->do("delete from ____subscribed_cols____ where pubname =3D $q=
pub") || return 'Failed to delete subscribed columns: ' . $self->{DBH}->err=
str;
	$self->{DBH}->do("delete from ____publications____ where tablename =3D $qt=
able and pubname =3D $qpub") || return 'Failed to delete from publications:=
 ' . $self->{DBH}->errstr;

	#if this is the last reference, we want to drop triggers, etc;
	if ($refcount <=3D 1) {
		$id =3D "_".$id."_";

		$self->{DBH}->do("alter table $table alter column ____rowver____ drop def=
ault") || return 'Failed to alter column default: ' . $self->{DBH}->errstr;
		$self->{DBH}->do("alter table $table alter column ____rowid____ drop defa=
ult") || return 'Failed to alter column default: ' . $self->{DBH}->errstr;
		$self->{DBH}->do("alter table $table alter column ____stamp____ drop defa=
ult") || return 'Failed to alter column default: ' . $self->{DBH}->errstr;

		my $trigger =3D $id.'_ver_upd';
		$self->{DBH}->do("drop trigger $trigger on $table") || return 'Failed to =
drop trigger: ' . $self->{DBH}->errstr;

		my $trigger =3D $id.'_ver_ins';
		$self->{DBH}->do("drop trigger $trigger on $table") || return 'Failed to =
drop trigger: ' . $self->{DBH}->errstr;

		my $trigger =3D $id.'_del_row';
		$self->{DBH}->do("drop trigger $trigger on $table") || return 'Failed to =
drop trigger: ' . $self->{DBH}->errstr;

		my $sequence =3D $id.'_rowid_seq';
		$self->{DBH}->do("drop sequence $sequence") || return 'Failed to drop seq=
uence: ' . $self->{DBH}->errstr;

		my $index =3D $id.'____rowid____idx';
		$self->{DBH}->do("drop index $index") || return 'Failed to drop index: ' =
. $self->{DBH}->errstr;
		$self->{DBH}->do("delete from ____tables____ where tablename =3D $qtable"=
) || return 'remove entry from tables: ' . $self->{DBH}->errstr;
	}
return undef;
}





#Subscribe user/node to a publication
	# Accepts 3 arguements: Username, Nodename, Publication
	# 	NOTE: the remaining arguments can be supplied as column names to which =
the user/node should be subscribed
	# Return undef if ok, else returns an error string

sub subscribe {
	my $self =3D shift;
	my $user =3D shift || die 'You must provide user, node and publication as =
arguments';
	my $node =3D shift || die 'You must provide user, node and publication as =
arguments';
	my $pub =3D shift || die 'You must provide user, node and publication as a=
rguments';
	my @cols =3D @_;

	my $quser =3D $self->{DBH}->quote($user);
	my $qnode =3D $self->{DBH}->quote($node);
	my $qpub =3D $self->{DBH}->quote($pub);

	my $sql =3D "select tablename from ____publications____ where pubname =3D =
$qpub";
	my ($table) =3D $self->GetOneRow($sql);
	return "Publication $pub does not exist." if not defined $table;
	my $qtable =3D $self->{DBH}->quote($table);

	@cols =3D $self->GetTableCols($table) if !@cols; # get defaults if cols we=
re not spefified by caller

	$self->{DBH}->do("insert into ____subscribed____ (username, nodename,pubna=
me,last_ver,refreshonce) values('$user', '$node','$pub',0, true)") || retur=
n 'Failes to create subscription: ' . $self->{DBH}->errstr;=09

	foreach my $col (@cols) {
		$self->{DBH}->do("insert into ____subscribed_cols____ (username, nodename=
, pubname, col_name) values ('$user','$node','$pub','$col')") || return 'Fa=
iles to subscribe column: ' . $self->{DBH}->errstr;=09
	}

	return undef;
}


#Unsubscribe user/node to a publication
	# Accepts 3 arguements: Username, Nodename, Publication
	# Return undef if ok, else returns an error string

sub unsubscribe {
	my $self =3D shift;
	my $user =3D shift || die 'You must provide user, node and publication as =
arguments';
	my $node =3D shift || die 'You must provide user, node and publication as =
arguments';
	my $pub =3D shift || die 'You must provide user, node and publication as a=
rguments';
	my @cols =3D @_;

	my $quser =3D $self->{DBH}->quote($user);
	my $qnode =3D $self->{DBH}->quote($node);
	my $qpub =3D $self->{DBH}->quote($pub);

	my $sql =3D "select tablename from ____publications____ where pubname =3D =
$qpub";
	my $table =3D $self->GetOneRow($sql);
	return "Publication $pub does not exist." if not defined $table;

	$self->{DBH}->do("delete from ____subscribed_cols____ where pubname =3D $q=
pub and username =3D $quser and nodename =3D $qnode") || return 'Failed to =
remove column subscription: '. $self->{DBH}->errstr;
	$self->{DBH}->do("delete from ____subscribed____ where pubname =3D $qpub a=
nd username =3D $quser and nodename =3D $qnode") || return 'Failed to remov=
e subscription: '. $self->{DBH}->errstr;


	return undef;
}



#INSTALL creates the necessary management tables.=20=20
	#returns undef if everything is ok, else returns a string describing the e=
rror;
sub INSTALL {
my $self =3D shift;

#check to see if management tables are already installed

my ($test) =3D $self->GetOneRow("select * from pg_class where relname =3D '=
____publications____'");
if (defined($test)) {
	return 'It appears that synchronization manangement tables are already ins=
talled here.  Please uninstall before reinstalling.';
};



#install the management tables, etc.

$self->{DBH}->do("create table ____publications____ (pubname text primary k=
ey,description text, tablename text, sync_order int4, whereclause text)") |=
| return $self->{DBH}->errstr();

$self->{DBH}->do("create table ____subscribed_cols____ (nodename text, user=
name text, pubname text, col_name text, description text, primary key(noden=
ame, username, pubname,col_name))") || return $self->{DBH}->errstr();

$self->{DBH}->do("create table ____subscribed____ (nodename text, username =
text, pubname text, last_session timestamp, post_ver int4, last_ver int4, w=
hereclause text, sanity_limit int4 default 0, sanity_delete int4 default 0,=
 sanity_update int4 default 0, sanity_insert int4 default 50, readonly bool=
ean, disabled boolean, fullrefreshonly boolean, refreshonce boolean, primar=
y key(nodename, username, pubname))") || return $self->{DBH}->errstr();

$self->{DBH}->do("create table ____last_stable____ (version int4, username =
text, nodename text, primary key(version, username, nodename))") || return =
$self->{DBH}->errstr();

$self->{DBH}->do("create table ____tables____ (tablename text, table_id int=
4, refcount int4, primary key(tablename, table_id))") || return $self->{DBH=
}->errstr();

$self->{DBH}->do("create sequence ____table_id_seq____") || return $self->{=
DBH}->errstr();

$self->{DBH}->do("alter table ____tables____ alter column table_id set defa=
ult nextval('____table_id_seq____')") || return $self->{DBH}->errstr();

$self->{DBH}->do("create table ____deleted____ (rowid int4, tablename text,=
 rowver int4, stamp timestamp, primary key (rowid, tablename))") || return =
$self->{DBH}->errstr();

$self->{DBH}->do("create table ____collision____ (rowid text, tablename tex=
t, rowver int4, stamp timestamp, faildate timestamp default now(),data text=
,reason text, action text, username text, nodename text,queue text)") || re=
turn $self->{DBH}->errstr();

$self->{DBH}->do("create sequence ____version_seq____") || return $self->{D=
BH}->errstr();

$self->{DBH}->do("create table ____sync_log____ (username text, nodename te=
xt, stamp timestamp, message text)") || return $self->{DBH}->errstr();

$self->{DBH}->do("create function sync_insert_ver() returns opaque as
'begin
if new.____rowver____ isnull then
new.____rowver____ :=3D ____version_seq____.last_value;
end if;
if new.____stamp____ isnull then
new.____stamp____ :=3D now();
end if;
return NEW;
end;' language 'plpgsql'") || return $self->{DBH}->errstr();

$self->{DBH}->do("create function sync_update_ver() returns opaque as
'begin
if new.____rowver____ =3D old.____rowver____ then
new.____rowver____ :=3D ____version_seq____.last_value;
end if;
if new.____stamp____ =3D old.____stamp____ then
new.____stamp____ :=3D now();
end if;
return NEW;
end;' language 'plpgsql'") || return $self->{DBH}->errstr();


$self->{DBH}->do("create function sync_delete_row() returns opaque as=20
'begin=20
insert into ____deleted____ (rowid,tablename,rowver,stamp) values
(old.____rowid____, TG_RELNAME, old.____rowver____,old.____stamp____);=20
return old;=20
end;' language 'plpgsql'") || return $self->{DBH}->errstr();

return undef;
}

#removes all management tables & related stuff
	#returns undef if ok, else returns an error message as a string
sub UNINSTALL {
my $self =3D shift;

#Make sure all tables are unpublished first
my $sth =3D $self->{DBH}->prepare("select pubname from ____publications____=
");
$sth->execute;
my $pub;
while (($pub) =3D $sth->fetchrow_array) {
	$self->unpublish($pub);=09
}
$sth->finish;

$self->{DBH}->do("drop table ____publications____") || return $self->{DBH}-=
>errstr();
$self->{DBH}->do("drop table ____subscribed_cols____") || return $self->{DB=
H}->errstr();
$self->{DBH}->do("drop table ____subscribed____") || return $self->{DBH}->e=
rrstr();
$self->{DBH}->do("drop table ____last_stable____") || return $self->{DBH}->=
errstr();
$self->{DBH}->do("drop table ____deleted____") || return $self->{DBH}->errs=
tr();
$self->{DBH}->do("drop table ____collision____") || return $self->{DBH}->er=
rstr();
$self->{DBH}->do("drop table ____tables____") || return $self->{DBH}->errst=
r();
$self->{DBH}->do("drop table ____sync_log____") || return $self->{DBH}->err=
str();

$self->{DBH}->do("drop sequence ____table_id_seq____") || return $self->{DB=
H}->errstr();
$self->{DBH}->do("drop sequence ____version_seq____") || return $self->{DBH=
}->errstr();

$self->{DBH}->do("drop function sync_insert_ver()") || return $self->{DBH}-=
>errstr();
$self->{DBH}->do("drop function sync_update_ver()") || return $self->{DBH}-=
>errstr();
$self->{DBH}->do("drop function sync_delete_row()") || return $self->{DBH}-=
>errstr();

return undef;

}

sub DESTROY {
	my $self =3D shift;

	$self->{DBH}->disconnect;
	$self->{DBLOG}->disconnect;
	return undef;
}

############# Helper Subs ############

sub GetOneRow {
	my $self =3D shift;
	my $sql =3D shift || die 'Must provide sql select statement';
	my $sth =3D $self->{DBH}->prepare($sql) || return undef;
	$sth->execute || return undef;
	my @row =3D $sth->fetchrow_array;
	$sth->finish;
	return @row;
}

#call this with second non-zero value to get hidden columns
sub GetTableCols {
	my $self =3D shift;
	my $table =3D shift || die 'Must provide table name';
	my $wanthidden =3D shift;
	my $sql =3D "select * from $table where 0 =3D 1";
	my $sth =3D $self->{DBH}->prepare($sql) || return undef;
	$sth->execute || return undef;
	my @row =3D @{$sth->{NAME}};
	$sth->finish;
	return @row if $wanthidden;
	my @cols;
	foreach my $col (@row) {
		next if $col eq '____rowver____';
		next if $col eq '____stamp____';
		next if $col eq '____rowid____';
		push @cols, $col;=09
	}
	return @cols;
}


1; #happy require

------=_NextPart_000_0062_01C0541E.125CAF30--


From pgsql-hackers-owner+M9917@postgresql.org Mon Jun 11 15:53:25 2001
Return-path: <pgsql-hackers-owner+M9917@postgresql.org>
Received: from postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.10.1/8.10.1) with ESMTP id f5BJrPL01206
	for <pgman@candle.pha.pa.us>; Mon, 11 Jun 2001 15:53:25 -0400 (EDT)
Received: from postgresql.org.org (webmail.postgresql.org [216.126.85.28])
	by postgresql.org (8.11.3/8.11.1) with SMTP id f5BJrPE67753;
	Mon, 11 Jun 2001 15:53:25 -0400 (EDT)
	(envelope-from pgsql-hackers-owner+M9917@postgresql.org)
Received: from mail.greatbridge.com (mail.greatbridge.com [65.196.68.36])
	by postgresql.org (8.11.3/8.11.1) with ESMTP id f5BJmLE65620
	for <pgsql-hackers@postgresql.org>; Mon, 11 Jun 2001 15:48:21 -0400 (EDT)
	(envelope-from djohnson@greatbridge.com)
Received: from j2.us.greatbridge.com (djohnsonpc.us.greatbridge.com [65.196.69.70])
	by mail.greatbridge.com (8.11.2/8.11.2) with SMTP id f5BJm2Q28847
	for <pgsql-hackers@postgresql.org>; Mon, 11 Jun 2001 15:48:02 -0400
From: Darren Johnson <djohnson@greatbridge.com>
Date: Mon, 11 Jun 2001 19:46:44 GMT
Message-ID: <20010611.19464400@j2.us.greatbridge.com>
Subject: [HACKERS] Postgres Replication
To: pgsql-hackers@postgresql.org
Reply-To: Darren Johnson <djohnson@greatbridge.com>
X-Mailer: Mozilla/3.0 (compatible; StarOffice/5.2;Linux)
X-Priority: 3 (Normal)
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by postgresql.org id f5BJmLE65621
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR

We have been researching replication for several months now, and
I have some opinions to share to the community for feedback,
discussion, and/or participation. Our goal is to get a replication
solution for PostgreSQL that will meet most needs of users
and applications alike (mission impossible theme here :). 

My research work along with others contributors has been collected
and presented here http://www.greatbridge.org/genpage?replication_top
If there is something missing, especially PostgreSQL related
work, I would like to know about it, and my apologies to any
one who got left off the list. This work is ongoing and doesn't
draw a conclusion, which IMHO should be left up to the user,
but I'm offering my opinions to spur discussion and/or feed back
from this list, and try not to offend any one.

Here's my opinion: of the approaches we've surveyed, the most
promising one is the Postgres-R project from the Information and
Communication Systems Group, ETH  in Zurich, Switzerland, originally 
produced by Bettina Kemme, Gustavo Alonso, and others.  Although 
Postgres-R is a synchronous approach, I believe it is the closest to 
the goal mentioned above. Here is an abstract of the advantages.

1) Postgres-R is built on the PostgreSQL-6.4.2 code base.  The 
replication 
functionality is an optional parameter, so there will be insignificant 
overhead for non replication situations. The replication and 
communication
managers are the two new modules added to the PostgreSQL code base.

2) The replication manager's main function is controlling the
replication protocol via a message handling process. It receives
messages from the local and remote backends and forwards write
sets and decision messages via the communication manager to the
other servers. The replication manager controls all the transactions 
running on the local server by keeping track of the states, including 
which protocol phase (read, send, lock, or write) the transaction is
in. The replication manager maintains a two way channel
implemented as buffered sockets to each backend.

3) The main task of the communication manager is to provide simple
socket based interface between the replication manager and the
group communication system (currently Ensemble). The
communication system is a cluster of servers connected via
the communication manager.  The replication manager also maintains
three one-way channels to the communication system: a broadcast
channel to send messages, a total-order channel to receive
totally orders write sets, and a no-order channel to listen for
decision messages from the communication system. Decision
messages can be received at any time where the reception of
totally ordered write sets can be blocked in certain phases.

4) Based on a two phase locking approach, all dead lock situations
are local and detectable by Postgres-R code base, and aborted.

5) The write set messages used to send database changes to other
servers, can use either the SQL statements or the actual tuples
changed. This is a parameter based on number of tuples changed
by a transaction. While sending the tuple changes reduces
overhead in query parse, plan and execution, there is a negative
effect in sending a large write set across the network.

6) Postgres-R uses a synchronous approach that keeps the data on 
all sites consistent and provides serializability. The user does not 
have to bother with conflict resolution, and receives the same 
correctness and consistency of a centralized system.

7) Postgres-R could be part of a good fault-resilient and load 
distribution 
solution.  It is peer-to-peer based and incurs low overhead propagating 
updates to the other cluster members.  All replicated databases locally 
process queries.

8) Compared to other synchronous replication strategies (e.g., standard 
distributed 2-phase-locking + 2-phase-commit), Postgres-R has much 
better performance using 2-phase-locking.


There are some issues that are not currently addressed by
Postgres-R, but some enhancements made to PostgreSQL since the
6.4.2 tree are very favorable to addressing these short comings.

1) The addition of WAL in 7.1 has the information for recovering 
failed/off-line servers, currently all the servers would have to be 
stopped, and a copy would be used to get all the servers synchronized
before starting again. 

2)Being synchronous, Postgres-R would not be a good solution 
for off line/WAN scenarios where asynchronous replication is 
required.  There are some theories on this issue which involve servers
connecting and disconnecting from the cluster.

3)As in any serialized synchronous approach there is  change in the 
flow of execution of a transaction; while most of these changes can 
be solved by calling newly developed functions at certain time points, 
synchronous replica control is tightly coupled with the concurrency 
control.
Hence, especially in PostgreSQL 7.2 some parts of the concurrency control
(MVCC) might have to be adjusted. This can lead to a slightly more 
complicated maintenance than a system that does not change the backend.

4)Partial replication is not addressed. 


Any feedback on this post will be appreciated.

Thanks,

Darren 

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
    (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)

From pgsql-hackers-owner+M9923@postgresql.org Mon Jun 11 18:14:23 2001
Return-path: <pgsql-hackers-owner+M9923@postgresql.org>
Received: from postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.10.1/8.10.1) with ESMTP id f5BMENL18644
	for <pgman@candle.pha.pa.us>; Mon, 11 Jun 2001 18:14:23 -0400 (EDT)
Received: from postgresql.org.org (webmail.postgresql.org [216.126.85.28])
	by postgresql.org (8.11.3/8.11.1) with SMTP id f5BMEQE14877;
	Mon, 11 Jun 2001 18:14:26 -0400 (EDT)
	(envelope-from pgsql-hackers-owner+M9923@postgresql.org)
Received: from spoetnik.xs4all.nl (spoetnik.xs4all.nl [194.109.249.226])
	by postgresql.org (8.11.3/8.11.1) with ESMTP id f5BM6ME12270
	for <pgsql-hackers@postgresql.org>; Mon, 11 Jun 2001 18:06:23 -0400 (EDT)
	(envelope-from reinoud@xs4all.nl)
Received: from KAYAK (kayak [192.168.1.20])
	by spoetnik.xs4all.nl (Postfix) with SMTP id 865A33E1B
	for <pgsql-hackers@postgresql.org>; Tue, 12 Jun 2001 00:06:16 +0200 (CEST)
From: reinoud@xs4all.nl (Reinoud van Leeuwen)
To: pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Postgres Replication
Date: Mon, 11 Jun 2001 22:06:07 GMT
Organization: Not organized in any way
Reply-To: reinoud@xs4all.nl
Message-ID: <3b403d96.562404297@192.168.1.10>
References: <20010611.19464400@j2.us.greatbridge.com>
In-Reply-To: <20010611.19464400@j2.us.greatbridge.com>
X-Mailer: Forte Agent 1.5/32.451
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by postgresql.org id f5BM6PE12276
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR

On Mon, 11 Jun 2001 19:46:44 GMT, you wrote:

>We have been researching replication for several months now, and
>I have some opinions to share to the community for feedback,
>discussion, and/or participation. Our goal is to get a replication
>solution for PostgreSQL that will meet most needs of users
>and applications alike (mission impossible theme here :). 
>
>My research work along with others contributors has been collected
>and presented here http://www.greatbridge.org/genpage?replication_top
>If there is something missing, especially PostgreSQL related
>work, I would like to know about it, and my apologies to any
>one who got left off the list. This work is ongoing and doesn't
>draw a conclusion, which IMHO should be left up to the user,
>but I'm offering my opinions to spur discussion and/or feed back
>from this list, and try not to offend any one.
>
>Here's my opinion: of the approaches we've surveyed, the most
>promising one is the Postgres-R project from the Information and
>Communication Systems Group, ETH  in Zurich, Switzerland, originally 
>produced by Bettina Kemme, Gustavo Alonso, and others.  Although 
>Postgres-R is a synchronous approach, I believe it is the closest to 
>the goal mentioned above. Here is an abstract of the advantages.
>
>1) Postgres-R is built on the PostgreSQL-6.4.2 code base.  The 
>replication 
>functionality is an optional parameter, so there will be insignificant 
>overhead for non replication situations. The replication and 
>communication
>managers are the two new modules added to the PostgreSQL code base.
>
>2) The replication manager's main function is controlling the
>replication protocol via a message handling process. It receives
>messages from the local and remote backends and forwards write
>sets and decision messages via the communication manager to the
>other servers. The replication manager controls all the transactions 
>running on the local server by keeping track of the states, including 
>which protocol phase (read, send, lock, or write) the transaction is
>in. The replication manager maintains a two way channel
>implemented as buffered sockets to each backend.

what does "manager controls all the transactions" mean? I hope it does
*not* mean that a bug in the manager would cause transactions not to
commit...

>
>3) The main task of the communication manager is to provide simple
>socket based interface between the replication manager and the
>group communication system (currently Ensemble). The
>communication system is a cluster of servers connected via
>the communication manager.  The replication manager also maintains
>three one-way channels to the communication system: a broadcast
>channel to send messages, a total-order channel to receive
>totally orders write sets, and a no-order channel to listen for
>decision messages from the communication system. Decision
>messages can be received at any time where the reception of
>totally ordered write sets can be blocked in certain phases.
>
>4) Based on a two phase locking approach, all dead lock situations
>are local and detectable by Postgres-R code base, and aborted.

Does this imply locking over different servers? That would mean a
grinding halt when a network outage occurs...

>5) The write set messages used to send database changes to other
>servers, can use either the SQL statements or the actual tuples
>changed. This is a parameter based on number of tuples changed
>by a transaction. While sending the tuple changes reduces
>overhead in query parse, plan and execution, there is a negative
>effect in sending a large write set across the network.
>
>6) Postgres-R uses a synchronous approach that keeps the data on 
>all sites consistent and provides serializability. The user does not 
>have to bother with conflict resolution, and receives the same 
>correctness and consistency of a centralized system.
>
>7) Postgres-R could be part of a good fault-resilient and load 
>distribution 
>solution.  It is peer-to-peer based and incurs low overhead propagating 
>updates to the other cluster members.  All replicated databases locally 
>process queries.
>
>8) Compared to other synchronous replication strategies (e.g., standard 
>distributed 2-phase-locking + 2-phase-commit), Postgres-R has much 
>better performance using 2-phase-locking.

Coming from a Sybase background I have some experience with
replication. The way it works in Sybase Replication server is as
follows:
- for each replicated database, there is a "log reader" process that
reads the WAL and captures only *committed transactions* to the
replication server. (it does not make much sense to replicate other
things IMHO :-).
- the replication server stores incoming data in a que ("stable
device"), until it is sure it has reached its final destination

- a replication server can send data to another replication server in
a compact (read: WAN friendly) way. A chain of replication servers can
be made, depending on network architecture)

- the final replication server makes a almost standard client
connection to the target database and translates the compact
transactions back to SQL statements. By using masks, extra
functionality can be built in. 

This kind of architecture has several advantages:
- only committed transactions are replicated which saves overhead
- it does not have very much impact on performance of the source
server (apart from reading the WAL)
- since every replication server has a stable device, data is stored
when the network is down and nothing gets lost (nor stops performing)
- because only the log reader and the connection from the final
replication server are RDBMS specific, it is possible to replicate
from MS to Oracle using a Sybase replication server (or different
versions etc).

I do not know how much of this is patented or copyrighted, but the
architecture seems elegant and robust to me. I have done
implementations of bi-directional replication too. It *is* possible
but does require some funky setup and maintenance. (but it is better
that letting offices on different continents working on the same
database :-)

just my 2 EURO cts  :-)


-- 
__________________________________________________
"Nothing is as subjective as reality"
Reinoud van Leeuwen       reinoud@xs4all.nl
http://www.xs4all.nl/~reinoud
__________________________________________________

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org

From pgsql-hackers-owner+M9924@postgresql.org Mon Jun 11 18:41:51 2001
Return-path: <pgsql-hackers-owner+M9924@postgresql.org>
Received: from postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.10.1/8.10.1) with ESMTP id f5BMfpL28917
	for <pgman@candle.pha.pa.us>; Mon, 11 Jun 2001 18:41:51 -0400 (EDT)
Received: from postgresql.org.org (webmail.postgresql.org [216.126.85.28])
	by postgresql.org (8.11.3/8.11.1) with SMTP id f5BMfsE25092;
	Mon, 11 Jun 2001 18:41:54 -0400 (EDT)
	(envelope-from pgsql-hackers-owner+M9924@postgresql.org)
Received: from spider.pilosoft.com (p55-222.acedsl.com [160.79.55.222])
	by postgresql.org (8.11.3/8.11.1) with ESMTP id f5BMalE23024
	for <pgsql-hackers@postgresql.org>; Mon, 11 Jun 2001 18:36:47 -0400 (EDT)
	(envelope-from alex@pilosoft.com)
Received: from localhost (alexmail@localhost)
	by spider.pilosoft.com (8.9.3/8.9.3) with ESMTP id SAA06092;
	Mon, 11 Jun 2001 18:46:05 -0400 (EDT)
Date: Mon, 11 Jun 2001 18:46:05 -0400 (EDT)
From: Alex Pilosov <alex@pilosoft.com>
To: Reinoud van Leeuwen <reinoud@xs4all.nl>
cc: pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Postgres Replication
In-Reply-To: <3b403d96.562404297@192.168.1.10>
Message-ID: <Pine.BSO.4.10.10106111828450.9902-100000@spider.pilosoft.com>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR

On Mon, 11 Jun 2001, Reinoud van Leeuwen wrote:

> On Mon, 11 Jun 2001 19:46:44 GMT, you wrote:

> what does "manager controls all the transactions" mean? I hope it does
> *not* mean that a bug in the manager would cause transactions not to
> commit...
Well yeah it does. Bugs are a fact of life. :)

> >4) Based on a two phase locking approach, all dead lock situations
> >are local and detectable by Postgres-R code base, and aborted.
> 
> Does this imply locking over different servers? That would mean a
> grinding halt when a network outage occurs...
Don't know, but see below.

> Coming from a Sybase background I have some experience with
> replication. The way it works in Sybase Replication server is as
> follows:
> - for each replicated database, there is a "log reader" process that
> reads the WAL and captures only *committed transactions* to the
> replication server. (it does not make much sense to replicate other
> things IMHO :-).
> - the replication server stores incoming data in a que ("stable
> device"), until it is sure it has reached its final destination
> 
> - a replication server can send data to another replication server in
> a compact (read: WAN friendly) way. A chain of replication servers can
> be made, depending on network architecture)
> 
> - the final replication server makes a almost standard client
> connection to the target database and translates the compact
> transactions back to SQL statements. By using masks, extra
> functionality can be built in. 
> 
> This kind of architecture has several advantages:
> - only committed transactions are replicated which saves overhead
> - it does not have very much impact on performance of the source
> server (apart from reading the WAL)
> - since every replication server has a stable device, data is stored
> when the network is down and nothing gets lost (nor stops performing)
> - because only the log reader and the connection from the final
> replication server are RDBMS specific, it is possible to replicate
> from MS to Oracle using a Sybase replication server (or different
> versions etc).
> 
> I do not know how much of this is patented or copyrighted, but the
> architecture seems elegant and robust to me. I have done
> implementations of bi-directional replication too. It *is* possible
> but does require some funky setup and maintenance. (but it is better
> that letting offices on different continents working on the same
> database :-)
Yes, the above architecture is what almost every vendor of replication
software uses. And I'm sure if you worked much with Sybase, you hate the
garbage that their repserver is :). 

The architecture of postgres-r and repserver are fundamentally different
for a good reason: repserver only wants to replicate committed
transactions, while postgres-r is more of a 'clustering' solution (albeit
they don't say this word), and is capable to do much more than simple rep
server. 

I.E. you can safely put half of your clients to second server in a
replicated postgres-r cluster without being worried that a conflict (or a
wierd locking situation) may occur.

Try that with sybase, it is fundamentally designed for one-way
replication, and the fact that you can do one-way replication in both
directions doesn't mean its safe to do that!

I'm not sure how postgres-r handles network problems. To be useful, a good
replication solution must have an option of "no network->no updates" as
well as "no network->queue updates and send them later". However, it is
far easier to add queuing to a correct 'eager locking' database than it is
to add proper locking to a queue-based replicator.

-alex


---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

From pgsql-hackers-owner+M9932@postgresql.org Mon Jun 11 22:17:54 2001
Return-path: <pgsql-hackers-owner+M9932@postgresql.org>
Received: from postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.10.1/8.10.1) with ESMTP id f5C2HsL15803
	for <pgman@candle.pha.pa.us>; Mon, 11 Jun 2001 22:17:54 -0400 (EDT)
Received: from postgresql.org.org (webmail.postgresql.org [216.126.85.28])
	by postgresql.org (8.11.3/8.11.1) with SMTP id f5C2HtE86836;
	Mon, 11 Jun 2001 22:17:55 -0400 (EDT)
	(envelope-from pgsql-hackers-owner+M9932@postgresql.org)
Received: from femail15.sdc1.sfba.home.com (femail15.sdc1.sfba.home.com [24.0.95.142])
	by postgresql.org (8.11.3/8.11.1) with ESMTP id f5C2BXE85020
	for <pgsql-hackers@postgresql.org>; Mon, 11 Jun 2001 22:11:33 -0400 (EDT)
	(envelope-from djohnson@greatbridge.com)
Received: from greatbridge.com ([65.2.95.27])
          by femail15.sdc1.sfba.home.com
          (InterMail vM.4.01.03.20 201-229-121-120-20010223) with ESMTP
          id <20010612021124.OZRG17243.femail15.sdc1.sfba.home.com@greatbridge.com>;
          Mon, 11 Jun 2001 19:11:24 -0700
Message-ID: <3B257969.6050405@greatbridge.com>
Date: Mon, 11 Jun 2001 22:07:37 -0400
From: Darren Johnson <djohnson@greatbridge.com>
User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; m18) Gecko/20001108 Netscape6/6.0
X-Accept-Language: en
MIME-Version: 1.0
To: Alex Pilosov <alex@pilosoft.com>, Reinoud van Leeuwen <reinoud@xs4all.nl>
cc: pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Postgres Replication
References: <Pine.BSO.4.10.10106111828450.9902-100000@spider.pilosoft.com>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR


Thanks for the feedback.  I'll try to address both your issues here.

>> what does "manager controls all the transactions" mean? 
> 
The replication manager controls the transactions by serializing the 
write set messages. 
This ensures all transactions are committed in the same order on each 
server, so bugs
here are not allowed  ;-)

>> I hope it does
>> *not* mean that a bug in the manager would cause transactions not to
>> commit...
> 
> Well yeah it does. Bugs are a fact of life. :

> 
>>> 4) Based on a two phase locking approach, all dead lock situations
>>> are local and detectable by Postgres-R code base, and aborted.
>> 
>> Does this imply locking over different servers? That would mean a
>> grinding halt when a network outage occurs...
> 
> Don't know, but see below.

There is a branch of the Postgres-R code that has some failure detection 
implemented,
so we will have to merge this functionality with the version of 
Postgres-R we have, and
test this issue.  I'll let you the results.

>> 
>> - the replication server stores incoming data in a que ("stable
>> device"), until it is sure it has reached its final destination
> 
I like this idea for recovering servers that have been down a short 
period of time, using WAL
to recover transactions missed during the outage.

>> 
>> This kind of architecture has several advantages:
>> - only committed transactions are replicated which saves overhead
>> - it does not have very much impact on performance of the source
>> server (apart from reading the WAL)
>> - since every replication server has a stable device, data is stored
>> when the network is down and nothing gets lost (nor stops performing)
>> - because only the log reader and the connection from the final
>> replication server are RDBMS specific, it is possible to replicate
>> from MS to Oracle using a Sybase replication server (or different
>> versions etc).
> 
There are some issues with the "log reader" approach:
1) The databases are not synchronized until the log reader completes its 
processing.
2) I'm not sure about Sybase, but the log reader sends SQL statements to 
the other servers
which are then parsed, planned and executed.  This over head could be 
avoided if only
the tuple changes are replicated.
3) Works fine for read only situations, but peer-to-peer applications 
using this approach
must be designed with a conflict resolution scheme. 

Don't get me wrong, I believe we can learn from the replication 
techniques used by commercial
databases like Sybase, and try to implement the good ones into 
PostgreSQL.  Postgres-R is
a synchronous approach which out performs the traditional approaches to 
synchronous replication.
Being based on PostgreSQL-6.4.2, getting this approach in the 7.2 tree 
might be better than
reinventing the wheel.

Thanks again,

Darren


Thanks again,

Darren


---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://www.postgresql.org/search.mpl

From pgsql-hackers-owner+M9936@postgresql.org Tue Jun 12 03:22:51 2001
Return-path: <pgsql-hackers-owner+M9936@postgresql.org>
Received: from postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.10.1/8.10.1) with ESMTP id f5C7MoL11061
	for <pgman@candle.pha.pa.us>; Tue, 12 Jun 2001 03:22:50 -0400 (EDT)
Received: from postgresql.org.org (webmail.postgresql.org [216.126.85.28])
	by postgresql.org (8.11.3/8.11.1) with SMTP id f5C7MPE35441;
	Tue, 12 Jun 2001 03:22:25 -0400 (EDT)
	(envelope-from pgsql-hackers-owner+M9936@postgresql.org)
Received: from reorxrsm.server.lan.at (zep3.it-austria.net [213.150.1.73])
	by postgresql.org (8.11.3/8.11.1) with ESMTP id f5C72ZE25009
	for <pgsql-hackers@postgresql.org>; Tue, 12 Jun 2001 03:02:36 -0400 (EDT)
	(envelope-from ZeugswetterA@wien.spardat.at)
Received: from gz0153.gc.spardat.at (gz0153.gc.spardat.at [172.20.10.149])
	by reorxrsm.server.lan.at (8.11.2/8.11.2) with ESMTP id f5C72Qu27966
	for <pgsql-hackers@postgresql.org>; Tue, 12 Jun 2001 09:02:26 +0200
Received: by sdexcgtw01.f000.d0188.sd.spardat.at with Internet Mail Service (5.5.2650.21)
	id <M3L15341>; Tue, 12 Jun 2001 09:02:21 +0200
Message-ID: <11C1E6749A55D411A9670001FA68796336831B@sdexcsrv1.f000.d0188.sd.spardat.at>
From: Zeugswetter Andreas SB  <ZeugswetterA@wien.spardat.at>
To: "'Darren Johnson'" <djohnson@greatbridge.com>,
   pgsql-hackers@postgresql.org
Subject: AW: [HACKERS] Postgres Replication
Date: Tue, 12 Jun 2001 09:02:20 +0200
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2650.21)
Content-Type: text/plain;
	charset="iso-8859-1"
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR


> Although 
> Postgres-R is a synchronous approach, I believe it is the closest to 
> the goal mentioned above. Here is an abstract of the advantages.

If you only want synchronous replication, why not simply use triggers ?
All you would then need is remote query access and two phase commit,
and maybe a little script that helps create the appropriate triggers.

Doing a replicate all or nothing approach that only works synchronous
is imho not flexible enough.

Andreas

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://www.postgresql.org/search.mpl

From pgsql-hackers-owner+M9945@postgresql.org Tue Jun 12 10:18:29 2001
Return-path: <pgsql-hackers-owner+M9945@postgresql.org>
Received: from postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.10.1/8.10.1) with ESMTP id f5CEISL06372
	for <pgman@candle.pha.pa.us>; Tue, 12 Jun 2001 10:18:28 -0400 (EDT)
Received: from postgresql.org.org (webmail.postgresql.org [216.126.85.28])
	by postgresql.org (8.11.3/8.11.1) with SMTP id f5CEIQE77517;
	Tue, 12 Jun 2001 10:18:26 -0400 (EDT)
	(envelope-from pgsql-hackers-owner+M9945@postgresql.org)
Received: from krypton.netropolis.org ([208.222.215.99])
	by postgresql.org (8.11.3/8.11.1) with ESMTP id f5CEDuE75514
	for <pgsql-hackers@postgresql.org>; Tue, 12 Jun 2001 10:13:56 -0400 (EDT)
	(envelope-from root@generalogic.com)
Received: from [132.216.183.103] (helo=localhost)
	by krypton.netropolis.org with esmtp (Exim 3.12 #1 (Debian))
	id 159ouq-0003MU-00
	for <pgsql-hackers@postgresql.org>; Tue, 12 Jun 2001 10:13:08 -0400
To: pgsql-hackers@postgresql.org
Subject: Re: AW: [HACKERS] Postgres Replication
In-Reply-To: <20010612.13321600@j2.us.greatbridge.com>
References: <Pine.BSF.4.33.0106120605130.411-100000@mobile.hub.org>
	<20010612.13321600@j2.us.greatbridge.com>
X-Mailer: Mew version 1.94.2 on Emacs 20.7 / Mule 4.0 (HANANOEN)
MIME-Version: 1.0
Content-Type: Text/Plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <20010612123623O.root@generalogic.com>
Date: Tue, 12 Jun 2001 12:36:23 +0530
From: root <root@generalogic.com>
X-Dispatcher: imput version 20000414(IM141)
Lines: 47
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR


Hello

I have hacked up a replication layer for Perl code accessing a
database throught the DBI interface. It works pretty well with MySQL
(I can run pre-bender slashcode replicated, haven't tried the more
recent releases).

Potentially this hack should also work with Pg but I haven't tried
yet. If someone would like to test it out with a complex Pg app and
let me know how it went that would be cool.

The replication layer is based on Eric Newton's Recall replication
library (www.fault-tolerant.org/recall), and requires that all
database accesses be through the DBI interface.

The replicas are live, in that every operation affects all the
replicas in real time. Replica outages are invisible to the user, so
long as a majority of the replicas are functioning. Disconnected
replicas can be used for read-only access.

The only code modification that should be required to use the
replication layer is to change the DSN in connect():

  my $replicas = '192.168.1.1:7000,192.168.1.2:7000,192.168.1.3:7000';
  my $dbh = DBI->connect("DBI:Recall:database=$replicas");

You should be able to install the replication modules with:

perl -MCPAN -eshell
cpan> install Replication::Recall::DBServer

and then install DBD::Recall (which doesn't seem to be accessible from
the CPAN shell yet, for some reason), by:

wget http://www.cpan.org/authors/id/AGUL/DBD-Recall-1.10.tar.gz
tar xzvf DBD-Recall-1.10.tar.gz
cd DBD-Recall-1.10
perl Makefile.PL
make install

I would be very interested in hearing about your experiences with
this...

Thanks

#!

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

From pgsql-hackers-owner+M9938@postgresql.org Tue Jun 12 05:12:54 2001
Return-path: <pgsql-hackers-owner+M9938@postgresql.org>
Received: from postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.10.1/8.10.1) with ESMTP id f5C9CrL15228
	for <pgman@candle.pha.pa.us>; Tue, 12 Jun 2001 05:12:53 -0400 (EDT)
Received: from postgresql.org.org (webmail.postgresql.org [216.126.85.28])
	by postgresql.org (8.11.3/8.11.1) with SMTP id f5C9CnE91297;
	Tue, 12 Jun 2001 05:12:49 -0400 (EDT)
	(envelope-from pgsql-hackers-owner+M9938@postgresql.org)
Received: from mobile.hub.org (SHW39-29.accesscable.net [24.138.39.29])
	by postgresql.org (8.11.3/8.11.1) with ESMTP id f5C98DE89175
	for <pgsql-hackers@postgresql.org>; Tue, 12 Jun 2001 05:08:13 -0400 (EDT)
	(envelope-from scrappy@hub.org)
Received: from localhost (scrappy@localhost)
	by mobile.hub.org (8.11.3/8.11.1) with ESMTP id f5C97f361630;
	Tue, 12 Jun 2001 06:07:46 -0300 (ADT)
	(envelope-from scrappy@hub.org)
X-Authentication-Warning: mobile.hub.org: scrappy owned process doing -bs
Date: Tue, 12 Jun 2001 06:07:41 -0300 (ADT)
From: The Hermit Hacker <scrappy@hub.org>
To: Zeugswetter Andreas SB <ZeugswetterA@wien.spardat.at>
cc: "'Darren Johnson'" <djohnson@greatbridge.com>,
   <pgsql-hackers@postgresql.org>
Subject: Re: AW: [HACKERS] Postgres Replication
In-Reply-To: <11C1E6749A55D411A9670001FA68796336831B@sdexcsrv1.f000.d0188.sd.spardat.at>
Message-ID: <Pine.BSF.4.33.0106120605130.411-100000@mobile.hub.org>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR


which I believe is what the rserv implementation in contrib currently does
... no?

its funny ... what is in contrib right now was developed in a weekend by
Vadim, put in contrib, yet nobody has either used it *or* seen fit to
submit patches to improve it ... ?

On Tue, 12 Jun 2001, Zeugswetter Andreas SB wrote:

>
> > Although
> > Postgres-R is a synchronous approach, I believe it is the closest to
> > the goal mentioned above. Here is an abstract of the advantages.
>
> If you only want synchronous replication, why not simply use triggers ?
> All you would then need is remote query access and two phase commit,
> and maybe a little script that helps create the appropriate triggers.
>
> Doing a replicate all or nothing approach that only works synchronous
> is imho not flexible enough.
>
> Andreas
>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: Have you searched our list archives?
>
> http://www.postgresql.org/search.mpl
>

Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org


---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org

From pgsql-hackers-owner+M9940@postgresql.org Tue Jun 12 09:39:08 2001
Return-path: <pgsql-hackers-owner+M9940@postgresql.org>
Received: from postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.10.1/8.10.1) with ESMTP id f5CDd8L03200
	for <pgman@candle.pha.pa.us>; Tue, 12 Jun 2001 09:39:08 -0400 (EDT)
Received: from postgresql.org.org (webmail.postgresql.org [216.126.85.28])
	by postgresql.org (8.11.3/8.11.1) with SMTP id f5CDcmE58175;
	Tue, 12 Jun 2001 09:38:48 -0400 (EDT)
	(envelope-from pgsql-hackers-owner+M9940@postgresql.org)
Received: from mail.greatbridge.com (mail.greatbridge.com [65.196.68.36])
	by postgresql.org (8.11.3/8.11.1) with ESMTP id f5CDYAE56164
	for <pgsql-hackers@postgresql.org>; Tue, 12 Jun 2001 09:34:10 -0400 (EDT)
	(envelope-from djohnson@greatbridge.com)
Received: from j2.us.greatbridge.com (djohnsonpc.us.greatbridge.com [65.196.69.70])
	by mail.greatbridge.com (8.11.2/8.11.2) with SMTP id f5CDXeQ03585;
	Tue, 12 Jun 2001 09:33:40 -0400
From: Darren Johnson <djohnson@greatbridge.com>
Date: Tue, 12 Jun 2001 13:32:16 GMT
Message-ID: <20010612.13321600@j2.us.greatbridge.com>
Subject: Re: AW: [HACKERS] Postgres Replication
To: The Hermit Hacker <scrappy@hub.org>
cc: Zeugswetter Andreas SB <ZeugswetterA@wien.spardat.at>,
   <pgsql-hackers@postgresql.org>
Reply-To: Darren Johnson <djohnson@greatbridge.com>
In-Reply-To: <Pine.BSF.4.33.0106120605130.411-100000@mobile.hub.org>
References: <Pine.BSF.4.33.0106120605130.411-100000@mobile.hub.org>
X-Mailer: Mozilla/3.0 (compatible; StarOffice/5.2;Linux)
X-Priority: 3 (Normal)
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by postgresql.org id f5CDYAE56166
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR


> which I believe is what the rserv implementation in contrib currently 
does
> ... no?

We tried rserv, PG Link (Joseph Conway), and PosrgreSQL Replicator.  All
these projects are trigger based asynchronous replication.  They all have
some advantages over the current functionality of Postgres-R some of 
which I believe can be addressed:

1) Partial replication - being able to replicate just one or part of a
table(s)
2) They make no changes to the PostgreSQL code base. (Postgres-R can't 
address this one ;)
3) PostgreSQL Replicator has some very nice conflict resolution schemes.


Here are some disadvantages to using a "trigger based" approach:

1) Triggers simply transfer individual data items when they are modified,
they do not keep track of transactions.
2) The execution of triggers within a database imposes a performance 
overhead to that database.
3) Triggers require careful management by database administrators.  
Someone needs to keep track of all the "alarms" going off.
4) The activation of triggers in a database cannot be easily 
rolled back or undone.



> On Tue, 12 Jun 2001, Zeugswetter Andreas SB wrote:

> > Doing a replicate all or nothing approach that only works synchronous
> > is imho not flexible enough.
> >


I agree.  Partial and asynchronous replication need to be addressed, 
and some of the common functionality of Postgres-R could possibly 
be used to meet those needs. 
 

Thanks for your feedback,

Darren

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/users-lounge/docs/faq.html

From pgsql-hackers-owner+M9969@postgresql.org Tue Jun 12 16:53:45 2001
Return-path: <pgsql-hackers-owner+M9969@postgresql.org>
Received: from postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.10.1/8.10.1) with ESMTP id f5CKriL23104
	for <pgman@candle.pha.pa.us>; Tue, 12 Jun 2001 16:53:44 -0400 (EDT)
Received: from postgresql.org.org (webmail.postgresql.org [216.126.85.28])
	by postgresql.org (8.11.3/8.11.1) with SMTP id f5CKrlE87423;
	Tue, 12 Jun 2001 16:53:47 -0400 (EDT)
	(envelope-from pgsql-hackers-owner+M9969@postgresql.org)
Received: from sectorbase2.sectorbase.com (sectorbase2.sectorbase.com [63.88.121.62] (may be forged))
	by postgresql.org (8.11.3/8.11.1) with SMTP id f5CHWkE69562
	for <pgsql-hackers@postgresql.org>; Tue, 12 Jun 2001 13:32:46 -0400 (EDT)
	(envelope-from vmikheev@SECTORBASE.COM)
Received: by sectorbase2.sectorbase.com with Internet Mail Service (5.5.2653.19)
	id <MX6MWMV8>; Tue, 12 Jun 2001 10:30:29 -0700
Message-ID: <3705826352029646A3E91C53F7189E32016670@sectorbase2.sectorbase.com>
From: "Mikheev, Vadim" <vmikheev@SECTORBASE.COM>
To: "'Darren Johnson'" <djohnson@greatbridge.com>,
   The Hermit Hacker
  <scrappy@hub.org>
cc: Zeugswetter Andreas SB <ZeugswetterA@wien.spardat.at>,
   pgsql-hackers@postgresql.org
Subject: RE: AW: [HACKERS] Postgres Replication
Date: Tue, 12 Jun 2001 10:30:27 -0700
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR

> Here are some disadvantages to using a "trigger based" approach:
> 
> 1) Triggers simply transfer individual data items when they 
> are modified, they do not keep track of transactions.

I don't know about other *async* replication engines but Rserv
keeps track of transactions (if I understood you corectly).
Rserv transfers not individual modified data items but
*consistent* snapshot of changes to move slave database from
one *consistent* state (when all RI constraints satisfied)
to another *consistent* state.

> 4) The activation of triggers in a database cannot be easily
> rolled back or undone.

What do you mean?

Vadim

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
    (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)

From pgsql-hackers-owner+M9967@postgresql.org Tue Jun 12 16:42:11 2001
Return-path: <pgsql-hackers-owner+M9967@postgresql.org>
Received: from postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.10.1/8.10.1) with ESMTP id f5CKgBL17982
	for <pgman@candle.pha.pa.us>; Tue, 12 Jun 2001 16:42:11 -0400 (EDT)
Received: from postgresql.org.org (webmail.postgresql.org [216.126.85.28])
	by postgresql.org (8.11.3/8.11.1) with SMTP id f5CKgDE80566;
	Tue, 12 Jun 2001 16:42:13 -0400 (EDT)
	(envelope-from pgsql-hackers-owner+M9967@postgresql.org)
Received: from mail.greatbridge.com (mail.greatbridge.com [65.196.68.36])
	by postgresql.org (8.11.3/8.11.1) with ESMTP id f5CIVdE07561
	for <pgsql-hackers@postgresql.org>; Tue, 12 Jun 2001 14:31:39 -0400 (EDT)
	(envelope-from djohnson@greatbridge.com)
Received: from j2.us.greatbridge.com (djohnsonpc.us.greatbridge.com [65.196.69.70])
	by mail.greatbridge.com (8.11.2/8.11.2) with SMTP id f5CIUfQ10080;
	Tue, 12 Jun 2001 14:30:41 -0400
From: Darren Johnson <djohnson@greatbridge.com>
Date: Tue, 12 Jun 2001 18:29:20 GMT
Message-ID: <20010612.18292000@j2.us.greatbridge.com>
Subject: RE: AW: [HACKERS] Postgres Replication
To: "Mikheev, Vadim" <vmikheev@SECTORBASE.COM>
cc: The Hermit Hacker <scrappy@hub.org>,
   Zeugswetter Andreas SB
	<ZeugswetterA@wien.spardat.at>,
   pgsql-hackers@postgresql.org
Reply-To: Darren Johnson <djohnson@greatbridge.com>
	<3705826352029646A3E91C53F7189E32016670@sectorbase2.sectorbase.com>
References: <3705826352029646A3E91C53F7189E32016670@sectorbase2.sectorbase.com>
X-Mailer: Mozilla/3.0 (compatible; StarOffice/5.2;Linux)
X-Priority: 3 (Normal)
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by postgresql.org id f5CIVdE07562
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR



> > Here are some disadvantages to using a "trigger based" approach:
> >
> > 1) Triggers simply transfer individual data items when they
> > are modified, they do not keep track of transactions.

> I don't know about other *async* replication engines but Rserv
> keeps track of transactions (if I understood you corectly).
> Rserv transfers not individual modified data items but
> *consistent* snapshot of changes to move slave database from
> one *consistent* state (when all RI constraints satisfied)
> to another *consistent* state.

I thought Andreas did a good job of correcting me here. Transaction-
based replication with triggers do not apply to points 1 and 4.  I
should have made a distinction between non-transaction and 
transaction based replication with triggers.  I was not trying to
single out rserv or any other project, and I can see how my wording 
implies this misinterpretation (my apologies).
 

> > 4) The activation of triggers in a database cannot be easily
> > rolled back or undone.

> What do you mean?

Once the trigger fires, it is not an easy task  to abort that 
execution via rollback or undo.  Again this is not an issue 
with a transaction-based trigger approach.


Sincerely,

Darren

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
    (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)

From pgsql-hackers-owner+M9943@postgresql.org Tue Jun 12 10:03:02 2001
Return-path: <pgsql-hackers-owner+M9943@postgresql.org>
Received: from postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.10.1/8.10.1) with ESMTP id f5CE32L04619
	for <pgman@candle.pha.pa.us>; Tue, 12 Jun 2001 10:03:02 -0400 (EDT)
Received: from postgresql.org.org (webmail.postgresql.org [216.126.85.28])
	by postgresql.org (8.11.3/8.11.1) with SMTP id f5CE31E70430;
	Tue, 12 Jun 2001 10:03:01 -0400 (EDT)
	(envelope-from pgsql-hackers-owner+M9943@postgresql.org)
Received: from fizbanrsm.server.lan.at (zep4.it-austria.net [213.150.1.74])
	by postgresql.org (8.11.3/8.11.1) with ESMTP id f5CDoQE64062
	for <pgsql-hackers@postgresql.org>; Tue, 12 Jun 2001 09:50:26 -0400 (EDT)
	(envelope-from ZeugswetterA@wien.spardat.at)
Received: from gz0153.gc.spardat.at (gz0153.gc.spardat.at [172.20.10.149])
	by fizbanrsm.server.lan.at (8.11.2/8.11.2) with ESMTP id f5CDoJe11224
	for <pgsql-hackers@postgresql.org>; Tue, 12 Jun 2001 15:50:19 +0200
Received: by sdexcgtw01.f000.d0188.sd.spardat.at with Internet Mail Service (5.5.2650.21)
	id <M3L15S4T>; Tue, 12 Jun 2001 15:50:15 +0200
Message-ID: <11C1E6749A55D411A9670001FA68796336831F@sdexcsrv1.f000.d0188.sd.spardat.at>
From: Zeugswetter Andreas SB  <ZeugswetterA@wien.spardat.at>
To: "'Darren Johnson'" <djohnson@greatbridge.com>,
   The Hermit Hacker
  <scrappy@hub.org>
cc: pgsql-hackers@postgresql.org
Subject: AW: AW: [HACKERS] Postgres Replication
Date: Tue, 12 Jun 2001 15:50:09 +0200
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2650.21)
Content-Type: text/plain;
	charset="iso-8859-1"
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR


> Here are some disadvantages to using a "trigger based" approach:
> 
> 1) Triggers simply transfer individual data items when they 
> are modified, they do not keep track of transactions.
> 2) The execution of triggers within a database imposes a performance 
> overhead to that database.
> 3) Triggers require careful management by database administrators.  
> Someone needs to keep track of all the "alarms" going off.
> 4) The activation of triggers in a database cannot be easily 
> rolled back or undone.

Yes, points 2 and 3 are a given, although point 2 buys you the functionality
of transparent locking across all involved db servers.
Points 1 and 4 are only the case for a trigger mechanism that does 
not use remote connection and 2-phase commit. 

Imho an implementation that opens a separate client connection to the 
replication target is only suited for async replication, and for that a WAL 
based solution would probably impose less overhead.

Andreas

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
    (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)

From pgsql-hackers-owner+M9946@postgresql.org Tue Jun 12 10:47:09 2001
Return-path: <pgsql-hackers-owner+M9946@postgresql.org>
Received: from postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.10.1/8.10.1) with ESMTP id f5CEl9L08144
	for <pgman@candle.pha.pa.us>; Tue, 12 Jun 2001 10:47:09 -0400 (EDT)
Received: from postgresql.org.org (webmail.postgresql.org [216.126.85.28])
	by postgresql.org (8.11.3/8.11.1) with SMTP id f5CEihE88714;
	Tue, 12 Jun 2001 10:44:43 -0400 (EDT)
	(envelope-from pgsql-hackers-owner+M9946@postgresql.org)
Received: from mail.greatbridge.com (mail.greatbridge.com [65.196.68.36])
	by postgresql.org (8.11.3/8.11.1) with ESMTP id f5CEd6E85859
	for <pgsql-hackers@postgresql.org>; Tue, 12 Jun 2001 10:39:06 -0400 (EDT)
	(envelope-from djohnson@greatbridge.com)
Received: from j2.us.greatbridge.com (djohnsonpc.us.greatbridge.com [65.196.69.70])
	by mail.greatbridge.com (8.11.2/8.11.2) with SMTP id f5CEcgQ04905;
	Tue, 12 Jun 2001 10:38:42 -0400
From: Darren Johnson <djohnson@greatbridge.com>
Date: Tue, 12 Jun 2001 14:37:18 GMT
Message-ID: <20010612.14371800@j2.us.greatbridge.com>
Subject: Re: AW: AW: [HACKERS] Postgres Replication
To: Zeugswetter Andreas SB <ZeugswetterA@wien.spardat.at>
cc: pgsql-hackers@postgresql.org
Reply-To: Darren Johnson <djohnson@greatbridge.com>
	<11C1E6749A55D411A9670001FA68796336831F@sdexcsrv1.f000.d0188.sd.spardat.at>
References: <11C1E6749A55D411A9670001FA68796336831F@sdexcsrv1.f000.d0188.sd.spardat.at>
X-Mailer: Mozilla/3.0 (compatible; StarOffice/5.2;Linux)
X-Priority: 3 (Normal)
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by postgresql.org id f5CEd6E85860
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR



> Imho an implementation that opens a separate client connection to the
> replication target is only suited for async replication, and for that a 
WAL
> based solution would probably impose less overhead.


Yes there is significant overhead with opening a connection to a 
client, so Postgres-R creates a pool of backends at start up, 
coupled with the group communication system (Ensemble) that
significantly reduces this issue.


Very good points,

Darren



---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://www.postgresql.org/search.mpl

From pgsql-hackers-owner+M9982@postgresql.org Tue Jun 12 19:04:06 2001
Return-path: <pgsql-hackers-owner+M9982@postgresql.org>
Received: from postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.10.1/8.10.1) with ESMTP id f5CN46E10043
	for <pgman@candle.pha.pa.us>; Tue, 12 Jun 2001 19:04:06 -0400 (EDT)
Received: from postgresql.org.org (webmail.postgresql.org [216.126.85.28])
	by postgresql.org (8.11.3/8.11.1) with SMTP id f5CN4AE62160;
	Tue, 12 Jun 2001 19:04:10 -0400 (EDT)
	(envelope-from pgsql-hackers-owner+M9982@postgresql.org)
Received: from spoetnik.xs4all.nl (spoetnik.xs4all.nl [194.109.249.226])
	by postgresql.org (8.11.3/8.11.1) with ESMTP id f5CMxaE60194
	for <pgsql-hackers@postgresql.org>; Tue, 12 Jun 2001 18:59:36 -0400 (EDT)
	(envelope-from reinoud@xs4all.nl)
Received: from KAYAK (kayak [192.168.1.20])
	by spoetnik.xs4all.nl (Postfix) with SMTP id 435353E1B
	for <pgsql-hackers@postgresql.org>; Wed, 13 Jun 2001 00:59:28 +0200 (CEST)
From: reinoud@xs4all.nl (Reinoud van Leeuwen)
To: pgsql-hackers@postgresql.org
Subject: Re: AW: AW: [HACKERS] Postgres Replication
Date: Tue, 12 Jun 2001 22:59:23 GMT
Organization: Not organized in any way
Reply-To: reinoud@xs4all.nl
Message-ID: <3b499c5b.652202125@192.168.1.10>
References: <11C1E6749A55D411A9670001FA68796336831F@sdexcsrv1.f000.d0188.sd.spardat.at>
In-Reply-To: <11C1E6749A55D411A9670001FA68796336831F@sdexcsrv1.f000.d0188.sd.spardat.at>
X-Mailer: Forte Agent 1.5/32.451
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by postgresql.org id f5CMxcE60196
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR

On Tue, 12 Jun 2001 15:50:09 +0200, you wrote:

>
>> Here are some disadvantages to using a "trigger based" approach:
>> 
>> 1) Triggers simply transfer individual data items when they 
>> are modified, they do not keep track of transactions.
>> 2) The execution of triggers within a database imposes a performance 
>> overhead to that database.
>> 3) Triggers require careful management by database administrators.  
>> Someone needs to keep track of all the "alarms" going off.
>> 4) The activation of triggers in a database cannot be easily 
>> rolled back or undone.
>
>Yes, points 2 and 3 are a given, although point 2 buys you the functionality
>of transparent locking across all involved db servers.
>Points 1 and 4 are only the case for a trigger mechanism that does 
>not use remote connection and 2-phase commit. 
>
>Imho an implementation that opens a separate client connection to the 
>replication target is only suited for async replication, and for that a WAL 
>based solution would probably impose less overhead.

Well as I read back the thread I see 2 different approaches to
replication:

1: tight integrated replication. 
pro:
- bi-directional (or multidirectional): updates are possible
everywhere
- A cluster of servers allways has the same state. 
- it does not matter to which server you connect
con:
- network between servers will be a bottleneck, especially if it is a
WAN connection
- only full replication possible
- what happens if one server is down? (or the network between) are
commits still possible

2: async replication
pro:
- long distance possible
- no problems with network outages
- only changes are replicated, selects do not have impact 
- no locking issues accross servers
- partial replication possible (many->one (datawarehouse), or one-many
(queries possible everywhere, updates only central) 
- goof for failover situations (backup server is standing by)
con:
- bidirectional replication hard to set up (you'll have to implement
conflict resolution according to your business rules)
- different servers are not guaranteed to be in the same state.

I can think of some scenarios where I would definitely want to
*choose* one of the options. A load-balanced web environment would
likely want the first option, but synchronizing offices in different
continents might not work with 2-phase commit over the network....

And we have not even started talking about *managing* replicated
environments. A lot of fail-over scenarios stop planning after the
backup host has take control. But how to get back? 
-- 
__________________________________________________
"Nothing is as subjective as reality"
Reinoud van Leeuwen       reinoud@xs4all.nl
http://www.xs4all.nl/~reinoud
__________________________________________________

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org

From pgsql-hackers-owner+M9986@postgresql.org Tue Jun 12 19:48:48 2001
Return-path: <pgsql-hackers-owner+M9986@postgresql.org>
Received: from postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.10.1/8.10.1) with ESMTP id f5CNmmE13125
	for <pgman@candle.pha.pa.us>; Tue, 12 Jun 2001 19:48:48 -0400 (EDT)
Received: from postgresql.org.org (webmail.postgresql.org [216.126.85.28])
	by postgresql.org (8.11.3/8.11.1) with SMTP id f5CNmqE76673;
	Tue, 12 Jun 2001 19:48:52 -0400 (EDT)
	(envelope-from pgsql-hackers-owner+M9986@postgresql.org)
Received: from sss.pgh.pa.us ([192.204.191.242])
	by postgresql.org (8.11.3/8.11.1) with ESMTP id f5CNdQE73923
	for <pgsql-hackers@postgresql.org>; Tue, 12 Jun 2001 19:39:26 -0400 (EDT)
	(envelope-from tgl@sss.pgh.pa.us)
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
	by sss.pgh.pa.us (8.11.3/8.11.3) with ESMTP id f5CNdI016442;
	Tue, 12 Jun 2001 19:39:18 -0400 (EDT)
To: reinoud@xs4all.nl
cc: pgsql-hackers@postgresql.org
Subject: Re: AW: AW: [HACKERS] Postgres Replication 
In-Reply-To: <3b499c5b.652202125@192.168.1.10> 
References: <11C1E6749A55D411A9670001FA68796336831F@sdexcsrv1.f000.d0188.sd.spardat.at> <3b499c5b.652202125@192.168.1.10>
Comments: In-reply-to reinoud@xs4all.nl (Reinoud van Leeuwen)
	message dated "Tue, 12 Jun 2001 22:59:23 +0000"
Date: Tue, 12 Jun 2001 19:39:18 -0400
Message-ID: <16439.992389158@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR

reinoud@xs4all.nl (Reinoud van Leeuwen) writes:
> Well as I read back the thread I see 2 different approaches to
> replication:
> ...
> I can think of some scenarios where I would definitely want to
> *choose* one of the options.

Yes.  IIRC, it looks to be possible to support a form of async
replication using the Postgres-R approach: you allow the cluster
to break apart when communications fail, and then rejoin when
your link comes back to life.  (This can work in principle, how
close it is to reality is another question; but the rejoin operation
is the same as crash recovery, so you have to have it anyway.)

So this seems to me to allow getting most of the benefits of the async
approach.  OTOH it is difficult to see how to go the other way: getting
the benefits of a synchronous solution atop a basically-async
implementation doesn't seem like it can work.

			regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://www.postgresql.org/search.mpl

From pgsql-hackers-owner+M9997@postgresql.org Wed Jun 13 09:05:56 2001
Return-path: <pgsql-hackers-owner+M9997@postgresql.org>
Received: from postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.10.1/8.10.1) with ESMTP id f5DD5tE28260
	for <pgman@candle.pha.pa.us>; Wed, 13 Jun 2001 09:05:55 -0400 (EDT)
Received: from postgresql.org.org (webmail.postgresql.org [216.126.85.28])
	by postgresql.org (8.11.3/8.11.1) with SMTP id f5DD5xE12437;
	Wed, 13 Jun 2001 09:05:59 -0400 (EDT)
	(envelope-from pgsql-hackers-owner+M9997@postgresql.org)
Received: from fizbanrsm.server.lan.at (zep4.it-austria.net [213.150.1.74])
	by postgresql.org (8.11.3/8.11.1) with ESMTP id f5DD19E00635
	for <pgsql-hackers@postgresql.org>; Wed, 13 Jun 2001 09:01:10 -0400 (EDT)
	(envelope-from ZeugswetterA@wien.spardat.at)
Received: from gz0153.gc.spardat.at (gz0153.gc.spardat.at [172.20.10.149])
	by fizbanrsm.server.lan.at (8.11.2/8.11.2) with ESMTP id f5DD13m08153
	for <pgsql-hackers@postgresql.org>; Wed, 13 Jun 2001 15:01:03 +0200
Received: by sdexcgtw01.f000.d0188.sd.spardat.at with Internet Mail Service (5.5.2650.21)
	id <M6AB97MY>; Wed, 13 Jun 2001 15:00:02 +0200
Message-ID: <11C1E6749A55D411A9670001FA687963368322@sdexcsrv1.f000.d0188.sd.spardat.at>
From: Zeugswetter Andreas SB  <ZeugswetterA@wien.spardat.at>
To: "'reinoud@xs4all.nl'" <reinoud@xs4all.nl>, pgsql-hackers@postgresql.org
Subject: AW: AW: AW: [HACKERS] Postgres Replication
Date: Wed, 13 Jun 2001 11:55:48 +0200
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2650.21)
Content-Type: text/plain;
	charset="iso-8859-1"
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR


> Well as I read back the thread I see 2 different approaches to
> replication:
> 
> 1: tight integrated replication. 
> pro:
> - bi-directional (or multidirectional): updates are possible everywhere
> - A cluster of servers allways has the same state. 
> - it does not matter to which server you connect
> con:
> - network between servers will be a bottleneck, especially if it is a
> WAN connection
> - only full replication possible

I do not understand that point, if it is trigger based, you 
have all the flexibility you need. (only some tables, only some rows,
different rows to different targets ....), 
(or do you mean not all targets, that could also be achieved with triggers)

> - what happens if one server is down? (or the network between) are
> commits still possible

No, updates are not possible if one target is not reachable, 
that would not be synchronous and would again need business rules
to resolve conflicts.

Allowing updates when a target is not reachable would require admin 
intervention.

Andreas

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

From pgsql-hackers-owner+M10005@postgresql.org Wed Jun 13 11:15:48 2001
Return-path: <pgsql-hackers-owner+M10005@postgresql.org>
Received: from postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.10.1/8.10.1) with ESMTP id f5DFFmE08382
	for <pgman@candle.pha.pa.us>; Wed, 13 Jun 2001 11:15:48 -0400 (EDT)
Received: from postgresql.org.org (webmail.postgresql.org [216.126.85.28])
	by postgresql.org (8.11.3/8.11.1) with SMTP id f5DFFoE53621;
	Wed, 13 Jun 2001 11:15:50 -0400 (EDT)
	(envelope-from pgsql-hackers-owner+M10005@postgresql.org)
Received: from mail.greatbridge.com (mail.greatbridge.com [65.196.68.36])
	by postgresql.org (8.11.3/8.11.1) with ESMTP id f5DEk7E38930
	for <pgsql-hackers@postgresql.org>; Wed, 13 Jun 2001 10:46:07 -0400 (EDT)
	(envelope-from djohnson@greatbridge.com)
Received: from j2.us.greatbridge.com (djohnsonpc.us.greatbridge.com [65.196.69.70])
	by mail.greatbridge.com (8.11.2/8.11.2) with SMTP id f5DEhfQ22566;
	Wed, 13 Jun 2001 10:43:41 -0400
From: Darren Johnson <djohnson@greatbridge.com>
Date: Wed, 13 Jun 2001 14:44:11 GMT
Message-ID: <20010613.14441100@j2.us.greatbridge.com>
Subject: Re: AW: AW: AW: [HACKERS] Postgres Replication
To: Zeugswetter Andreas SB <ZeugswetterA@wien.spardat.at>
cc: "'reinoud@xs4all.nl'" <reinoud@xs4all.nl>, pgsql-hackers@postgresql.org
Reply-To: Darren Johnson <djohnson@greatbridge.com>
	<11C1E6749A55D411A9670001FA687963368322@sdexcsrv1.f000.d0188.sd.spardat.at>
References: <11C1E6749A55D411A9670001FA687963368322@sdexcsrv1.f000.d0188.sd.spardat.at>
X-Mailer: Mozilla/3.0 (compatible; StarOffice/5.2;Linux)
X-Priority: 3 (Normal)
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by postgresql.org id f5DEk8E38931
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR


> > - only full replication possible

> I do not understand that point, if it is trigger based, you
> have all the flexibility you need. (only some tables, only some rows,
> different rows to different targets ....),
> (or do you mean not all targets, that could also be achieved with 
triggers)

Currently with Postgres-R, it is one database replicating all tables to 
all servers in the group communication system.  There are some ways 
around
this by invoking the -r option when a SQL statement should be replicated, 
and leaving the -r option off for non-replicated scenarios.  IMHO this is
not a good solution.  

A better solution will need to be implemented, which involves a 
subscription table(s) with relation/server information.  There are two
ideas for subscribing and receiving replicated data.

1) Receiver driven propagation - A simple solution where all 
transactions are propagated and the receiving servers will reference
the subscription information before applying updates.

2) Sender driven propagation - A more optimal and complex solution 
where servers do not receive any messages regarding data items for 
which they have not subscribed


> > - what happens if one server is down? (or the network between) are
> > commits still possible

> No, updates are not possible if one target is not reachable,

AFAIK, Postgres-R can still replicate if one target is not reachable,
but only to the remaining servers ;).  

There is a scenario that could arise if a server issues a lock 
request then fails or goes off line.  There is code that checks 
for this condition, which needs to be merged with the branch we have.

> that would not be synchronous and would again need business rules
> to resolve conflicts.

Yes the failed server would not be synchronized, and getting this
failed server back in sync needs to be addressed.

> Allowing updates when a target is not reachable would require admin
> intervention.

In its current state yes, but our goal would be to eliminate this
requirement as well.



Darren

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

From pgsql-hackers-owner+M18443=candle.pha.pa.us=pgman@postgresql.org Mon Feb  4 19:16:17 2002
Return-path: <pgsql-hackers-owner+M18443=candle.pha.pa.us=pgman@postgresql.org>
Received: from server1.pgsql.org (www.postgresql.org [64.49.215.9])
	by candle.pha.pa.us (8.11.6/8.10.1) with SMTP id g150GGP03822
	for <pgman@candle.pha.pa.us>; Mon, 4 Feb 2002 19:16:16 -0500 (EST)
Received: (qmail 77444 invoked by alias); 5 Feb 2002 00:16:11 -0000
Received: from unknown (HELO postgresql.org) (64.49.215.8)
  by www.postgresql.org with SMTP; 5 Feb 2002 00:16:11 -0000
Received: from snoopy.mohawksoft.com (h0050bf7a618d.ne.mediaone.net [24.147.138.78])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id g150Esl77040
	for <pgsql-hackers@postgresql.org>; Mon, 4 Feb 2002 19:14:54 -0500 (EST)
	(envelope-from markw@mohawksoft.com)
Received: from mohawksoft.com (localhost [127.0.0.1])
	by snoopy.mohawksoft.com (8.11.6/8.11.6) with ESMTP id g150AWh08676
	for <pgsql-hackers@postgresql.org>; Mon, 4 Feb 2002 19:10:33 -0500
Message-ID: <3C5F22F8.C9B958F0@mohawksoft.com>
Date: Mon, 04 Feb 2002 19:10:32 -0500
From: mlw <markw@mohawksoft.com>
X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.17 i686)
X-Accept-Language: en
MIME-Version: 1.0
To: PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: [HACKERS] Replication
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR

I re-wrote RServ.pm to C, and wrote a replication daemon. It works, but it
works like the whole rserv project. I don't like it.

OK, what the hell do we need to do to get PostgreSQL replicating?

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

From pgsql-hackers-owner+M18445=candle.pha.pa.us=pgman@postgresql.org Mon Feb  4 19:57:01 2002
Return-path: <pgsql-hackers-owner+M18445=candle.pha.pa.us=pgman@postgresql.org>
Received: from server1.pgsql.org (www.postgresql.org [64.49.215.9])
	by candle.pha.pa.us (8.11.6/8.10.1) with SMTP id g150v0P06518
	for <pgman@candle.pha.pa.us>; Mon, 4 Feb 2002 19:57:00 -0500 (EST)
Received: (qmail 90440 invoked by alias); 5 Feb 2002 00:56:59 -0000
Received: from unknown (HELO postgresql.org) (64.49.215.8)
  by www.postgresql.org with SMTP; 5 Feb 2002 00:56:59 -0000
Received: from www1.navtechinc.com ([192.234.226.140])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id g150rMl89885
	for <pgsql-hackers@postgresql.org>; Mon, 4 Feb 2002 19:53:22 -0500 (EST)
	(envelope-from ssinger@navtechinc.com)
Received: from pcNavYkfAdm1.ykf.navtechinc.com (wall [192.234.226.190])
	by www1.navtechinc.com (8.9.3/8.9.3) with ESMTP id AAA06047;
	Tue, 5 Feb 2002 00:53:22 GMT
Received: from localhost (ssinger@localhost)
	by pcNavYkfAdm1.ykf.navtechinc.com (8.9.3/8.9.3) with ESMTP id AAA10675;
	Tue, 5 Feb 2002 00:52:43 GMT
Date: Tue, 5 Feb 2002 00:52:43 +0000 (GMT)
From: Steven <ssinger@navtechinc.com>
X-X-Sender: <ssinger@pcNavYkfAdm1.ykf.navtechinc.com>
To: mlw <markw@mohawksoft.com>
cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Replication
In-Reply-To: <3C5F22F8.C9B958F0@mohawksoft.com>
Message-ID: <Pine.LNX.4.33.0202050040190.24027-100000@pcNavYkfAdm1.ykf.navtechinc.com>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR

On Mon, 4 Feb 2002, mlw wrote:

I've developed a replacement for Rserv and we are planning on releasing 
it as open source(ie as a contrib module).  

Like Rserv its trigger based but its much more flexible.
The key adventages it has over Rserv is that it has
-Support for multiple slaves
-It Perserves transactions while doing the mirroring. Ie  If rows A,B are 
originally added in the same transaction they will be mirrored in the same 
transaction.

We have plans on adding filtering based on data/selective mirroring as 
well. (Ie only rows with COUNTRY='Canada' go to 
slave A, and  rows with COUNTRY='China' go to slave B).
But I'm not sure when I'll get to that.

Support for conflict resolution(If allow edits to be made on the slaves) 
would be nice.

I hope to be able to send a tarball with the source to the pgpatches list 
within the next few days.

We've been using the system operationally for a number of months and have
been happy with it.

> I re-wrote RServ.pm to C, and wrote a replication daemon. It works, but it
> works like the whole rserv project. I don't like it. 
> OK, what the hell do we need to do to get PostgreSQL replicating?
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster
> 

-- 
Steven Singer                                       ssinger@navtechinc.com
Aircraft Performance Systems                Phone:  519-747-1170 ext 282
Navtech Systems Support Inc.                AFTN:   CYYZXNSX SITA: YYZNSCR
Waterloo, Ontario                           ARINC:  YKFNSCR


---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
    (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)

From pgsql-hackers-owner+M18447=candle.pha.pa.us=pgman@postgresql.org Mon Feb  4 20:06:57 2002
Return-path: <pgsql-hackers-owner+M18447=candle.pha.pa.us=pgman@postgresql.org>
Received: from server1.pgsql.org (www.postgresql.org [64.49.215.9])
	by candle.pha.pa.us (8.11.6/8.10.1) with SMTP id g1516vP07508
	for <pgman@candle.pha.pa.us>; Mon, 4 Feb 2002 20:06:57 -0500 (EST)
Received: (qmail 92753 invoked by alias); 5 Feb 2002 01:06:55 -0000
Received: from unknown (HELO postgresql.org) (64.49.215.8)
  by www.postgresql.org with SMTP; 5 Feb 2002 01:06:55 -0000
Received: from inflicted.crimelabs.net (crimelabs.net [66.92.101.112])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id g150vhl91978
	for <pgsql-hackers@postgresql.org>; Mon, 4 Feb 2002 19:57:44 -0500 (EST)
	(envelope-from bpalmer@crimelabs.net)
Received: from mizer.crimelabs.net (mizer.crimelabs.net [192.168.88.10])
	by inflicted.crimelabs.net (Postfix) with ESMTP
	id 9D6EE8779; Mon,  4 Feb 2002 19:57:46 -0500 (EST)
Date: Mon, 4 Feb 2002 19:57:34 -0500 (EST)
From: bpalmer <bpalmer@crimelabs.net>
To: mlw <markw@mohawksoft.com>
cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Replication
In-Reply-To: <3C5F22F8.C9B958F0@mohawksoft.com>
Message-ID: <Pine.BSO.4.43.0202041955420.17121-100000@mizer.crimelabs.net>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR

>
> OK, what the hell do we need to do to get PostgreSQL replicating?

I hope you understand that replication,  done right,  is a massive
project.  I know that Darren any myself (and the rest of the pg-repl
folks) have been waiting till 7.2 went gold till we did anymore work.  I
think we hope to have master / slave replicatin working for 7.3 and then
target multimaster for 7.4.  At least that's the hope.

- Brandon

----------------------------------------------------------------------------
 c: 646-456-5455                                            h: 201-798-4983
 b. palmer,  bpalmer@crimelabs.net           pgp:crimelabs.net/bpalmer.pgp5


---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
    (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)

From pgsql-hackers-owner+M18449=candle.pha.pa.us=pgman@postgresql.org Mon Feb  4 21:16:56 2002
Return-path: <pgsql-hackers-owner+M18449=candle.pha.pa.us=pgman@postgresql.org>
Received: from server1.pgsql.org (www.postgresql.org [64.49.215.9])
	by candle.pha.pa.us (8.11.6/8.10.1) with SMTP id g152GtP10503
	for <pgman@candle.pha.pa.us>; Mon, 4 Feb 2002 21:16:55 -0500 (EST)
Received: (qmail 6711 invoked by alias); 5 Feb 2002 02:16:53 -0000
Received: from unknown (HELO postgresql.org) (64.49.215.8)
  by www.postgresql.org with SMTP; 5 Feb 2002 02:16:53 -0000
Received: from snoopy.mohawksoft.com (h0050bf7a618d.ne.mediaone.net [24.147.138.78])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id g151qSl99469
	for <pgsql-hackers@postgresql.org>; Mon, 4 Feb 2002 20:52:28 -0500 (EST)
	(envelope-from markw@mohawksoft.com)
Received: from mohawksoft.com (localhost [127.0.0.1])
	by snoopy.mohawksoft.com (8.11.6/8.11.6) with ESMTP id g151lph09147;
	Mon, 4 Feb 2002 20:47:51 -0500
Message-ID: <3C5F39C7.970F4549@mohawksoft.com>
Date: Mon, 04 Feb 2002 20:47:51 -0500
From: mlw <markw@mohawksoft.com>
X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.17 i686)
X-Accept-Language: en
MIME-Version: 1.0
To: Steven <ssinger@navtechinc.com>
cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Replication
References: <Pine.LNX.4.33.0202050040190.24027-100000@pcNavYkfAdm1.ykf.navtechinc.com>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR

Steven wrote:
> 
> On Mon, 4 Feb 2002, mlw wrote:
> 
> I've developed a replacement for Rserv and we are planning on releasing
> it as open source(ie as a contrib module).
> 
> Like Rserv its trigger based but its much more flexible.
> The key adventages it has over Rserv is that it has
> -Support for multiple slaves
> -It Perserves transactions while doing the mirroring. Ie  If rows A,B are
> originally added in the same transaction they will be mirrored in the same
> transaction.

I did a similar thing. I took the rserv trigger "as is," but rewrote the
replication support code. What I eventually did was write a "snapshot daemon"
which created snapshot files. Then a "slave daemon" which would check the last
snapshot applied and apply all the snapshots, in order, as needed. One would
run one of these daemons per slave server.

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/users-lounge/docs/faq.html

From pgsql-hackers-owner+M18448=candle.pha.pa.us=pgman@postgresql.org Mon Feb  4 20:57:25 2002
Return-path: <pgsql-hackers-owner+M18448=candle.pha.pa.us=pgman@postgresql.org>
Received: from server1.pgsql.org (www.postgresql.org [64.49.215.9])
	by candle.pha.pa.us (8.11.6/8.10.1) with SMTP id g151vOP09239
	for <pgman@candle.pha.pa.us>; Mon, 4 Feb 2002 20:57:24 -0500 (EST)
Received: (qmail 99828 invoked by alias); 5 Feb 2002 01:57:19 -0000
Received: from unknown (HELO postgresql.org) (64.49.215.8)
  by www.postgresql.org with SMTP; 5 Feb 2002 01:57:19 -0000
Received: from snoopy.mohawksoft.com (h0050bf7a618d.ne.mediaone.net [24.147.138.78])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id g151s0l99529
	for <pgsql-hackers@postgresql.org>; Mon, 4 Feb 2002 20:54:00 -0500 (EST)
	(envelope-from markw@mohawksoft.com)
Received: from mohawksoft.com (localhost [127.0.0.1])
	by snoopy.mohawksoft.com (8.11.6/8.11.6) with ESMTP id g151nah09156;
	Mon, 4 Feb 2002 20:49:37 -0500
Message-ID: <3C5F3A30.A4C46FB8@mohawksoft.com>
Date: Mon, 04 Feb 2002 20:49:36 -0500
From: mlw <markw@mohawksoft.com>
X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.17 i686)
X-Accept-Language: en
MIME-Version: 1.0
To: bpalmer <bpalmer@crimelabs.net>
cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Replication
References: <Pine.BSO.4.43.0202041955420.17121-100000@mizer.crimelabs.net>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR

bpalmer wrote:
> 
> >
> > OK, what the hell do we need to do to get PostgreSQL replicating?
> 
> I hope you understand that replication,  done right,  is a massive
> project.  I know that Darren any myself (and the rest of the pg-repl
> folks) have been waiting till 7.2 went gold till we did anymore work.  I
> think we hope to have master / slave replicatin working for 7.3 and then
> target multimaster for 7.4.  At least that's the hope.

I do know how hard replication is. I also understand how important it is.

If you guys have a project going, and need developers, I am more than willing.

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/users-lounge/docs/faq.html

From pgsql-hackers-owner+M18450=candle.pha.pa.us=pgman@postgresql.org Mon Feb  4 21:42:13 2002
Return-path: <pgsql-hackers-owner+M18450=candle.pha.pa.us=pgman@postgresql.org>
Received: from server1.pgsql.org (www.postgresql.org [64.49.215.9])
	by candle.pha.pa.us (8.11.6/8.10.1) with SMTP id g152gCP11957
	for <pgman@candle.pha.pa.us>; Mon, 4 Feb 2002 21:42:13 -0500 (EST)
Received: (qmail 14229 invoked by alias); 5 Feb 2002 02:42:09 -0000
Received: from unknown (HELO postgresql.org) (64.49.215.8)
  by www.postgresql.org with SMTP; 5 Feb 2002 02:42:09 -0000
Received: from www1.navtechinc.com ([192.234.226.140])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id g152SBl10682
	for <pgsql-hackers@postgresql.org>; Mon, 4 Feb 2002 21:28:11 -0500 (EST)
	(envelope-from ssinger@navtechinc.com)
Received: from pcNavYkfAdm1.ykf.navtechinc.com (wall [192.234.226.190])
	by www1.navtechinc.com (8.9.3/8.9.3) with ESMTP id CAA06384;
	Tue, 5 Feb 2002 02:28:13 GMT
Received: from localhost (ssinger@localhost)
	by pcNavYkfAdm1.ykf.navtechinc.com (8.9.3/8.9.3) with ESMTP id CAA10682;
	Tue, 5 Feb 2002 02:27:35 GMT
Date: Tue, 5 Feb 2002 02:27:35 +0000 (GMT)
From: Steven <ssinger@navtechinc.com>
X-X-Sender: <ssinger@pcNavYkfAdm1.ykf.navtechinc.com>
To: mlw <markw@mohawksoft.com>
cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Replication
In-Reply-To: <3C5F39C7.970F4549@mohawksoft.com>
Message-ID: <Pine.LNX.4.33.0202050159591.26756-100000@pcNavYkfAdm1.ykf.navtechinc.com>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR


DBMirror doesn't use snapshot's instead it records a log of transactions 
that are committed to the database in a pair of tables.  
In the case of an INSERT this is the row that is being added.
In the case of a delete the primary key of the row being deleted.

And in the case of an UPDATE, the primary key before the update along with 
all of the data the row should have after an update.

Then for each slave database a perl script walks though the transactions 
that are pending for that host and reconstructs SQL to send the row edits 
to that host.  A record of the fact that transaction Y has been sent to 
host X is also kept.

When transaction X has been sent to all of the hosts that are in the 
system it is then deleted from the Pending tables.

I suspect that all of the information I'm storing in the Pending tables is 
also being stored by Postgres in its log but I haven't investigated how 
the information could be extracted(or how long it is kept for).  That 
would  reduce the extra storage overhead that the replication system 
imposes.

As I remember(Its been a while since I've looked at it) RServ uses OID's 
in its tables to point to the data that needs to be replicated.  We tried 
a similar approach but found difficulties with doing partial updates.






On Mon, 4 Feb 2002, mlw wrote:

> I did a similar thing. I took the rserv trigger "as is," but rewrote the
> replication support code. What I eventually did was write a "snapshot daemon"
> which created snapshot files. Then a "slave daemon" which would check the last
> snapshot applied and apply all the snapshots, in order, as needed. One would
> run one of these daemons per slave server.




 

-- 
Steven Singer                                       ssinger@navtechinc.com
Aircraft Performance Systems                Phone:  519-747-1170 ext 282
Navtech Systems Support Inc.                AFTN:   CYYZXNSX SITA: YYZNSCR
Waterloo, Ontario                           ARINC:  YKFNSCR


---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
    (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)

From pgsql-hackers-owner+M18554=candle.pha.pa.us=pgman@postgresql.org Thu Feb  7 02:49:48 2002
Return-path: <pgsql-hackers-owner+M18554=candle.pha.pa.us=pgman@postgresql.org>
Received: from server1.pgsql.org (www.postgresql.org [64.49.215.9])
	by candle.pha.pa.us (8.11.6/8.10.1) with SMTP id g177nlP04347
	for <pgman@candle.pha.pa.us>; Thu, 7 Feb 2002 02:49:47 -0500 (EST)
Received: (qmail 22556 invoked by alias); 7 Feb 2002 07:49:49 -0000
Received: from unknown (HELO postgresql.org) (64.49.215.8)
  by www.postgresql.org with SMTP; 7 Feb 2002 07:49:49 -0000
Received: from linuxworld.com.au (www.linuxworld.com.au [203.34.46.50])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id g177QfE19572
	for <pgsql-hackers@postgresql.org>; Thu, 7 Feb 2002 02:26:42 -0500 (EST)
	(envelope-from swm@linuxworld.com.au)
Received: from localhost (swm@localhost)
	by linuxworld.com.au (8.11.4/8.11.4) with ESMTP id g177RiU06086;
	Thu, 7 Feb 2002 18:27:45 +1100
Date: Thu, 7 Feb 2002 18:27:44 +1100 (EST)
From: Gavin Sherry <swm@linuxworld.com.au>
To: mlw <markw@mohawksoft.com>
cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Replication
In-Reply-To: <3C5F22F8.C9B958F0@mohawksoft.com>
Message-ID: <Pine.LNX.4.21.0202071751240.5160-100000@linuxworld.com.au>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR

On Mon, 4 Feb 2002, mlw wrote:

> I re-wrote RServ.pm to C, and wrote a replication daemon. It works, but it
> works like the whole rserv project. I don't like it.
> 
> OK, what the hell do we need to do to get PostgreSQL replicating?

The trigger model is not a very sophisticated one. I think I have a better
-- though more complicated -- one. This model would be able to handle
multiple masters and master->slave.

First of all, all machines in the cluster would have to be aware all the
machines in the cluster. This would have to be stored in a new system
table.

The FE/BE protocol would need to be modified to accepted parsed node trees
generated by pg_analyze_and_rewrite(). These could then be dispatched by 
the executing server, inside of pg_exec_query_string, to all other servers
in the cluster (excluding itself). Naturally, this dispatch would need to
be non-blocking.

pg_exec_query_string() would need to check that nodetags to make sure
selects and perhaps some commands are not dispatched.

Before the executing server runs finish_xact_command(), it would check
that the query was successfully executed on all machines otherwise
abort. Such a system would need a few configuration options: whether or
not you abort on failed replication to slaves, the ability to replicate
only certain tables, etc.

Naturally, this would slow down writes to the system (possibly a lot
depending on the performance difference between the executing machine and
the least powerful machine in the cluster), but most usages of postgresql
are read intensive, not write.

Any reason this model would not work?

Gavin


---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

From pgsql-hackers-owner+M18558=candle.pha.pa.us=pgman@postgresql.org Thu Feb  7 08:31:00 2002
Return-path: <pgsql-hackers-owner+M18558=candle.pha.pa.us=pgman@postgresql.org>
Received: from server1.pgsql.org (www.postgresql.org [64.49.215.9])
	by candle.pha.pa.us (8.11.6/8.10.1) with SMTP id g17DUxP13923
	for <pgman@candle.pha.pa.us>; Thu, 7 Feb 2002 08:30:59 -0500 (EST)
Received: (qmail 91796 invoked by alias); 7 Feb 2002 13:30:55 -0000
Received: from unknown (HELO postgresql.org) (64.49.215.8)
  by www.postgresql.org with SMTP; 7 Feb 2002 13:30:55 -0000
Received: from snoopy.mohawksoft.com (h0050bf7a618d.ne.mediaone.net [24.147.138.78])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id g17Cw0E87782
	for <pgsql-hackers@postgresql.org>; Thu, 7 Feb 2002 07:58:01 -0500 (EST)
	(envelope-from markw@mohawksoft.com)
Received: from mohawksoft.com (localhost [127.0.0.1])
	by snoopy.mohawksoft.com (8.11.6/8.11.6) with ESMTP id g17CqNt16887;
	Thu, 7 Feb 2002 07:52:24 -0500
Message-ID: <3C627887.CC9FF837@mohawksoft.com>
Date: Thu, 07 Feb 2002 07:52:23 -0500
From: mlw <markw@mohawksoft.com>
X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.17 i686)
X-Accept-Language: en
MIME-Version: 1.0
To: Gavin Sherry <swm@linuxworld.com.au>
cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Replication
References: <Pine.LNX.4.21.0202071751240.5160-100000@linuxworld.com.au>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR

Gavin Sherry wrote:
> Naturally, this would slow down writes to the system (possibly a lot
> depending on the performance difference between the executing machine and
> the least powerful machine in the cluster), but most usages of postgresql
> are read intensive, not write.
> 
> Any reason this model would not work?

What, then is the purpose of replication to multiple masters?

I can think of only two reasons why you want replication. (1) Redundancy, make
sure that if one server dies, then another server has the same data and is used
seamlessly. (2) Increase performance over one system.

In reason (1) I submit that a server load balance which sits on top of
PostgreSQL, and executes writes on both servers while distributing reads would
be best. This is a HUGE project. The load balancer must know EXACTLY how the
system is configured, which includes all functions and everything. 

In reason (2) your system would fail to provide the scalability that would be
needed. If writes take a long time, but reads are fine, what is the difference
between the trigger based replicator?

I have in the back of my mind, an idea of patching into the WAL stuff, and
using that mechanism to push changes out to the slaves.

Where one machine is still the master, but no trigger stuff, just a WAL patch.
Perhaps some shared memory paradigm to manage WAL visibility? I'm not sure
exactly, the idea hasn't completely formed yet.

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/users-lounge/docs/faq.html

From pgsql-hackers-owner+M18574=candle.pha.pa.us=pgman@postgresql.org Thu Feb  7 12:51:42 2002
Return-path: <pgsql-hackers-owner+M18574=candle.pha.pa.us=pgman@postgresql.org>
Received: from server1.pgsql.org (www.postgresql.org [64.49.215.9])
	by candle.pha.pa.us (8.11.6/8.10.1) with SMTP id g17HpfP16661
	for <pgman@candle.pha.pa.us>; Thu, 7 Feb 2002 12:51:41 -0500 (EST)
Received: (qmail 62955 invoked by alias); 7 Feb 2002 17:50:42 -0000
Received: from unknown (HELO postgresql.org) (64.49.215.8)
  by www.postgresql.org with SMTP; 7 Feb 2002 17:50:42 -0000
Received: from www1.navtechinc.com ([192.234.226.140])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id g17HnTE62256
	for <pgsql-hackers@postgresql.org>; Thu, 7 Feb 2002 12:49:29 -0500 (EST)
	(envelope-from ssinger@navtechinc.com)
Received: from pcNavYkfAdm1.ykf.navtechinc.com (wall [192.234.226.190])
	by www1.navtechinc.com (8.9.3/8.9.3) with ESMTP id RAA07908;
	Thu, 7 Feb 2002 17:49:31 GMT
Received: from localhost (ssinger@localhost)
	by pcNavYkfAdm1.ykf.navtechinc.com (8.9.3/8.9.3) with ESMTP id RAA05687;
	Thu, 7 Feb 2002 17:48:52 GMT
Date: Thu, 7 Feb 2002 17:48:51 +0000 (GMT)
From: Steven Singer <ssinger@navtechinc.com>
X-X-Sender: <ssinger@pcNavYkfAdm1.ykf.navtechinc.com>
To: Gavin Sherry <swm@linuxworld.com.au>
cc: mlw <markw@mohawksoft.com>,
   PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Replication
In-Reply-To: <Pine.LNX.4.21.0202071751240.5160-100000@linuxworld.com.au>
Message-ID: <Pine.LNX.4.33.0202071735360.6435-100000@pcNavYkfAdm1.ykf.navtechinc.com>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR



What you describe sounds like a form of a two-stage commit protocol.

If the command worked on two of the replicated databases but failed on a 
third then the executing server would have to be able to undo the command
on the replicated databases as well as itself.

The problems with two stage commit type approches to replication are 
1) Speed as you mentioned.  Write speed isn't a concern for some 
applications but it is very important in others.

and 
2) All of the databases must be able to communicate with each other at 
all times in order for any edits to work.   If the servers are 
connected over some sort of WAN that periodically has short outages this 
is a problem.   Also if your using replication because you want to be able 
to take down one of the databases for short periods of time without 
bringing down the others your in trouble.


btw: I posted the alternative to Rserv that I mentioned the other day to 
the  pg-patches mailing list.  If anyone is intreasted you should be able 
to grab it off the archives.

On Thu, 7 Feb 2002, Gavin Sherry wrote:

> 
> First of all, all machines in the cluster would have to be aware all the
> machines in the cluster. This would have to be stored in a new system
> table.
> 
> The FE/BE protocol would need to be modified to accepted parsed node trees
> generated by pg_analyze_and_rewrite(). These could then be dispatched by 
> the executing server, inside of pg_exec_query_string, to all other servers
> in the cluster (excluding itself). Naturally, this dispatch would need to
> be non-blocking.
> 
> pg_exec_query_string() would need to check that nodetags to make sure
> selects and perhaps some commands are not dispatched.
> 
> Before the executing server runs finish_xact_command(), it would check
> that the query was successfully executed on all machines otherwise
> abort. Such a system would need a few configuration options: whether or
> not you abort on failed replication to slaves, the ability to replicate
> only certain tables, etc.
> 
> Naturally, this would slow down writes to the system (possibly a lot
> depending on the performance difference between the executing machine and
> the least powerful machine in the cluster), but most usages of postgresql
> are read intensive, not write.
> 
> Any reason this model would not work?
> 
> Gavin
> 
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster
> 

-- 
Steven Singer                                       ssinger@navtechinc.com
Aircraft Performance Systems                Phone:  519-747-1170 ext 282
Navtech Systems Support Inc.                AFTN:   CYYZXNSX SITA: YYZNSCR
Waterloo, Ontario                           ARINC:  YKFNSCR


---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org

From pgsql-hackers-owner+M18590=candle.pha.pa.us=pgman@postgresql.org Thu Feb  7 17:50:42 2002
Return-path: <pgsql-hackers-owner+M18590=candle.pha.pa.us=pgman@postgresql.org>
Received: from server1.pgsql.org (www.postgresql.org [64.49.215.9])
	by candle.pha.pa.us (8.11.6/8.10.1) with SMTP id g17MoeP27121
	for <pgman@candle.pha.pa.us>; Thu, 7 Feb 2002 17:50:40 -0500 (EST)
Received: (qmail 39930 invoked by alias); 7 Feb 2002 22:50:17 -0000
Received: from unknown (HELO postgresql.org) (64.49.215.8)
  by www.postgresql.org with SMTP; 7 Feb 2002 22:50:17 -0000
Received: from odin.fts.net (wall.icgate.net [209.26.177.2])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id g17Ma4E38041
	for <pgsql-hackers@postgresql.org>; Thu, 7 Feb 2002 17:36:04 -0500 (EST)
	(envelope-from fharvell@odin.fts.net)
Received: from odin.fts.net (fharvell@localhost)
	by odin.fts.net (8.11.6/8.11.6) with ESMTP id g17MZhR17707;
	Thu, 7 Feb 2002 17:35:43 -0500
Message-ID: <200202072235.g17MZhR17707@odin.fts.net>
X-Mailer: exmh version 2.2 06/23/2000 with nmh-1.0.4
From: F Harvell <fharvell@fts.net>
To: mlw <markw@mohawksoft.com>
cc: Gavin Sherry <swm@linuxworld.com.au>,
   PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Replication 
In-Reply-To: Message from mlw
    of "Thu, 07 Feb 2002 07:52:23 EST."
    <3C627887.CC9FF837@mohawksoft.com> 
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Date: Thu, 07 Feb 2002 17:35:43 -0500
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR

I'm not that familiar with the whole replication issues in PostgreSQL,
however, I would be partial to replication that was based upon the
playback of the (a?) journal file.  (I believe that the WAL is a
journal file.)

By being based upon a journal file, it would be possible to accomplish
two significant items.  First, it would be possible to "restore" a
database to an exact state just before a failure.  Most commercial
databases provide the ability to do this.  Banks, etc. log the journal
files directly to tape to provide a complete transaction history such
that they can rebuild their database from any given snapshot.  (Note
that the journal file needs to be "editable" as a failure may be
"delete from x" with a missing where clause.)

This leads directly into the second advantage, the ability to have a
replicated database operating anywhere, over any connection on any
server.  Speed of writes would not be a factor.  In essence, as long
as the replicated database had a snapshot of the database and then was
provided with all journal files since the snapshot, it would be
possible to build a current database.  If the replicant got behind in
the processing, it would catch up when things slowed down.

In my opionion, the first advantage is in many ways most important.
Replication becomes simply the restoration of the database in realtime
on a second server.  The "replication" task becomes the definition of
a protocol for distributing the journal file.  At least one major
database vendor does replication (shadowing) in exactly this mannor.

Maybe I'm all wet and the journal file and journal playback already
exists.  If so, IMHO, basing replication off of this would be the
right direction.


On Thu, 07 Feb 2002 07:52:23 EST, mlw wrote:
> 
> I have in the back of my mind, an idea of patching into the WAL stuff, and
> using that mechanism to push changes out to the slaves.
> 
> Where one machine is still the master, but no trigger stuff, just a WAL patch.
> Perhaps some shared memory paradigm to manage WAL visibility? I'm not sure
> exactly, the idea hasn't completely formed yet.
> 



---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

From pgsql-hackers-owner+M18605=candle.pha.pa.us=pgman@postgresql.org Fri Feb  8 00:50:08 2002
Return-path: <pgsql-hackers-owner+M18605=candle.pha.pa.us=pgman@postgresql.org>
Received: from server1.pgsql.org (www.postgresql.org [64.49.215.9])
	by candle.pha.pa.us (8.11.6/8.10.1) with SMTP id g185o7P27878
	for <pgman@candle.pha.pa.us>; Fri, 8 Feb 2002 00:50:07 -0500 (EST)
Received: (qmail 17348 invoked by alias); 8 Feb 2002 05:50:03 -0000
Received: from unknown (HELO postgresql.org) (64.49.215.8)
  by www.postgresql.org with SMTP; 8 Feb 2002 05:50:03 -0000
Received: from lakemtao03.mgt.cox.net (mtao3.east.cox.net [68.1.17.242])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id g185cTE15241
	for <pgsql-hackers@postgresql.org>; Fri, 8 Feb 2002 00:38:29 -0500 (EST)
	(envelope-from darren.johnson@cox.net)
Received: from cox.net ([68.10.181.230]) by lakemtao03.mgt.cox.net
          (InterMail vM.5.01.04.05 201-253-122-122-105-20011231) with ESMTP
          id <20020208053833.YKTV6710.lakemtao03.mgt.cox.net@cox.net>
          for <pgsql-hackers@postgresql.org>;
          Fri, 8 Feb 2002 00:38:33 -0500
Message-ID: <3C636232.6060206@cox.net>
Date: Fri, 08 Feb 2002 00:29:22 -0500
From: Darren Johnson <darren.johnson@cox.net>
User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; m18) Gecko/20001108 Netscape6/6.0
X-Accept-Language: en
MIME-Version: 1.0
To: PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Replication
References: <Pine.LNX.4.33.0202071735360.6435-100000@pcNavYkfAdm1.ykf.navtechinc.com>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR


 >
 > The problems with two stage commit type approches to replication are

IMHO the biggest problem with two phased commit is it doesn't scale.
The more servers
you add to the replica the slower it goes.  Also there's the potential
for dead locks across
server boundaries.

 >
 > 2) All of the databases must be able to communicate with each other at
 > all times in order for any edits to work.   If the servers are
 > connected over some sort of WAN that periodically has short outages this
 > is a problem.   Also if your using replication because you want to be 
able
 > to take down one of the databases for short periods of time without
 > bringing down the others your in trouble.

All true for two phased commit protocol.  To have multi master
replication, you must have all
systems communicating, but you can use a multicast group communication
system instead of
2PC.  Using total order messaging, you can ensure all changes are
delivered to all servers in the
replica in the same order.   This group communication system also allows
failures to be detected
while other servers in the replica continue processing.

A few of us are working with this theory, and trying to integrate with
7.2.  There is a working
model for 6.4, but its very limited.  (insert, update, and deletes)  We
are currently hosted at

http://gborg.postgresql.org/project/pgreplication/projdisplay.php
But the site has been down the last 2 days.  I've contacted the web
master, but haven't seen
any results yet.  If any one knows what going on with gborg, I'd
appreciate a status.

Darren


---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
    (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)

From pgsql-hackers-owner+M18617=candle.pha.pa.us=pgman@postgresql.org Fri Feb  8 06:20:44 2002
Return-path: <pgsql-hackers-owner+M18617=candle.pha.pa.us=pgman@postgresql.org>
Received: from server1.pgsql.org (www.postgresql.org [64.49.215.9])
	by candle.pha.pa.us (8.11.6/8.10.1) with SMTP id g18BKhP06132
	for <pgman@candle.pha.pa.us>; Fri, 8 Feb 2002 06:20:43 -0500 (EST)
Received: (qmail 90815 invoked by alias); 8 Feb 2002 11:20:40 -0000
Received: from unknown (HELO postgresql.org) (64.49.215.8)
  by www.postgresql.org with SMTP; 8 Feb 2002 11:20:40 -0000
Received: from laptop.kieser.demon.co.uk (kieser.demon.co.uk [62.49.6.72])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id g18B9ZE89589
	for <pgsql-hackers@postgresql.org>; Fri, 8 Feb 2002 06:09:36 -0500 (EST)
	(envelope-from brad@kieser.net)
Received: from laptop.kieser.demon.co.uk (localhost.localdomain [127.0.0.1])
	by laptop.kieser.demon.co.uk (Postfix) with SMTP
	id 598393A132; Fri,  8 Feb 2002 11:09:36 +0000 (GMT)
From: Bradley Kieser <brad@kieser.net>
Date: Fri, 08 Feb 2002 11:09:36 GMT
Message-ID: <20020208.11093600@laptop.kieser.demon.co.uk>
Subject: Re: [HACKERS] Replication
To: Darren Johnson <darren.johnson@cox.net>
cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
In-Reply-To: <3C636232.6060206@cox.net>
References: <Pine.LNX.4.33.0202071735360.6435-100000@pcNavYkfAdm1.ykf.navtechinc.com> <3C636232.6060206@cox.net>
X-Mailer: Mozilla/3.0 (compatible; StarOffice/5.2;Linux)
X-Priority: 3 (Normal)
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by postgresql.org id g18BJoF90352
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR

Darren,
Given that different replication strategies will probably be developed 
for PG, do you envisage DBAs to be able to select the type of replication 
for their installation? I.e. Replication being selectable rther like 
storage structures?

Would be a killer bit of flexibility, given how enormous the impact of 
replication will be to corporate adoption of PG.

Brad 


>>>>>>>>>>>>>>>>>> Original Message <<<<<<<<<<<<<<<<<<

On 2/8/02, 5:29:22 AM, Darren Johnson <darren.johnson@cox.net> wrote 
regarding Re: [HACKERS] Replication:


>  >
>  > The problems with two stage commit type approches to replication are

> IMHO the biggest problem with two phased commit is it doesn't scale.
> The more servers
> you add to the replica the slower it goes.  Also there's the potential
> for dead locks across
> server boundaries.

>  >
>  > 2) All of the databases must be able to communicate with each other at
>  > all times in order for any edits to work.   If the servers are
>  > connected over some sort of WAN that periodically has short outages this
>  > is a problem.   Also if your using replication because you want to be
> able
>  > to take down one of the databases for short periods of time without
>  > bringing down the others your in trouble.

> All true for two phased commit protocol.  To have multi master
> replication, you must have all
> systems communicating, but you can use a multicast group communication
> system instead of
> 2PC.  Using total order messaging, you can ensure all changes are
> delivered to all servers in the
> replica in the same order.   This group communication system also allows
> failures to be detected
> while other servers in the replica continue processing.

> A few of us are working with this theory, and trying to integrate with
> 7.2.  There is a working
> model for 6.4, but its very limited.  (insert, update, and deletes)  We
> are currently hosted at

> http://gborg.postgresql.org/project/pgreplication/projdisplay.php
> But the site has been down the last 2 days.  I've contacted the web
> master, but haven't seen
> any results yet.  If any one knows what going on with gborg, I'd
> appreciate a status.

> Darren


> ---------------------------(end of broadcast)---------------------------
> TIP 2: you can get off all lists at once with the unregister command
>     (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org

From pgsql-hackers-owner+M18642=candle.pha.pa.us=pgman@postgresql.org Fri Feb  8 12:40:36 2002
Return-path: <pgsql-hackers-owner+M18642=candle.pha.pa.us=pgman@postgresql.org>
Received: from server1.pgsql.org (www.postgresql.org [64.49.215.9])
	by candle.pha.pa.us (8.11.6/8.10.1) with SMTP id g18HeZP08450
	for <pgman@candle.pha.pa.us>; Fri, 8 Feb 2002 12:40:35 -0500 (EST)
Received: (qmail 74089 invoked by alias); 8 Feb 2002 17:40:30 -0000
Received: from unknown (HELO postgresql.org) (64.49.215.8)
  by www.postgresql.org with SMTP; 8 Feb 2002 17:40:30 -0000
Received: from lakemtao03.mgt.cox.net (mtao3.east.cox.net [68.1.17.242])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id g18HbwE73437
	for <pgsql-hackers@postgresql.org>; Fri, 8 Feb 2002 12:37:58 -0500 (EST)
	(envelope-from darren.johnson@cox.net)
Received: from cox.net ([68.10.181.230]) by lakemtao03.mgt.cox.net
          (InterMail vM.5.01.04.05 201-253-122-122-105-20011231) with ESMTP
          id <20020208173804.DKQS6710.lakemtao03.mgt.cox.net@cox.net>;
          Fri, 8 Feb 2002 12:38:04 -0500
Message-ID: <3C63FB71.206@cox.net>
Date: Fri, 08 Feb 2002 11:23:13 -0500
From: Darren Johnson <darren.johnson@cox.net>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; m18) Gecko/20010131 Netscape6/6.01
X-Accept-Language: en
MIME-Version: 1.0
To: Bradley Kieser <brad@kieser.net>
cc: pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Replication
References: <Pine.LNX.4.33.0202071735360.6435-100000@pcNavYkfAdm1.ykf.navtechinc.com> <3C636232.6060206@cox.net> <20020208.11093600@laptop.kieser.demon.co.uk>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR

> 
> Given that different replication strategies will probably be developed 
> for PG, do you envisage DBAs to be able to select the type of replication 
> for their installation? I.e. Replication being selectable rther like 
> storage structures?

I can't speak for other replication solutions, but we are using the 
--with-replication or
-r parameter when starting postmaster.  Some day I hope there will be 
parameters for
master/slave partial/full and sync/async,  but it will be some time 
before we cross those
bridges.

Darren




---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org

From pgsql-hackers-owner+M18658=candle.pha.pa.us=pgman@postgresql.org Fri Feb  8 14:42:40 2002
Return-path: <pgsql-hackers-owner+M18658=candle.pha.pa.us=pgman@postgresql.org>
Received: from server1.pgsql.org (www.postgresql.org [64.49.215.9])
	by candle.pha.pa.us (8.11.6/8.10.1) with SMTP id g18JgdP28166
	for <pgman@candle.pha.pa.us>; Fri, 8 Feb 2002 14:42:39 -0500 (EST)
Received: (qmail 18650 invoked by alias); 8 Feb 2002 19:42:39 -0000
Received: from unknown (HELO postgresql.org) (64.49.215.8)
  by www.postgresql.org with SMTP; 8 Feb 2002 19:42:39 -0000
Received: from enigma.trueimpact.net (enigma.trueimpact.net [209.82.45.201])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id g18JYBE17341
	for <pgsql-hackers@postgresql.org>; Fri, 8 Feb 2002 14:34:11 -0500 (EST)
	(envelope-from rjonasz@trueimpact.com)
Received: from nietzsche.trueimpact.net (unknown [209.82.45.200])
	by enigma.trueimpact.net (Postfix) with ESMTP id A785066B04
	for <pgsql-hackers@postgresql.org>; Fri,  8 Feb 2002 14:33:28 -0500 (EST)
Date: Fri, 8 Feb 2002 14:34:34 -0500 (EST)
From: Randall Jonasz <rjonasz@trueimpact.com>
X-X-Sender: <rjonasz@nietzsche.trueimpact.net>
To: PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Replication
In-Reply-To: <3C627887.CC9FF837@mohawksoft.com>
Message-ID: <20020208142932.H6545-100000@nietzsche.trueimpact.net>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR

I've been looking into database replication theory lately and have found
some interesting papers discussing various approaches.  (Here's
one paper that struck me as being very helpful,
http://citeseer.nj.nec.com/460405.html )  So far I favour an
eager replication system which is predicated on a read local/write all
available. The system should not depend on two phase commit or primary
copy algorithms.  The former leads to the whole system being as quick as
the slowest machine.  In addition, 2 phase commit involves 2n messages for
each transaction which does not scale well at all.  This idea will also
have to take into account a crashed node which did not ack a transaction.
The primary copy algorithms I've seen suffer from a single point of
failure and potential bottlenecks at the primary node.

Instead I like the master to master or peer to peer algorithm as discussed
in the above paper.  This approach accounts for network partitions, nodes
leaving and joining a cluster and the ability to commit a transaction once
the communication module has determined the total order of the said
transaction, i.e. no need for waiting for acks.   This scales well and
research has shown it to increase the number of transactions/second a
database cluster can handle over a single node.

Postgres-R is another interesting approach which I think should be taken
seriously. Anyone interested can read a paper on this at
http://citeseer.nj.nec.com/330257.html

Anyways, my two cents

Randall Jonasz
Software Engineer
Click2net Inc.


On Thu, 7 Feb 2002, mlw wrote:

> Gavin Sherry wrote:
> > Naturally, this would slow down writes to the system (possibly a lot
> > depending on the performance difference between the executing machine and
> > the least powerful machine in the cluster), but most usages of postgresql
> > are read intensive, not write.
> >
> > Any reason this model would not work?
>
> What, then is the purpose of replication to multiple masters?
>
> I can think of only two reasons why you want replication. (1) Redundancy, make
> sure that if one server dies, then another server has the same data and is used
> seamlessly. (2) Increase performance over one system.
>
> In reason (1) I submit that a server load balance which sits on top of
> PostgreSQL, and executes writes on both servers while distributing reads would
> be best. This is a HUGE project. The load balancer must know EXACTLY how the
> system is configured, which includes all functions and everything.
>
> In reason (2) your system would fail to provide the scalability that would be
> needed. If writes take a long time, but reads are fine, what is the difference
> between the trigger based replicator?
>
> I have in the back of my mind, an idea of patching into the WAL stuff, and
> using that mechanism to push changes out to the slaves.
>
> Where one machine is still the master, but no trigger stuff, just a WAL patch.
> Perhaps some shared memory paradigm to manage WAL visibility? I'm not sure
> exactly, the idea hasn't completely formed yet.
>
> ---------------------------(end of broadcast)---------------------------
> TIP 5: Have you checked our extensive FAQ?
>
> http://www.postgresql.org/users-lounge/docs/faq.html
>
>


---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/users-lounge/docs/faq.html

From pgsql-hackers-owner+M18660=candle.pha.pa.us=pgman@postgresql.org Fri Feb  8 15:20:32 2002
Return-path: <pgsql-hackers-owner+M18660=candle.pha.pa.us=pgman@postgresql.org>
Received: from server1.pgsql.org (www.postgresql.org [64.49.215.9])
	by candle.pha.pa.us (8.11.6/8.10.1) with SMTP id g18KKSP03731
	for <pgman@candle.pha.pa.us>; Fri, 8 Feb 2002 15:20:29 -0500 (EST)
Received: (qmail 28961 invoked by alias); 8 Feb 2002 20:20:27 -0000
Received: from unknown (HELO postgresql.org) (64.49.215.8)
  by www.postgresql.org with SMTP; 8 Feb 2002 20:20:27 -0000
Received: from inflicted.crimelabs.net (crimelabs.net [66.92.101.112])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id g18KC7E27667
	for <pgsql-hackers@postgresql.org>; Fri, 8 Feb 2002 15:12:07 -0500 (EST)
	(envelope-from bpalmer@crimelabs.net)
Received: from mizer.crimelabs.net (mizer.crimelabs.net [192.168.88.10])
	by inflicted.crimelabs.net (Postfix) with ESMTP
	id 1066F8787; Fri,  8 Feb 2002 15:12:08 -0500 (EST)
Date: Fri, 8 Feb 2002 15:12:00 -0500 (EST)
From: bpalmer <bpalmer@crimelabs.net>
To: Randall Jonasz <rjonasz@trueimpact.com>
cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Replication
In-Reply-To: <20020208142932.H6545-100000@nietzsche.trueimpact.net>
Message-ID: <Pine.BSO.4.43.0202081510130.21860-100000@mizer.crimelabs.net>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR

I've not looked at the first paper,  but I wil.

> Postgres-R is another interesting approach which I think should be taken
> seriously. Anyone interested can read a paper on this at
> http://citeseer.nj.nec.com/330257.html

I would point you to the info on gborg,  but it seems to be down at the
moment.

- Brandon

----------------------------------------------------------------------------
 c: 646-456-5455                                            h: 201-798-4983
 b. palmer,  bpalmer@crimelabs.net           pgp:crimelabs.net/bpalmer.pgp5


---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

From pgsql-hackers-owner+M18666=candle.pha.pa.us=pgman@postgresql.org Fri Feb  8 17:41:03 2002
Return-path: <pgsql-hackers-owner+M18666=candle.pha.pa.us=pgman@postgresql.org>
Received: from server1.pgsql.org (www.postgresql.org [64.49.215.9])
	by candle.pha.pa.us (8.11.6/8.10.1) with SMTP id g18Mf2P18046
	for <pgman@candle.pha.pa.us>; Fri, 8 Feb 2002 17:41:03 -0500 (EST)
Received: (qmail 63057 invoked by alias); 8 Feb 2002 22:41:02 -0000
Received: from unknown (HELO postgresql.org) (64.49.215.8)
  by www.postgresql.org with SMTP; 8 Feb 2002 22:41:02 -0000
Received: from lakemtao03.mgt.cox.net (mtao3.east.cox.net [68.1.17.242])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id g18MR9E60361
	for <pgsql-hackers@postgresql.org>; Fri, 8 Feb 2002 17:27:11 -0500 (EST)
	(envelope-from darren.johnson@cox.net)
Received: from cox.net ([68.10.181.230]) by lakemtao03.mgt.cox.net
          (InterMail vM.5.01.04.05 201-253-122-122-105-20011231) with ESMTP
          id <20020208222634.GTRG6710.lakemtao03.mgt.cox.net@cox.net>;
          Fri, 8 Feb 2002 17:26:34 -0500
Message-ID: <3C643F0F.70303@cox.net>
Date: Fri, 08 Feb 2002 16:11:43 -0500
From: Darren Johnson <darren.johnson@cox.net>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; m18) Gecko/20010131 Netscape6/6.01
X-Accept-Language: en
MIME-Version: 1.0
To: Randall Jonasz <rjonasz@trueimpact.com>
cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Replication
References: <20020208142932.H6545-100000@nietzsche.trueimpact.net>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR


> I've been looking into database replication theory lately and have found
> some interesting papers discussing various approaches.  (Here's
> one paper that struck me as being very helpful,
> http://citeseer.nj.nec.com/460405.html )


Here is another one from that same group, that addresses  the WAN issues.

> http://www.cnds.jhu.edu/pub/papers/cnds-2002-1.pdf


enjoy,

Darren




---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org

From pgsql-hackers-owner+M18674=candle.pha.pa.us=pgman@postgresql.org Fri Feb  8 19:20:30 2002
Return-path: <pgsql-hackers-owner+M18674=candle.pha.pa.us=pgman@postgresql.org>
Received: from server1.pgsql.org (www.postgresql.org [64.49.215.9])
	by candle.pha.pa.us (8.11.6/8.10.1) with SMTP id g190KTP26980
	for <pgman@candle.pha.pa.us>; Fri, 8 Feb 2002 19:20:29 -0500 (EST)
Received: (qmail 88124 invoked by alias); 9 Feb 2002 00:20:27 -0000
Received: from unknown (HELO postgresql.org) (64.49.215.8)
  by www.postgresql.org with SMTP; 9 Feb 2002 00:20:27 -0000
Received: from localhost.localdomain (bgp01077650bgs.wanarb01.mi.comcast.net [68.40.135.112])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id g190H3E87489
	for <pgsql-hackers@postgresql.org>; Fri, 8 Feb 2002 19:17:03 -0500 (EST)
	(envelope-from camber@ais.org)
Received: from localhost (camber@localhost)
	by localhost.localdomain (8.11.6/8.11.6) with ESMTP id g190H0P18427;
	Fri, 8 Feb 2002 19:17:00 -0500
X-Authentication-Warning: localhost.localdomain: camber owned process doing -bs
Date: Fri, 8 Feb 2002 19:17:00 -0500 (EST)
From: Brian Bruns <camber@ais.org>
X-X-Sender: <camber@localhost.localdomain>
To: Randall Jonasz <rjonasz@trueimpact.com>
cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Replication
In-Reply-To: <20020208142932.H6545-100000@nietzsche.trueimpact.net>
Message-ID: <Pine.LNX.4.33.0202081904190.18420-100000@localhost.localdomain>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR

> > I have in the back of my mind, an idea of patching into the WAL stuff, and
> > using that mechanism to push changes out to the slaves.
> >
> > Where one machine is still the master, but no trigger stuff, just a WAL patch.
> > Perhaps some shared memory paradigm to manage WAL visibility? I'm not sure
> > exactly, the idea hasn't completely formed yet.
> >

FWIW, Sybase Replication Server does just such a thing.  

They have a secondary log marker (prevents the log from truncating past 
the oldest unreplicated transaction).  A thread within the system called 
the "rep agent" (but it use to be a separate process call the LTM), reads 
the log and forwards it to the rep server, once the rep server has the 
whole transaction and it is written to a stable device (aka synced to 
disk) the rep server responds to the LTM telling him it's OK to move the 
log marker forward.

Anyway, once the replication server proper has the transaction it uses a 
publish/subscribe methodology to see who wants get the update.

Bidirectional replication is done by making two oneway replications.  The 
whole thing is table based, it marks the tables as replicated or not in 
the database to save the trip to the repserver on un replicated tables.

Plus you can take parts of a database (replicate all rows where the 
country is "us" to this server and all the rows with "uk" to that server).  
Or opposite you can roll up smaller regional databases to bigger ones, 
it's very flexible.


Cheers,

Brian


---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster