Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to content

Commit 8cb2c01

Browse files
bmomjianbmomjian
committed
Add.
1 parent a05eae0 commit 8cb2c01

File tree

1 file changed

+184
-7
lines changed

1 file changed

+184
-7
lines changed

‎doc/TODO.detail/replication

Lines changed: 184 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@ From owner-pgsql-hackers@hub.org Fri Dec 24 10:01:18 1999
4343
Received: from renoir.op.net (root@renoir.op.net [207.29.195.4])
4444
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id LAA11295
4545
for <pgman@candle.pha.pa.us>; Fri, 24 Dec 1999 11:01:17 -0500 (EST)
46-
Received: from hub.org (hub.org [216.126.84.1]) by renoir.op.net (o1/$Revision: 1.5 $) with ESMTP id KAA20310 for <pgman@candle.pha.pa.us>; Fri, 24 Dec 1999 10:39:18 -0500 (EST)
46+
Received: from hub.org (hub.org [216.126.84.1]) by renoir.op.net (o1/$Revision: 1.6 $) with ESMTP id KAA20310 for <pgman@candle.pha.pa.us>; Fri, 24 Dec 1999 10:39:18 -0500 (EST)
4747
Received: from localhost (majordom@localhost)
4848
by hub.org (8.9.3/8.9.3) with SMTP id KAA61760;
4949
Fri, 24 Dec 1999 10:31:13 -0500 (EST)
@@ -129,7 +129,7 @@ From owner-pgsql-hackers@hub.org Fri Dec 24 18:31:03 1999
129129
Received: from renoir.op.net (root@renoir.op.net [207.29.195.4])
130130
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id TAA26244
131131
for <pgman@candle.pha.pa.us>; Fri, 24 Dec 1999 19:31:02 -0500 (EST)
132-
Received: from hub.org (hub.org [216.126.84.1]) by renoir.op.net (o1/$Revision: 1.5 $) with ESMTP id TAA12730 for <pgman@candle.pha.pa.us>; Fri, 24 Dec 1999 19:30:05 -0500 (EST)
132+
Received: from hub.org (hub.org [216.126.84.1]) by renoir.op.net (o1/$Revision: 1.6 $) with ESMTP id TAA12730 for <pgman@candle.pha.pa.us>; Fri, 24 Dec 1999 19:30:05 -0500 (EST)
133133
Received: from localhost (majordom@localhost)
134134
by hub.org (8.9.3/8.9.3) with SMTP id TAA57851;
135135
Fri, 24 Dec 1999 19:23:31 -0500 (EST)
@@ -212,7 +212,7 @@ From owner-pgsql-hackers@hub.org Fri Dec 24 21:31:10 1999
212212
Received: from renoir.op.net (root@renoir.op.net [207.29.195.4])
213213
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id WAA02578
214214
for <pgman@candle.pha.pa.us>; Fri, 24 Dec 1999 22:31:09 -0500 (EST)
215-
Received: from hub.org (hub.org [216.126.84.1]) by renoir.op.net (o1/$Revision: 1.5 $) with ESMTP id WAA16641 for <pgman@candle.pha.pa.us>; Fri, 24 Dec 1999 22:18:56 -0500 (EST)
215+
Received: from hub.org (hub.org [216.126.84.1]) by renoir.op.net (o1/$Revision: 1.6 $) with ESMTP id WAA16641 for <pgman@candle.pha.pa.us>; Fri, 24 Dec 1999 22:18:56 -0500 (EST)
216216
Received: from localhost (majordom@localhost)
217217
by hub.org (8.9.3/8.9.3) with SMTP id WAA89135;
218218
Fri, 24 Dec 1999 22:11:12 -0500 (EST)
@@ -486,7 +486,7 @@ From owner-pgsql-hackers@hub.org Sun Dec 26 08:31:09 1999
486486
Received: from renoir.op.net (root@renoir.op.net [207.29.195.4])
487487
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id JAA17976
488488
for <pgman@candle.pha.pa.us>; Sun, 26 Dec 1999 09:31:07 -0500 (EST)
489-
Received: from hub.org (hub.org [216.126.84.1]) by renoir.op.net (o1/$Revision: 1.5 $) with ESMTP id JAA23337 for <pgman@candle.pha.pa.us>; Sun, 26 Dec 1999 09:28:36 -0500 (EST)
489+
Received: from hub.org (hub.org [216.126.84.1]) by renoir.op.net (o1/$Revision: 1.6 $) with ESMTP id JAA23337 for <pgman@candle.pha.pa.us>; Sun, 26 Dec 1999 09:28:36 -0500 (EST)
490490
Received: from localhost (majordom@localhost)
491491
by hub.org (8.9.3/8.9.3) with SMTP id JAA90738;
492492
Sun, 26 Dec 1999 09:21:58 -0500 (EST)
@@ -909,7 +909,7 @@ From owner-pgsql-hackers@hub.org Thu Dec 30 08:01:09 1999
909909
Received: from renoir.op.net (root@renoir.op.net [207.29.195.4])
910910
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id JAA10317
911911
for <pgman@candle.pha.pa.us>; Thu, 30 Dec 1999 09:01:08 -0500 (EST)
912-
Received: from hub.org (hub.org [216.126.84.1]) by renoir.op.net (o1/$Revision: 1.5 $) with ESMTP id IAA02365 for <pgman@candle.pha.pa.us>; Thu, 30 Dec 1999 08:37:10 -0500 (EST)
912+
Received: from hub.org (hub.org [216.126.84.1]) by renoir.op.net (o1/$Revision: 1.6 $) with ESMTP id IAA02365 for <pgman@candle.pha.pa.us>; Thu, 30 Dec 1999 08:37:10 -0500 (EST)
913913
Received: from localhost (majordom@localhost)
914914
by hub.org (8.9.3/8.9.3) with SMTP id IAA87902;
915915
Thu, 30 Dec 1999 08:34:22 -0500 (EST)
@@ -1006,7 +1006,7 @@ From owner-pgsql-patches@hub.org Sun Jan 2 23:01:38 2000
10061006
Received: from renoir.op.net (root@renoir.op.net [207.29.195.4])
10071007
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id AAA16274
10081008
for <pgman@candle.pha.pa.us>; Mon, 3 Jan 2000 00:01:28 -0500 (EST)
1009-
Received: from hub.org (hub.org [216.126.84.1]) by renoir.op.net (o1/$Revision: 1.5 $) with ESMTP id XAA02655 for <pgman@candle.pha.pa.us>; Sun, 2 Jan 2000 23:45:55 -0500 (EST)
1009+
Received: from hub.org (hub.org [216.126.84.1]) by renoir.op.net (o1/$Revision: 1.6 $) with ESMTP id XAA02655 for <pgman@candle.pha.pa.us>; Sun, 2 Jan 2000 23:45:55 -0500 (EST)
10101010
Received: from hub.org (hub.org [216.126.84.1])
10111011
by hub.org (8.9.3/8.9.3) with ESMTP id XAA13828;
10121012
Sun, 2 Jan 2000 23:40:47 -0500 (EST)
@@ -1424,7 +1424,7 @@ From owner-pgsql-hackers@hub.org Tue Jan 4 10:31:01 2000
14241424
Received: from renoir.op.net (root@renoir.op.net [207.29.195.4])
14251425
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id LAA17522
14261426
for <pgman@candle.pha.pa.us>; Tue, 4 Jan 2000 11:31:00 -0500 (EST)
1427-
Received: from hub.org (hub.org [216.126.84.1]) by renoir.op.net (o1/$Revision: 1.5 $) with ESMTP id LAA01541 for <pgman@candle.pha.pa.us>; Tue, 4 Jan 2000 11:27:30 -0500 (EST)
1427+
Received: from hub.org (hub.org [216.126.84.1]) by renoir.op.net (o1/$Revision: 1.6 $) with ESMTP id LAA01541 for <pgman@candle.pha.pa.us>; Tue, 4 Jan 2000 11:27:30 -0500 (EST)
14281428
Received: from localhost (majordom@localhost)
14291429
by hub.org (8.9.3/8.9.3) with SMTP id LAA09992;
14301430
Tue, 4 Jan 2000 11:18:07 -0500 (EST)
@@ -1728,3 +1728,180 @@ Betreff: [HACKERS] failing over with postgresql
17281728
>
17291729

17301730

1731+
From pgsql-hackers-owner+M3662@postgresql.org Tue Jan 23 16:23:34 2001
1732+
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
1733+
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id QAA04456
1734+
for <pgman@candle.pha.pa.us>; Tue, 23 Jan 2001 16:23:34 -0500 (EST)
1735+
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
1736+
by mail.postgresql.org (8.11.1/8.11.1) with SMTP id f0NLKf004705;
1737+
Tue, 23 Jan 2001 16:20:41 -0500 (EST)
1738+
(envelope-from pgsql-hackers-owner+M3662@postgresql.org)
1739+
Received: from sectorbase2.sectorbase.com ([208.48.122.131])
1740+
by mail.postgresql.org (8.11.1/8.11.1) with SMTP id f0NLAe003753
1741+
for <pgsql-hackers@postgresql.org>; Tue, 23 Jan 2001 16:10:40 -0500 (EST)
1742+
(envelope-from vmikheev@SECTORBASE.COM)
1743+
Received: by sectorbase2.sectorbase.com with Internet Mail Service (5.5.2653.19)
1744+
id <DG1W4Q8F>; Tue, 23 Jan 2001 12:49:07 -0800
1745+
Message-ID: <8F4C99C66D04D4118F580090272A7A234D32AF@sectorbase1.sectorbase.com>
1746+
From: "Mikheev, Vadim" <vmikheev@SECTORBASE.COM>
1747+
To: "'dom@idealx.com'" <dom@idealx.com>, pgsql-hackers@postgresql.org
1748+
Subject: RE: [HACKERS] Re: AW: Re: MySQL and BerkleyDB (fwd)
1749+
Date: Tue, 23 Jan 2001 13:10:34 -0800
1750+
MIME-Version: 1.0
1751+
X-Mailer: Internet Mail Service (5.5.2653.19)
1752+
Content-Type: text/plain;
1753+
charset="iso-8859-1"
1754+
Precedence: bulk
1755+
Sender: pgsql-hackers-owner@postgresql.org
1756+
Status: ORr
1757+
1758+
> I had thought that the pre-commit information could be stored in an
1759+
> auxiliary table by the middleware program ; we would then have
1760+
> to re-implement some sort of higher-level WAL (I thought of the list
1761+
> of the commands performed in the current transaction, with a sequence
1762+
> number for each of them that would guarantee correct ordering between
1763+
> concurrent transactions in case of a REDO). But I fear I am missing
1764+
1765+
This wouldn't work for READ COMMITTED isolation level.
1766+
But why do you want to log commands into WAL where each modification
1767+
is already logged in, hm, correct order?
1768+
Well, it has sense if you're looking for async replication but
1769+
you need not in two-phase commit for this and should aware about
1770+
problems with READ COMMITTED isolevel.
1771+
1772+
Back to two-phase commit - it's easiest part of work required for
1773+
distributed transaction processing.
1774+
Currently we place single commit record to log and transaction is
1775+
committed when this record (and so all other transaction records)
1776+
is on disk.
1777+
Two-phase commit:
1778+
1779+
1. For 1st phase we'll place into log "prepared-to-commit" record
1780+
and this phase will be accomplished after record is flushed on disk.
1781+
At this point transaction may be committed at any time because of
1782+
all its modifications are logged. But it still may be rolled back
1783+
if this phase failed on other sites of distributed system.
1784+
1785+
2. When all sites are prepared to commit we'll place "committed"
1786+
record into log. No need to flush it because of in the event of
1787+
crash for all "prepared" transactions recoverer will have to
1788+
communicate other sites to know their statuses anyway.
1789+
1790+
That's all! It is really hard to implement distributed lock- and
1791+
communication- managers but there is no problem with logging two
1792+
records instead of one. Period.
1793+
1794+
Vadim
1795+
1796+
From pgsql-hackers-owner+M3665@postgresql.org Tue Jan 23 17:05:26 2001
1797+
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
1798+
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id RAA05972
1799+
for <pgman@candle.pha.pa.us>; Tue, 23 Jan 2001 17:05:24 -0500 (EST)
1800+
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
1801+
by mail.postgresql.org (8.11.1/8.11.1) with SMTP id f0NM31008120;
1802+
Tue, 23 Jan 2001 17:03:01 -0500 (EST)
1803+
(envelope-from pgsql-hackers-owner+M3665@postgresql.org)
1804+
Received: from candle.pha.pa.us (candle.navpoint.com [162.33.245.46])
1805+
by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id f0NLsU007188
1806+
for <pgsql-hackers@postgresql.org>; Tue, 23 Jan 2001 16:54:30 -0500 (EST)
1807+
(envelope-from pgman@candle.pha.pa.us)
1808+
Received: (from pgman@localhost)
1809+
by candle.pha.pa.us (8.9.0/8.9.0) id QAA05300;
1810+
Tue, 23 Jan 2001 16:53:53 -0500 (EST)
1811+
From: Bruce Momjian <pgman@candle.pha.pa.us>
1812+
Message-Id: <200101232153.QAA05300@candle.pha.pa.us>
1813+
Subject: Re: [HACKERS] Re: AW: Re: MySQL and BerkleyDB (fwd)
1814+
In-Reply-To: <8F4C99C66D04D4118F580090272A7A234D32AF@sectorbase1.sectorbase.com>
1815+
"from Mikheev, Vadim at Jan 23, 2001 01:10:34 pm"
1816+
To: "Mikheev, Vadim" <vmikheev@SECTORBASE.COM>
1817+
Date: Tue, 23 Jan 2001 16:53:53 -0500 (EST)
1818+
CC: "'dom@idealx.com'" <dom@idealx.com>, pgsql-hackers@postgresql.org
1819+
X-Mailer: ELM [version 2.4ME+ PL77 (25)]
1820+
MIME-Version: 1.0
1821+
Content-Transfer-Encoding: 7bit
1822+
Content-Type: text/plain; charset=US-ASCII
1823+
Precedence: bulk
1824+
Sender: pgsql-hackers-owner@postgresql.org
1825+
Status: OR
1826+
1827+
[ Charset ISO-8859-1 unsupported, converting... ]
1828+
> > I had thought that the pre-commit information could be stored in an
1829+
> > auxiliary table by the middleware program ; we would then have
1830+
> > to re-implement some sort of higher-level WAL (I thought of the list
1831+
> > of the commands performed in the current transaction, with a sequence
1832+
> > number for each of them that would guarantee correct ordering between
1833+
> > concurrent transactions in case of a REDO). But I fear I am missing
1834+
>
1835+
> This wouldn't work for READ COMMITTED isolation level.
1836+
> But why do you want to log commands into WAL where each modification
1837+
> is already logged in, hm, correct order?
1838+
> Well, it has sense if you're looking for async replication but
1839+
> you need not in two-phase commit for this and should aware about
1840+
> problems with READ COMMITTED isolevel.
1841+
>
1842+
1843+
I believe the issue here is that while SERIALIZABLE ISOLATION means all
1844+
queries can be run serially, our default is READ COMMITTED, meaning that
1845+
open transactions see committed transactions, even if the transaction
1846+
committed after our transaction started. (FYI, see my chapter on
1847+
transactions for help, http://www.postgresql.org/docs/awbook.html.)
1848+
1849+
To do higher-level WAL, you would have to record not only the queries,
1850+
but the other queries that were committed at the start of each command
1851+
in your transaction.
1852+
1853+
Ideally, you could number every commit by its XID your log, and then
1854+
when processing the query, pass the "committed" transaction ids that
1855+
were visible at the time each command began.
1856+
1857+
In other words, you can replay the queries in transaction commit order,
1858+
except that you have to have some transactions committed at specific
1859+
points while other transactions are open, i.e.:
1860+
1861+
XID Open XIDS Query
1862+
500 UPDATE t SET col = 3;
1863+
501 500 BEGIN;
1864+
501 500 UPDATE t SET col = 4;
1865+
501 UPDATE t SET col = 5;
1866+
501 COMMIT;
1867+
1868+
This is a silly example, but it shows that 500 must commit after the
1869+
first command in transaction 501, but before the second command in the
1870+
transaction. This is because UPDATE t SET col = 5 actually sees the
1871+
changes made by transaction 500 in READ COMMITTED isolation level.
1872+
1873+
I am not advocating this. I think WAL is a better choice. I just
1874+
wanted to outline how replaying the queries in commit order is
1875+
insufficient.
1876+
1877+
> Back to two-phase commit - it's easiest part of work required for
1878+
> distributed transaction processing.
1879+
> Currently we place single commit record to log and transaction is
1880+
> committed when this record (and so all other transaction records)
1881+
> is on disk.
1882+
> Two-phase commit:
1883+
>
1884+
> 1. For 1st phase we'll place into log "prepared-to-commit" record
1885+
> and this phase will be accomplished after record is flushed on disk.
1886+
> At this point transaction may be committed at any time because of
1887+
> all its modifications are logged. But it still may be rolled back
1888+
> if this phase failed on other sites of distributed system.
1889+
>
1890+
> 2. When all sites are prepared to commit we'll place "committed"
1891+
> record into log. No need to flush it because of in the event of
1892+
> crash for all "prepared" transactions recoverer will have to
1893+
> communicate other sites to know their statuses anyway.
1894+
>
1895+
> That's all! It is really hard to implement distributed lock- and
1896+
> communication- managers but there is no problem with logging two
1897+
> records instead of one. Period.
1898+
1899+
Great.
1900+
1901+
1902+
--
1903+
Bruce Momjian | http://candle.pha.pa.us
1904+
pgman@candle.pha.pa.us | (610) 853-3000
1905+
+ If your life is a hard drive, | 830 Blythe Avenue
1906+
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026
1907+

0 commit comments

Comments
 (0)