Get rid of global_tx.c in_table implementation.

It was invented to deal with recovery-resolving deadlocks possible only on (at
it seems) on 5+ nodes like
- 1 prepares T1 on 1, 2, 3
- 4 prepares conflicting T2 on 4, 5
- Everyone fails, and only 2, 3, 5 go up. They can't recover
  without resolving at least one of xacts first because T1 and T2 conflict; and
  'my orphaned xact which never got PRECOMMITted can be aborted directly'
  doesn't help here as no T1 neigher T2 authored by live nodes.

To break out of this cycle, we could participate in voting normally, persisting
xact as in_table before even getting PREPARE. However, it was very cumbersome
and ugly; the process of in_table -> PREPARE state migration was especially
baroque. Implementing accurately batch wal scan would also bring a lot of pain.

There is a better idea: use generations to directly abort one of the
xacts above. The connectivity clique forming majority will eventually elect its
generation in which one of the nodes will be donor, i.e. will be immediately
MTM_GEN_ONLINE. Any prepare which it doesn't have is subject to direct ABORT; it
is the fundamental property of generations that node which is online in
generation n has all committable PREPAREs of all <n gens.

Author: arssher

multimaster--1.0.sql

@@ -204,17 +204,6 @@ CREATE FUNCTION mtm.pg_lsn_to_bigint(lsn pg_lsn) RETURNS bigint AS $$
 $$
 LANGUAGE sql;
 
-CREATE TABLE mtm.gtx_proposals(
-    gid text primary key not null,
-    -- defines the horizon after which record is not needed and can be deleted
-    -- gtx_proposals contains only neighbours xacts, so this is truly origin_lsn
-    -- origin node (coordinator) is encoded in gid
-    prepare_origin_lsn pg_lsn,
-    state text not null
-);
-/* proposals tables are private to nodes, don't stream them  */
-INSERT INTO mtm.local_tables VALUES('mtm', 'gtx_proposals');
-
 CREATE OR REPLACE FUNCTION mtm.alter_sequences() RETURNS boolean AS
 $$
 DECLARE

src/commit.c

@@ -143,8 +143,7 @@ mtm_commit_cleanup(int status, Datum arg)
 		Assert(GetMyGlobalTx() != NULL || on_exit);
 		if (GetMyGlobalTx() == NULL)
 			mtm_commit_state.gtx = GlobalTxAcquire(mtm_commit_state.gid,
-												   false,
-												   NULL);
+												   false);
 		/*
 		 * If we managed to prepare the xact, tell resolver to deal with it
 		 */
@@ -445,7 +444,7 @@ MtmTwoPhaseCommit(void)
 
 		/* prepare transaction on our node */
 
-		mtm_commit_state.gtx = GlobalTxAcquire(mtm_commit_state.gid, true, NULL);
+		mtm_commit_state.gtx = GlobalTxAcquire(mtm_commit_state.gid, true);
 		Assert(mtm_commit_state.gtx->state.status == GTXInvalid);
 		/*
 		 * PREPARE doesn't happen here; ret 0 just means we were already in
@@ -463,11 +462,10 @@ MtmTwoPhaseCommit(void)
 		ReleasePB(); /* don't hold generation switch anymore */
 		mtm_commit_state.gtx->prepared = true;
 		/* end_lsn of PREPARE */
-		mtm_commit_state.gtx->coordinator_end_lsn = XactLastCommitEnd;
 		mtm_log(MtmTxFinish, "TXFINISH: %s prepared at %X/%X",
 				mtm_commit_state.gid,
-				(uint32) (mtm_commit_state.gtx->coordinator_end_lsn >> 32),
-				(uint32) mtm_commit_state.gtx->coordinator_end_lsn);
+				(uint32) (XactLastCommitEnd >> 32),
+				(uint32) (XactLastCommitEnd));
 
 		/*
 		 * By definition of generations, we must collect PREPARE ack from