Minor doc fixes, pt_cleanup().

Author: arssher

readme.md

@@ -172,7 +172,7 @@ on shardlord with usual `CREATE TABLE` DDL and execute
 `shardman.create_hash_partitions` function, for example
 
 ```plpgsql
-CRATE TABLE horns (id int, branchness int);
+CRATE TABLE horns (id int primary key, branchness int);
 select create_hash_partitions('horns', 'id', 30, redundancy = 1)
 ```
 
@@ -181,7 +181,7 @@ On successfull execution, table `horns` will be hash-sharded into 30 partitions
 replica will be spawn and added to logical replication channel between nodes
 holding primary and replica.
 
-After table was sharded, each `pg_shardman` node can execute any DML statement
+After table was sharded, each `pg_shardman` worker node can execute any DML statement
 involving it. You are free to issue queries involving data from more than one
 shard, remote or local. It works using standard Postgres inheritance mechanism:
 all partitions are derived from parent table. If a partition is located at some
@@ -194,6 +194,9 @@ on all nodes in parallel: foreign data wrappers do not support parallel scan
 because of using cursors. Execution of OLAP queries at `pg_shardman` might not
 be that efficient. `pg_shardman` is oriented mainly on OLTP workload.
 
+Shardlord doesn't hold any data and can't read it, this node is used only for
+managing the cluster.
+
 Presently internal Postgres function is used for hashing; there is no easy way
 for client app to determine the node for particular record to perform local
 reads/writes.
@@ -249,6 +252,10 @@ We don't track replication mode for each table separately, and changing
 `shardman.sync_replication` GUC during cluster operation might lead to a mess.
 It is better to set it permanently for now.
 
+`pg_shardman` currently doesn't bothers itself with configuring replication
+identities. It is strongly recommended to use primary key as sharding key to
+avoid problems with `UPDATE` and `DELETE` operations.
+
 ### Balancing the load
 
 Newly added nodes are initially empty. To bring some sense into their existence,
@@ -388,8 +395,11 @@ also create new indices. It must be done per each partition separately,
 see
 [pathman docs](https://github.com/postgrespro/pg_pathman/wiki/How-do-I-create-indexes)
 
+`monitor` function on shardlord can continiously track failed nodes and resolve
+distributed deadlocks, see the reference.
+
 If shardlord has failed during command execution or you just feel that something
-goes wrong, run `shadman.recover()` function. It will verify nodes state
+goes wrong, run `shardman.recover()` function. It will verify nodes state
 against current shardlord's metadata and try to fix up things, i.e. reconfigure
 LR channels, repair FDW, etc.
 
@@ -505,6 +515,9 @@ more than one copy of the partition per node. Distributed table is always
 created empty, it doesn't matter had the original table on shardlord had any
 data or not.
 
+Column(s) participating in `expr` must be marked `NOT NULL`, as in
+`pg_pathman`. Generally we strongly recommend to use primary key there.
+
 ```plpgsql
 rm_table(rel_name name)
 ```
@@ -640,7 +653,7 @@ Data is not touched by this command.
 ## Some limitations:
   * You should not touch `synchronous_standby_names` manually while using pg_shardman.
   * The shardlord itself can't be worker node for now.
-  * All [limitations of `pg_pathman`](https://github.com/postgrespro/pg_pathman/wiki/Known-limitations),
+  * All [limitations](https://github.com/postgrespro/pg_pathman/wiki/Known-limitations)  (and some features) of `pg_pathman`,
 	e.g. we don't support global primary keys and foreign keys to sharded tables.
   * All [limitations of logical replications](https://www.postgresql.org/docs/10/static/logical-replication-restrictions.html).
     `TRUNCATE` statements on sharded tables will not be replicated.

tests/python/tests.py

@@ -249,6 +249,11 @@ def pt_replicas_integrity(self, required_replicas=None):
                     DBNAME, sum_query(part_name))
                 self.assertEqual(part_sum, replica_sum)
 
+    def pt_cleanup(self):
+        self.lord.safe_psql(DBNAME, "select shardman.rm_table('pt')")
+        self.lord.safe_psql(DBNAME, "drop table pt;")
+        self.lord.destroy_cluster();
+
     # tests
 
     def test_add_node(self):
@@ -361,9 +366,7 @@ def test_set_redundancy(self):
         # must be exactly two replicas for each partition
         self.pt_replicas_integrity(required_replicas=2)
 
-        self.lord.safe_psql(DBNAME, "select shardman.rm_table('pt')")
-        self.lord.safe_psql(DBNAME, "drop table pt;")
-        self.lord.destroy_cluster()
+        self.pt_cleanup()
 
     def test_rebalance(self):
         self.lord.create_cluster(2, 2)
@@ -413,9 +416,7 @@ def test_rebalance(self):
         self.pt_everyone_sees_the_whole()
         self.pt_replicas_integrity()
 
-        self.lord.safe_psql(DBNAME, "select shardman.rm_table('pt')")
-        self.lord.safe_psql(DBNAME, "drop table pt;")
-        self.lord.destroy_cluster()
+        self.pt_cleanup()
 
     def test_deadlock_detector(self):
         self.lord.create_cluster(2)
@@ -486,7 +487,7 @@ def xact_2():
         try:
             with self.lord.connect() as con:
                 con.execute("set statement_timeout = '3s'")
-                con.execute("select shardman.monitor(deadlock_check_timeout_sec => 1)")
+                con.execute("select shardman.monitor(check_timeout_sec => 1)")
         except:
             pass
         t1.join(5)
@@ -495,12 +496,11 @@ def xact_2():
         self.assertTrue(not t2.is_alive())
         self.assertTrue(xact_1_aborted or xact_2_aborted)
 
-        self.lord.safe_psql(DBNAME, "select shardman.rm_table('pt')")
-        self.lord.safe_psql(DBNAME, "drop table pt;")
-        self.lord.destroy_cluster();
+        self.pt_cleanup()
 
+    # WIP
     def test_worker_failover(self):
-        self.lord.create_cluster(3, 2)
+        self.lord.create_cluster(2, 2)
         self.lord.safe_psql(
             DBNAME, 'create table pt(id int primary key, payload int default 1);')
         self.lord.safe_psql(