EDB Postgres Distributed 6.0.0 release notes v6.0.0

New built-in connection manager which handles routing of connections automatically and allows enforcing of read-only connections to non-leader.

The PGD CLI now allows initial cluster setup as well as adding nodes from command-line using pgd node setup command.

Transform the sequences in distributed based on the bdr.default_sequence_kind GUC when creating/joining a bdr group instead of when creating the node as done in older versions.

Set the startvalue for galloc sequences to the following valid number after the last used by the local sequence. With this change, when creating distributed sequences and specifically galloc, there is no need to adjust the startvalue based on what might be already used.

Link to a detailed google doc is provided below

The node_uuid uniquely identifies instance of a node of a given name. Random node_uuid is generated when node is created and remains constant for the lifetime of the node. The node_id column is now derived from node_uuid instead of node name.

For the time being a node needs to be fully parted before before node of the same name can be rejoined, this may be relaxed in future releases to permit rejoin as soon as part_node process for the old instance has commenced and before it completed.

For the time being upgrades from older PGD versions and mixed-version operation in clusters with older PGD nodes are not supported. This limitation will be addressed in future releases.

Replication origin and slot names now use node uuid and thus correspond to particular incarnation of a node of a given name. Similarly node group uuid is used instead of group name. Hash of database name is used in lieu of database name.

Please note that origin and node names should be treated as opaque identifiers from user's perspective, one shouldn't rely on the structure of these names nor expect these to be particularly meaningful to a human operator.

The new naming scheme is as follows:

Slots Naming Convention

• normal slot to a node => bdr_node_<targetuuid>_<dbhash>
• join slot for node => bdr_node_<targetuuid>_<dbhash>_tmp
• group slot for a topgroup => bdr_group_<topgroupuuid>_<dbhash>
• slot for any forwarding + lead to lead => bdr_node_<targetuuid>_<originidhex>_<dbhash>
• analytics slot => bdr_analytics_<groupuuid>_<dbhash>
• decoding slot => bdr_decoder_<topgroupuuid>_<dbhash>

Origins Naming Convention:

• normal origin to a node => bdr_<originuuid>_<dbhash>
• fwd origin to a source node => bdr_<originuuid>_<sourceoidhex>_<dbhash>

Ensure that no more than three node groups (one top group and two subgroups) can exist at any given time. If the limit is exceeded, an error is raised.

Don't allow PGD Essential clusters to join more than 4 data nodes.

bdr.wait_node_confirm_lsn() will look at the confirmed_flush_lsn of the given node when available, otherwise it will query pg_replication_origin_progress() of that node, and wait for the specified LSN to be reached by said node.

So far subscriber-only nodes could only be added to node groups of type "subscriber-only". But now a subscriber-only node can be added to a data node group too. Only node_kind='subscriber_only' needs to be specified while doing a create_node as before. the join_node_group can be done on a data node group.

Returns name of analytics slot. This merely produces the correct name irrespective of whether analytics feature is in use.

Added to complement the addition of the node_uuid column to bdr.node and bdr.local_node

Both pk_exists and multiple_unique_conflicts conflict types can now resolve more than one conflicting row by removing any old rows that are part of the conflict. The multiple_unique_conflicts now defaults to update_if_newer resolver, so it does not throw error by default anymore.

The backend_type now shows consistent worker type for PGD workers without the extra process identification. The wait_event_type and wait_event include more wait events now, instead of showing "extension" for some events. Also, connection management related columns are added to show real client address/port and whether the session is read-only.

From PGD 6.0.0, bdr.part_node functionality is enhanced to remove the parted node’s metadata automatically from all nodes in the cluster.

• For local node, it will remove all the node metadata, including information about remote nodes.
• For remote node, it removes only metadata for that specific node. Hence with this release
• A node will remain in PART_CLEANUP state till group slots of all nodes are caught up to all the transactions originating from the PARTED node
• A node will not remain in PARTED state as the node is removed as soon as it moves to PARTED state.

Add the --summary and --options flags to pgd node show command to filter the output of the pgd node show command. This also maintains symmetry with other show commands.

Add the bdr.lock_table_locking and bdr.truncate_locking GUCs to list of GUCs verfied in pgd cluster verify command.

Changing a column's type command which causes the whole table to be rewritten and the change isn't binary coercible is now supported:

CREATE TABLE foo (c1 int,c2 int, c3 int, c4 box, UNIQUE(c1, c2) INCLUDE(c3,c4)); ALTER TABLE foo ALTER c1 TYPE bigint; – results into table rewrite

This also includes support for ALTER TYPE when using the USING clause:

CREATE TABLE foo (id serial primary key,data text); ALTER TABLE foo ALTER data TYPE BYTEA USING data::bytea;

Table rewrites can hold an AccessExclusiveLock for extended periods on larger tables.

The restrictions on non-immutable ALTER TABLE... ADD COLUMN calls have been removed.

The new bdr.node_group_config_summary view contains detailed information about group options, including effective value, source of the effective value, default value, whether the value can be inherited, etc. This is in similar spirit to pg_settings

New lock type leader DML lock is used by default for locking DDL statements that need to block DML. This lock locks on write-leaders only, no requiring all nodes to participate in the locking operation. Old behavior can be restored by adjusting bdr.ddl_locking configuration parameter.

Functions run_on_nodes, run_on_all_nodes and run_on_group now sets bdr.xact_replication to off by default.

The auto value for bdr.default_replica_identity changed to REPLICA IDENTITY FULL. This setting prevents some edge cases in conflict detection between inserts, updates and deletes across node crashes and recovery.

When the PGD group is created and the database of the initial PGD node is not empty (i.e. has some tables with data) the REPLICA IDENTITY of all tables will be set according to bdr.default_replica_identity.

Tablespace operations CREATE/ALTER/DROP TABLESPACE are now replicated as a DDL operation. Where users are running a configuration with multiple nodes on the same machine, you will need to enable the developer option allow_in_place_tablespace.

Improve the formating of the log messages to be more readable and symmetrical with Postgres log messages.

Add the recommended_value column to the result of the pgd cluster verify --settings command. The column will not be displayed in tabular output but will be displayed in JSON output.

The output for the commands with tabular output will be sorted by the resource name. The commands that display more than one resource, the output will be sorted by each resource column in order.

Remove the old (PGD 5 and below) CLI commands, which were deprecated but supported for backward compatibility.

Previously, non-data nodes would attempt to handle, but not process commit scope logic, which could lead to confusing, albeit harmless log messages.

This provides greater visibility into the node cloning process and assists with debugging possible issues.

Routing (and by extension raft) is now enabled by default on data-groups (subgroups with data nodes).

As it is not possible to roll back a group join, it can not form part of an idempotent transaction.

pause_in_standby has been deprecated since PGD 5.0.0. Logical standby nodes should be specified as such when executing bdr.create_node()

Unlogged BDR sequences may display unexpected behaviour following a server crash. Existing unlogged BDR sequences may be converted to logged ones.

The pgd cluster show command failed with error for clock drift if only one node is up and running in a N node cluster. The command is fixed to return valid output for other components viz., health and summary while reporting a valid error for clock-drift.

The pgd node show command crashed if a non-existent node is specified to the command. The command is fixed to fail gracefully with appropriate error message.

When parting a given node, that same node may have subscriptions whose origin was already parted and the group dropped. Previously this would break PGD, and has since been fixed.

The num_writers option, used in bdr.alter_node_group_option() and bdr.alter_node_group_config() should be positive or -1.