Consistency of the Neighbor-Net Algorithm

Abstract

Background

Neighbor-Net is a novel method for phylogenetic analysis that is currently being widely used in areas such as virology, bacteriology, and plant evolution. Given an input distance matrix, Neighbor-Net produces a phylogenetic network, a generalization of an evolutionary or phylogenetic tree which allows the graphical representation of conflicting phylogenetic signals.

Results

In general, any network construction method should not depict more conflict than is found in the data, and, when the data is fitted well by a tree, the method should return a network that is close to this tree. In this paper we provide a formal proof that Neighbor-Net satisfies both of these requirements so that, in particular, Neighbor-Net is statistically consistent on circular distances.

1 Background

Phylogenetics is concerned with the construction and analysis of evolutionary or phylogenetic trees and networks to understand the evolution of species, populations and individuals [1]. Neighbor-Net is a phylogenetic analysis and data representation method introduced in [2]. It is loosely based on the popular Neighbor-Joining (NJ) method of Saitou and Nei [3], but with one fundamental difference: whereas NJ constructs phylogenetic trees, Neighbor-Net constructs phylogenetic networks. The method is widely used, in areas such as virology [4], bacteriology [5], plant evolution [6] and even linguistics [7].

Evolutionary processes such as hybridization between species, lateral transfer of genes, recombination within a population, and convergent evolution can all lead to evolutionary histories that are distinctly non tree-like. Moreover, even when the underlying evolution is tree-like, the presence of conflicting or ambiguous signal can make a single tree representation inappropriate. In these situations, phylogenetic network methods can be particularly useful (see e.g. [8]).

Phylogenetic networks are a generalization of phylogenetic trees (see Figure 1 for a typical example of a phylogenetic network). In case there are many conflicting phylogenetic signals supported by the data, Neighbor-Net can represent this conflict graphically. In particular a single network can represent several trees simultaneously, indicate whether or not the data is substantially tree-like, and give evidence for possible reticulation or hybridization events. Evolutionary hypotheses suggested by the network can be tested directly using more detailed phylogenetic analyses and specialized biochemical methods (e.g. DNA fingerprinting or chromosome painting).

Figure 1

A phylogenetic network. The network was generated by Neighbor-Net for a sequence-based data set comprising of Salmonella isolates that originally appeared in [17]. A detailed network-based analysis of this data is presented in [2], where the strains indicated in bold-face are tested for the presence of recombination. Note that the network is planar (that is, it can be drawn in the plane without any crossing edges), and that parallel edges in the network represent bipartitions of the data.

For any network construction method, it is vital that the network does not depict more conflict than is found in the data and that, if there are conflicting signals, then these should be represented by the network. At the same time, when the data is fitted well by a tree, the method should return a network that is close to being a tree. This is essential not just to avoid false inferences, but for the application of networks in statistical tests of the extent to which the data is tree-like [9].

In this paper we provide a proof that these properties all hold for Neighbor-Net. Formally, we prove that if the input to NeighborNet is a circular distance function (distance matrix) [10], then the method returns a network that exactly represents the distance. Circular distance functions are more general than additive (patristic) distances on trees and, thus, as a corollary, if Neighbor-Net is given an additive distance it will return the corresponding tree. In this sense, Neighbor-Net is a statistically consistent method.

The paper is structured as follows: In Section 2 we introduce some basic notation, and in Section 3 we review the Neighbor-Net algorithm. In Section 4 we prove that Neighbor-Net is consistent (Theorem 4.1).

2 Preliminaries

In this section we present some notation that will be needed to describe the Neighbor-Net algorithm. We will assume some basic facts concerning phylogenetic trees, more details concerning which may be found in [11].

Throughout this paper, X will denote a finite set with cardinality n. A split S = {A, B} (of X) is a bipartition of X. We let Ϭ = Ϭ(X) = {{A, X\A}|∅ ⊂ A ⊂ X} denote the set of all splits of X, and call any non-empty subset of Ϭ(X) a split system. A split weight function on X is a map ω: Ϭ(X) → ℝ_≥0. We let Ϭ _ω denote the set {S ∈ Ϭ|ω(S) > 0}, the support of ω.

Let Θ = x₁, ..., x _n be an ordering of X. A split S = {A, B} is compatible with Θ if there exist i, j ∈ {1, ..., n}, i ≤ j, such that A = {x _i , ..., x _j } or B = {x _i , ..., x _j }. Note that if a split is compatible with an ordering Θ it is also compatible with its reversal x _n , ..., x₂, x₁ and with ordering x₂, ..., x _n , x₁. We let Ϭ_Θ denote the set of those splits in Ϭ(X) which are compatible with ordering Θ. A split system Ϭ' is compatible with Θ if Ϭ' ⊆ Ϭ_Θ. In addition a split system Ϭ' ⊆ Ϭ(X) is circular if there exists an ordering Θ of X such that Ϭ' is compatible with Θ. Note that any split system corresponding to a phylogenetic tree is circular [[11], Ch. 3], and so circular split systems can be regarded as a generalization of split systems induced by phylogenetic trees. A split weight function ω is called circular if the split system Ϭ _ω is circular. A distance function on X is a map d: X × X → ℝ_≥0 such that for all x, y ∈ X both d(x, x) = 0 and d(x, y) = d(y, x) hold. Note that any split weight function ω on X induces a distance function d _ω on X as follows: For a split S = {A, B} ∈ Ϭ(X) define the distance function or split metric d _S by

$$d_S(x,y)=\{\begin{array}{ll}0\hfill & \text{if }\{x,y\}\subseteq A\text{ or }\{x,y\}\subseteq B\hfill \\ 1\hfill & \text{otherwise},\hfill \end{array}$$

and put

$$d_{\omega }(x,y)={\displaystyle \sum _{S\in \mathfrak{S}(X)}\omega (S)d_S(x,y)}$$

for all x, y ∈ X. A distance function d is called circular if there exits a circular split weight function ω such that d = d _ω . An ordering Θ of X is said to be compatible with d if there exists ω such that d = d _ω and Ϭ _ω ⊆ Ϭ_Θ. Note that the representation of a circular distance function d is unique, i.e., if d = $d_{{\omega }_1}$ and d = $d_{{\omega }_2}$ for circular split weight functions ω₁ and ω₂ then ω₁ = ω₂ holds [10].

Circular distances were introduced in [10] and have been further studied in, for example, [12] and [13]. Just as any tree-like distance function on X can be uniquely represented by a phylogenetic tree [[11], ch. 7], any circular distance function d can be represented by a planar phylogenetic network such as the one pictured in Figure 1[14]. The program SplitsTree [9] allows the automatic generation of such a network for d by computing a circular split weight function ω with d = d _ω .

3 Description of the Neighbor-Net algorithm

In this section we present a detailed description of the Neighbor-Net algorithm, as implemented in the current version of SplitsTree [9]. The Neighbor-Net algorithm was originally described in [2], where the reader may find a more informal description for how it works. For the convenience of the reader we will use the same notation as in [2] where possible.

In Figure 2 we present pseudo-code for the Neighbor-Net algorithm. The aim of the algorithm is, for a given input distance function d, to compute a circular split weight function ω so that the distance function d _ω gives a good approximation to d. The resulting distance function d _ω can then be represented by a planar phylogenetic network as indicated in the last section.

Figure 2

The Neighbor-Net algorithm. Pseudo-code for the Neighbor-Net algorithm detailing the procedure FINDORDERING.

To this end, NEIGHBOR-NET first computes an ordering Θ of X, and then applies a non-negative least-squares procedure to find a best fit for d within the set of distance functions {d _ϕ |ϕ:Ϭ(X) → ℝ_≥0, Ϭ _ϕ ⊆ Ϭ_Θ}. More details concerning the least-squares procedure may be found in [2]: Here we will concentrate on the description of the key computation for finding an ordering Θ of X, which is detailed in the procedure FINDORDERING.

An (ordered) cluster is a non-empty finite set C together with an ordering Θ _C = c₁, ..., c _k of the elements in C, k = |C|. Two elements a, b ∈ C are called neighbors if there exists i ∈ {1, ..., k - 1} such that a = c _i and b = c_i+1, or b = c _i and a = c_i+1. The input of the procedure FINDORDERING consists of a set $ℭ$ of mutually disjoint clusters, together with a distance function d on the set $Y={\cup }_{C\in ℭ}C$. The ordering Θ = y₁, ..., y _n of Y that is returned by FINDORDERING must be compatible with the collection $ℭ$ of ordered clusters, that is, for every cluster C ∈ $ℭ$ there must exist i, j ∈ {1, ..., n}, i ≤ j, with the property that Θ _C = y _i , ..., y _j or Θ _C = y _j , ..., y _i .

The procedure FINDORDERING calls itself recursively. Apart from the base case (line 5 of Figure 2), where the recursion bottoms out, two different cases are considered – the reduction and selection cases (lines 7–15 and lines 17–22 of Figure 2, respectively). In the reduction case a cluster C ∈ $ℭ$ with k = |C| ≥ 3 is replaced by a smaller cluster C'. In particular, in lines 7–11 we let Θ _C = c₁, ..., c _k be the ordering of C with c₁ = x, c₂ = y, c₃ = z, and put C' = (C\{x, y, z}) ∪ {u, v} and Θ_C'= u, v, c₄, ..., c _k , where u and v are two new elements not contained in Y. Then, in lines 12–14, we define a distance function d' on the set Y' = (Y\{x, y, z}) ∪ {u, v} using the formulae:

$$\begin{array}{ll}{d}^{\prime }(a,b)=d(a,b)\hfill & \text{for }\{a,b\}\subseteq Y^{\prime }\backslash \{u,v\}\hfill \\ d^{\prime }(u,a)=(\alpha +\beta )d(x,a)+\gamma d(y,a)\hfill & \text{for }a\in Y^{\prime }\backslash \{u,v\}\hfill \\ d^{\prime }(v,a)=\alpha d(y,a)+(\beta +\gamma )d(z,a)\hfill & \text{for }a\in Y^{\prime }\backslash \{u,v\}\hfill \\ d^{\prime }(u,v)=\alpha d(x,y)+\beta d(x,z)+\gamma d(y,z)\hfill \end{array}$$

(1)

where α, β and γ are positive real numbers satisfying α + β + γ = 1 (note that these formulae slightly differ from the ones given in [2] in which there is a typographical error). In the current implementation of Neighbor-Net the values α = β = γ = 1/3 are used.

When FINDORDERING is recursively called with the new collection ${ℭ}^{\prime }$ of clusters and distance function d' it returns an ordering ${\Theta }^{\prime }={y^{\prime }}_1,\mathrm{...},{y^{\prime }}_{n-1}$ of Y' that is compatible with ${ℭ}^{\prime }$. Thus, there exists i ∈ {1, ..., n - 2} such that either u = ${y^{\prime }}_i$ and v = ${y^{\prime }}_{i+1}$ or v = ${y^{\prime }}_i$ and u = ${y^{\prime }}_{i+1}$. The resulting ordering Θ of Y is then defined (in line 14) as follows:

$$\Theta =\{\begin{array}{ll}{y^{\prime }}_1,\mathrm{...},{y^{\prime }}_{i-1},x,y,z,{y^{\prime }}_{i+2},\mathrm{...},{y^{\prime }}_{n-1}\hfill & \text{if }u={y^{\prime }}_i\text{ and }v={y^{\prime }}_{i+1}\hfill \\ {y^{\prime }}_1,\mathrm{...},{y^{\prime }}_{i-1},z,y,x,{y^{\prime }}_{i+2},\mathrm{...},{y^{\prime }}_{n-1}\hfill & \text{if }u={y^{\prime }}_{i+1}\text{ and }v={y^{\prime }}_i.\hfill \end{array}$$

(2)

This completes the description of the reduction case.

We now describe the selection case. Note that in view of line 6 this case only applies if every cluster in $ℭ$ contains at most two elements. In lines 17–18, two clusters C₁, C₂ ∈ $ℭ$ are selected and replaced by the single cluster C' = C₁ ∪ C₂. The clusters C₁ and C₂ are selected as follows: We define a distance function $\overline{d}$ on the set of clusters $ℭ$ by

$$\overline{d}(A,B)=\{\begin{array}{ll}0\hfill & \text{if }A=B\hfill \\ \frac{1}{\left|A\right|\left|B\right|}{\displaystyle {\sum }_{a\in A}{\displaystyle {\sum }_{b\in B}d(a,b)}}\hfill & \text{if }A\ne B,\hfill \end{array}$$

and select C₁, C₂ ∈ $ℭ$, C₁ ≠ C₂ that minimize the quantity

$$Q(C_1,C_2)=(m-2)\overline{d}(C_1,C_2)-{\displaystyle \sum _{C\in ℭ\backslash \left\{C_1\right\}}\overline{d}(C_1,C)-{\displaystyle \sum _{C\in ℭ\backslash \left\{C_2\right\}}\overline{d}(C_2,C)}}$$

(3)

where m is the number of clusters in $ℭ$. The function Q that is used to select pairs of clusters is called the Q-criterion. Note that this is a direct generalization of the selection criterion used in the NJ algorithm [2]. However, using only this criterion yields a method that is not consistent as illustrated in Figure 3. So, once the clusters C₁ and C₂ have been selected we use a second criterion to determine an ordering Θ_C'in line 19 for the new cluster C'. In particular, for every x ∈ C₁ ∪ C₂ we define

Figure 3

A network representing a circular distance. A circular distance d on the set {u, v, ..., z} for which NeighborNet using only the Q-criterion employed in NJ to cluster elements would be inconsistent. Distances are given by shortest paths in the network. The pairs u, v and x, y would be clustered together first and then the pair z, w. However it is not hard to show that z and w are not adjacent in any ordering of {u, v, ..., z} that is compatible with d.

$$R(x)={\displaystyle \sum _{C\in ℭ\backslash \{C_1,C_2\}}\overline{d}(\{x\},C)}+{\displaystyle \sum _{y\in (C_1\cup C_2)\backslash \left\{x\right\}}d(x,y)},$$

put $\widehat{m}$ = m + |C₁| + |C₂| - 2, and select x₁ ∈ C₁ and x₂ ∈ C₂ that minimize the quantity

$$\widehat{Q}$$

[d](x₁, x₂) = (

$$\widehat{m}$$

- 2)d(x₁, x₂) - R(x₁) - R(x₂).

We then choose an ordering Θ_C'in which x₁ and x₂ are neighbors and for which every two elements that were neighbors in C₁ or C₂ remain neighbors. This completes the description of the selection case, and hence the description of the procedure FINDORDERING.

4 Neighbor-Net is consistent

In this section we prove the consistency of Neighbor-Net:

Theorem 4.1 If d: X × X → ℝ_≥0 is a circular distance function, then the output of the Neighbor-Net algorithm is a circular split weight function ω: Ϭ(X) → ℝ_≥0 with the property that d = d _ω .

The key part of the Neighbor-Net algorithm is the procedure FINDORDERING. We will show that, for a circular distance function d = d _ω on X, the call FINDORDERING({{x}|x ∈ X}, d) will produce an ordering Θ of X that is compatible with d. The non-negative least squares procedure finds the distance function in {d _ϕ |ϕ: Ϭ(X) → ℝ_≥0, Ϭ _ϕ ⊆ Ϭ_Θ} that is closest to d. As this set of distance functions includes d _ω , the least squares procedure returns exactly d = d _ω , proving the theorem.

We focus, then, on the proof that FINDORDERING behaves as required:

Theorem 4.2 Let d: Y × Y → ℝ_≥0 be a distance function that is induced by a circular split weight function ω: Ϭ(Y) → ℝ_≥0. In addition, let $ℭ$ be a collection of mutually disjoint clusters with the property that Y = $Y={\cup }_{C\in ℭ}C$, and assume there exists an ordering of Y that is compatible with ω and with $ℭ$. Then FINDORDERING($ℭ$, d) will compute an ordering that is compatible with the collection of clusters $ℭ$ and with the split weight function ω.

We present the proof of this result in the remainder of this section. Suppose that the algorithm FINDORDERING is called with input $ℭ$ and d and that there exists an ordering that is compatible with $ℭ$ and d. Let $Y={\cup }_{C\in ℭ}C$. We prove Theorem 4.2 by induction, first on |Y|, the cardinality of Y, and then on |$ℭ$|, the number of clusters in $ℭ$.

The base case of the induction is |Y| ≤ 3. In this case the set of splits Ϭ_Θ equals Ϭ(Y) for every ordering of Y. In particular, any ordering of Y that is compatible with $ℭ$ is also compatible with ω.

We now assume that |Y| > 3 and make the following induction hypothesis:

If there exists an ordering compatible with distance function d' and ordered clusters ${ℭ}^{\prime }$, where either |${\cup }_{C\in {ℭ}^{\prime }}C$| < |Y|, or |${\cup }_{C\in {ℭ}^{\prime }}C$| = |Y| and |${ℭ}^{\prime }$| < |$ℭ$|, then FINDORDERING(${ℭ}^{\prime }$, d') will return an ordering compatible with ${ℭ}^{\prime }$ and d'.

There are two cases to consider. In the first case, $ℭ$ contains some cluster C with |C| ≥ 3. In the second case, $ℭ$ contains only clusters C with |C| ≤ 2.

4.1 Case 1: The reduction case

Suppose that there is C ∈ $ℭ$ with |C| ≥ 3. This is the reduction case in the description of the algorithm. The procedure FINDORDERING constructs a new set of clusters ${ℭ}^{\prime }$ (in line 11) and a new distance function d' (in line 12). We first show that, if there is an ordering compatible with $ℭ$ and d, then there is also an ordering compatible with ${ℭ}^{\prime }$ and d'.

Proposition 4.3 If ${ℭ}^{\prime }$ and d' are constructed according to lines 7–12 of the procedure FINDORDERING then there exists an ordering compatible with ${ℭ}^{\prime }$ and d'.

Proof: Suppose that $\tilde{\Theta }$ = y₁, ..., y _n is an ordering of Y that is compatible with $ℭ$ and d, where, without loss of generality, we have Θ _C = y₁, ..., y _k . Let ${\tilde{\Theta }}^{\prime }$ = u, v, y₄, ..., y _n = z₁, ..., z_n-1, which is an ordering of Y' = ${\cup }_{C\in {ℭ}^{\prime }}C$. We claim that the ordering ${\tilde{\Theta }}^{\prime }$ is compatible with the collection ${ℭ}^{\prime }$ and with the distance function d'.

Since $ℭ$ is compatible with $\tilde{\Theta }$ it is straight-forward to check that ${ℭ}^{\prime }$ is compatible with ${\tilde{\Theta }}^{\prime }$. Hence, we only need to show that ${\tilde{\Theta }}^{\prime }$ is compatible with d'. We will use a 4-point condition that was first studied in a different context by Kalmanson [15] and has been shown to characterize circular distances in [12]. To be more precise, it suffices to show that, for every four elements $z_{i_1},z_{i_2},z_{i_3},z_{i_4}$, i₁ <i₂ <i₃ <i₄,

$$\begin{array}{l}{d}^{\prime }(z_{i_1},z_{i_3})+d^{\prime }(z_{i_2},z_{i_4})\ge d^{\prime }(z_{i_1},z_{i_2})+d^{\prime }(z_{i_3},z_{i_4})\text{ and}\hfill \\ d^{\prime }(z_{i_1},z_{i_3})+d^{\prime }(z_{i_2},z_{i_4})\ge d^{\prime }(z_{i_1},z_{i_4})+d^{\prime }(z_{i_2},z_{i_3}).\hfill \end{array}$$

Case 1: |{$z_{i_1},z_{i_2},z_{i_3},z_{i_4}$} ∩ {u, v}| = 0. The above inequalities follow immediately since d is circular, and d and d' as well as $\tilde{\Theta }$ and ${\tilde{\Theta }}^{\prime }$ coincide on Y'\{u, v}.

Case 2: |{$z_{i_1},z_{i_2},z_{i_3},z_{i_4}$} ∩ {u, v}| = 1. Consider the situation $z_{i_1}$ = u. Then

$$\begin{array}{l}{d}^{\prime }(z_{i_1},z_{i_3})+d^{\prime }(z_{i_2},z_{i_4})\hfill \\ =(\alpha +\beta )d(x,z_{i_3})+\gamma d(y,z_{i_3})+(\alpha +\beta +\gamma )d(z_{i_2},z_{i_4})\hfill \\ \ge (\alpha +\beta )d(x,z_{i_2})+\gamma d(y,z_{i_2})+(\alpha +\beta +\gamma )d(z_{i_3},z_{i_4})\hfill \\ =d^{\prime }(z_{i_1},z_{i_2})+d^{\prime }(z_{i_3},z_{i_4}).\hfill \end{array}$$

The other inequalities can be derived in a completely analogous way.

Case 3: |{$z_{i_1},z_{i_2},z_{i_3},z_{i_4}$} ∩ {u, v}| = 2. Then we have $z_{i_1}$ = u and $z_{i_2}$ = v and

$$\begin{array}{l}{d}^{\prime }(z_{i_1},z_{i_3})+d^{\prime }(z_{i_2},z_{i_4})\hfill \\ =(\alpha +\beta )d(x,z_{i_3})+\gamma d(y,z_{i_3})+\alpha d(y,z_{i_4})+(\beta +\gamma )d(z,z_{i_4})\hfill \\ \ge \alpha d(x,y)+\beta d(x,z)+\gamma d(y,z)+(\alpha +\beta +\gamma )d(z_{i_3},z_{i_4})\hfill \\ =d^{\prime }(z_{i_1},z_{i_2})+d^{\prime }(z_{i_3},z_{i_4}).\hfill \end{array}$$

The other inequality $d^{\prime }(z_{i_1},z_{i_3})+d^{\prime }(z_{i_2},z_{i_4})\ge d^{\prime }(z_{i_1},z_{i_4})+d^{\prime }(z_{i_2},z_{i_3})$ can be shown to hold in a similar way.   ■

The procedure FINDORDERING calls itself recursively with ${ℭ}^{\prime }$ and d' as input. An ordering of Y', the union of ${ℭ}^{\prime }$, is returned. By Proposition 4.3 and the induction hypothesis, this ordering Θ' is compatible with ${ℭ}^{\prime }$ and d'. It is used to construct an ordering Θ on Y, in line 14, which becomes the output of the procedure.

Proposition 4.4 The ordering Θ is compatible with collection $ℭ$ and with the distance function d.

Proof: Since ${ℭ}^{\prime }$ is compatible with Θ' it is straight-forward to check that $ℭ$ is compatible with Θ. Hence we only need to show that Θ is compatible with d.

Let orderings $\tilde{\Theta }$ = y₁, ..., y _n of Y and ${\tilde{\Theta }}^{\prime }$ = z₁, ..., z_n-1of Y' be as in the proof of Proposition 4.3 and let ω be the split weight function such that d = d _ω . Then $\tilde{\Theta }$ is compatible with all splits S such that ω(S) > 0. Now consider some split S = {A, B} such that ω(S) > 0 and assume that y _n ∈ B. Then there exists i, j ∈ {1, ..., n - 1}, i ≤ j, such that A = {y _i , ..., y _j }. Note also that, since the distance function d' is compatible with ordering ${\tilde{\Theta }}^{\prime }$ = z₁, ..., z_n-1of Y' and, hence, is circular, there exists a unique circular split weight function ω': Ϭ(Y') → ℝ_≥0 with the property that d' = d_ω'. We divide the remaining argument into five cases.

Case 1: j ≤ 3. Then, clearly, S is compatible with Θ.

Case 2: j ≥ 4 and i = 1. Define A' = {z₁, ..., z_j-1} and the split S' = {A', Y'\A'} of Y'. Then we can express ω'(S') in terms of d' as follows (cf. [12]):

$$\begin{array}{c}2{\omega }^{\prime }(S^{\prime })=d^{\prime }(z_1,z_j)+d^{\prime }(z_{j-1},z_{n-1})-d^{\prime }(z_1,z_{j-1})-d^{\prime }(z_j,z_{n-1})\\ =(\alpha +\beta )d(y_1,y_{j+1})+\gamma d(y_2,y_{j+1})+d(y_j,y_n)\\ -(\alpha +\beta )d(y_1,y_j)-\gamma d(y_2,y_j)-d(y_{j+1},y_n)\\ \ge (\alpha +\beta +\gamma )(d(y_1,y_{j+1})+d(y_j,y_n)-d(y_1,y_j)-d(y_{j+1},y_n))\\ =2\omega (S)\end{array}$$

Thus, ω'(S') > 0. Hence, the split S' is compatible with the ordering Θ' of Y'. But then the split S is compatible with the ordering Θ of Y.

Case 3: j ≥ 4 and 2 ≤ i ≤ 3. We only consider the situation when i = 2; the situation i = 3 is completely analogous. Define A' = {z₂, ..., z_j-1} and the split S' = {A', Y'\A'} of Y'. With a similar calculation as made for Case 2 we obtain ω'(S') ≥ (α + β)ω(S). Hence, ω'(S') > 0 and, thus, S' is compatible with Θ'. But then S is compatible with Θ.

Case 4: j ≥ 4 and i = 4. This case is similar to Case 2. Define A' = {z₄, ..., z_j-1} and S' = {A', Y'\A'}. We obtain ω'(S') ≥ ω(S). Hence, as for Case 2, ω'(S') > 0 and, thus, S is compatible with Θ.

Case 5: j ≥ i ≥ 5. Define the split S' = {A, Y'\A}. Then we have ω'(S') = ω'(S') > 0. Hence, S' is compatible with Θ' and, thus, S is compatible with Θ.   ■

4.2 Case 2: The selection case

Now suppose that there are no clusters C ∈ $ℭ$ with |C| ≥ 3. This is the selection case in the description of the algorithm.

In line 17 the algorithm selects two clusters that minimize (3):

$$Q(C_1,C_2)=(m-2)\overline{d}(C_1,C_2)-{\displaystyle \sum _{C\in ℭ\backslash \left\{C_1\right\}}\overline{d}(C_1,C)-{\displaystyle \sum _{C\in ℭ\backslash \left\{C_2\right\}}\overline{d}(C_2,C)}},$$

where

$$\overline{d}(A,B)=\{\begin{array}{ll}0\hfill & \text{if }A=B\hfill \\ \frac{1}{\left|A\right|\left|B\right|}{\displaystyle {\sum }_{a\in A}{\displaystyle {\sum }_{b\in B}d(a,b)}}\hfill & \text{if }A\ne B.\hfill \end{array}$$

Note that $\overline{d}$ is a distance function defined on the set of clusters $ℭ$. We will first show that $\overline{d}$ is circular. We do this in two steps: Proposition 4.5 and Proposition 4.6.

Proposition 4.5 Let d: M × M → ℝ_≥0 be a circular distance function and Θ = x₁, ..., x _n be an ordering of M that is compatible with d. Let M' = (M\{x₁, x₂}) ∪ {y} where y is a new element not contained in M. Define a distance function d': M' × M' → ℝ_≥0 as follows:

$$\begin{array}{ll}{d}^{\prime }(a,b)=d(a,b)\hfill & \text{for }\{a,b\}\subseteq M^{\prime }\backslash \left\{y\right\}\hfill \\ d^{\prime }(y,a)=\lambda d(x_1,a)+(1-\lambda )d(x_2,a)\hfill & \text{for }a\in M^{\prime }\backslash \left\{y\right\},\hfill \end{array}$$

where λ is a real number with the property that 0 <λ < 1. Then the following hold:

(i) d' is circular and compatible with ordering y, x₃, ..., x _n of M'.

(ii) If z₁, ..., z_n-1is an ordering of M' that is compatible with d' then at least one of the orderings x₁, x₂, z₂, ..., z_n-1or x₂, x₁, z₂, ..., z_n-1of M is compatible with d.

Proof: (i) and (ii) can be proven using convexity arguments, or in a way analogous to our proof of Propositions 4.3 and 4.4, respectively.   ■

Proposition 4.6 The distance function $\overline{d}$, defined on the individual clusters in $ℭ$, is a circular distance. Moreover, for every ordering D₁, ..., D _k of $ℭ$ that is compatible with $\overline{d}$ there exist orderings Θ _i of D _i , i ∈ {1, ..., k}, such that the ordering Θ₁, ..., Θ _k of Y is compatible with distance function d.

Proof: We use multiple applications of Proposition 4.5, once for each cluster in $ℭ$ with two elements, and with λ = $\frac{1}{2}$ in each case.   ■

We now have the more difficult task of showing that clusters C₁ and C₂ selected by the Q-criterion, that is by minimizing (3), are adjacent in at least one ordering of the clusters that is compatible with $\overline{d}$, as described in Proposition 4.6. This is the most technical part of the proof. The key step is the inequality established in Lemma 4.7. This is used to prove Theorem 4.8, which establishes that the Q-criterion when applied to a circular distance will always select a pair of elements that are adjacent in at least one ordering compatible with the circular distance. As a corollary it will follow that there exists an ordering of the clusters in $ℭ$ compatible with $\overline{d}$ where C₁ and C₂ are adjacent.

Lemma 4.7 Let Θ = x₁, x₂, ..., x _n be an ordering of M that is compatible with circular distance d on M and suppose that 3 ≤ r ≤ ⌈n/2⌉. Let S = {A, M\A} be a split compatible with Θ where A = {x _i , ..., x _j }. Define Q _S : M × M → ℝ by

$$Q_S(x_i,x_j)=(n-2)d_S(x_i,x_j)-{\displaystyle \sum _{k=1}^n{d}_S(x_i,x_k)}-{\displaystyle \sum _{k=1}^n{d}_S(x_j,x_k)}$$

and let

$$\lambda (S)={\displaystyle \sum _{l=1}^{r-1}{Q}_S(x_l,x_{l+1})-(r-1)Q_S(x_1,x_r)}.$$

• (i) If min{|A|, |M\A|} > 1 and |A ∩ {x₁, x _r }| = 1 then λ(S) < 0.

• (ii) Any other split S compatible with Θ satisfies λ(S) ≤ 0.

Proof: Expanding λ(S) gives

$$\begin{array}{ll}\lambda (S)=(n-2){\displaystyle \sum _{l=1}^{r-1}{d}_S(x_l,x_{l+1})}\hfill & -(r-1)(n-2)d_S(x_1,x_r)\hfill \\ +(r-2){\displaystyle \sum _{i=1}^n{d}_S(x_1,x_l)}\hfill & -2{\displaystyle \sum _{l=2}^{r-1}{\displaystyle \sum _{k=1}^n{d}_S(x_l,x_k)}}\hfill \\ +(r-2){\displaystyle \sum _{l=1}^n{d}_S(x_r,x_l)}.\hfill \end{array}$$

We divide the rest of our argument into five cases which are summarized in Table 1. For these cases straight-forward calculations yield the entries of Table 2. Using Table 2 we compute λ(S) in each case.

Table 1 List of cases in the proof of Lemma 4.7

Table 2 Precomputed expressions used in the proof of Lemma 4.7

Case (i): We obtain λ(S) = 2(j - 1)(j + 1 - r) + 2(j - 1)(j + 1 - n). Hence, λ(S) = 0 if j = 1 and λ(S) < 0 if j ≥ 2.

Case (ii): We obtain λ(S) = 0.

Case (iii): We obtain λ(S) = (j - i)(4(j - i) - 2n + 8). Thus, since j - i ≤ r - 3 ≤ (n + 1)/2 - 3, λ(S) = 0 if i = j and λ(S) < 0 if i <j.

Case (iv): We obtain λ(S) = 2(i - r)(n - 2 - (j - i)) + 2(2 - i)(j - i). Thus, since j - i ≤ n - 3, λ(S) < 0 if i <r. If i = r then λ(S) = 0 if j = r and λ(S) < 0 otherwise.

Case (v): We obtain λ(S) = 0.   ■

Theorem 4.8 Let M be a set of n elements and d: M × M → ℝ_≥0 be a circular distance function. Suppose that x, y minimize

$$Q(x,y)=(n-2)d(x,y)-{\displaystyle \sum _{z\in M}d(x,z)}-{\displaystyle \sum _{z\in M}d(y,z)}.$$

Then there is an ordering of M that is compatible with d in which x and y are adjacent.

Proof: Let Θ = x₁, ..., x _n be an ordering of M that is compatible with d. Suppose that Q(x₁, x _r ) ≤ Q(x, y) for all x, y where, without loss of generality, 2 ≤ r ≤⌈n/2⌉. If r = 2 then we are done, so we assume r ≥ 3. Let ω be the (circular) split weight function for which d = d _ω , so Θ is compatible with ω. Let Θ* be the ordering obtained by removing x _r from Θ and re-inserting it immediately after x₁. We claim that Θ* is also compatible with ω.

As in Lemma 4.7, for any split S compatible with Θ we define

$$\lambda (S)={\displaystyle \sum _{l=1}^{r-1}{Q}_S(x_l,x_{l+1})-(r-1)Q_S(x_1,x_r).}$$

By the choice of x₁ and x _r we have

$$(r-1)Q(x_1,x_r)\le {\displaystyle \sum _{l=1}^{r-1}Q(x_l,x_{l+1})}.$$

Since Q is linear, and d = Σ_S∈Ϭ(X)ω(S)d _S by Lemma 4.7 we have

$$\begin{array}{c}0\le {\displaystyle \sum _{l=1}^{r-1}Q(x_l,x_{l+1})-(r-1)Q(x_1,x_r)}\\ ={\displaystyle \sum _S\omega (S)\left({\displaystyle \sum _{l=1}^{r-1}{Q}_S(x_l,x_{l+1})-(r-1)Q_S(x_1,x_r)}\right)}\\ ={\displaystyle \sum _S\omega (S)\lambda (S)\le 0}.\end{array}$$

Now consider any split S compatible with Θ but not Θ*. Then S satisfies the conditions in Lemma 4.7 (i), giving λ(S) < 0 and hence ω(S) = 0. Thus there are no splits in the support of ω that are not compatible with Θ*, and Θ* is compatible with ω and, hence, d. Thus x₁ and x _r are adjacent in an ordering Θ* compatible with d.   ■

Corollary 4.9 Let C₁ and C₂ be the two clusters selected in line 17 of procedure FINDORDERING. Then there exists an ordering Θ* = D₁, ..., D _k of $ℭ$ such that D₁ = C₁, D₂ = C₂ and $\overline{d}$ is compatible with Θ*.

After selecting C₁ and C₂ the procedure FINDORDERING removes these clusters from the collection and replaces them with their union C' = C₁ ∪ C₂. It also assigns an ordering Θ_C'to the cluster.

FINDORDERING is then called recursively. The following is directly analogous to Proposition 4.3.

Proposition 4.10 There exists an ordering of Y that is compatible with collection ${ℭ}^{\prime }$ and split weight function ω.

Proof: We already know by Proposition 4.9 and Proposition 4.6 that there exists an ordering $\tilde{\Theta }$ = y₁, ..., y _n of Y that is compatible with $ℭ$ and ω and, in addition, also satisfies one of the following properties:

$$\begin{array}{ll}{C}_1=\left\{y_1\right\}\text{ and }{C}_2=\left\{y_2\right\}\hfill & C_1=\left\{y_1\right\}\text{ and }{C}_2=\{y_2,y_3\}\hfill \\ C_1=\{y_1,y_2\}\text{ and }{C}_2=\left\{y_3\right\}\hfill & C_1=\{y_1,y_2\}\text{ and }{C}_2\{y_3,y_4\}.\hfill \end{array}$$

If x₁ ∈ C₁ and x₂ ∈ C₂ are selected such that $\tilde{\Theta }$ is also compatible with ${ℭ}^{\prime }$ then we are done. Otherwise we have to construct a suitable new ordering ${\tilde{\Theta }}^{\prime }$ of Y. There are, up to symmetric situations with roles of C₁ and C₂ swapped, only two cases we need to consider.

Case 1: C₁ = {y₁, y₂}, x₁ = y₁ and x₂ = y₃. We want to show that ordering ${\tilde{\Theta }}^{\prime }$ = y₂, y₁, y₃, ..., y _n is compatible with ω. To this end we first show that $\widehat{Q}$[d](y₂, y₃) ≤ $\widehat{Q}$[d](y₁, y₃). It suffices to establish this inequality for all split metrics d _S with S ∈ ${\mathfrak{S}}_{\tilde{\Theta }}$. Define the set of splits

Ϭ' = {{{y₂, ..., y _i }, Y\{y₂, ..., y _i }}|3 ≤ i ≤ n - 1}.

By a case analysis similar to the one applied in the proof of Lemma 4.7 we obtain the following:

• $\widehat{Q}$[d _S ](y₂, y₃) = $\widehat{Q}$[d _S ](y₁, y₃) if S ∈ ${\mathfrak{S}}_{\widehat{\Theta }}$\Ϭ', and

• $\widehat{Q}$[d _S ](y₂, y₃) <$\widehat{Q}$[d _S ](y₁, y₃) if S ∈ Ϭ'.

But then, since $\widehat{Q}$[d](y₁, y₃) is minimum, $\widehat{Q}$[d](y₂, y₃) = $\widehat{Q}$[d](y₁, y₃). Thus, by the above strict inequality, for every split S ∈ Ϭ' we must have ω(S) = 0. Hence, ω is compatible with ${\tilde{\Theta }}^{\prime }$.

Case 2: C₁ = {y₁, y₂}, C₂ = {y₃, y₄}, x₁ = y₁, x₂ = y₄ and n ≥ 5. We want to show that ${\tilde{\Theta }}^{\prime }$ = y₂, y₁, y₄, y₃, y₅, ..., y _n is compatible with ω. A similar argument to the one used in Case 1 shows that for every split S in

Ϭ' = {{{y₂, ..., y _i }, Y\{y₂, ..., y _i }}|3 ≤ i ≤ n - 1} ∪ {{{y₄, ..., y _i }, Y\{y₂, ..., y _i }}|5 ≤ i ≤ n}

we must have ω(S) = 0. Thus, ω is compatible with ${\tilde{\Theta }}^{\prime }$.   ■

Now, by Proposition 4.10, we can apply the induction hypothesis and conclude that the recursive call FINDORDERING(${ℭ}^{\prime }$, d) will return an ordering Θ compatible with ${ℭ}^{\prime }$ and d. Since Θ will order C' according to Θ_C'(or its reverse), we have that Θ is compatible with C₁ and C₂. Thus Θ is compatible with $ℭ$ and d, completing the proof of Theorem 4.2.   □

Remark 4.11 Note that we have shown that Corollary 4.9 holds under the assumption that (in view of line 6) every cluster in $ℭ$ contains at most two elements. However, it is possible to prove this result in the more general setting where clusters can have arbitrary size. In principle, this could yield a consistent variation of the Neighbor-Net algorithm that is analogous to the recently introduced QNet algorithm [16], where, instead of reducing the size of clusters when they have more than two elements, the reduction case is skipped entirely and clusters are pairwise combined until only one cluster is left. However, we suspect that such a method would probably not work well in practice since the reduced distances have smaller variance than the original distances.