A Lightweight Method for Tackling Unknown Participation Statistics in Federated Averaging

We consider the problem of finding $\mathbf{x}\in\mathbb{R}^{d}$ that minimizes the distributed finite-sum objective:

where each individual (local) objective $F_{n}(\mathbf{x})$ is only computable at the client $n$. This problem often arises in the context of federated learning (FL) (Kairouz et al., 2021, Li et al., 2020a, Yang et al., 2019), where $F_{n}(\mathbf{x})$ is defined on client $n$’s local dataset, $f(\mathbf{x})$ is the global objective, and $\mathbf{x}$ is the parameter vector of the model being trained. Each client keeps its local dataset to itself, which is not shared with other clients or the server. It is possible to extend (1) to weighted average with positive coefficients multiplied to each $F_{n}(\mathbf{x})$, but for simplicity, we consider such coefficients to be included in $\{F_{n}(\mathbf{x}):\forall n\}$ (see Appendix A.1) and do not write them out.

Federated averaging (FedAvg) is a commonly used algorithm for minimizing (1), which alternates between local updates at each client and parameter aggregation among multiple clients with the help of a server (McMahan et al., 2017). However, there are several challenges in FedAvg, including data heterogeneity and partial participation of clients, which can cause performance degradation and even non-convergence if the FedAvg algorithm is improperly configured.

Unknown, Uncontrollable, and Heterogeneous Participation of Clients. Most existing works on FL with partial client participation assume that the clients participate according to a known or controllable random process (Karimireddy et al., 2020, Yang et al., 2021, Chen et al., 2022, Fraboni et al., 2021a, Li et al., 2020b; c). In practice, however, it is common for clients to have heterogeneous and time-varying computation power and network bandwidth, which depend on both the inherent characteristics of each client and other tasks that concurrently run in the system. This generally leads to heterogeneous participation statistics across clients, which are difficult to know a priori due to their complex dependency on various factors in the system (Wang et al., 2021). It is also generally impossible to fully control the participation statistics, due to the randomness of whether a client can successfully complete a round of model updates (Bonawitz et al., 2019).

The problem of having heterogeneous and unknown participation statistics is that it may cause the result of FL to be biased towards certain local objectives, which diverges from the optimum of the original objective in (1). In FL, data heterogeneity across clients is a common phenomenon, resulting in diverse local objectives $\{F_{n}(\mathbf{x})\}$. The participation heterogeneity is often correlated with data heterogeneity, because the characteristics of different user populations may be correlated with how powerful their devices are. Intuitively, when some clients participate more frequently than others, the final FL result will be benefiting the local objectives of those frequently participating clients, causing a possible discrimination for clients that participate less frequently.

A few recent works aiming at addressing this problem are based on the idea of global variance reduction by saving the most recent updates of all the clients, which requires a substantial amount of additional memory in the order of $Nd$, i.e., the total number of clients times the dimension of the model parameter vector (Yang et al., 2022, Yan et al., 2020, Gu et al., 2021, Jhunjhunwala et al., 2022). This additional memory consumption is either incurred at the server or evenly distributed to all the clients. For practical FL systems with many clients, this causes unnecessary memory usage that affects the overall capability and performance of the system. Therefore, we ask the following important question in this paper:

Is there a lightweight method that provably minimizes the original objective in (1), when the participation statistics of clients are unknown, uncontrollable, and heterogeneous?

We leverage the insight that we can apply different weights to different clients’ updates in the parameter aggregation stage of FedAvg. If this is done properly, the effect of heterogeneous participation can be canceled out so that we can minimize (1), as shown in existing works that assume known participation statistics (Chen et al., 2022, Fraboni et al., 2021a, Li et al., 2020b; c). However, in our setting, we do not know the participation statistics a priori, which makes it challenging to compute (estimate) the optimal aggregation weights. It is also non-trivial to quantify the impact of estimation error on convergence.

Our Contributions. We thoroughly analyze this problem and make the following novel contributions.

1. 1.

   To motivate the need for adaptive weighting in parameter aggregation, we show that FedAvg with non-optimal weights minimizes a different objective (defined in (2)) instead of (1).

2. 2.

   We propose a lightweight procedure for estimating the optimal aggregation weight at each client $n$ as part of the overall FL process, based on client $n$’s participation history. We name this new algorithm FedAU, which stands for FedAvg with adaptive weighting to support unknown participation statistics.

3. 3.

   We analyze the convergence upper bound of FedAU, using a novel method that first obtains a weight error term in the convergence bound and then further bounds the weight error term via a bias-variance decomposition approach. Our result shows that FedAU converges to an optimal solution of the original objective (1). In addition, a desirable linear speedup of convergence with respect to the number of clients is achieved when the number of FL rounds is large enough.

4. 4.

   We verify the advantage of FedAU in experiments with several datasets and baselines, with a variety of participation patterns including those that are independent, Markovian, and cyclic.

Related Work. Earlier works on FedAvg considered the convergence analysis with full client participation (Gorbunov et al., 2021, Haddadpour et al., 2019, Lin et al., 2020, Stich, 2019, Wang & Joshi, 2019; 2021, Yu et al., 2019, Malinovsky et al., 2023), which do not capture the fact that only a subset of clients participates in each round in practical FL systems. Recently, partial client participation has came to attention. Some works analyzed the convergence of FedAvg where the statistics or patterns of client participation are known or controllable (Fraboni et al., 2021a; b, Li et al., 2020c, Yang et al., 2021, Wang & Ji, 2022, Cho et al., 2023, Karimireddy et al., 2020, Li et al., 2020b, Chen et al., 2022, Rizk et al., 2022). However, as pointed out by Wang et al. (2021), Bonawitz et al. (2019), the participation of clients in FL can have complex dependencies on the underlying system characteristics, which makes it difficult to know or control each client’s behavior a priori. A recent work analyzed the convergence for a re-weighted objective (Patel et al., 2022), where the re-weighting is essentially arbitrary for unknown participation distributions. Some recent works (Yang et al., 2022, Yan et al., 2020, Gu et al., 2021, Jhunjhunwala et al., 2022) aimed at addressing this problem using variance reduction, by including the most recent local update of each client in the global update, even if they do not participate in the current round. These methods require a substantial amount of additional memory to store the clients’ local updates. In contrast, our work focuses on developing a lightweight algorithm that has virtually the same memory requirement as the standard FedAvg algorithm.

A related area is adaptive FL algorithms, where adaptive gradients (Reddi et al., 2021, Wang et al., 2022b; c) and adaptive local updates (Ruan et al., 2021, Wang et al., 2020) were studied. Some recent works viewed the adaptation of aggregation weights from different perspectives (Wu & Wang, 2021, Tan et al., 2022, Wang et al., 2022a), which do not address the problem of unknown participation statistics. All these methods are orthogonal to our work and can potentially work together with our algorithm. To the best of our knowledge, no prior work has studied weight adaptation in the presence of unknown participation statistics with provable convergence guarantees.

A uniqueness in our problem is that the statistics related to participation need to be collected across multiple FL rounds. Although Wang & Ji (2022) aimed at extracting a participation-specific term in the convergence bound, that approach still requires the aggregation weights in each round to sum to one (thus coordinated participation); it also requires an amplification procedure over multiple rounds for the bound to hold, making it difficult to tune the hyperparameters. In contrast, this paper considers uncontrolled and uncoordinated participation without sophisticated amplification mechanisms.

We begin by describing a generic FedAvg algorithm that includes a separate oracle for computing the aggregation weights, as shown in Algorithm 1. In this algorithm, there are a total of $T$ rounds, where each round $t$ includes $I$ steps of local stochastic gradient descent (SGD) at a participating client. For simplicity, we consider $I$ to be the same for all the clients, while noting that our algorithm and results can be extended to more general cases. We use $\gamma>0$ and $\eta>0$ to denote the local and global step sizes, respectively. The variable $\mathbf{x}_{0}$ is the initial model parameter, ${\rm I\kern-1.99997ptl}_{t}^{n}$ is an identity function that is equal to one if client $n$ participates in round $t$ and zero otherwise, and $\mathbf{g}_{n}(\cdot)$ is the stochastic gradient of the local objective $F_{n}(\cdot)$ for each client $n$.

The main steps of Algorithm 1 are similar to those of standard FedAvg, but with a few notable items as follows. 1) In Line 1, we clearly state that we do not have prior knowledge of the sampling process of client participation. 2) Line 1 calls a separate oracle to compute the aggregation weight $\omega_{t}^{n}$ ($\omega_{t}^{n}>0$) for client $n$ in round $t$. This computation is done on each client $n$ alone, without coordinating with other clients. We do not need to save the full sequence of participation record $\{{\rm I\kern-1.99997ptl}_{\tau}^{n}:\tau<t\}$, because it is sufficient to save an aggregated metric of the participation record for weight computation. In Section 3, we will see that we use the average participation interval for weight computation in FedAU, where the average can be computed in an online manner. We also note that we do not include ${\rm I\kern-1.99997ptl}_{t}^{n}$ in the current round $t$ for computing the weight, which is needed for the convergence analysis so that $\omega_{t}^{n}$ is independent of the local parameter $\mathbf{y}^{n}_{t,i}$ when the initial parameter of round $t$ (i.e., $\mathbf{x}_{t}$) is given. 3) The parameter aggregation is weighted by $\omega_{t}^{n}$ for each client $n$ in Line 1.

Objective Inconsistency with Improper Aggregation Weights. We first show that without weight adaptation, FedAvg minimizes an alternative objective that is generally different from (1).

A formal version of the theorem is given in Appendix B.4. Theorem 1 shows that, even in the special case where each client $n$ participates according to a Bernoulli distribution with probability $p_{n}$, choosing a constant aggregation weight such as $\omega_{n}=1,\forall n$ as in standard FedAvg causes the algorithm to converge to a different objective that is weighted by $p_{n}$. As mentioned earlier, this implicit weighting discriminates clients that participate less frequently. In addition, since the participation statistics (here, the probabilities $\{p_{n}\}$) of clients are unknown, the exact objective being minimized is also unknown, and it is generally unreasonable to minimize an unknown objective. This means that it is important to design an adaptive method to find the aggregation weights, so that we can minimize (1) even when the participation statistics are unknown, which is our focus in this paper.

The full proofs of all mathematical claims are in Appendix B.

In this section, we describe the computation of aggregation weights $\{\omega_{t}^{n}\}$ based on the participation history observed at each client, which is the core of our FedAU algorithm that extends FedAvg. Our goal is to choose $\{\omega_{t}^{n}\}$ to minimize the original objective (1) as close as possible.

Intuition. We build from the intuition in Theorem 1 and design an aggregation weight adaptation algorithm that works for general participation patterns, i.e., not limited to the Bernoulli distribution considered in Theorem 1. From (2), we see that if we can choose $\omega_{n}=\nicefrac{{1}}{{p_{n}}}$, the objective being minimized is the same as (1). We note that $p_{n}\approx\frac{1}{T}\sum_{t=0}^{T-1}{\rm I\kern-1.99997ptl}_{t}^{n}$ for each client $n$ when $T$ is large, due to ergodicity of the Bernoulli distribution considered in Theorem 1. Extending to general participation patterns that are not limited to the Bernoulli distribution, intuitively, we would like to choose the weight $\omega_{n}$ to be inversely proportional to the average frequency of participation. In this way, the bias caused by lower participation frequency is “canceled out” by the higher weight used in aggregation. Based on this intuition, our goal of aggregation weight estimation is as follows.

Some previous works have discovered this need of debiasing the skewness of client participation (Li et al., 2020c, Perazzone et al., 2022) or designing the client sampling scheme to ensure that the updates are unbiased (Fraboni et al., 2021a, Li et al., 2020b). However, in our work, we consider the more realistic case where the participation statistics are unknown, uncontrollable, and heterogeneous. In this case, we are unable to directly find the optimal aggregation weights because we do not know the participation statistics a priori.

Technical Challenge. If we were to know the participation pattern for all the $T$ rounds, an immediate solution to Problem 1 is to choose $\omega_{t}^{n}$ (for each client $n$) to be equal to $T$ divided by the number of rounds where client $n$ participates. We can see that this solution is equal to the average interval between every two adjacent participating rounds, assuming that the first interval starts right before the first round $t=0$. However, since we do not know the future participation pattern or statistics in each round $t$, we cannot directly apply this solution. In other words, in every round $t$, we need to perform an online estimation of the weight $\omega_{t}^{n}$ based on the participation history up to round $t-1$.

A challenge in this online setting is that the estimation accuracy is related to the number of times each client $n$ has participated until round $t-1$. When $t$ is small and client $n$ has not yet participated in any of the preceding rounds, we do not have any information about how to choose $\omega_{t}^{n}$. For an intermediate value of $t$ where client $n$ has participated only in a few rounds, we have limited information about the choice of $\omega_{t}^{n}$. In this case, if we directly use the average participation interval up to the $(t-1)$-th round, the resulting $\omega_{t}^{n}$ can be far from its optimal value, i.e., the estimation has a high variance if the client participation follows a random process. This is problematic especially when there exists a long interval between two rounds (both before the $(t-1)$-th round) where the client participates. Although the probability of the occurrence of such a long interval is usually low, when it occurs, it results in a long average interval for the first $t$ rounds when $t$ is relatively small, and using this long average interval as the value of $\omega_{t}^{n}$ may cause instability to the training process.

Key Idea. To overcome this challenge, we define a positive integer $K$ as a “cutoff” interval length. If a client has not participated for $K$ rounds, we consider $K$ to be a participation interval that we sample and start a new interval thereafter. In this way, we can limit the length of each interval by adjusting $K$. By setting $\omega_{t}^{n}$ to be the average of this possibly cutoff participation interval, we overcome the aforementioned challenge. From a theoretical perspective, we note that $\omega_{t}^{n}$ will be a biased estimation when $K<\infty$ and the bias will be larger when $K$ is smaller. In contrast, a smaller $K$ leads to a smaller variance of $\omega_{t}^{n}$, because we collect more samples in the computation of $\omega_{t}^{n}$ with a smaller $K$. Therefore, an insight here is that $K$ controls the bias-variance tradeoff¹¹1Note that we focus on the aggregation weights here, which is different from classical concept of the bias-variance tradeoff of the model. of the aggregation weight $\omega_{t}^{n}$. In Section 4, we will formally show this property and obtain desirable convergence properties of the weight error term and the overall objective function (1), by properly choosing $K$ in the theoretical analysis. Our experimental results in Section 5 also confirm that choosing an appropriate value of $K<\infty$ improves the performance in most cases.

Online Algorithm. Based on the above insight, we describe the procedure of computing the aggregation weights $\{\omega_{t}^{n}\}$, as shown in Algorithm 2. The computation is independent for each client $n$. In this algorithm, the variable $M_{n}$ denotes the number of (possibly cutoff) participation intervals that have been collected, and $S_{n}^{\diamond}$ denotes the the length of the last interval that is being computed. We compute the interval by incrementing $S_{n}^{\diamond}$ by one in every round, until the condition in Line 2 holds. When this condition holds, $S_{n}=S_{n}^{\diamond}$ is the actual length of the latest participation interval with possible cutoff. As explained above, we always start a new interval when $S_{n}^{\diamond}$ reaches $K$. Also note that we consider ${\rm I\kern-1.99997ptl}_{t-1}^{n}$ instead of ${\rm I\kern-1.99997ptl}_{t}^{n}$ in this condition and start the loop from $t=1$ in Line 2, to align with the requirement in Algorithm 1 that the weights are computed from the participation records before (not including) the current round $t$. For $t=0$, we always use $\omega_{t}^{n}=1$. In Line 2, we compute the weight using an online averaging method, which is equivalent to averaging over all the participation intervals that have been observed until each round $t$. With this method, we do not need to save all the previous participation intervals. Essentially, the computation in each round $t$ only requires three state variables that are scalars, including $M_{n}$, $S_{n}^{\diamond}$, and the previous round’s weight $\omega_{t-1}^{n}$. This makes this algorithm extremely memory efficient.

In the full FedAU algorithm, we plug in the result of $\omega_{n}^{t}$ for each round $t$ obtained from Algorithm 2 into Line 1 of Algorithm 1. In other words, ComputeWeight in Algorithm 1 calls one step of update that includes Lines 2–2 of Algorithm 2.

Compatibility with Privacy-Preserving Mechanisms. In our FedAU algorithm, the aggregation weight computation (Algorithm 2) is done individually at each client, which only uses the client’s participation states and does not use the training dataset or the model. When using these aggregation weights as part of FedAvg in Algorithm 1, the weight $\omega_{t}^{n}$ can be multiplied with the parameter update $\Delta_{t}^{n}$ at each client $n$ (and in each round $t$) before the update is transmitted to the server. In this way, methods such as secure aggregation (Bonawitz et al., 2017) can be applied directly, since the server only needs to compute a sum of the participating clients’ updates. Differentially private FedAvg methods (McMahan et al., 2018, Andrew et al., 2021) can be applied in a similar way.

Practical Implementation. We will see from our experimental results in Section 5 that a coarsely chosen value of $K$ gives a reasonably good performance in practice, which means that we do not need to fine-tune $K$. There are also other engineering tweaks that can be made in practice, such as using an exponentially weighted average in Line 2 of Algorithm 2 to put more emphasis on the recent participation characteristics of clients. In an extreme case where each client participates only once, a possible solution is to group clients that have similar computation power (e.g., same brand/model of devices) and are in similar geographical locations together. They may share the same state variables $M_{n}$, $S_{n}^{\diamond}$, and $\omega_{t-1}^{n}$ used for weight computation in Algorithm 2. We note that according to the lower bound derived by Yang et al. (2022), if each client participates only once, it is impossible to have an algorithm to converge to the original objective without sharing additional information.

Assumptions 1–3 are commonly used in the literature for the convergence analysis of FL algorithms (Yang et al., 2021, Wang & Ji, 2022, Cho et al., 2023). Our consideration of independent participation across clients in Assumption 4 is more realistic than the conventional setting of sampling among all the clients with or without replacement (Li et al., 2020c, Yang et al., 2021), because it is difficult to coordinate the participation across a large number of clients in practical FL systems.

Challenge in Analyzing Time-Dependent Participation. Regarding the assumption on the independence of ${\rm I\kern-1.99997ptl}_{t}^{n}$ across time (round) $t$ in Assumption 4, the challenge in analyzing the more general time-dependent participation is due to the complex interplay between the randomness in stochastic gradient noise, participation identities $\{{\rm I\kern-1.99997ptl}_{t}^{n}\}$, and estimated aggregation weights $\{\omega_{t}^{n}\}$. In particular, the first step in our proof of the general descent lemma (see Appendix B.3, the specific step is in (B.3.6)) would not hold if ${\rm I\kern-1.99997ptl}_{t}^{n}$ is dependent on the past, because the past information is contained in $\mathbf{x}_{t}$ and $\{\omega_{t}^{n}\}$ that are conditions of the expectation. We emphasize that this is a purely theoretical limitation, and this time-independence of client participation has been assumed in the majority of works on FL with client sampling (Fraboni et al., 2021a; b, Karimireddy et al., 2020, Li et al., 2020b; c, Yang et al., 2021). The novelty in our analysis is that we consider the true values of $\{p_{n}\}$ to be unknown to the system. Our experimental results in Section 5 show that FedAU provides performance gains also for Markovian and cyclic participation patterns that are both time-dependent.

Assumption 5 is only needed for Theorem 2 (stated below) and not for Theorem 1. Here, the bounded global gradient assumption is a relaxed variant of the bounded stochastic gradient assumption commonly used in adaptive gradient algorithms (Reddi et al., 2021, Wang et al., 2022b; c). Although focusing on very different problems, our FedAU method shares some similarities with adaptive gradient methods in the sense that we both adapt the weights used in model updates, where the adaptation is dependent on some parameters that progressively change during the training process; see Appendix A.2 for some further discussion. For the nearly optimal weights assumption, we can see that it holds if $\nicefrac{{8}}{{9p_{n}}}\leq\omega_{t}^{n}\leq\nicefrac{{10}}{{9p_{n}}}$, which means a toleration of a relative error of $\nicefrac{{1}}{{9}}\approx 11\%$ from the optimal weight $\nicefrac{{1}}{{p_{n}}}$. Theorem 2 holds under either of these two additional assumptions.

The proof of Theorem 2 includes a novel step to obtain $\frac{1}{NT}\sum_{t=0}^{T-1}\sum_{n=1}^{N}\mathbb{E}\big{[}\left(p_{n}\omega_{t}^{n}-1\right)^{2}\big{]}$ (ignoring the other constants), referred to as the weight error term, that characterizes how the aggregation weights $\{\omega_{n}^{t}\}$ affect the convergence. Next, we focus on $\{\omega_{t}^{n}\}$ obtained from Algorithm 2.

The proof of Theorem 3 is based on analyzing the unique statistical properties of the possibly cutoff participation interval $S_{n}$ obtained in Algorithm 2. The first term of the bound in (7) is related to the variance of $\omega_{t}^{n}$. This term increases linearly in $K$, because when $K$ gets larger, the minimum number of samples of $S_{n}$ that are used for computing $\omega_{t}^{n}$ gets smaller, thus the variance upper bound becomes larger. The second term of the bound in (7) is related to the bias of $\omega_{t}^{n}$, which measures how far $\mathbb{E}\left[\omega_{t}^{n}\right]$ departs from the desired quantity of $\nicefrac{{1}}{{p_{n}}}$. Since $0<p_{n}\leq 1$, this term decreases exponentially in $K$. This result confirms the bias-variance tradeoff of $\omega_{t}^{n}$ that we mentioned earlier.

The result in Corollary 4 is the convergence upper bound of the full FedAU algorithm. Its proof involves further bounding (7) in Theorem 3, when choosing $K=\log_{c}T$, and plugging back the result along with the values of $\gamma$ and $\eta$ into Theorem 2. It shows that, with properly estimated aggregation weights $\{\omega_{t}^{n}\}$ using Algorithm 2, the error approaches zero as $T\rightarrow\infty$, although the actual participation statistics are unknown. The first two terms of the bound in (8) dominate when $T$ is large enough, which are related to the stochastic gradient variance $\sigma^{2}$ and gradient divergence $\delta^{2}$. The error caused by the fact that $\{p_{n}\}$ is unknown is captured by the third term of the bound in (8), which has an order of $\mathcal{O}(\nicefrac{{\log^{2}T}}{{T}})$. We also see that, as long as we maintain $T$ to be large enough so that the first two terms of the bound in (8) dominate, we can achieve the desirable property of linear speedup in $N$. This means that we can keep the same convergence error by increasing the number of clients ($N$) and decreasing the number of rounds ($T$), to the extent that $T$ remains large enough. Our result also recovers existing convergence bounds for FedAvg in the case of known participation probabilities (Karimireddy et al., 2020, Yang et al., 2021); see Appendix A.3 for details.