Access Characteristic-Guided Remote Swapping Across Mobile Devices

In mobile systems, memory swapping occurs when the available memory achieves the watermark threshold. During memory swapping, inactive pages are evicted from the main memory and delivered to the swap area as I/O requests by a kernel thread named kswapd. This process is called swap-out. When a swapped-out page is demanded, the system will trigger a page fault and the page is swapped back to the memory through a new I/O request. This process is called swap-in. Based on the location of the swap area, memory swapping can be classified into two categories: local swapping and remote swapping. Local swapping means the system allocates the swap area in the local flash-based storage (i.e., flash-based swapping), while remote swapping means that the swap area is in the remote device memory, as shown in Figure 1.

For the local swapping [20, 27, 33, 44, 47], memory pages are moved between DRAM and local storage through the I/O block layer. However, due to the read/write interference and limited endurance for flash devices [21, 30, 35, 37, 46], the system performance is significantly affected by swapping-induced I/Os. To address this issue, remote swapping is proposed to alleviate the memory pressure with the aid of remote device memory [3, 6, 9, 19, 28, 31, 38]. For remote swapping, when the system faces memory pressure, the victim memory pages will be swapped out to a remote device [36]. The memory pages are swapped between the main memory and the remote swap area through the remote I/O virtual block layer based on the typical socket (e.g., network block device (NBD)). Another type of memory swapping in mobile systems is compression-based swapping (i.e., ZRAM [23]), which compresses inactive memory pages and stores them in a separate memory area. And then, when the application accesses the compressed pages, they are decompressed back into memory. ZRAM is faster than traditional flash-based swapping mechanisms because it avoids long-latency I/O accesses. However, the disadvantage of this approach is that compressing memory pages still consumes memory capacity, while compression/decompression costs CPU cycles. Many commercial mobile operating systems (e.g., Android) enable this mechanism by default [15]. However, in practice, Android does not actively exploit this mechanism by default because the Android low memory killer daemon (lmkd) [2, 14] is usually triggered first to reclaim memory space before swapping occurs.

Android is a widely used mobile system based on Linux. Process management in Android is specially designed to ensure smooth application. First, Android divides all processes into two categories: system-service and application-specific. The system-service processes provide general services to support application running. The application-specific process starts running when the application is started. Second, to determine which process should be terminated when memory is insufficient, Android will sort them in order of importance according to the components running in each process and their running states. Briefly, it includes four types [16]. (1) A foreground process represents the processes being used by the user. (2) A visible process is doing work that the user is currently aware of. (3) Service processes are not directly visible to the user, while they generally provide support for foreground applications (e.g., background network activity). (4) A cached process is not needed temporarily and is cached in memory or a swap device.

Remote Swapping System. Many previous works adopted remote swapping to release the system memory pressure. Sprite [36] presented a network operating system and provided sharing, flexibility, and high performance to networked workstations by a set of kernel calls. In terms of the virtual memory system, it used ordinary files for backing storage to simplify process migration, to share backing storage between workstations, and to capitalize on server caches. In the data center, recent studies [4, 6, 19, 31, 38, 42, 43] leverage remote swapping to implement memory disaggregation based on RDMA. With the development of mobile devices and wireless communication technologies (e.g., 5G and WiFi-Direct), users often own multiple mobile devices and the performance of remote I/O has been significantly improved, while mobile devices have always suffered from memory pressure. For mobile devices, remote swapping is able to expand the limited physical memory and avoid the disadvantages of traditional memory swapping, which has been widely studied. The problem of energy consumption of remote swapping in mobile devices is addressed [3]. CloudSwap [9] made use of the cloud resource to release the memory pressure of local devices to enable the memory-oblivious app caching. Moreover, two cloud-assisted prefetching schemes were adopted to reduce remote swap-in latency. Different from this work, we use a page filtering strategy based on application characteristics to ensure efficient page transfer between different mobile devices without the intervention of cloud resources and the help of local storage devices. MobileSwap [28] built a dedicated network channel to improve the network throughput for remote swapping under existing network infrastructure. LegoSwap [26] further proposed elastic swap area management to ensure user interaction. Unlike these two, this paper does not seek to change the network itself, but aims to realize the network-aware remote paging based on the application characteristics to boost user experience.

Application Running Characteristic Analysis. Prior studies have been conducted to analyze application running characteristics to design a more efficient memory management mechanism [25, 27, 44]. Marvin [25] combines the upper-layer garbage collection (GC) of the system with the underlying swap mechanism to optimize memory management due to the gap between the system GC and the swap mechanism. SEAL [27] builds a hybrid swap system including local storage and memory by exploiting the page access characteristics of the different running stages of the application. lcSWAP [44] revisits swap I/O size based on page compressibility to reduce write stress. Unlike these works, ACR-Swap\(^+\) identifies the swapping characteristics of different processes of an application and distinguishes between system-service processes and application-specific processes. And then ACR-Swap\(^+\) analyzes the performance characteristics of remote swapping systems in different network environments.

The proposed methodology is more relevant to the work presented in [29]. Compared to that, the present work (i) further analyzes the swap-in characteristics of the system service process, including the swap-in frequency and the impact of running states on swap-in; (ii) redesigns the original page filtering and prefetch mechanism to account for forgotten system-service processes; (iii) clarifies the impact of network traffic on remote swapping and proposes a network traffic-aware remote paging scheme; (iv) redesigns the experiment on latest platform and provides in-depth analysis of evaluation results, impact on application caching ability and switching time, which is critical to user experience.

The basic idea of ACR-Swap\(^+\) is to partition the swap area into a local swap area (LSA) and a remote swap area (RSA). Then, based on the observations presented in Section 3, swapped data should be carefully placed between these two areas. Figure 6 shows the architecture of ACR-Swap\(^+\). Two critical components are implemented: LSA and RSA. LSA exposes a conventional frontswap interface [18] to the kernel virtual memory and consists of four components: a system service-aware page sifting, a running state-oriented prefetching, and a network traffic-aware remote paging. The system service-aware page sifting module classifies pages according to their belonging processes and places selected processes at LSA based on the observations in Section 3.1 (i.e., swap-in frequency). The running state-oriented prefetching module enables data to be swapped in from RSA based on the observations in Section 3.2 (i.e., the application running states). RSA is designed with the remote device memory, consisting of the remote stub and page pool. The remote stub is responsible for responding to LSA’s requests, and the page pool manages the memory pool at the remote device. The local stub is responsible for communicating with remote devices and delivering pages.

When facing memory pressure, pages are frequently swapped between memory and the swap area. For remote swapping, excessive remote paging will magnify the shortcomings of remote I/O, which will inevitably affect the user experience. Unlike the simple treatment of system-service processes in previous work [29], SPS adopts a system service-aware page identification and placement strategy for processes based on the observation in Section 3.1. The core idea is to prioritize the data placement of system-service processes with frequent Swap-ins in LSA to avoid excessive remote swapping operations. When the free space of LSA reaches a certain threshold during memory reclamation, application-specific processes are preferentially swapped out to RSA. Figure 7 shows the design of SPS. There are three challenges for the above design: First, when the kswapd thread reclaims memory, it cannot directly obtain the process information corresponding to the page. In this case, it is impossible to classify and place pages according to the types of processes. Second, the numerous system-service processes need to be further sifted to improve the efficiency of remote swapping. Third, setting the threshold for page transfers from LSA to RSA needs to be transparent and efficient to the user. To solve the above challenges, the page identification, system service sifting and threshold determination schemes are presented as follows.

Traditional memory swapping employs a read-ahead scheme, which simultaneously swaps in an aligned block of pages containing the faulted page (always eight pages). If the pages swapped in by read-ahead are accessed in the cache, it can provide superior access performance to improve user experience. However, even though the read-ahead schemes have been significantly improved [5, 40, 41, 45], it is not designed for remote swapping across mobile devices. To this end, ACR-Swap\(^+\) proposed RSP based on the swap-in access characteristics. The basic idea is to dynamically regulate the prefetching window size based on the application running states. In this way, the application can achieve different swap-in throughput at different running states, which can effectively hide the latency of remote I/O and avoid cache pollution.

Different from the previous work [29], it neglected to prefetch system-service processes. Since system services are swapped in more frequently than applications, efficient prefetching needs to take them into account. Considering that there is no apparent running state changes when the system-service processes perform swap-in, the previous RSP does not seem to be suitable for them. However, note that the system-service processes are often called during application switching. Therefore, the newly proposed RSP can utilize this feature to synchronously regulate the prefetching windows of all processes based on the application running states. However, there are at least two challenges to this basic idea: First, the switching of the application running state should be monitored accurately. Second, the prefetching window size should be carefully regulated. To solve the above issues, a monitoring application running state scheme and an N-size prefetching algorithm are proposed as follows.

Remote swapping performs well in an ideal laboratory environment [28], but it faces various external environment interference in real use cases, which will significantly degrade the user experience. Different wireless networks often have different performance characteristics in terms of bandwidth, latency, jitter, and packet loss. Specifically, first, mobile devices may be used at home, school, or other public places, and the networks in these places obviously have different network bandwidths and network delays [8]. Second, there is often more than one user using the devices under the same LAN. In particular, when other users are also performing network-intensive tasks (e.g., file sharing and screen casting), these tasks can easily compete with remote swapping for network resources. To evaluate the performance of remote swapping in a real network environment, we constructed network conditions under different scenarios, covering different locations (e.g., family network and laboratory network) and different users under the same network.

Figure 8 shows an evaluation of the swap throughput for different network states and use scenarios. Figure 8(a) shows that the remote swapping throughput reached the highest 517 Mbps under the ideal laboratory network. However, in the public network, such as the coffee network, the swap throughput may drop by 47.9%. Correspondingly, under different networks, the network bandwidth jumps up and down. This reveals that differences in network conditions themselves can significantly affect remote swapping. It is worth mentioning that mobile devices may be used at home, school or other public places, which seems inevitable. Figure 8(b) shows the throughput variation of remote swapping under another scenario. Under the same local area network, there is often more than one users using their mobile devices. However, when more and more users use their mobile devices under the local area network, the throughput of remote swapping continues to decline. For example, when there are 10 other users on the LAN using their mobile devices, the swap throughput drops by 42.7%. This is because other users inevitably compete with remote swapping tasks for network resources when using mobile devices, especially when other users are also performing network-intensive tasks (i.e., file sharing or screen casting).

In conclusion, current remote swapping is susceptible to performance fluctuations caused by changes in network traffic, which will seriously impair user experience.

To solve the above issue, a NaRP scheme is proposed. The basic idea is to divide the network traffic into different levels and then perform remote paging at various degrees to ensure the stability of remote swapping. Two challenges should be solved for this basic idea: First, network traffic varies in different environments and over time, which is difficult to quantify into individual levels. Second, even if the network traffic is well-graded, undesired network fluctuation will still exist. Thus, the system must be redesigned to accommodate different degrees of remote paging. To solve the above challenges, the grading network traffic and network license algorithm schemes are proposed as follows.

ACR-Swap\(^+\) is implemented on real mobile devices. Several independent kernel modules are implemented to the main functions, including common_network for providing common network services between local and remote devices, local_swap for enabling SPS, RSP, and NaRP on the local device, and remote_swap for providing remote memory on the remote device. System-native functions (e.g., try_to_unmap)are used in preference to obtain information for high compatibility. ACR-Swap\(^+\) introduces only a few system source code modifications (2,210 lines of C language in the kernel space and 105 lines of Java/C++ in the user space). To deploy ACR-Swap\(^+\) on off-the-shelf devices, some implementations are presented as follows.

First, LSA and RSA are implemented as follows. LSA is deployed in physical memory as a dedicated frontswap interface [18]. To the best of our knowledge, frontswap is designed for swapping at page granularity rather than supporting general block I/O operation. It strives to minimize context switches to other tasks while swapping operations completely. At the same time, a virtual network block device is created and formatted as a swap area, and the pages that are swapped into the partition are actually forwarded to the remote device. Specifically, the source code of boot.img is modified. After porting the recompiled image to the mobile device and rebooting, a customized frontswap and block interface are set in the systems. One is \(localswap\), which can be seen in the directory “/sys/module” and the other is \(remoteswap\), which can be seen in the directory “/dev/block”. \(localswap\) provides a transcendent memory interface for swap pages, responsible for page filtering store and selectively forwarding pages to \(remoteswap\) suffering memory pressure. Actually, RSA acts as the \(remoteswap\) corresponding to the actual swap area to temporarily store the swapped pages and responds to client-side read requests.

Second, due to the frequent page delivery between the local device and the remote device, effective page indexing is essential for swap area management. In the implementation, flags \(local\) and \(remote\) are used to maintain index pages located in LSA or RSA, respectively. The original swap-related code and corresponding mapping table are modified to support the two swap areas page indexing. Expressly, when a page locates in LSA, the address bit of PTE is set as \(local\). Finally, another assumption of this work is that the system performs remote swapping in only user-owned trusted devices. For such a scenario, ACR-Swap\(^+\) adopts an authentication mechanism that asks the local and remote devices’ owner(s) to authenticate the remote paging, after which ACR-Swap\(^+\) assumes that both systems are not malicious. Therefore, the security issues related to untrusted devices are out of our work scope.

Third, SPS adopts the page reverse mapping mechanism to obtain the process ID of the page to distinguish the system service process. Moreover, the swap-in frequency of each process is recorded to support the selection of victim system-service processes when SPS performs system service sifting. We maintain a two-dimensional array in the kernel to hold this information. RSP identifies the determine the running status of the process based on \(oom\_score\_adj\), when a page is swapped in. To support NaRP, we periodically detect the network bandwidth. Depending on the network bandwidth, different degrees of remote paging mechanisms are executed.

Due to mobility, the connection between the local and remote devices can be lost at any time. If not handled properly, disconnecting may induce multiple problems, such as the local system crashes and the remote system keeps monitoring. Based on this, a time-out mechanism is adopted to detect a disconnection. At regular intervals, the remote paging latency is measured. If the local device does not receive the acknowledgment before a certain threshold or the remote does not hear from the local, they both trigger a disconnection event [7]. For the remote, network disconnection is equivalent to killing the kernel thread that updates and maintains anonymous client memory. Then, the remote device cleans up the residuals of the disconnected client process, while the corresponding memory is inevitably reclaimed by the remote-side gradually. It is a future work that remote memory is not reclaimed and properly handled in the case of network disconnection. For the local device, we try to make the disconnection transparent to the application as much as possible. Specifically, we first update the corresponding PTE of the swapped pages and then enable the local storage as a swap area after the disconnection. It is worth mentioning that we put some system-service processes in the local memory, which can further reduce the impact on the system when the device is disconnected.

Another assumption of this work is that the system performs remote swapping in only user-owned trusted devices. For such a scenario, ACR-Swap\(^+\) simply adopts an authentication mechanism that asks the local and remote devices’ owner(s) to authenticate the remote paging, after which ACR-Swap\(^+\) assumes that both systems are not malicious. For scenarios with high security requirements, to optimize network security, ACR-Swap\(^+\) provides an optional data encryption interface to ensure user privacy. Specifically, ACR-Swap\(^+\) enables an encrypt kernel module based on the Linux crypto framework [34] to encrypt/decrypt pages. This module uses the AES acceleration hardware on mobile systems to provide secure multi-mobile device communication using 128-bit AES encryption. Once the module is enabled, the system generates a public/private key pair to facilitate the secure sharing of the AES session key generated at runtime. When sharing pages between local and remote devices, the data encryption and decryption functions crypto_skcipher_encrypt or crypto_skcipher_decrypt can be called to achieve secure data transmission. Since data security is not the main concern in this paper, the encryption feature is turned off by default.

In our experiments, we adopt two Google Pixel 6 smartphones with LineageOS 20.0 [1] (based on Android 13 and kernel 5.10) as our platform. It is equipped with an Octa-core ARM CPU, 8 GB DRAM, and 128 GB UFS 3.1 storage space. To deploy the remote swapping system, one device is configured as the local device while the other is configured as the remote device. All devices are connected to the same Wi-Fi network with a bandwidth of 1200 Mbps. To avoid interference, network functions except Wi-Fi are disabled. We selected widely used mobile applications as mentioned in Table 1, and conducted experiments by following the two steps: (1) We installed the pre-selected 40 applications (12 switching applications and 28 background running applications) on the device to begin each experiment. (2) We used adb [10] and logcat [13] commands to collect evaluation results while performing automated tests with UI Automator [17] that emulates UI touches of users based on scripts. We performed the same automated tests ten rounds and calculated the average.

The following seven schemes are evaluated to show the advantages of ACR-Swap\(^+\): (1) ZRAM represents the baseline adopted by mobile systems by default, which inevitably kills processes when the compressed memory is almost used up. (2) Local Swapping (LS) represents the traditional flash-based swapping, which stores the swapped-out pages in the local storage. (3) Remote Swapping (RS) represents the original remote swapping that creates the swap area in remote devices, and the swapped-out data will be stored in the remote device to release local memory pressure. (4) CloudSwap (CS) represents enhanced remote swapping [9]. We reproduce its design relevant to this paper including that using the fast local storage as a cache of read-intensive swapping pages while storing prefetch-enabled, write-intensive swapping pages in the cloud storage. (5) MobileSwap (MS) represents enhanced remote swapping [28] that includes resource-dedicated swapping for fast swapping among devices and app-aware swapping for network connectivity considerations. (6) ACR-Swap represents the state-of-the-art work ACR-Swap [29], which includes PPS (store system services locally) and AGP (adaptive prefetching for applications). (7) ACR-Swap\(^+\) is the proposed scheme, which includes SPS, RSP and NaRP. In the evaluation, ZRAM is set to 1GB for swap space. The swap area with LS, RS, and MS is set to 3GB. CS, ACR-Swap and ACR-Swap\(^+\) maintain a 1GB local swap area and a 2GB remote swap area, for fairness. Moreover, ACR-Swap and ACR-Swap\(^+\) set the LSA’s \(TH\) to 512MB for simplicity.

In the following, application caching capability and switching latency are evaluated to show the advantages of the proposed scheme.

To further understand the benefits of ACR-Swap\(^+\), SPS and RSP are evaluated as follows.

NaRP is designed to ensure that remote swapping has stable performance under unpredictable network conditions. To understand the benefit of NaRP, we compare the swap throughput under different network conditions, which is evaluated as follows. We use swap throughput to evaluate the efficiency of NaRP. To collect it, we deploy different remote swapping schemes in each of the five locations (representing different LAN connections). As in the above experiments, six different applications were switched back and forth, and the average swap throughput during application switching was calculated.

Figure 13 shows the swap throughput under various network conditions among multiple remote swapping schemes. First, the swap throughput corresponding to various remote swapping is faced with fluctuations under different network conditions. For example, compared to Net.2, the throughput of RS under Net.5 drops by 36.7%. Other existing schemes including CS, MS, and ACR-Swap also suffer performance losses of 35.7%, 20.2%, and 25.1%, respectively. Most importantly, ACR-Swap\(^+\) achieves the slightest performance fluctuation of only 5.1%. Thanks to the conscious remote paging design of the network, ACR-Swap\(^+\) realizes dynamic remote paging and effectively avoids the influence of network jitter on the system. Second, under the same network conditions, the swapping throughput of previous designs is also significantly different. ACR-Swap\(^+\) guarantees the swap throughput of 838 Mbps even under the worst network conditions.

To further understand ACR-Swap\(^+\), several sensitive studies are conducted, including the size of LSA, the \(TH\) for remote paging in SPS and prefetching window size in RSP.

While ACR-Swap\(^+\) effectively optimized the memory management with cross-device swapping, there are still several tradeoffs in the design. This section quantifies ACR-Swap\(^+\)’s Three types of overhead: (1) Performance overhead of reverse mapping in SPS design; (2) Memory overhead of system service sifting in SPS design; and (3) CPU overhead to manage the two-level framework.