FlexPointer: Fast Address Translation Based on Range TLB and Tagged Pointers

One of the primary goals of the virtual memory system is to provide each process with the illusion that it has a private address space [18, 19, 30]. In the page-based virtual memory, the OS partitions the address space of a process into virtual pages and maps them to physical pages, then hardware performs address translation according to the mappings at runtime. The architecture defines how software and hardware work in cooperation. In this section, we focus on the virtual memory system of the x86-64 architecture.

The x86-64 architecture supports three-page sizes, 4 KB, 2 MB, and 1 GB. The base page size is 4 KB, while 2 MB and 1 GB pages are called huge pages. The OS maintains a page table for every process, which is also stored in memory as part of the virtual address space. The page table uses PTEs for recording mappings from virtual pages to physical frames, as well as information for security and other purposes. To save space, the page table is usually organized as a radix tree. For every memory access, hardware must first translate from virtual address to physical address with the page table. The lookup process in the page table is called page table walks. Page table walks require multiple memory reads, e.g., translating a 4 KB page in a 4-level-paging x86 architecture requires four reads. To accelerate the translation, processors use TLBs to cache recently used PTEs. Since page walks are still costly, often requiring hundreds of cycles, processors also adopt MMU caches to contain recently walked upper-level PTEs.

The OS kernel has several software components for memory management. A user program gets virtual memory from the OS by calling syscalls like brk() or mmap(), and OS will either adjust existing virtual memory areas or create a new one. On the other hand, physical memory is managed by the buddy allocator, which tracks all of the free physical memory with free lists that contain power-of-two-sized blocks. In most cases, the program does not get physical memory immediately after calling such syscalls. Actual allocations of physical memory are delayed until the first access to the newly allocated virtual area. The first access to such an area will trigger a page fault and invoke the buddy allocator to get physical memory. This strategy is called demand paging.

Huge pages. Since one TLB entry records a recently used PTE of a single page, an intuitive approach to increase the TLB reach is using a larger page size to increase the coverage of a TLB entry. Huge pages are common in modern computer systems [44, 50, 59]. The x86-64 architecture supports huge pages of 2 MB and 1 GB, backed up by various OS mechanisms, e.g., Transparent Huge Pages (THP) [14] and HugeTLBFS [15] in Linux. Available page sizes in x86-64 are sparse, which can bring internal fragmentation problems. For example, an allocation request of 1 MB requires 256 PTEs when using 4 KB base pages, but it also wastes half of the memory space if served with a 2 MB huge page. The lack of available page sizes puts some of the allocations in a dilemma.

Some architectures provide more page size choices. Intel Itanium [12] separates the address space into eight areas and allows each area to configure its own page size. Itanium organizes huge pages with a hash page table, but without aggressive modification to the conventional 4-level page table, it can only be used to reduce page walk overheads. HP Tunable Base Page Size [20] allows OS to adjust the base page size. It still faces the internal fragmentation problem, so HP advises the base page size should be no more than 16 KB. Shadow Superpage [21, 55] adds a new translation level in the memory controller to merge non-contiguous physical pages into a huge page in a shadow memory space. Although it grants TLB a larger coverage, all memory traffic needs to be translated again in the memory controller, adding an extra latency for all memory accesses.

Tailored Page Sizes (TPS) [24] expand the available page sizes to all power-of-two sizes. A tailored page consists of \(2^N\) base pages and aligns on its size. For a properly aligned tailored page, TPS makes the PTE of the first base page a real PTE and others as alias PTEs. TPS utilizes the fact that larger pages have fewer page frame number (PFN) bits to identify from the real PTE the tailored page size. To correctly fetch the real PTE during a page walk, all alias PTEs are pointed to the real one, requiring one additional memory access in the page walk.

TLB coalescing. TLB coalescing approaches [48, 49, 56] leverage default OS allocator behaviors to pack multiple PTEs into one TLB entry. Because of locality, a default on-demand OS allocator tends to map a small set of contiguous virtual pages either to contiguous physical pages (Sub-blocked TLB [56] and CoLT [49]) or to a clustered set of physical pages (Clustered TLB [48]). The main drawback of such techniques is that the contiguity and cluster generated by a default allocator are limited.

Most of these techniques are pure hardware solutions. They check PTEs in the same cache line and determine if they can be packed into one TLB entry, so their effectiveness is also restricted by the relatively small search window. Anchor TLB [46] instead searches the page table to achieve a more global view. Anchor TLB inserts anchor entries at a certain interval, then searches for physical contiguity starting from anchor entries and records the end address of contiguous region. Upon a TLB miss, Anchor TLB searches the TLB again for a corresponding anchor entry and compares the address being translated with the end address. If the address is in the contiguous range, then it can be translated using the anchor entry. As the comparison increases the lookup cost, Anchor TLB only searches for anchor entries after an L2 TLB miss.

Segment-like techniques. Some early processors use segments to manage virtual memory. A segment can be viewed as the mapping between contiguous virtual memory to contiguous physical memory. Since a segment can be much larger than a page, some approaches partly bring back segmentation by separating special areas in the virtual address space to increase the translation coverage. Direct Segment [6, 22] allows the programmer to explicitly set one segment for a big-memory application. Two registers are added to mark the start and end of the segment. Any virtual addresses in this area can be translated by adding the offset between the virtual start address and the physical start address. Do-it-yourself Virtual Memory Translation (DVMT) [2] similarly adds two registers to separate a special area from the virtual address space, but it allows more complicated translations. DVMT launches a dedicated thread for translation when accessing an address in the special area. Direct Segment and DVMT both require intensive source code modifications, and their separated special virtual area will not appear in the conventional page table.

Redundant Memory Mappings (RMM) [33] pave another way by adding an additional range table. For an allocation large enough, RMM eagerly allocates contiguous physical pages for it instead of demand paging. This pre-allocation strategy creates large memory ranges that are both virtually and physically contiguous, and addresses in those ranges can be translated by adding the offset like in Direct Segment [6]. RMM adds a standalone range table to hold the information of such ranges, while the conventional page table still holds the complete memory mapping. Compared with Direct Segment, RMM supports multiple ranges and is transparent to programmers. However, RMM has to compare an address with all range boundaries to decide which range it belongs to. Such comparisons are expensive, so RMM only queries the range TLB after an L1 TLB miss.

Tagged pointers are widely used in the memory safety field. Detecting a memory safety violation often requires finding an object’s information using only a pointer. This is very similar to the indexing problem of memory ranges. The use of tagged pointers in memory protection techniques can be classified into two main categories. One uses the tag to search a metadata table [10, 35, 43, 52, 53, 61], and the other uses the tag and the address to calculate object boundaries [11, 37, 38]. The former needs to assign every object an ID, but since the object number is very huge in some programs, such techniques must either compress the ID or change the pointer representation for more spare bits. On the other hand, calculating the object boundary often puts extra alignment and/or size constraints on memory allocations, which can cause memory fragmentations.

FlexPointer adopts the range translation concept in RMM [33] and Direct Segment [6]. A range consists of contiguous virtual pages mapped to contiguous physical pages. Pages in a range have uniform protection bits, e.g., read/write/executable. A range is base-page-aligned and has an arbitrary number of pages. We use two addresses, BASE and LIMIT, to define a range. Because of the virtual and physical contiguity, addresses in a range share a common DELTA, which is defined as \((physical\_address-virtual\_address)\) . To translate a virtual address with range translation, the processor adds DELTA to it. Figure 1 shows parts of an application’s address spaces mapped with range translations.

A system using range translations has three main components. (i) The creation of memory ranges. (ii) The management of range information. (iii) The hardware that efficiently utilizes range translations. As stated before, FlexPointer creates a range and assigns it a unique ID when receiving an allocation request of a very large size. To assure the physical contiguity of a range, we use the eager paging strategy. Instead of allocating physical pages at the first access, eager paging allocates physical pages at the allocation request time. We instrument malloc() functions to catch allocation requests and modify kernel memory management functions, e.g., mmap(), to eagerly allocate physical memory. We also add a kernel range table to record range translations. In our current prototype, mappings in the range table are also kept in the page table to ensure the correctness of other memory subsystems.

Similarly, we use a range TLB to utilize range translations in hardware as in RMM [33]. But we can pass the range ID to hardware through the pointer tag, allowing the range TLB to work in parallel with the address generation. During a memory access, the processor decides whether to search the range TLB or the page TLB according to the pointer tag. The tag also decides which table to search when a range/page TLB miss happens.

Figure 2 shows how FlexPointer creates ranges. In FlexPointer, we divide allocation requests into two categories according to request sizes. For small allocation requests, they are served with the default malloc() procedure; while for large allocation requests, malloc() will directly call a newly added syscall, mmap_range(). mmap_range() creates a virtual memory area for the request and invokes the buddy allocator to eagerly allocate physical memory.

We implement a wrapper function for malloc() which judges allocation requests by a pre-defined threshold. According to a previous study, setting the threshold to 48 KB or lower may classify too many objects as large objects, e.g., more than 40,000 in gcc [13]. On the other hand, this study also shows that a 64 KB threshold can keep the number of large objects under 1,000 for many applications, which is also proved by the results in Table 1. So in our wrapper function, we set the threshold to 64 KB.

The eager paging strategy ensures the newly allocated virtual pages are backed by physical pages, but we also need to modify the buddy allocator to provide physical contiguity. Various OS proposals offer different allocation strategies to address the issue of maximizing memory contiguity [3, 39, 45, 62]. Among them, contiguity-aware (CA) paging [3] introduces an indexing structure, contiguity_map, to record physical contiguity in the system. We adopt this structure to help implement the buddy allocator. We put no rounding constraint or upper limit on the range size and only require a range to be aligned on the base page size (4 KB on x86 architectures).

When mmap_range() is called, it inserts a new entry into the range table and uses the index of the entry as the range ID. We maintain a FIFO free index list to help find the inserting position of a new entry. If the free index list is empty, the new entry is inserted at the end of the range table; otherwise, mmap_range() takes an index from the free list. In the munmap() syscall, the kernel checks high bits of the incoming address to judge if it is unmapping a range. For a tagged address, munmap() should reclaim the corresponding range table entry and put its index in the free list. We will discuss the structure of range table in Section 3.4.

Figure 3 illustrates our pointer format. Conventionally, the most significant bit of a 64-bit pointer is used for distinguishing user and kernel pointers, so we leave this bit alone. There are 15 unused bits in the pointer, and we use 13 bits of them for ID storage. The other 2 bits are left for future use, e.g., as selection bits to cooperate with other tagged-pointer techniques. Normally, the high bits of an untagged pointer are all 0s or 1s, so we exclude 0x0 and 0x1FFF from IDs. According to the object counts in Table 1, no benchmark examined produces more than 1,000 ranges when setting the threshold to 64 KB. This encoding scheme grants us 8,190 available IDs, fairly enough for current benchmarks.

FlexPointer adds a per-process range table to record range translations. Figure 4 shows the structure of a range table entry. BASE and LIMIT define the range in the virtual address space, and DELTA defines it in the physical space. Current systems use a 48-bit address space, so the virtual page number (VPN) and physical page number (PPN) are both 36 bits. Since DELTA represents the difference between VPN and PPN, it also takes 36 bits to store. In the conventional page table, a PTE uses its low 12 bits to record attributes of the page, e.g., r/w bits. We also adopt these attributes in our range table (Attr in Figure 4).

The range table is organized as an array. When handling a large allocation request, mmap_range() needs to assign the range an ID. As mentioned in Section 3.2, we use a free index list to guide the insertion of an entry. mmap_range() gets an index from the free list and uses it as the range ID, recording it in the high bits of the return pointer. If there is no free entry in the range table, mmap_range() should fall back to the default demand paging strategy.

We use Next Range ID (NRID) and Previous Range ID (PRID) to help handle mremap(). NRID and PRID are both 13 bits, the same length as the range ID in the pointer tag. When calling mremap() to enlarge a range, there may not be enough physical memory for in-place resizing. One possible solution is to fall back to normal paging. But to further exploit contiguity in such cases, we let mremap() create a sub-range: the extended part shall get a contiguous physical region, which may not be contiguous with the original range. A sub-range inherits the original range’s ID because they are the same object from the perspective of the user program. However, the sub-range should have its own range table entry because it has a different DELTA.

When mremap() is called, it first checks the high bits of the input pointer. For a tagged pointer, mremap() fetches the corresponding range table entry (last sub-range entry if multiple sub-ranges exist). Then mremap() checks whether there are enough free physical pages to resize in situ. If possible, mremap() initializes the pages and updates the range table entry. Otherwise, mremap() will search the free list for a free entry, record the translation information and link it to the last sub-range entry of the range. Sub-range entries are organized as a double-linked list, using NRID and PRID to connect with each other. The L bit indicates if a sub-range is the last one in a range. The index taken by a sub-range entry will never appear in the pointer tag until the sub-range gets freed.

A possible yet not observed scenario is that a program calls mremap() to expand an under-threshold object, and the expanded part exceeds the threshold. In that case, mremap() should create a sub-range for the incremented part and tag the return address. However, the tricky part is that now addresses within the original object are also tagged. A tagged address should be translated using the range table, but addresses in the original part are mapped with non-contiguous physical pages and should be translated with the page table. To solve this problem, we create a dummy sub-range to represent the original under-threshold object. We use the D bit in a range table entry to mark it as a dummy entry. In a dummy entry, we store an upper-level PTE (PDE or PDPTE in x86-64) address in DELTA and Attr to reduce the page table walk overhead. Moreover, the return address of mremap() is tagged with the index of the dummy entry because it is the first sub-range.

Like RMM [33], FlexPointer relies on the range TLB to utilize range translations. We organize the range TLB as a fully-associative cache, as shown in Figure 5. The field Du represents if the entry is a dummy one. With the help of range ID, we can greatly simplify the lookup of range TLB. In RMM, the range TLB performs two comparisons per entry to complete an address lookup, while FlexPointer only needs to perform a single equality test per entry. Moreover, because our range ID does not participate in the address generation, we can start the search in parallel with the address generation instead of after it. As a result, we can shift the range TLB lookup to an earlier stage.

Although indexing the range TLB only requires the range ID, we also need to record BASE and LIMIT in the range TLB entry. For one reason, we need to compare the address with BASE and LIMIT for safety concerns. Since we already get the DELTA for translation after the ID equality test, the comparison can be done in parallel with fetching from the data cache. Upon a comparison fail, we trigger an RTLB miss signal to cancel the cache access and start a range table walk. If the address is illegal, then the range walk cannot find a corresponding entry and will eventually trigger a security fault.

Another reason is that mremap() may create sub-ranges that share the same range ID. To simplify the range TLB lookup, we only record the most recently used sub-range, assuring that IDs are unique in the range TLB. As a result, we need to perform a boundary check to make sure that the address falls in the right sub-range. If the boundary check finds the address mismatching the sub-range, the processor will send an RTLB miss signal, cancel the cache access and start a range table walk to fetch the correct sub-range from memory.

If an application creates too many sub-ranges and frequently switches between them, the range TLB will suffer from frequent range table walks. We examined the benchmarks and found that most of them rarely use mremap(). gcc is an exception that frequently increases the range size by one page. The sub-range number is closely related to the system fragmentation level and the allocation strategy of the buddy allocator. For example, with a proper page reservation strategy, the buddy allocator may satisfy most remapping requests of gcc in situ. Another possible solution for the sub-range problem is adding another level of range TLB, as discussed in Section 4.6. In this article, we assume that the physical contiguity is abundant and mremap() will not generate sub-ranges. We leave a detailed analysis of the sub-range number and its impact on the range TLB for future work.

Figure 6 illustrates the hardware components and workflow of FlexPointer. At the address generation stage, the processor decides whether to query the range TLB or the page TLB according to the high bits of the address. If they are all 0s or 1s, then the address is translated with the original page TLB after the address generation. Otherwise, the processor uses the range ID in the pointer to search the range TLB at once. As we stated before, IDs are unique in the range TLB, so the processor gets only one DELTA when the ID hits in the range TLB. Next, the processor adds DELTA to the generated virtual address while comparing the virtual address with BASE and LIMIT. If it passes the range check, then the sum of DELTA and the virtual address is the correct physical address. Moreover, if an address passes the boundary check of a dummy entry, it will be sent to the page TLB for translation.

A range TLB miss happens in two cases. (i) There is no matching ID in the range TLB. (ii) The boundary check fails. When a range TLB miss happens, the processor handles it by starting a range table walk to fetch the corresponding range translation. To simplify the range TLB lookup, we want to keep IDs in the range TLB unique. So if the miss is caused by sub-range mismatching, the newly fetched translation should be updated where the previous sub-range lies instead of inserting a new entry.

During a range table walk, we first calculate the translation entry address by adding \((ID\lt \lt 5)\) to the base address of the range table (a range table entry takes 32 bytes, as shown in Figure 4). The range table base address is part of the program context and should be stored in a dedicated register, like storing the page table base in CR3. If the address to be translated falls in \([BASE,LIMIT)\) , then the entry is what we need and will be fetched into the range TLB, whether it is a dummy entry or not. For a dummy entry, we should query the page table for a correct translation and insert it into the page TLB. When the address falls out of \([BASE,LIMIT)\) , we check the L bit to see whether the current entry is the last sub-range entry. If not, we get the next sub-range entry with NRID and repeat the above procedure. But if it is the last one, then there must be a safety violation.

Although current benchmarks allocate only a few hundred ranges, future programs may require much more range IDs. In general, we want to associate the ID with an entity in high-level program semantics then it can pass more software information to the hardware, so it’s better to avoid the sharing of IDs. The ID uniqueness also simplifies the design of range TLB and range table. Under this circumstance, we can either use the remaining two bits to increase the ID limit, or let the OS dynamically adjust the allocation threshold when the number of available IDs falls below a certain watermark. We also provide an alternative pointer format in Section 5.2 that breaks the ID limit by using the position of a range as part of its ID, although complicating the range table at the same time. Moreover, FlexPointer is an enhancement to the traditional page table instead of a replacement, so on an ID shortage, the OS can choose to only use the page table.

Except for translating virtual addresses, the TLB in x86 processors is also responsible for updating the usage information of pages. In an x86 PTE, there is an accessed bit and a dirty bit. The TLB should set the accessed bit on the first access to a page and the dirty bit on the first write. However, the range TLB groups multiple pages in one range translation entry, so it is difficult to determine which page’s accessed/dirty bit to set on a hit. One solution is to only maintain range-level accessed/dirty bits. It is reasonable because in FlexPointer a range is created from a memory object, so from the perspective of program semantics, pages in a range can be treated as a whole. Although coarse-grained page usage information may induce overheads for swapping, studies also show that performance-critical big-memory applications tend to do little or no swapping [6, 36]. Also, since we, in fact, save many L2 page TLB entries, we can reuse them to record more fine-grained information, like compacting metadata of several pages in a bit vector as PRISM [4] does.

Copy-on-write is a virtual memory optimization for sharing pages between processes. With this optimization, the OS duplicates a page when one of the processes writes to it, ensuring that modifications are only visible to the process initiating the write. Conventional systems implement this mechanism by setting protection bits for shared pages in their PTEs, so a write to such pages will trigger a fault for the OS to duplicate the page. In FlexPointer, we can utilize the dummy entry to support this mechanism. Upon the first write, we can dissolve part of the range (including the page being written to) and copy the modified page. Then, we create a dummy sub-range for the dissolved part and update the PTEs of the pages in it so that future access to the dissolved part will be directed to the page table. In this way, we can retain the translation efficiency of the remaining part of the range and the flexibility of copy-on-write at the same time.

Linux requires physical pages to be initialized when allocated. Eager paging will increase the latency of an allocation request because the application must wait for all allocated pages to be initialized. Previous studies [6, 31] show that big-memory applications tend to allocate most of their memory at an early execution stage and those allocations generally have a long lifetime, so the overheads of initialization can be amortized. To further reduce the initialization overheads, we can adopt the reservation-based paging strategy used in FreeBSD [42] and other proposals [24, 44, 56]. We can use the dummy entry to represent reserved pages and promote them when needed. Another way to ease the impact of initialization is using a dedicated background thread to initialize free pages [45].

In this work, we follow the same assumption of RMM that physical memory contiguity is abundant. But in a long-running server, there are generally multiple processes in execution and a variety of workload mixes, so physical memory may become fragmented during the execution. The chance to find a suitable contiguous physical region drops as the system fragmentation increases, thus the efficiency of FlexPointer is limited in a fragmented environment.

Technically, the OS should fall back to the default paging strategy if it cannot find a sufficiently large range of free physical memory. But with the help of our sub-range mechanism, we can tolerate the fragmentation to a certain level. If there is not enough physical contiguity, we can split a range request into several smaller sub-ranges. But it also should be noted that too many sub-ranges can both lower the range TLB hit rate and slow the range table walk. So we suggest setting a sub-range number limit and treating a request that exceeds this limit as a signal for memory compaction. Moreover, many OS proposals [3, 39, 45, 62] can help increase the physical contiguity and ease the impact of memory fragmentations, increasing the chance of finding a suitable physical region.

As stated before, when an application creates too many sub-ranges, the performance of FlexPointer will suffer from frequent range table walks. Although current benchmarks rarely create sub-ranges, future programs may present a different memory usage pattern. On the other hand, when using the sub-range mechanism to tolerate memory fragmentations, a program may split ranges into too many small parts in a heavily fragmented environment. To avoid such performance degradation, we can add a second-level range TLB to hold sub-ranges sharing the same ID. As IDs are no longer unique in the L2 range TLB, we have to do the boundary check first to distinguish which sub-range to use before the translation, which will complicate the L2 range TLB structure. In this work, we focus on the environment where physical memory contiguity is abundant. Also, our results in Section 6.2 shows that a small L1 range TLB is sufficient to satisfy most benchmarks, so adding an L2 range TLB in this work provides little benefit.

In this article, we use the unused high bits in a pointer to accelerate the address translation, but some systems also use these bits for other purposes. For example, ARM Memory Tagging Extension (MTE) stores an address tag at the top of a pointer to detect memory safety violations [26]. Applying FlexPointer with other tagged-pointer-based techniques is a tricky task because we must avoid the conflict of different tags. Since we currently provide far more available range IDs than an application needs, we can compress the range ID and provide space for other tags. Moreover, most tagged-pointer systems only use part of the unused bits, so we can add selection bits in the tag to switch between them flexibly. But in general, applying FlexPointer with another tagged-pointer-based techniques is a question that needs to be discussed case-by-case.

Our current pointer format is designed towards a 64-bit system that adopts a 48-bit virtual address space, but future architectures may adopt a larger address space [29, 54]. Intel begins to support 5-level paging in Ice Lake, and AMD will also add the feature in future CPUs. When enabling the 5-level paging, the length of a virtual address increases to 57 bits. With the most significant bit used for distinguishing kernel and user pointers, unused bits decrease to 6 bits. Moreover, a program that requires a 57-bit address space may present a different memory usage pattern, such as allocating more large objects. Considering these problems, we provide an alternative encoding format in Figure 7.

This alternative format is inspired by FRAMER [43] and Cryptographic Capability Computing ( \(C^3\) ) [40]. FRAMER defines frames as memory blocks that are \(2^n\) -sized and aligned by their size. The wrapper frame of memory object x is the smallest frame that completely contains x. FRAMER proves that a given wrapper frame can contain at most one object [43], so we can use it as an alternative range ID. A frame can be represented by its start address and the binary logarithm of its size. The Anchor in Figure 8 is a \(2^{N+2}\) -aligned address. The wrapper frame of range A starts at Anchor and ends at Anchor+ \(2^{N+1}\) , represented as (Anchor, N+1). Similarly, the wrapper frame of range B is represented as (Anchor, N+2). We use N+1 and N+2 as the pointer tag for addresses in ranges A and B, respectively. To translate an address tagged with N, the processor first clears the lowest N bits to get the wrapper frame start address and then combines it with N to form an ID. As long as the access is in-bounds, this process can be done in parallel with the address generation.

By using the position of a memory range as part of its ID, this format breaks the ID number limit. It requires 6 bits to store the tag and does not put extra constraints on the allocation process. We also exclude 0x0 and 0x3F to distinguish tagged and untagged pointers. When using the alternative format, the range table structure should also be changed. The new ID is much longer than 13 bits, so organizing the range table as a simple array may take up too much space. A hash table may be a more suitable choice, but the specific hash function implementation also affects the efficiency.

In this article, our work focuses on the format using a 13-bit ID. The main reason is that current benchmarks have no hard requirement for 5-level paging, so we think it is not proper to assume program behaviors under the 5-level paging, especially memory usage patterns, with current benchmarks. As a result, we leave related research for future work and provide this format only as a possible solution for 5-level paging. The remaining parts of this article are all based on the format in Section 3.3.

Eager paging strategy allocates physical memory rightly after the creation of a virtual memory area. But if a program does not access all of the allocated virtual pages, un-accessed physical pages are wasted, causing internal fragmentations. We tracked and examined the target address of every load/store instruction to find actually allocated pages in memory ranges. When a page is accessed for the first time, it is counted as an allocated page because, since then, it needs to be backed by a physical page.

Table 4 shows the total page number requested by memory ranges and the number of allocated pages. We can find that most of the requested pages are actually accessed, so eager paging should not cause serious internal fragmentation problems. Here, we take the percentage of pages not allocated as the internal fragmentation. As shown in the table, the eager paging strategy leads to less than 10% internal fragmentations for most of the benchmarks.

We also compared with the internal fragmentations of adopting 2M/1G huge pages. Currently, using 1G pages in a program relies on the hugetlbfs [15] mechanism, which requires modifications to the source code and the program managing some of the memory on its own. Here, we assume using hugetlbfs and that the program can perfectly compact all the ranges into 2 M/1 G huge pages. We then estimate the number of huge pages by rounding up the sum of range sizes. Fragmentation results in Table 4 show that the eager paging strategy leads to far less internal fragmentation than using 1 G pages to support memory ranges. Adopting 2 M huge pages uses a similar amount of memory to the eager paging, but requires much more TLB entries.

We evaluate the TLB hit/miss rates of FlexPointer with a functional TLB simulator, which modeled L1/L2 page TLB and the page table walker. The simulator uses 4 KB pages. We use Simpoint [47] to locate simulation points and extract load/store instructions with PIN [41]. All simulation points consist of 1 billion instructions. We also validated MPKI of the trace (baseline configuration enabling only page TLBs) with performance counter measurements of native executions to ensure that the sampled regions are representative. The coverage of simulation points is over 90% on every benchmark.

According to the results from previous studies [13, 31] and our evaluations in Table 1, we set the large object threshold to 64 KB to constrain the number of ranges. We implement a malloc() wrapper function as discussed in Section 3.2 and replace the original malloc() with it. Then we use PIN to catch all calls to mmap(), mremap(), and munmap() as hints for range creations and reclaims. These function calls are recorded both in and out of simulation points to completely reflect range creations and reclaims.

We also implemented RMM [33] and CoLT [49] in our simulator. For CoLT, we assume an ideal situation where every TLB entries cover eight contiguous physical pages. We let FlexPointer and RMM create ranges according to the same threshold. For FlexPointer, all tagged addresses go to the range TLB while the others go to the page TLB; for RMM, the range TLB is searched only after an L1 TLB miss. In FlexPointer, a page walk happens either after the L2 page TLB misses or after the range TLB misses; in RMM, a page walk happens after the L2 page TLB and the range TLB both miss. Misses caused by sub-ranges (generated by remapping) are included in the range TLB misses.

Figures 9 and 10 show the reduction of L1 TLB miss number and page table walk number, respectively. RMM only searches the range TLB after an L1 TLB miss, so it cannot reduce L1 TLB misses. For most benchmarks, FlexPointer can reduce 70–99+% of L1 TLB misses and page walks. Table 5 illustrates how FlexPointer and RMM affect the miss rate of page TLB. We can find that FlexPointer mainly reduces the L1 TLB miss rate. For some benchmarks, the L2 miss rate instead increases. This is because the decrease in L1 TLB misses reduces the total number of L2 TLB accesses, and some untagged addresses still miss in both L1 and L2 page TLB.

For gcc, xalancbmk, and omnetpp, neither FlexPointer nor RMM shows a good reduction of TLB miss number. Referring to the coverage results in Table 1, we find that for FlexPointer and RMM, the coverage of large objects is consistent with the TLB miss and page walk reduction. We then examined the allocation patterns of these benchmarks and found that they allocate much more objects than others, and most of the objects are below the threshold. So these three benchmarks benefit little from FlexPointer and RMM because the memory coverage of ranges is low. But it should also be noted that, although the reduction of TLB misses and page walks is low, FlexPointer does not hurt the performance of these benchmarks.

On the other hand, CoLT can reduce many of the L1 TLB misses and page table walks for gcc, xalancbmk, and omnetpp, but cannot reduce as many on other benchmarks. Because CoLT increases the coverage of a TLB entry at a relatively small scale, its efficiency on benchmarks using only a few very large objects is limited. TLB coalescing techniques like CoLT only modifies the page TLB, so they are orthogonal to FlexPointer. We can combine them with FlexPointer to improve performance for applications with many small objects.

We also noted that FlexPointer can reduce more page walks than RMM for all benchmarks except cactuBSSN. We assumed that cactuBSSN suffers from thrashing, so we changed the range TLB capacity and tested again. As Figure 11 shows, the number of L1 range TLB misses drops drastically when increasing the L1 range TLB capacity from 50 to 51. Thus we can come to a conclusion that cactuBSSN needs a larger range TLB to avoid thrashing. Although cactuBSSN endures range TLB thrashing, FlexPointer still reduces over 70% of page walks.

One possible problem of adding a range TLB is that it may instead introduce many range table walks if its hit rate is poor. We investigated range TLB hit rates of benchmarks using a 16-entry range TLB configuration. As shown in Table 6, FlexPointer induces nearly no range TLB misses for all benchmarks. A benchmark experiencing thrashing like cactuBSSN can still achieve a hit rate over 93%. From the results, we can see that even a small range TLB can provide very high hit rates, so the range TLB miss will not become a bottleneck for FlexPointer. Table 6 also proves that a 16-entry range TLB is sufficient for most benchmarks.

The overall performance is evaluated with Mosmodel [1]. Mosmodel implemented a special memory allocator to generate different page size mixes and produced a performance model using performance counter measurements under different mixes. Denoting L2 TLB hits, L2 TLB misses, and page walk cycles as \(H\) , \(M\) , and \(C\) , respectively, Mosmodel is a third-degree polynomial model:

We first execute each benchmark completely and use perf to get the original performance counter data. Then, we adjust the three parameters of each benchmark proportionally according to the functional simulation results and input them into the Mosmodel performance model to predict execution cycles.

For FlexPointer, the data of H and M are only from the page TLB because our range TLB works before the L2 page TLB. We count the range TLB misses as page walks. In this article, we assume a system with a low fragmentation level, so range table walks can complete on the first access, which means that a range table walk cannot be more costly than a default page walk. We denote the L2 page TLB hit and miss in FlexPointer simulation as \(PHit_f\) and \(PMiss_f\) , and denote the range TLB miss as \(RMiss_f\) . The L2 TLB hit and L2 TLB miss of the baseline simulation (only page TLB) are denoted as \(PHit\) , and \(PMiss\) . The adjusted performance model parameters are denoted as \(C_p\) , \(H_p\) , and \(M_p\) . We adjust parameters as follows:

For RMM, we view the range TLB as a means to reduce page walks. When an address hits in the L2 page TLB, we omit the lookup result in the range TLB, so the number of L2 page TLB hits is the same as that in the baseline. As for an L2 page TLB miss, if it hits in the range TLB, we view it as a miss canceled by the range TLB and subtract it from the final miss number. Only when an address misses both in the L2 page TLB and the range TLB, it is counted as an L2 page TLB miss, denoted as \(PMiss_r\) . Now we adjust parameters as follows:

As for CoLT, since it only modify the page TLB, the adjustment of parameters is straightforward. Denoting the L2 page TLB hit and miss of CoLT as \(PHit_c\) and \(PMiss_c\) , respectively, we adjust its parameters as follows:

Figure 12 presents the speedups of FlexPointer, RMM, and CoLT. All performance are normalized to the baseline system. We also compare the results with the speedups of THP as a reference because the internal fragmentations of using 2 MB huge pages are close to the eager paging strategy. Speedups of THP are calculated by comparing the elapsed time when executing with/without enabling THP on the real machine.

Among all benchmarks, gups benefits the most from FlexPointer and RMM. FlexPointer grants a 180% performance improvement and RMM shows a 150% improvement. Considering that gups allocates a large area and does random access within it, this benchmark presents a memory usage pattern that best fits the range translation. On the other hand, CoLT improves little on gups because of the limited coverage of TLB entries and the poor memory locality.

For most of the benchmarks, FlexPointer shows a better performance than RMM and CoLT. Some benchmarks, like xz, graph500, XSBench, and NPB:CG, can be accelerated by FlexPointer for 10–20%. Improvement for the remaining benchmarks is not as significant, but most of them experience around 5% performance improvement with FlexPointer. For benchmarks with a high range coverage, FlexPointer can offer a higher speedup than THP, except for XSBench and NPB:MG. For these benchmarks, one possible reason is that THP mapped the stack with huge pages, while FlexPointer only works on the heap.

We also repeated the experiments on the remaining SPEC 2017 benchmarks to assess if FlexPointer causes performance degradation. Among them, exchange2, povray, perlbench, and nab allocate few or no memory range. They show similar results to gcc, xalancbmk, and omnetpp. Other benchmarks, like namd, parest, blender, wrf, and leela, can achieve a higher range coverage (around 40%). Remaining benchmarks (waves, lbm, x264, pop2, imagick, and fotonik3d) allocate even more large objects (range coverage over 80%). These benchmarks already have high TLB hit rates when using 4 KB pages, so they experience only a little improvement, around 1–4%. None of these benchmarks experiences performance degradation.

Overall, FlexPointer offers an average performance improvement of around 14% (5.9% if excluding gups), while RMM and CoLT offer an 11% (4.7% if excluding gups) and 3% average improvement, respectively. On the other hand, THP offers a 12% (6.0% if excluding gups) average improvement.