The Vienna LTE simulators - Enabling reproducibility in wireless communications research

Abstract

In this article, we introduce MATLAB-based link and system level simulation environments for UMTS Long-Term Evolution (LTE). The source codes of both simulators are available under an academic non-commercial use license, allowing researchers full access to standard-compliant simulation environments. Owing to the open source availability, the simulators enable reproducible research in wireless communications and comparison of novel algorithms. In this study, we explain how link and system level simulations are connected and show how the link level simulator serves as a reference to design the system level simulator. We compare the accuracy of the PHY modeling at system level by means of simulations performed both with bit-accurate link level simulations and PHY-model-based system level simulations. We highlight some of the currently most interesting research questions for LTE, and explain by some research examples how our simulators can be applied.

1. Introduction

Reproducibility is one of the pillars of scientific research. Although reproducibility has a long tradition in most nature sciences and theoretical sciences, such as mathematics, it is only recently that reproducible research has become more and more important in the field of signal processing [1, 2]. In contrast to results in fields of purely theoretical sciences, results of signal processing research articles can be reproduced only if a comprehensive description of the investigated algorithms (including the setting of all necessary parameters), as well as eventually required input data are fully available. Owing to the lack of space, a fully comprehensive description of the algorithm is often omitted in research articles. Even if an algorithm is explained in detail, for instance, by a pseudo code, initialization values are often not fully defined. Moreover, it is often not possible to include in an article all the necessary resources, such as data, which were processed by the presented algorithms. Ideally, all resources, including source code of the presented algorithms, should be made available for download to enable other researchers (and also reviewers of articles) to reproduce the results presented. Unfortunately, researcher's reality does not resemble this ideal situation, a circumstance that has recently been quite openly complained about [3].

In the past few years, several researchers have started to build up online resource databases in which simulation code and data are provided, see for example [4, 5]. However, it is still not a common practice in signal processing research. We are furthermore convinced that reproducibility should also play an important role in the review process of an article. Although thorough checking is very possibly impractical, it would make the presented studies more transparent to the review process. Reproducibility becomes even more important when the systems that are simulated become more and more complex, as it is the case in the evaluation of wireless communication systems. When algorithms for wireless systems are evaluated, authors often claim to use a standard-compliant transmission system and simply make reference to the corresponding technical specification. Since technical specifications are usually extensive, including a cornucopia of options, it is not always clear which parts of a specification were actually implemented and which parts were omitted for the sake of simplicity reasons. The situation of trying to reproduce someone else's results to compare them to one's own algorithm but not being able to do so (or only after extensive effort to discover the unreported details of the actual implementation) is familiar to most researchers. Without access to the details of the implementation, including all assumptions, comparisons of algorithms, developed by different researchers, are very difficult, if not impossible to carry out. A way out of this dilemma is offered by a publicly available simulation environment. In this study, we present such an open-source simulation environment that supports link and system level simulations of the Universal Mobile Telecommunications System (UMTS) Long-Term Evolution (LTE), specifically designed to support reproducibility. The development and publishing of this LTE simulation environment is based on our previous very good experience with a WiMAX physical layer simulator [6].

Furthermore, such simulators can be used as a reference for validation of algorithms, for example, when designing transmitter or receiver chips [7]. We also have used our simulators for generating LTE signals that are required to include realistic signals in related research [8], or as a reference for LTE-compliant measurements. In such cases, the simulator can serve not only as a data pump, but also as a vehicle to evaluate the received data.

LTE, the current evolutionary step in the third Generation Partnership Project (3GPP) roadmap for future wireless cellular systems, was introduced in 3GPP Release 8 [9]. Besides the definition of the novel physical layer, LTE also contains many other remarkable innovations. Most notable are (i) the redevelopment of the system architecture, now called System Architecture Evolution (SAE), (ii) the definition of network self-organization, and (iii) the introduction of home base-stations. The main reasons for these profound changes in the Radio Access Network (RAN) system design are to provide higher spectral efficiency, lower delay (latency), and more multi-user flexibility than the currently deployed networks.

In the development and standardization of LTE, as well as in the implementation process of equipment manufacturers, simulations are necessary to test and optimize algorithms and procedures. This has to be carried out on the physical layer (link level) and in the network (system level) context:

• 1) Link level simulations allow for the investigation of channel estimation, tracking, and prediction algorithms, as well as synchronization algorithms [10, 11]; Multiple-Input Multiple-Output (MIMO) gains; Adaptive Modulation and Coding (AMC); and feedback techniques [12, 13]. Furthermore, receiver structures (typically neglecting inter-cell interference and impact of scheduling, as this increases simulation complexity and runtime dramatically) [14], modeling of channel encoding and decoding [15], physical-layer modeling crucial, for system level simulations [16], and the like are typically analyzed on link level. Although MIMO broadcast channels have been investigated quite extensively over the past few years [17, 18], there are still a lot of open questions that need to be resolved, both in theory and in practical implementation. For example, LTE offers the flexibility to adjust many transmission parameters, but it is not clear up to now how to exploit the available Degrees of Freedom (DoF) to achieve the optimum performance. Some recent theoretical results point out how to proceed in this matter [18, 19], but practical results for LTE are still missing.

• 2) System level simulations focus more on network-related issues, such as resource allocation and scheduling [20, 21], multi-user handling, mobility management, admission control [22], interference management [23, 24], and network planning optimization [25, 26]. Furthermore, in a multi-user oriented system, such as LTE, it is not directly clear which figures of merit should be used to assess the performance of the system. The classical measures of (un)coded Bit Error Ratio (BER), (un)coded BLock Error Ratio (BLER), and throughput are not covering multi-user scenario properties. More comprehensive measures of the LTE performance are, for example, fairness, multi-user diversity, or DoF [27]. However, these theoretical concepts have to be mapped to performance values that can be evaluated by means of simulations [28, 29].

Around the world, many research facilities and vendors are investigating the above mentioned aspects of LTE. For that purpose, commercially available simulators applied in industry [30–32], as well simulators applied in academia [33], have been developed. Also, probably all major equipment vendors have implemented their own, proprietary simulators. Regardless of the simulation tools being commercial/noncommercial, the development framework (C, C++, MATLAB, WM-SIM [33],...), or their claimed performance/flexibility, one fact is shared by all of the simulators. Their closed implementation disables access to implementation details and thus to any assumption that may have been included. As such, the reliability of the results relies purely on the faith of a proper implementation. Independent validation of results in such closed simulation environments is not easy, very time-consuming, and often not feasible. Since the results were obtained with closed tools, simply repeating the same experiment is a daunting task. Transparency not only in the results, but also in the tools employed, thus greatly magnifies the credibility of the results.

The two simulators [34, 35] described in Sections 2 and 3 of this article are freely available at our homepage http://www.nt.tuwien.ac.at/ltesimulator/ under an open, free for non-commercial academic use license, which facilitates academic research and enables a closer cooperation between different universities and research facilities. In addition, developed algorithms can be shared under the same license again, making the comparison of algorithms easier, reproducible, and therefore refutable and more credible. To the best of the authors' knowledge, our two simulators are the first to be published in the context of LTE, including source code under an academic use license. Thus, the simulators provide opportunities for many institutions to directly apply their ideas and algorithms in the context of LTE. The availability of the simulators, together with the possibility to include links to the utilized simulator version and any resources needed furthermore, enables researchers to quickly reproduce published results [2].

The remainder of this article is organized as follows. In Sections 2 and 3, we describe the Vienna LTE Simulators and how they relate to each other. In Section 4, we provide a validation of the two simulators. Exemplary simulation results are shown in Section 5. Finally, we conclude the article in Section 6.

2. The Vienna LTE link level simulator

In this section, we describe the overall structure of the Vienna LTE Link Level Simulator, currently (January 2011) released in version 1.6r917. Furthermore, we present the capabilities of the simulator and provide some examples of its application.

A. Structure of the simulator

The link level simulator can be divided into three basic building blocks, namely, transmitter, channel model, and receiver (see Figure 1). Depending on the type of simulation, one or several instances of these basic building blocks are employed. The transmitter and receiver blocks are linked by the channel model, which is used to transmit the downlink data, while signaling and uplink feedback is assumed to be error-free. Since signaling is stronger protected than data, by means of lower coding rates and/or lower-order modulations, the assumption of error-free signaling is in fact quite realistic. Equivalently, errors on the signaling channels will only occur when the data channels are already facing severe performance degradation--a point of operation usually not targeted in investigations.

Figure 1

LTE link level simulator overall structure, as implemented in the Vienna LTE link level simulator. The simulator comprises by one or more transmitter blocks, channel modeling for each link, and receiver blocks. The feedback channel is implemented as a delayed error-free signaling channel.

In the downlink, the signaling information passed on by the transmitter to the receiver contains coding, HARQ, scheduling, and precoding parameters. In the uplink, Channel Quality Indicator (CQI), Precoding Matrix Indicator (PMI), and Rank Indicator (RI) are signalled, which together form the Channel State Information (CSI) feedback. All simulation scenarios (see Section 2-B) support the feedback of CQI, PMI, and RI, although it is also possible to set some or all of them to fixed values. Such a setting is required for specific simulations, such as throughput evaluation of a single Modulation and Coding Scheme (MCS).

A standard-compliant implementation of the downlink control channels would not affect the overall structure of our simulator and just requires the insertion of the control channels in the relevant resource elements [36]. On the other hand, non-error-free feedback transmissions would require a physical layer implementation of the LTE uplink, which is currently not in the scope of the simulator. (A first release of the uplink, however, is currently being implemented in the simulator and will be released soon.)

1) Transmitter

The layout of the transmitter is shown in Figure 2, which is also a graphical representation of the transmitter description defined in the TS36' standard series [36–38]. Based on User Equipment (UE) feedback values, a scheduling algorithm assigns Resource Blocks (RBs) to UEs and sets an appropriate MCS (coding rates between 0.076 and 0.926 with 4, 16, or 64-QAM modulation [38]), the MIMO transmission mode (Transmit Diversity (TxD), Open Loop Spatial Multiplexing (OLSM), or Closed Loop Spatial Multiplexing (CLSM)), and the precoding/number of spatial layers for all served users. Such a channel adaptive scheduling allows for the exploitation of frequency diversity, time diversity, spatial diversity, and multi-user diversity.

Figure 2

LTE downlink transmitter implementation in the Vienna LTE link level simulator, as specified in [36–38].

Given the number of available DoF, the specific implementation of the scheduler algorithm has a large impact on the system performance and has been a hot topic in research [39–41]. In Section 5-B, we provide performance evaluations of several schedulers.

2) Channel model

The Vienna LTE Link Level Simulator supports block- and fast-fading channels. In the block-fading case, the channel is constant during the duration of one subframe (1 ms). In the fast-fading case, time-correlated channel impulse responses are generated for each sample of the transmit signal. Currently (January 2011), the simulator supports the following channel models:

• 1) Additive White Gaussian Noise (AWGN);

• 2) Flat Rayleigh fading;

• 3) Power Delay Profile-based channel models, such as ITU Pedestrian B, or ITU Vehicular A [42];

• 4) Winner Phase II+ [43]

The most sophisticated of these channel models is the Winner Phase II+ model. It is an evolution of the 3GPP spatial channel model, and introduces additional features, such as support for arbitrary 3D antenna patterns.

3) Receiver

Figure 3 shows our implementation of the UE receiver. After disassembling the RBs according to the UE resource allocation, MIMO Orthogonal Frequency Division Multiplexing (OFDM) detection is carried out. The simulator currently supports Zero-Forcing (ZF), Linear Minimum Mean Squared Error (LMMSE), and soft sphere decoding as detection algorithms. The detected soft bits are decoded to obtain the data bits and several figures of merit, such as coded/uncoded BER, BLER, and throughput.

Figure 3

LTE downlink receiver structure, as implemented in the Vienna LTE link level simulator.

Currently, four different types of channel estimators are supported within the simulator: (i) Least Squares (LS), (ii) Minimum Mean Squared Error (MMSE), (iii) Approximate LMMSE [44], and (iv) genie-driven (near) perfect channel knowledge based on all transmitted symbols.

LTE requires UE feedback to adapt the transmission to the current channel conditions. The LTE standard specifies three feedback indicators for that purpose: CQI, RI, and PMI [36]. The CQI is employed to choose the appropriate MCS, such as to achieve a predefined target BLER, whereas the RI and the PMI are utilized for MIMO pre-processing. Specifically, the RI informs the eNodeB about the preferred number of parallel spatial data streams, while the PMI signals the preferred precoder that is stemming from a finite code book as specified in [36]. Very similar feedback values are also employed in other systems such as WiMAX and WiFi. The simulator provides algorithms that utilize the estimated channel coefficients to evaluate these feedback indicators [13]. Researchers and engineers working on feedback algorithms can implement other algorithms using the provided feedback functions as a starting point to define their own functions.

Given this receiver structure, the simulator allows the investigation of various aspects, such as frequency synchronization [45], channel estimation [44], or interference awareness [46].

B. Complexity

Link level simulators are in practice a direct standard-compliant implementation of the Physical (PHY) layer procedures, including segmentation, channel coding, MIMO, transmit signal generation, pilot patterns, and synchronization sequences. Therefore, implementation complexity and simulation time are in general high. To obtain a simulator with readable and maintainable code, a high-level language (MATLAB) has been chosen. This choice enabled us to develop the simulator in a fraction of the time required for an implementation in other languages such as C. Furthermore, MATLAB ensures cross-platform compatibility. While MATLAB is certainly slower than C, by means of code optimization (vectorization) and parallelization by the MATLAB Parallel/Distributed Computing Toolbox, simulation runtime can be greatly reduced. Severely difficult-to-vectorize and often-called functions are implemented in C and linked to the MATLAB code by means of MEX functions. Such functions include the channel coding/decoding [47], Cyclic Redundancy Check (CRC) computation [48], and soft sphere decoding.

Furthermore, it is possible to adjust the scale of the simulation to the specific needs. This is achieved by introducing three different simulation types with largely different computational complexity (Figure 4):

Figure 4

Three possible scenarios in the Vienna LTE link level simulator allow us to adjust the scale of the simulation complexity: single-downlink, single-cell multi-user, and multi-cell multi-user.

1) Single-downlink

This simulation type only covers the link between one eNodeB and one UE. Such a set-up allows for the investigation of channel tracking, channel estimation [44], synchronization [11, 49], MIMO gains, AMC and feedback optimization [13], receiver structures [14] (neglecting interference and impact of the scheduling,^a modeling of channel encoding and decoding [15, 50], and physical layer modeling [51], which can be used for system level abstraction of the physical layer. To start a simple single-downlink simulation, run the file LTE _sim _batch _single _downlink.m.

2) Single-cell multi-user

This simulation covers the links between one eNodeB and multiple UEs. This set-up additionally allows for the investigation of receiver structures that take into account the influence of scheduling, multi-user MIMO resource allocation, and multi-user gains. Furthermore, this set-up allows researchers to investigate practically achievable multi-user rate regions. In the current implementation, the simulator fully evaluates the receivers of all users. However, if receiver structures are being investigated, the computational complexity of the simulation can considerably be reduced by only evaluating the user of interest. In order to enable a functional scheduler, it is sufficient to compute just the feedback parameters for all other users. To start a simple single-cell multi-user simulation, run the file LTE _sim _batch _single _cell _multi _user.m.

3) Multi-cell multi-user

This simulation is by far the computationally most demanding scenario and covers the links between multiple eNodeBs and UEs. This set-up allows for the realistic investigation of interference-aware receiver techniques [52], interference management (including cooperative transmissions [53] and interference alignment [54, 55]), and network-based algorithms such as joint resource allocation and scheduling. Furthermore, despite the vast computational efforts needed, such simulations are crucial to verify system level simulations. To start a simple multi-cell multi-user simulation, run the file LTE _sim _batch _multi _cell _multi _user.m.

The simulation time, which depends mainly on the desired precision and statistical accuracy of the simulation results, the selected bandwidth, the transmission mode, and the chosen modulation order, is for most users a crucial factor. It should be noted that by a smart choice of the simulation settings, the simulation time can be decreased (e.g., when investigating channel estimation performance, the smallest bandwidth can be sufficient).

3. The Vienna LTE system level simulator

In this section, we describe the overall structure of the Vienna LTE System Level Simulator, currently developed (January 2011) version 1.3r427. We furthermore show how the PHY layer procedures have been abstracted in a low complexity manner.

A. Structure of the simulator

In system level simulations, the performance of a whole network is analyzed. In LTE, such a network consists of a multitude of eNodeBs that cover a specific area in which many mobile terminals are located and/or moving around. While simulations of individual physical layer links allow for the investigation of MIMO gains, AMC feedback, modeling of the channel code, and retransmissions [13, 44, 45, 50, 56], it is not possible to reflect the effects of cell planning, scheduling, or interference in a large scale with dozens of eNodeBs and hundreds of users. Simply performing physical layer simulations of the radio links between all terminals and base-stations is unfeasible for system level investigations because of the vast amount of computational power required. Thus, the physical layer has to be abstracted by simplified models capturing its essential dynamics with high accuracy at low complexity.

Based on the standard approach in the literature [51, 57], our simulator consists of two parts: (i) a link measurement model, and (ii) a link performance model. The link measurement model reflects the link quality, given by the UE measurement reports, and is required to carry out link adaptation and resource allocation. The chosen link quality measure is evaluated per subcarrier. Based on the Signal to Interference and Noise Ratio (SINR), the UE computes the feedback (PMI, RI, and CQI), which is employed for link adaptation at the eNodeB as described in Section 2-A. The scheduling algorithm assigns resources to users to optimize the performance of the system (e.g., in terms of throughput) based on this feedback [21]. Based on the link measurement model, the link performance model predicts the BLER of the link, based on the receiver SINR and the transmission parameters (e.g., modulation and coding). Figure 5 illustrates the interaction between the two models and the several physical layer parameters.

Figure 5

Schematic block diagram of the LTE system level simulator. Link quality is evaluated by means of the link-measurement model, while the link-performance model maps it to BLER and outputs link throughput and error distribution.

Implementation-wise, the simulator follows the structure shown in Figure 6. Each network element is represented by a suitable class object, whose interactions are described below.

Figure 6

Schematic class diagram showing the implementation of the link-to-system model in the LTE System Level Simulator and depicting the relationship between the several elements/classes in the simulator. Implementation-wise, both the link-measurement and the link-performance model reside in the UE class.

In order to generate the network topology, transmission sites are generated, to which three eNodeBs are appended, i.e., sectors, each containing a scheduler (see Figure 6). In the simulator, traffic modeling assumes full buffers in the downlink. A scheduler assigns PHY resources, precoding matrices, and a suitable MCS to each UE attached to an eNodeB. The actual assignment depends on the scheduling algorithm and the received UE feedback.

At the UE side, the received subcarrier post-equalization symbol SINR is calculated in the link measurement model. The SINR is determined by the signal, interference, and noise power levels, which are dependent on the cell layout (defined by the eNodeB positions, large-scale (macroscopic, macro-scale) pathloss, shadow fading [58]), and the time-variant small-scale (microscopic, micro-scale) fading [59].

The CQI feedback report is calculated based on the subcarrier SINRs and the target transport BLER. The CQI reports are generated by an SINR-to-CQI mapping [35] and made available to the eNodeB implementation via a feedback channel with adjustable delay. At the transmitter, the appropriate MCS is selected by the CQI to achieve the targeted BLER during the transmission. Especially in high mobility scenarios, the feedback delay caused by computation and signaling timings can lead to a performance degradation if the channel state changes significantly during the delay. In the link performance model, an AWGN-equivalent SINR (γAWGN) is obtained via Mutual Information Effective Signal to Interference and Noise Ratio Mapping (MIESM) [60–62]. In a second step, γ_AWGN is mapped to BLER via AWGN link performance curves [34, 35]. The BLER value acts as a probability for computing ACK/NACKs, which are combined with the Transport Block (TB) size to compute the link throughput. The simulation output consists of traces, containing link throughput and error ratios for each user, as well as cell aggregates, from which statistical distributions of throughputs and errors can be extracted.

B. Complexity

One desirable functionality of a system level simulator is the ability to precalculate as many of the simulation parameters as possible. This not only reduces the computational load while carrying out a simulation, but also offers repeatability by loading an already partly precalculated scenario.

The precalculations involved in the LTE system level simulator are the generation of (i) eNodeB-dependent large-scale pathloss maps, (ii) site-dependent shadow fading maps, and (iii) time-dependent small-scale fading traces for each eNodeB-UE pair.

1) Pathloss and fading maps

The large-scale pathloss and the shadow fading are modeled as position-dependent maps. The large-scale pathloss is calculated according to well-known models [58, 63] and combined with the antenna gain pattern of the corresponding eNodeB. Space-correlated shadow fading is obtained from a log-normal random distribution using a low-complexity variant of the Cholesky decomposition [64]. Inter-site map correlation for shadow fading is similarly obtained. Figure 7 shows exemplary large-scale pathloss and shadow fading maps.

Figure 7

eNodeB- and site-dependance of the large-scale pathloss and shadow fading. Left: Large-scale pathloss and antenna gain map [dB] corresponding to the lower-leftmost eNodeB. Right: space-correlated shadow fading corresponding to the site [dB].

2) Time-dependent fading trace

While the large-scale pathloss and the shadow fading are modeled as position-dependent trace, the small-scale fading is modeled as a time-dependent trace. The calculation of this latter trace is based on the transmitter precoding, the small-scale fading MIMO channel matrix, and the receive filter. Currently, the receiver modeling is based on a linear ZF receiver. The small-scale fading trace consists of the signal power and the interference power after the receive filter. The break-down into these two parts significantly reduces the computational effort since it avoids many complex multiplications required when directly working with MIMO channel matrices on system level [16, 35, 51].

4. Validation of the simulators

The validation of the simulators was performed in two steps. First, in Section 4-A we compared the link level throughput with the minimum performance requirements stated by 3GPP in the technical specification TS 36.101 [65]. Second, in Section 4-B, we cross-validated the link and the system level simulators by comparing their results against each other. Other means of validation are being discussed in Section 4-C.

A. 3GPP minimum performance requirements

The technical specification TS 36.101 [65] defines minimum performance requirements for a UE that utilizes a dual-antenna receiver. These requirements have to be met by real devices and therefore have to be surpassed by our simulator, in which not every conceivable influential factor is incorporated.^b Such factors may include frequency and timing synchronization as well as other non-ideal effects, such as quantization or non-ideality of the manufactured physical components (e.g., I/Q imbalances, phase noise, and power amplifier nonlinearities).

In particular, TS 36.101 specifies reference measurement channels for the Physical Downlink Shared Channel (PDSCH) (comprising bandwidth, AMC scheme, overhead,...) and propagation conditions (power delay profiles, Doppler frequencies, and antenna correlation). The considered simulation scenarios are completely specified by referring to sections and test numbers in TS 36.101. For example, in TS 36.101 Section 8.2.1.1.1, the tests for a single transmit antenna N _T = 1 and dual receive antenna N _R = 2 scenario are defined. By referring to test number one in Section 8.2.1.1.1 of TS 36.101, the AMC mode is defined as Quadrature Phase Shift Keying (QPSK) with a target coding rate of 1/3, Extended Vehicular A (EVehA) channel model with a Doppler frequency of 5 Hz, and low antenna correlation. For our simulations presented in this article, we selected four test scenarios with a bandwidth of 10 MHz but different transmit modes (single antenna port transmission, OLSM, and TxD), different AMC schemes, and different channel models. Hybrid Automatic Repeat reQuest (HARQ) is supported by at most three retransmissions. The most important parameters of the test scenarios are listed in Table 1. The first scenario (8.2.1.1.1/1) refers to the test scenario described above. The OLSM scenario (8.2.1.3.2/1) utilizes a rank two transmission, that is, transmission of two spatial streams.

Table 1 Test scenarios of 3GPP TS 36.101.

Simulation results for the considered scenarios are shown in Figure 8. The dashed horizontal lines correspond to 70% of the maximum throughput values for which TS 36.101 defines a channel signal-to-noise ratio (SNR) requirement (shown as crosses in Figure 8). For all the considered test scenarios, the link level simulator outperforms the minimum requirements by approximately 2-3 dB. The small vertical bars within the markers in 8 are the 99% confidence intervals of the simulated mean throughput. Since the confidence intervals are much smaller than the distances between the individual throughput curves, we know that a repeated simulation with different seeds of the random number generators will lead to similar results and conclusions. Figure 8 can be reproduced by calling the script Reproducibility_RAN_sims.m included in the link level simulator.

Figure 8

Throughput simulations of the test scenarios in 3GPP TS 36.101 and comparison to the minimum performance requirements (marked with crosses). The small vertical bars within the circular markers indicate the 99% confidence intervals. Reproducible by running Reproducibility _RAN _sims.m.

B. Link and system level cross-comparison

Next, we cross-compare the performance of the link and system level simulators. We consider a single user single-cell scenario with different antenna configurations and transmit modes, as summarized in Table 2.

Table 2 Test scenarios for the cross-comparison of the link and system level simulators (SU CASE)

Depending on the channel conditions, we adapt the AMC scheme, the transmission rank, and the precoding matrices. For this purpose, we utilize the UE feedback schemes originally presented in [13]. In order to create an equivalent simulation scenario on link and system level, we do not employ shadow fading. While on link level the SNR is usually directly specified, on system level the SNR is a function of the user location in the cell. Without shadow fading, the user SNR on system level becomes a function of the distance between base-station and the user. This can be utilized to indirectly select appropriate SNR values in the system level simulator. The results of the link and system level comparisons are shown in Figure 9. For all the considered simulation scenarios, we obtain an excellent match between the results of the two simulators, confirming the validity of our Link Error Prediction (LEP) model [50] on system level. Figure 9 can be reproduced by running the script Reproducibility _LLvsSL _batch.m provided in the system level simulator package. Further comparisons between link and system level simulator results are shown in Section 5-B.

Figure 9

Cross-comparison of throughput results obtained with the link level and the system level simulators. The small vertical bars within the circular markers indicate the 99% confidence intervals. Reproducible by running Reproducibility _LLvsSL _batch.m.

In Table 2 we compare the simulation times of the link level simulator to those of the system level simulator. The simulations were conducted on a single core of a 2.66 GHz Quad Core CPU. The table also states the simulation speed-up, defined as the ratio of the simulation times required with the link level and the system level simulator, respectively. The speed-up of the system level simulator for a Single-Input Single-Output (SISO) system equals four. This speed-up is rather small because equalization, demodulation, and decoding (tasks that are abstracted on system level) have low complexity in a SISO system. With increasing system complexity also the speed-up increases. We expected the largest speed-up in the CLSM scenario, because it utilizes the largest antenna configuration. However, we measured the largest speed-up of almost 18 in the OLSM simulation scenario. The reason is, that the precoder changes from one subcarrier to the next, while in the CLSM scenario, we assumed wideband feedback meaning that the same precoder is employed on all the subcarriers [13].

The link level simulator supports the parallel computing capabilities of MATLAB. With these features, it is possible to run several MATLAB instances in parallel on the multiple cores of a modern CPU. The simulation time of the link level simulator then decreases linearly with the number of CPU cores, while the system level simulator is currently not capable of parallel computing.

C. Further validation means

For a basic validation of the correctness of the results produced by the simulator, we checked the uncoded BER and throughput performance over frequency flat Rayleigh fading and AWGN channels, as the theoretical performance of these channels is known [66]. Furthermore, we cross-checked our results with those produced by the other industry simulators, by comparing with corresponding publications of the 3GPP RAN WG1, e.g., [28, 29]. Still, an open issue is to prove a correct functionality of each part of the simulator. Evaluation of the simulators has also been made possible for the whole research community, allowing everybody to modify the code to meet individual requirements and to check the code for correctness [67–69], as the simulator's changelog reflects. The first versions of the simulators have been released in May 2009 (link level simulator) and in March 2010 (system level simulator), respectively. To facilitate the exchange of bugs and/or results often referred to as "crowdsourcing," a forumc is also provided. While the authors acknowledge this is not a perfect form of validation, neither is any other.

5. Exemplary results

In this section, we show two exemplary simulation results obtained with the Vienna LTE simulators. First, we present a link level throughput simulation in which we compare the throughput of the different MIMO schemes to theoretic bounds. Based on this simulation setup, researchers can investigate algorithms such as channel estimation, detection, or synchronization. Second, we compare the performance of different state-of-the-art schedulers in a single-cell multi-user environment. These schedulers serve as reference for researchers investigating advanced scheduling techniques.

A. Link level throughput

Before presenting the link level throughput results of the different LTE MIMO schemes, we introduce theoretic bounds for the throughput. We identify three bounds, namely, the mutual information, the channel capacity, and the so-called achievable mutual information. Depending on the type of channel state information available at the transmitter (only receive SNR, full, or quantized), an ideal transmission system is expected to attain one of these bounds.

1) Mutual information

The mutual information is the theoretic bound for the data throughput if only the receive SNR but no further channel state information is available at the transmitter side [70]:

(1)

where B_sub denotes the bandwidth occupied by a single data subcarrier, H _k the N_R×N_T (= number of receive antennas × number of transmit antennas) dimensional MIMO channel matrix of the k-th subcarrier, the energy of noise and interference at the receiver, N_tot the total number of usable subcarriers, and an identity matrix of size equal to the number of receive antennas N R. In Equation (1), we normalized the transmit power to one and the channel matrix according to . Therefore, Equation (1) does not show a dependence on the transmit power and the number N _T of transmit antennas.

The bandwidth B_sub of a subcarrier is calculated as

(2)

where N_s is the number of OFDM symbols in one subframe (usually equal to 14 when the normal cyclic prefix length is selected), T_sub the subframe duration (1 ms), and T_cp the time required for the transmission of all cyclic prefixes within one subframe. Note that we are calculating the mutual information for all usable subcarriers of the OFDM system, thereby taking into account the loss in spectral efficiency caused by the guard band carriers. If different transmission systems that apply different modulation formats are to be compared, however, a fair comparison then requires calculating the mutual information over the entire system bandwidth instead of calculating it only over the usable bandwidth.

Current communication systems employ adaptive modulation and coding schemes to optimize the data throughput. For a specific receive SNR, assuming an optimum receiver, the modulation and coding scheme that maximizes the data throughput can be selected. Thus, if the transmitter knows the receive SNR, a throughput equal to the mutual information should be achieved.

2) Channel capacity

For calculating the channel capacity of a frequency selective MIMO channel [66], consider the singular value decomposition of the channel matrix H _k scaled by the standard deviation σ_n of the additive white Gaussian noise impairment:

(3)

The optimum, capacity-achieving, frequency-dependent precoding at the transmitter is given by the unitary matrix V _k. If this precoding matrix is applied at the transmitter and also the optimum receive filter is employed, then the MIMO channel is separated into min(N_R, N_T) (with N R denoting the number of receive antennas and N_T the number of transmit antennas) independent SISO channels, each with a gain of , m = 1...min(N_R, N_T), k = 1... N_tot. The channel capacity is obtained by optimally distributing the available transmit power over these parallel SISO subchannels. The optimum power distribution P_{k, m}is the solution of the optimization problem:

(4)

where the second equation is a transmit power constraint that ensures an average transmit power equal to the number of data subcarriers: P_t = N_tot. Note that owing to the definition of in Equation (3), the power distribution P_{k, m}and thus P_t remain dimensionless. We calculate the power coefficients maximizing Equation (4) by the water-filling algorithm described in [66]. In order to achieve a throughput equal to the channel capacity, the transmitter needs full channel state information and has to apply the optimum precoder. Furthermore, the receiver needs to apply the optimum receive filter to separate the parallel SISO subchannels.

3) Achievable mutual information

Both mutual information and channel capacity do not consider system design losses caused, for example, by the transmission of cyclic prefix or reference symbols, or the quantization of the transmitter precoding. In order to obtain a tighter bound for the link level throughput, we therefore consider these effects in the definition of the so-called achievable mutual information. In the case of open-loop transmission, in which space-time coding is employed at the transmitter, we obtain for the achievable mutual information:

(5)

with N_L denoting the number of spatial transmission layers. The N_RN_L×N T dimensional matrix is the effective channel matrix including the space-time coding [71]. The factor F accounts for the inherent system losses due to the transmission of the cyclic prefix and the reference symbols. In detail, the factor F is calculated as

(6)

where N_ref is the number of reference symbols per resource block, and N_sc = 12 is the number of subcarriers in each RB. In LTE, the number of reference symbols depends on the number of transmit antennas. Therefore, the efficiency factor F decreases with increasing number of transmit antennas (see Table 3).

Table 3 pilot symbols and efficiency factor F in LTE

In the case of closed-loop transmission, a channel-adapted precoding matrix W is chosen from a set (defined in the standard) and applied to the transmit signal. We calculate the achievable mutual information for closed-loop transmission as

(7)

In Figure 10, the throughput of a 2×2 LTE system with 5 MHz bandwidth, perfect channel knowledge, and a Soft Sphere Decoder (SSD) receiver is shown and compared to the previously presented theoretic bounds. The difference between channel capacity and mutual information is only small, and, therefore, even knowledge of the full channel state information at the transmitter does not considerably increase the potential performance. In contrast, the difference between the mutual information and the achievable mutual information is much larger, resulting in a loss of 56% at an SNR of 15 dB. Most (41%) of this loss is due to the restrictions implied by the standard, as indicated by the achievable mutual information curves in Figure 10. At a rate of 16 Mbit/s, the difference between achievable mutual information and simulated throughput is approximately 4 dB. These findings are similar to the results we obtained when analyzing the performance of WiMAX and High-Speed Downlink Packet Access (HSDPA) in [72].

Figure 10

Throughput of a 2×2 system with 5-MHz bandwidth compared to the channel capacity, the mutual information, and the achievable mutual information. The small vertical bars within the circular markers indicate the 99% confidence intervals. Reproducible by running Physical _Layer _batch.m.

Figure 10 furthermore shows that, for SNRs lower than 14 dB, the TxD mode outperforms OLSM. Only at larger SNRs, above 20 dB, where the throughput of the TxD mode saturates, OLSM benefits from the second spatial stream and outperforms TxD.

Figure 10 can be reproduced by executing the script Physical _Layer _batch.m provided in the Vienna LTE Link Level Simulator package.

B. LTE scheduling

In this section, the performance of various multiuser LTE scheduling techniques is compared by means of link level and system level simulations. By appropriately selecting the simulation parameters in the link level, as well as the system level, we are able to show that the results obtained by the two simulators are equivalent.

In particular, we consider in the Vienna LTE System Level Simulator one sector of a single-cell SISO system with 20 randomly positioned users. The user positions yield the large-scale path loss and shadow fading coefficients of all the users, and as a consequence, the average receive SNRs, which are distributed in a range of 2.7 dB to 36 dB. These average receive SNRs of the 20 users are set in the Vienna LTE Link Level Simulator to ensure the same propagation environment as on system level. Further simulation parameters of both simulators are summarized in Table 4.

Table 4 Link and system level parameters for the scheduling simulations

The simulation results are averaged over 2,500 small-scale fading and noise realizations. In order to guarantee exactly the same channel realizations for all scheduler simulations on system level, the user positions, as well as the small- and large-scale fading realizations are loaded from pre-generated files. On link level, the seeds of the random number generators for fading and noise generation are set at the beginning of each simulation.

A performance comparison of different scheduling strategies is shown in Figures 11 and 12 in terms of total sector throughput and fairness (Jain's fairness index [73]). The figures show that the results produced by the link and system level simulators are very similar for both throughput and fairness. The largest difference between the results of the two simulators is less than 2%, while the 99% confidence intervals (too small to be identified in the figures) of the simulated throughput are much smaller. Thus, we conclude that the system level simulator is properly calibrated; that is, the approximation and modeling of the link level does not result in large errors on system level.

Figure 11

Comparison of system throughput obtained with different scheduling strategies with link and system level simulations. Reproducible by running Reproducibility _Schedulers _batch.m.

Figure 12

Comparison of fairness obtained with different scheduling strategies with link and system level simulations. Reproducible by running Reproducibility _Schedulers _batch.m.

The considered schedulers pursue different goals for resource allocation. The best CQI scheduler tries to maximize total throughput and completely ignores fairness by just assigning resources to the users with the best channel conditions. This is reflected in the simulation results in Figures 11 and 12, showing the highest system throughput and the lowest fairness for the best CQI scheduler. In contrast, the maxmin-scheduler assigns the resources in a way that equal throughput for all users is guaranteed, thereby maximizing Jain's fairness index [73]. Round robin scheduling does not consider the user equipment feedback and cyclically assigns the same amount of resources to each user. Thus, ignoring the user equipment feedback results in the worst throughput performance of all schedulers consider here. The proportional fair scheduler emphasizes multiuser diversity by scheduling the user who has the best current channel realization relative to its own average. The resource fair scheduling strategy guarantees an equal amount of resources for all users while trying to maximize the total throughput. In the simulations, the proportional fair strategy outperforms resource fair in terms of throughput as well as fairness thereby resulting in a good tradeoff between throughput and fairness. Further details about the implemented schedulers, as well as more simulation results, can be found in [21].

The presented simulation results can be reproduced by calling the script Reproducibility _Schedulers _batch.m that can be found in the directory "paper scripts" of the link level and the system level simulator, respectively. More examples of the Vienna LTE simulators also in the context of LTE-Advanced are presented in [74].

6. Conclusions

In this paper, we presented the Vienna LTE Simulators, consisting of a link level and a system level simulator. Both simulators are available under a non-commercial open source academic-use license and thereby enable researchers to implement and test algorithms in the context of LTE. The open source availability of the simulators facilitates researchers to reproduce published results in the context of LTE, and thus supports the comparison of novel algorithms with previous state-of-the-art. So far (July 2011), the simulators have been downloaded more than 18,000 times from all over the world.

Endnotes

^aNote that the scheduler in a multi-user system will change the statistics of the individual user's channel, thus influencing the receiver performance. ^bAfter all, the purpose of a simulation model is to abstract and thus simplify complex situations.^chttp://www.nt.tuwien.ac.at/forum and http://www.nt.tuwien.ac.at/forum.