Graphql Vs Rest Api

Bachelor Degree Project
Evaluating GraphQL over

REST within an .NET Web
API
A controlled experiment conducted by integrating
with the Swedish Companies Registration Office
Author: Rickard Marjanovic

Supervisor: Tobias Ohlsson
Semester: VT 2022
Subject: Computer Science
Abstract
It is only a matter of time before the Swedish Companies Registration Office makes
digital registration of annual reports/auditors reports mandatory. At the time of writing,
there are only ten public integrators currently able to handle this requirement. In
collaboration with one of the big four accounting firms, this project aims to evaluate
performance of the response time while using GraphQL versus REST in Web APIs. The
application under test is a .NET application integrating to the Swedish Companies
Registration Office API. Through a controlled experiment using two different GraphQL
frameworks, HotChocolate and GraphQL for .NET, this thesis provides a knowledge
base for the partnered accounting firm, developers and other stakeholders that are
evaluating the use of GraphQL in their future applications. Results from the experiment
indicate that HotChocolate in its current version is not only faster than its competitor
GraphQL for .NET, but also faster than REST. This is surprising, given that other
related work seems to suggest that this is not always the case. Testing of GraphQL for
.NET gives a more traditional result when compared to other related work. Given the
results, a senior developer of HotChocolate was contacted to gain insight to why the
framework outperforms not only GraphQL for .NET but also REST. The senior
developer states that a large amount of effort has been put in to make the GraphQL
execution engine more optimized, something that is corroborated by this thesis
experiment. HotChocolate is also periodically measuring and comparing performance
benchmarks to other libraries to conclude on its performance in different scenarios. The
analysis of the experiment concluded that there exists another important variable
previously not identified in other research, more precisely the chosen framework, that
has a large impact on performance and can impact both memory allocation and response
time.
Keywords: Swedish Companies Registration Office, Controlled Experiment, Clean

Architecture, RESTful API, GraphQL, HotChocolate, GraphQL for .NET.
Acknowledgements
This bachelor thesis marks the completion of a long journey, which I would not have
been able to embark on, less able to finish if it were not for the endless support of my
family, colleagues and friends.
I am very grateful to my supervisor, Professor T. Ohlsson, for his support and
knowledge throughout this project. A special thanks to the partnered accounting firm
and to the developers Max Fiske, Daniel Gustafsson and Joakim Ahlén for their time
and guidance.
Now begins a new chapter. The world is mine.
Content
1. Introduction 6
1.1 Background 6
1.2 Related work 7
1.3 Problem formulation 8
1.4 Motivation 8
1.5 Result 8
1.6 Scope/Limitation 8
1.7 Target group 9
1.8 Outline 9
2. Method 10
2.1 Controlled Experiment 10
2.1.1 Dependent and independent variables 10
2.2.1 Hypothesis statement 10
2.2 Other Method Considerations 12
2.3 Reliability and Validity 12
2.4 Ethical Considerations 13
3. Theoretical Background 14
3.1 REST 14
3.2 GraphQL 14
3.3 Clean Architecture 15
3.4 CQRS 16
3.5 Mediator 17
4. Implementation 18
4.1 Software and Hardware 18
4.2 General structure 18
4.3 ScroW.Api 19
4.4 ScroW.Application 21
4.5 ScroW.Infrastructure 21
4.6 ScroW.Api.PerformanceTests 22
4.7 ScroW.Api.IntegrationTests 24
5. Results 25
5.1 General result of implementation 25
5.2 Code reviews 26
5.3 Results of experiments 26
6. Analysis 30
6.1 Performance benchmarks 30
6.1.1 Testing of the endpoint BasicInformation (no modification) 30
6.1.2 Testing of the endpoint BasicInformation (over-fetching) 31
6.1.3 Testing of the endpoint BasicInformation (under-fetching) 32
6.1.4 Testing of the endpoint CreateToken (no modification) 33
6.2 Statistical test considerations 33
6.3 Statistical tests 34
6.4 Null hypotheses 36
7. Discussion 38
7.1 REST vs GraphQL for .NET 38
7.2 REST vs HotChocolate 39
8. Conclusions and Future Work 40

8.1 Future Work 40
References 42
Appendix A - Developer backgrounds and Code reviews 46

Max Fiske 46
Background 46
Code review 46
Daniel Gustafsson 46
Background 46
Code review 46
Joakim Ahlén 47
Background 47
Code review 47
Appendix B - Repository and benchmark outputs 48
Appendix C - Summarized results of experiments 49

1. Introduction
The auditor report, a result of an independent audit performed by a certified public
accountant, is used by a broad audience ranging from private investors, banks, tax
authorities, customers, and vendors alike to understand if a company's annual report is
free from any significant errors [1]. Due to it being a yearly requirement by Swedish
law, limited companies are each year required to file copies of signed versions of both
their annual reports and the auditors report no later than seven months after the
company's fiscal year has passed [2, Para. 8:3]. The administrative task of receiving and
processing the filed copies is currently under the responsibility of the Swedish
Companies Registration Office [2, Para. 8:1].
1.1 Background
Sweden is currently ranked the third most digital country in the European Union,
according to a press release made by the Swedish government [3]. One example of
where Sweden is looking to become even more digital is how the Swedish Companies
Registration Office was taxed by the Swedish government to implement digital
registration of annual reports and auditor reports [4][5]. There are currently ten
independent public integrators capable of digital registration [6]. This is suggested to be
problematic since the Swedish Companies Registration Office petitioned the Swedish
government to make digital filing of annual reports (including auditor reports)
mandatory by limited companies starting their fiscal year 2021-07-01 [7]. It is probably
unlikely that the government will approve such a petition with so little time left in the
fiscal year for companies to prepare. However, ten integrators are supposed to handle
the total load of, at the time of writing, around 600 000 annual reports (limited
companies with or without auditors reports) per year for digital registrations [8].
Performance and reliability of service is important when dealing with financial services
required by law and when the reporting is used so broadly [9]. To this end, new ways of
integrating to the Swedish Companies Registration Office is needed.
One way of integrating would be to create an API (application programming
interface), used together with the commonly used architecture style REST
(REpresentational State Transfer) to communicate through the HTTP protocol [10],
[11]. This is, however, not the only alternative, where a relatively newer technology,
GraphQL (a query language), originally developed by Facebook, was created to solve
issues commonly faced in REST APIs [12]. These include, among other, issues like
over-fetching (e.g. requesting too much data, or in other words requesting data that is
not used) or under-fetching (e.g. requesting to little data, or in other words not receiving
enough data from one endpoint, forcing another call to another) [12].
The choice of architectural style is important, since requesting too little or too much
data, or relying on a separate engine to execute and validate incoming requests, causes
implications on the response time needed to perform a request of data [12] [13]. This
thesis aims at evaluating the performance of APIs when implementing traditional REST
versus GraphQL, while integrating to the Swedish Companies Registration Office. Two
different frameworks are used to implement GraphQL, HotChocolate [14] and GraphQL
for .NET [15] and are evaluated together with REST. While the two frameworks in the
end are producing responses in the same way, they are different in the way they require
configuration (as presented in 5.1 General result of implementation and 7. Discussion),
available features, as well as the inner workings of the GraphQL frameworks [14], [15].
The REST API is developed by the author as a part of 4. Implementation using the
ASP.NET framework [16]. There are already a few studies related to the performance
6
implications in general when using the two different architectural types, however,
research is limited on the specific area of .NET Web APIs. By conducting an
experiment on performance of APIs between REST and GraphQL, this thesis aims to
provide detailed knowledge on which architectural style is better suited in an .NET Web
API.
The project is conducted in collaboration with one of the big four accounting firms in
the world (undisclosed at the accounting firms discretion).
1.2 Related work

Even though the use of GraphQL is recent, it was originally developed by Facebook and
released to the public in 2015, the technology quickly became very popular amongst
developers and has been widely adopted due to its flexible nature [17].
In 2018 [18] published a paper, analyzing the performance of GraphQL compared to
RESTful technologies, focusing on response time and throughput on a broad
perspective by creating comparative GraphQL queries returning the same data as REST.
In the paper [18] concluded that REST was faster than GraphQL when performing
requests containing the same data. Furthermore [19] published a similar study on the
topic in 2019. They however focused on request per second, time per request, time per
concurrent request, and transfer rate within a NodeJS development environment also
creating comparative GraphQL queries returning the same data as REST but instead
changing the workload [19]. In this study [19] migrated three different applications to
GraphQL and testing concluded that in two of the three tested applications GraphQL
was faster than REST. The above papers focus on performance in a general context,
with only the second study applying specific constraints other than the architecture type
(GraphQL vs REST). Furthermore they are not testing different types of queries,
whereas this paper focuses on performance implications of differently modified
GraphQL queries compared to REST to allow for a more detailed understanding on the
implications between the different technologies in a specific environment. Nevertheless,
the world of development is ever-changing and newer versions of frameworks might
radically change the outcomes. E.g the newest release of .NET, more specifically .NET
6 released in 2021 [15], which is supposedly generally faster than its .NET 5
counterpart [16] released in 2020 [15]. A systematic study in 2018 by [20] focuses on
the complexity and semantics of GraphQL by analyzing the time complexity of its
evaluation, enumeration and size-computation. The study showed, among other things,
that it is possible to calculate the complexity of the GraphQL query before its execution,
making it very specific to studying the complexity of the actual queries and not the
complexity or maintainability of an actual implementation [20]. Another study by [21]
was published in 2020 analyzing the effort of and perceptions of developers when
implementing specific queries in the two architecture types. An interesting article,
touching upon complexity through the developers perception of actually implementing
different types of endpoints within the application [21]. Lasty in 2021, [22] performed a
combined quantitative analysis of the technology’s performance (response time) and
qualitative study with developers concluding that there are no significant differences in
performance nor in maintainability between the two architectural types. As noted in [22]
the API used was developed in Ruby on Rails, and as such a .NET C# implementation
could yield different results.
To the authors knowledge there are no contributions that analyze performance by
testing differently modified queries within a .NET Web API.
7
1.3 Problem formulation
There has been considerable interest on the topic of REST APIs and GraphQL over the
years, however, the specific case of performance analysis in an .NET Web API
environment has gained little to no interest. In consequence, any developer, company or
other interest party looking to choose between one of the two architectural models will
have limited research data to support their decisions. The fact that environments are
ever changing also makes it clear that performance analysis needs to be done in a more
defined environment to better give assurance to any decision maker evaluating the two
architectural models. This thesis therefore will answer the following questions:
1. What are the performance implications when using GraphQL over REST in Web
APIs?
Focusing on the arithmetic mean of the response time, four different REST requests
will be compared with GraphQL:
1. Unmodified GET requests
2. Over-fetching of GET requests
3. Under-fetching of GET requests
4. Unmodified POST requests
1.4 Motivation
Digitalisation is without doubt something that has and continues to change the world as
we know it. By partnering with one of the big four accounting firms in the world the
author intends to integrate to the Swedish Companies Registration Offices APIs as a test
case for this thesis, meaning that this thesis at the same time will solve an issue of future
digital registration needs, potentially caused by slow adoption and availability of
technology. Furthermore as pointed out in the related work there currently are limited
contributions on the topic GraphQL vs REST in an .NET environment, meaning that
this thesis would contribute scientifically in that area. The reason why .NET was chosen
as the implementation was due to the fact that the partnered auditing firm currently is
using .NET in their development.
1.5 Result
This thesis evaluates the performance implications of using GraphQL over REST by
implementing an application in an .NET 6 Web API that integrates to the Swedish
Companies Registration Office. By conducting a controlled experiment using a well
known and respected framework for benchmark testing, BenchmarkDotNet, the results
are presented in graphs and tables and later analyzed and compared to the result of other
related work. The purpose and main contribution of the performed testing was to
analyze the performance implications of two different frameworks (GraphQL for .NET
and HotChocolate) compared to REST. This in order to provide a knowledge base for
the partnered accounting firm, developers and other stakeholders that are evaluating the
use of GraphQL in their future applications.
1.6 Scope/Limitation
Due to the complexity of creating a GraphQL execution engine, one usually uses
publicly available frameworks to create a separate server that handles all GraphQL
related queries. For this contribution, GraphQL for .NET [15] and HotChocolate [14]
have been chosen for implementation and testing. The reason being that these two
frameworks are the two most popular in the .NET environment, and also pointed out as
8
the most valuable by the partnered accounting firm [23]. The author also intends to
remove external dependencies when performing testing (e.g. the external API to the
Swedish Companies Registration Office), making the study focus on internal
performance. Due to the fact that the Swedish Companies Registration Office is only
using GET and POST requests in their API the conducted experiment only focuses on
those two methods. Furthermore due to time constraints the selected four cases for
comparison are not covering all possible scenarios of an API (e.g only testing
unmodified POST requests).
1.7 Target group

The primary audience and/or target group is naturally by default one of the big four
accounting firms which the author partnered with. Furthermore the target group is any
developer evaluating implementing GraphQL over REST APIs in .NET to see the
performance this brings to their applications.
1.8 Outline
The rest of the thesis is organized as follows. Chapter 2 describes the chosen research
methodology and hypothesis statements, including any reliability, validity and ethical
considerations. Chapter 3 presents a theoretical background on REST and GraphQL, as
well as important design principles chosen in the implementation. Chapter 4 describes
how the implementation of the application under test has been performed. Chapter 5
contains all the detailed results retrieved through BenchmarkDotNet, summarized in
tables. Additionally the general result of the implementation is presented together with a
result of the performed code reviews. Chapter 6 analyses the result and presents box
plots and tables of all benchmark data. Furthermore statistical analysis is performed to
conclude if the data is statistically significant or not. In chapter 7 the result is compared
to the related work of other studies as well as analyzing potential differences between
the actual result and the hypotheses. Lastly, chapter 8 presents conclusions as well as
proposals for possible future work.
9
2. Method
The research method used in this project is a controlled experiment, a well known
research strategy for computer science and used in other theses as well as related work
to test hypotheses [17], [22], [24]. In order to conduct the experiments an application
was implemented to test the performance of GraphQL versus REST. See 4.
Implementation. This chapter is divided into several subsections, ranging from an
explanation of the method, different hypotheses statement to ethical considerations
taken into account for this thesis.
2.1 Controlled Experiment

This thesis aims to evaluate the performance of two different architectural types, REST
APIs vs GraphQ retrieving data from the Swedish Companies Registration Offices API.
The results can support decision makers when choosing which architectural type to
select. One approach used when comparing two different architectural types is to
conduct a controlled experiment which means to set up an environment where one or
several independent variables are systematically manipulated and the dependent
variable/variables are measured. Controlled experiments are often used in research
regarding the comparison of architectural structures [17], [22], [24] and will therefore
be used for this thesis. The controlled experiment conducted by the author was defined
and conducted as outlined in [25] through definition and planning (2.1.1 Dependent and
independent variables and 2.2.1 Hypothesis statement) as well as operation, analysis
and presentation (5.3 Results of experiments and 6. Analysis). The experiment was
executed using the framework BenchmarkDotNet [26] and based on the implementation
described in 4. Implementation. Furthermore, a statistical analysis was performed on the
results to conclude statistical significance of the data. An initial Levene’s test in 6.2
Statistical test considerations concluded that the variances between GraphQL and REST
were significantly different, as such a non-parametric independent group test was
deemed the most appropriate [27]. To that end, a Kruskal–Wallis one-way analysis of
variance, was conducted to be able to conclude on the significance between groups
(REST, HotChocolate and GraphQL for .NET) on data where variances are significantly
different [28].
2.1.1 Dependent and independent variables

The dependent variable under this study was the response time measured as the mean of
all responses in microseconds for each test. The independent variables that were
controlled in the experiment were (1) if the underlying API call to the Swedish
Companies Registration Office was a GET or POST method, (2) the different
modifications that were performed to the GraphQL query string to simulate any
potential over/under-fetching scenarios.
2.2.1 Hypothesis statement

From the literature study presented in 1. Introduction and 3. Theoretical Background,
the following informal hypotheses have been identified.
1. The inner workings of GraphQL is more complex than REST APIs, and thus the
response time of REST APIs are faster than that of GraphQL when requesting
identical data.
2. GraphQL is designed to only return what is requested (no more, no less), and
thus the response time of GraphQL is faster than that of REST APIs when
10
requesting only a subsample of the data that a REST API is exposing (GraphQL
avoids over-fetching).
3. GraphQL can reduce the need to make multiple round trip requests (GraphQL
avoids under-fetching) and is thus faster than REST APIs if multiple endpoints
need to be called to fetch the required data.
4. Since there are minor differences in response time between GET and POST
requests, the response time of REST APIs are faster than that of GraphQL when
performing a REST POST request and requesting identical data.
Based on the above informal hypotheses the following formal statements have been
designed:
1. Null hypothesis, 𝐻0: REST APIs (RAPI) GET requests are slower or identical in
response time (mean time measured in microseconds) compared to GraphQL
(GQL) POST requests when requesting the exact data that is available from one
endpoint.
𝐻0: 𝑇𝐼𝑀𝐸(𝐺𝐸𝑇(𝑅𝐴𝑃𝐼)) ≥ 𝑇𝐼𝑀𝐸(𝑃𝑂𝑆𝑇(𝐺𝑄𝐿)) when requesting the exact
data that is available from one endpoint.
Alternative hypothesis, 𝐻1: 𝑇𝐼𝑀𝐸(𝐺𝐸𝑇(𝑅𝐴𝑃𝐼)) < 𝑇𝐼𝑀𝐸(𝑃𝑂𝑆𝑇(𝐺𝑄𝐿)) when
requesting the exact data that is available from one endpoint.
Measures needed: architectural type (RAPI or GQL) and time (TIME) in
microseconds.
2. Null hypothesis, 𝐻0: GraphQL (GQL) POST requests are slower or identical in
response time (mean time measured in microseconds) compared to REST APIs
(RAPI) GET requests when requesting less data than available from one
endpoint.
𝐻0: 𝑇𝐼𝑀𝐸(𝐺𝐸𝑇(𝑅𝐴𝑃𝐼)) ≤ 𝑇𝐼𝑀𝐸(𝑃𝑂𝑆𝑇(𝐺𝑄𝐿)) when requesting less data
than available from one endpoint.
Alternative hypothesis, 𝐻1: 𝑇𝐼𝑀𝐸(𝐺𝐸𝑇(𝑅𝐴𝑃𝐼)) > 𝑇𝐼𝑀𝐸(𝑃𝑂𝑆𝑇(𝐺𝑄𝐿)) when
requesting less data than available from one endpoint.
microseconds.
3. Null hypothesis, 𝐻0: GraphQL (GQL) POST requests are slower or identical in
response time (mean time measured in microseconds) compared to REST APIs
(RAPI) GET requests when requesting more data than available from one
endpoint.
𝐻0: 𝑇𝐼𝑀𝐸(𝐺𝐸𝑇(𝑅𝐴𝑃𝐼)) ≤ 𝑇𝐼𝑀𝐸(𝑃𝑂𝑆𝑇(𝐺𝑄𝐿)) when requesting more data
than available from one endpoint.
Alternative hypothesis,
𝐻1: 𝑇𝐼𝑀𝐸(𝐺𝐸𝑇(𝑅𝐴𝑃𝐼)) > 𝑇𝐼𝑀𝐸(𝑃𝑂𝑆𝑇(𝐺𝑄𝐿(𝐺𝐸𝑇))) when requesting more
data than available from one endpoint.
microseconds.
4. Null hypothesis, 𝐻0: REST APIs (RAPI) POST requests are slower or identical
in response time (mean time measured in microseconds) compared to GraphQL
(GQL) POST requests when requesting the exact data that is available from one
endpoint.
11
𝐻0: 𝑇𝐼𝑀𝐸(𝑃𝑂𝑆𝑇(𝑅𝐴𝑃𝐼)) ≥ 𝑇𝐼𝑀𝐸(𝑃𝑂𝑆𝑇(𝐺𝑄𝐿)) when requesting the exact
data that is available from one endpoint.
Alternative hypothesis, 𝐻1: 𝑇𝐼𝑀𝐸(𝑃𝑂𝑆𝑇(𝑅𝐴𝑃𝐼)) < 𝑇𝐼𝑀𝐸(𝑃𝑂𝑆𝑇(𝐺𝑄𝐿))
when requesting the exact data that is available from one endpoint..
microseconds.
2.2 Other Method Considerations

A few other research methods were considered and evaluated for this thesis, among
others a Systematic Literature Review, Case Study, Interviews and Design Science. In
regards to Systematic Literature Review, Case Study, or Interviews they weren't suited
as, while they would be able to compare e.g. implementations, they would not be able to
give any viable performance measurements. Furthermore, a Case Study would be
needed to be conducted in its natural environment, which would include integrating to,
and using, Azure Cloud Services in addition to integrating to the Swedish Companies
Registration Office. It was agreed with the partnered accounting firm that it was not a
possible nor an appropriate method for the scope of this thesis, due to time constraint
and security reasons. Similarly, Design Science was not deemed appropriate as the
objective did not include inventing any new technology, but rather reusing already
developed technology.
2.3 Reliability and Validity

Development of a software can be performed in various ways with different
performance and time implications to all solutions, basically making the experiment less
valid. In order to reduce the risk caused by this the principles of Clean Architecture [29]
will be used, forcing the implementation of both the REST API and GraphQL solutions
to be made in an architecturally similar way. The Clean Architecture principles are used
by the partnered accounting firm in their daily development, and as such these
principles will be adopted by the author as well in the design and development of the
thesis application.
In short, Clean Architecture advocates the use of different established design
principles (e.g. SOLID [30]) combined with structuring the application into different
layers to make it independent from frameworks, databases, external agencies, and the
UI, as well as being easily testable [29]. See 3.3 Clean Architecture for a more thorough
explanation. The usage of Clean Architecture will also have performance and code
implications in itself, but comparing the two solutions to one another will become more
valid as it is only the relative performance implications that can be argued will be
affected.
Furthermore, to increase the reliability and validity of the experiment, the
implementations will be reviewed by three independent, senior developers, all with
between 9 to 21 years of experience working with backend development. (as can be
seen in Appendix A - Developer backgrounds and Code reviews). This will add
assurance that the code has been designed and developed according to best practices
within an .NET 6 Web API.
Actual benchmark testing is performed with the framework BenchmarkDotNet. It
can be argued that relying on an external application can decrease validity and
reliability of the results. In this case however, BenchmarkDotNet is used by many
established applications e.g. ASP.NET Core, Entity Framework Core, ML.NET,
12
MediatR, GraphQL for .NET, and HotChocolate [26]. Furthermore, the framework is
built using best practice principles as outlined in [31], including but not limited to
determining the appropriate number of method invocations, warm-up iterations and
calculating statistics [26]. This argues that the framework BenchmarkDotNet is well
suited for use in this application and that its result can be trusted.
The environment where the tests are executed can also be considered a risk if it is not
isolated before running the code. To mitigate this risk the author intends to isolate the
environment as much as possible and shut down any application not needed before
performing the tests.
2.4 Ethical Considerations

The code produced will be reviewed by three different independent developers from
different software development companies. The personal information of the reviewers
and the companies they represent can be considered sensitive. Confidentiality and
participation consent was gathered from all parties and they all agreed to keep their
names and the companies they work for public. Additionally, a senior developer from
HotChocolate was contacted to get further understanding of the identified results.
However, confidentiality consent was not received, and as such his name and position
have been omitted in the thesis.
This bachelor thesis was performed on behalf of an external company. To this end,
there is a risk that sensitive data will be included in the application solution and/or
thesis, which should not become publicly available. In order to account for this,
company representatives will be consulted, and if any part of either the implemented
solution or the thesis itself is deemed to violate the company's integrity, it will be
excluded at the company’s discretion.
13
3. Theoretical Background
This section gives a theoretical background on important design principles and terms
used in the development of the application used for testing, named Swedish Companies
Registration Office Wrapper (ScroW), and aims to give the reader a deeper
understanding of the choices used in the implementation of the .NET 6 Web API built to
integrate to the Swedish Companies Registration Office.
3.1 REST
REST (REpresentational State Transfer) is a software architectural style and a contract
that defines a set of constraints of how an API (application programming interface)
client can interact with a server [10], [11], using the http protocol. There are basically
four parts that should make up a REST request, (1) the endpoint (or URL) to access the
data, (2) the HTTP method used to access the data, (3) the header data and lastly (4) the
body data [10].
Each datapoint is accessed/requested through an URL, commonly called an endpoint,
which returns a specific set of data defined on the server side [10]. A request is made to
the server through several methods, where the most common are GET (get access to a
resource), POST (create a new resource), PUT (update a resource), PATCH (partially
update a resource) and DELETE (delete a resource) [10]. The Swedish Companies
Registration Office is only using/providing GET and POST requests through their APIs
which in turn is mimicked by the developed application. The header provides
information to both the client and server and is used to decorate a response with e.g.
which content type is expected, or proof that the client has access to the particular
endpoint. Lastly, in some cases (e.g. for the POST requests used in the Swedish
Companies Registration Office API) the server expects a body to be included in the
request with additional information needed to process the request [10].
The illustration below is the body of a POST request to an endpoint with the
URL“/api/scro/createtoken” from the built application where the required values are
provided for the fields “organizationNumber” and “socialSecurityNumber”.
{
"organizationNumber": "1234567890",
"socialSecurityNumber": "1234567890"
}
Figure 3.1: Body of a POST request
3.2 GraphQL
In short GraphQL can be described as a query language and a runtime which is capable
of executing and resolving incoming queries [12]. The responsibility of control is
shifted from the server to client, enabling a more dynamic handling compared to e.g.
REST, which provides defined accessible resources that always return the same data
[32]. As mentioned in chapter 1.1 Background, some of GraphQL’s strengths include
solving issues in REST, like over-fetching and under-fetching, but in addition, it can
also retrieve data from all kinds of different sources through one request and using only
a single endpoint [12].
For GraphQL to work, mapping (or referencing) of the GraphQL type objects has to
be made to the actual object, and its properties, which varies depending on the
implementation or framework used [33]. In turn, the different types need to be mapped
14
to a schema definition for the execution and validation to be made [33]. This can be
done in several ways and is dependent on the framework used to implement GraphQL.
E.g. HotChocolate allows annotation, code-first, and a schema-first approach when
implementing, while GraphQl for .NET only supports code-first and a schema-first
approach [14], [15].
In essence, to request data through GraphQL, a REST API POST request is made to
the server containing a body with a query, operation type (e.g. query or mutation) and
root object with chosen fields to define what is to be resolved and from which defined
area [34]. The GraphQL execution engine interpers the query (either a query or
mutation) and provides the data requested in a response back to the client [34]. To
receive data using the defined type system, the queries are executed by a GraphQL
server which resolves each field using a defined resolver. The resolver can fetch data
from any data source (ranging from external APIs to databases), repeating this process
until all the fields in the query have been fulfilled, and lastly returning the data into a
response [32].
The illustration below is the body of a POST request, containing a query requesting
data for the fields “organizationNumber” and “name” in the defined root object/type
“basicInformation” taking the string input parameter “$organizationNumber”.
{
"query": "query($organizationNumber: String!) {
basicInformation (organizationNumber:$organizationNumber) {
organizationNumber
name
}
}"
"variables": {
"organizationNumber": "1234567890"
}
}
Figure 3.2: Body of a POST request, containing a query
3.3 Clean Architecture

This architectural style is heavily based on several commonly known architectural
principles (e.g. Single Responsibility, Open Closed, Liskov Substitution, Interface
Segregation and Dependency Inversion) helps structuring the application in different
ways so that among other things, the business logic is encapsulated as the center of the
application and making it independent from implementation details [29]. The key
principles of clean architecture include separation of concern, testability as well as
independence (loose coupling) of frameworks, databases, the UI and any external
agency [29]. Clean architecture is using different circles/layers to illustrate typical
concerns as illustrated in figure 3.3, reworked from [35].
15
Figure 3.3: A simplified representation of the different separation of concerns used in Clean Architecture
As shown in figure, the Application Core focuses on and contains interfaces, entities,
and business logic, the Infrastructure contains external dependencies (e.g. a database)
depends on the Core, and implements interfaces from the Core, while lastly the outer
layer User Interface (e.g. a Web API or Frontend application) also depends on the Core
[29].
Dependencies are pointed inwards, which means that e.g. the User Interface can
depend on the Application Core, but the Application Core cannot depend on the User
Interface [29].
3.4 CQRS
The CQRS (Command Query Responsibility Segregation) is a way to separate read and
update operations by using different models to handle each task [36]. This means that
each task is separated into its own isolated part, independent of other features,
commands or queries. Commands are specifically used for performing actions (e.g.
updating or modifying a database), while queries instead are used to retrieve data. The
CQRS design pattern brings the benefit of (1) scalability - making it easy to add or
modify features in an application, (2) performance -being able to specifically improve a
specific area of code (with e.g. caching), and lastly (3) added simplicity - by making
sure that each model does only one thing and is only used by that specific part of the
code [37]. Nevertheless, there are a few drawbacks, one being the risk that projects
easily become unnecessarily large with an increased risk of redundant/duplicate code.
16
3.5 Mediator
Another way of promoting loose coupling in an application is by using a behavioral
design pattern called the mediator pattern. This pattern manages interactions between
objects through a coordinator, while maintaining isolation between the classes [38]. In
.NET the most popular implementation of the mediator pattern is MediatR, also
commonly used together with the CQRS design pattern (see above 3.4 CQRS) [39].
17
4. Implementation
The following section will present the implementation of the application, called ScroW
(Swedish Companies Registration Office Wrapper), which aims at integrating with the
Swedish Companies Registration Office using both REST and GraphQL. Design
choices for the application will be presented in addition to the test written to evaluate
performance.
4.1 Software and Hardware

All development and testing has been performed on a computer with the following
hardware specifications:
OS: Windows 10.0.19044.1645 (21H2)

Processor: Intel Core i7-5930K CPU 3.50GHz
RAM: 64 GB
Hard Drive: Samsung SSD 950 PRO 512 GB
Furthermore, all implementations have been performed using the framework .NET 6.
Regarding GraphQL, the frameworks used and tested were GraphQL (or GraphQL for
.Net/GraphQL.Net) version 5.1.1 [40] and HotChocolate version 12.7.1 [41]. All of
which, at the time of the applications implementation, were the latest versions.
BenchmarkDotNet version 0.13.1 [42] was used together with Moq version 4.17.2
for the performance testing[43]. Please note that these packages are not comprehensive
for the actual implementation of the application. For a full list of packages used please
see Appendix B - Repository and benchmark outputs for the repository containing all
public code.
When running the benchmark application to run the tests, the computer was started
with only the necessary applications needed to run the operating system and the tests
itself. All other background applications were shut down to not cause any interference
while running the tests, as pointed out by through the code review performed (see
Appendix A - Developer backgrounds and Code reviews)
4.2 General structure

Using a common .NET solution, ScroW.sln, the project was structured into five
different areas using Clean architecture principles. The folder and project structure was
used as illustrated in figure 4.1, with three different projects making up the application
and two projects used for testing.
18
ScroW
src
Api
ScroW.Api
Core
ScroW.Application
Infrastructure
ScroW.Infrastructure
tests
ScroW.Api.PerformanceTests
ScroW.Api.IntegrationTests
Figure 4.1: Structure of the ScroW project.
4.3 ScroW.Api
The presentation layer of the application, ScroW.Api, was created using a .NET 6
version of the Web API SDK to ease development and make use of the built-in
functionality provided by .NET [44]. Using the Mediator pattern, the different
controllers/GraphQL query executors did not include any core logic but instead used the
Mediator to send through the data to its correct area in the core project, see code
snippets in figure 4.2 and 4.3.
The REST and GraphQL for .NET implementations require the creation of separate
controllers, handling logic e.g. endpoint definition and input parameters. Unlike the
framework HotChocolate which only needs registering in the Startup class due to its
behind the scenes working, setting everything up automatically. The code snippet in
figure 4.2 shows how implementation is made with REST (only showing one method
and limiting applied attributes for brevity) and subsequently, GraphQL for .NET in the
code snippet in figure 4.3.
[Route("api/[controller]")]
[Produces("application/json")]
public class ScroController : ControllerBase
{
public readonly IMediator mediator;
public ScroController(IMediator mediator) => this.mediator = mediator;
[HttpGet("BasicInformation/{organizationNumber}")]
[ProducesResponseType(typeof(BasicInformationResponse), 200)]
public async Task<IActionResult> BasicInformationAsync(
[Required][FromRoute] string organizationNumber)
{
return Ok(await mediator.Send(new BasicInformationQuery
{
OrganizationNumber = organizationNumber
}));
}
}
Figure 4.2: Controller class showing a method for a REST endpoint..
19
[Route("api/[controller]")]
[Produces("application/json")]
public class GraphQLController : ControllerBase
{
private readonly ISchema schema;
private readonly IDocumentExecuter executer;
public GraphQLController(IDocumentExecuter executer, ISchema schema)

{
this.executer = executer;
this.schema = schema;
}
[HttpPost("graphqlnet")]
public async Task<IActionResult> GraphQL([FromBody] GraphQLRequest r)
{
var result = await executer.ExecuteAsync(s =>
{
s.Schema = schema;
s.Variables = r?.Variables;
s.Query = r?.Query;
s.OperationName = r?.OperationName;
s.RequestServices = HttpContext.RequestServices;
s.CancellationToken = HttpContext.RequestAborted;
});
return new ExecutionResultActionResult(result);

}
}
Figure 4.3: Controller class showing a method for the GraphQL endpoint for GraphQL for .NET.
The two GraphQL implementations differ in how they are set up. GraphQL for .NET
requires the most setup by using the code-first approach, with a need to define each
object type and how they interact through queries/mutations and finally the schema
[45]. This is, however, automatically set up by HotChocolate. The only thing needed for
the HotChocolate implementation is defining the different queries/mutations so
HotChocolate can register each object type (queries/mutations) and create a schema
automatically using an annotation based approach [46], [47]. The following code
presens how GraphQL for .NET and subsequently HotChocolate was added to the
service collection in the Startup class as well as the mapping of endpoints to either the
controller (REST and GraphQL for .NET) or its separate endpoint (HotChocolate).
20
services.AddGraphQL(builder => builder
AddSelfActivatingSchema<ScroWSchema>()
.AddClrTypeMappings()
.AddSystemTextJson()
.AddGraphTypes(typeof(ScroWSchema).Assembly)
);
services.AddHttpContextAccessor();
services.AddGraphQLServer()
.AddQueryType<HotChocolate.Query>()
.AddMutationType<HotChocolate.Mutation>()
.ModifyRequestOptions(opt =>
{
opt.IncludeExceptionDetails = true;
}
);
Figure 4.4: Service registration for both GraphQL for .NET and subsequently HotChocolate.
app.UseEndpoints(endpoints =>
{
endpoints.MapControllers();
endpoints.MapGraphQL()
.WithOptions(new GraphQLServerOptions
{
EnableGetRequests = false
});
});
Figure 4.5: Registering Controllers (REST and GraphQL for .NET) and HotChocolate to a middleware.
4.4 ScroW.Application
In a .NET class project, using Mediator and CQRS the business logic was separated
from the presentation layer. The ScroW.Application project is the core of the application,
this is also shown by the folder structure. The application project contains contracts
(e.g. interfaces without the actual implementation), models used by the Swedish
Companies Registration Office API, mapping profiles between models and data transfer
objects (DTOs), serialization/deserialization logic etc. Even though there are
frameworks handling CQRS, CQRS was implemented using a folder structure. Mapping
between models and DTOs was handled using the library AutoMapper, which is
commonly used in .NET projects for this purpose [48].
4.5 ScroW.Infrastructure
Since the core layer of the application is oblivious as to how data is retrieved, it is
common to break out external dependencies (e.g. a database, frameworks, etc) to its
separate project. The only identified external dependency is the API to the Swedish
Companies Registration Office contacted to retrieve data processed in the core layer.
Using a contract from the core layer, a typed HttpClient [49] was created that was
specialized in retrieving the data from the Swedish Companies Registration Office.
21
4.6 ScroW.Api.PerformanceTests
The benchmark tests have been performed using a separate .NET console project
focused on benchmark tests called ScroW.Api.PerformanceTests (see Appendix B -
Repository and benchmark outputs for a reference to the repository, including the raw
test results). In order to test at a level that includes the application’s supporting
infrastructure (e.g. HttpRequests, Controllers, GraphQL executor engine, Mediator etc)
throughout the whole application, a generic custom WebApplicationFactory was used,
see below in figure 4.6. This allows the web host configurations to be modified as
needed when creating an in memory version of the application [50].
public class CustomWebApplicationFactory<TStartup> :

WebApplicationFactory<TStartup> where TStartup : class
{
protected override void ConfigureWebHost(IWebHostBuilder builder)
{
builder.ConfigureServices(services =>
{
var mockMediator = new Mock<IMediator>();
services.AddSingleton(mockMediator);
services.AddSingleton(mockMediator.Object);
});
base.ConfigureWebHost(builder);
}
protected override IHost CreateHost(IHostBuilder builder)
{
builder.UseContentRoot(Directory.GetCurrentDirectory());
return base.CreateHost(builder);
}
}
Figure 4.6: A custom WebApplicationFactory used to modify the web host by adding a mocked mediator
The most important configuration of the custom factory was the setup of a mock
Mediator using Moq, a framework to mock different areas of the application [51]. The
Mediator was mocked in order to control the results that were sent back when calling
different endpoints through REST and GraphQL. Since the application was contacting
the Swedish Companies Registration Office through an API this dependency was
removed since it only created uncertainty, increased latency and wouldn’t help nor
affect the benchmarking of REST and GraphQL. Using the WebApplicationFactory an
HttpClient was created which enabled making API calls to the application, just as if it
would have been an external application. However, all dependencies and eventual
overhead caused by contacting the different endpoint types was retained, allowing for
benchmarking REST vs GraphQL. Since the Mediator was mocked and configured to
return known data, the above ScroW.Application project was circumvented entirely and
was not important for the result of the benchmarking, but needed for running the
application.
The benchmark testing was performed with the framework BenchmarkDotNet. Not
much was needed in terms of configuration, which is also one of the key strengths of the
framework [26]. The code was structured as illustrated in the below figure 4.7, using all
common setup in an abstract base class, which each test class inherited from.
22
[InProcess]
[MemoryDiagnoser]
[MinColumn, MaxColumn]
[Config(typeof(TestConfig))]
public abstract class BenchmarkBase
{
public HttpClient client;
public Mock<IMediator> mockMediator;
[GlobalSetup]
public void GlobalSetup()
{
var factory = new CustomWebApplicationFactory<Startup>();
client = factory.CreateClient();
var scope = factory.Services.GetRequiredService<IServiceScopeFactory>().CreateScope();
mockMediator = scope.ServiceProvider.GetRequiredService<Mock<IMediator>>();
var expectedBasicInformationResult = MockResponse.GetExpectedBasicInformationResponse();

var expectedCaseStatusResponse = MockResponse.GetExpectedCaseStatusResponse();
var expectedCreateTokenResponse = MockResponse.GetExpectedCreateTokenResponse();
mockMediator
.Setup(m => m.Send(It.IsAny<BasicInformationQuery>(), It.IsAny<CancellationToken>()))
.ReturnsAsync(() => expectedBasicInformationResult);
mockMediator
.Setup(m => m.Send(It.IsAny<CaseStatusQuery>(), It.IsAny<CancellationToken>()))
.ReturnsAsync(() => expectedCaseStatusResponse);
mockMediator
.Setup(m => m.Send(It.IsAny<CreateTokenCommand>(), It.IsAny<CancellationToken>()))
.ReturnsAsync(() => expectedCreateTokenResponse);
}
}
Figure 4.7: Benchmark setup class registering mocked data to be retrieved when performing the different calls to the API.
Each separate test type was structured in its own class and was requesting exactly the
same data but from different endpoints, ultimately calling either GraphQL or REST.
Below figure 4.8 is an example of the “BasicInformation” tests with REST (GraphQL
methods have been omitted for brevity, see Appendix B for a reference to the test
repository).
public class GetBasicInformationEndpointBenchmarks : BenchmarkBase

{
[Benchmark(Baseline = true)]
public async Task Rest()
{
await client.GetAsync("/api/scro/basicinformation/1234567890");
}
}
Figure 4.8: Benchmark test class showing an example of a method to be tested.
23
4.7 ScroW.Api.IntegrationTests
The integration test project was structured in the exact same way as the performance test
project, however, without mocking the Mediator. This was due to the goal of the project,
testing the integration between ScroW and the Swedish Companies Registration Office
API. The tests were designed in the same way as the above performance tests, with the
only exception being the usage of asserts to be able to verify the correctness of what is
being returned from the API.
24
5. Results
This chapter is divided into three subsections. First presenting the results from
implementing ScroW using REST, HotChocolate or GraphQL for .NET. Secondly
presenting the result of the code review performed on the author's implemented
solution. Lastly presenting the performance results after running BenchmarkDotNet.
5.1 General result of implementation

The below table illustrates the result of the general analysis performed on the ScroW
project. As the code has been structured using Clean Architecture principles, the
presentation layer (ScroW.Api project) was the only project that contains classes and
code related to how data/information is transferred.
The number of classes metric was calculated by counting each instance class where
code that relates to the specific architectural type was included. The Lines of code
metric was instead calculated by counting the amount of code that was required by the
specific architectural type. As such, some code was excluded based on the fact that it
was not comparable between all architectural implementations. E.g. error handling with
an IActionFilter [52] was implemented for REST per the request of the partnered
accounting firm, but was excluded from the GraphQL implementations. Additionally
some code on error handling has been excluded from the lines of code column as it was
not required to make that architectural type to function.E.g. the “produces” attributes for
the error handling in the REST architectural type, used only for decorating the API
documentation in Swagger. Lastly, the startup class contains some code for GraphQL
that enables extra error information, which was not included as this code usually is not
enabled for production environments.
Table 5.1: Number of Classes and Lines of Code related to each architectural type
Method Classes Lines of Code
Rest 2 79
HotChocolate 3 81
GraphQLNet 21 373
Examples of code that has been removed from the calculation can be seen in the below
figures 5.1, 5.2 and 5.3.
[ProducesResponseType(typeof(ErrorMessage), 400)]
Figure 5.1: Attributes in the REST controller.
options.Filters.Add<HttpResponseExceptionFilter>();
Figure 5.2: Exception filter registration in the Startup class.
25
services.AddGraphQLServer()
.AddQueryType<HotChocolate.Query>()
.AddMutationType<HotChocolate.Mutation>()
.ModifyRequestOptions(opt =>
{
opt.IncludeExceptionDetails = true;
});
Figure 5.3: HotChocolate service registration, adding more verbose error messaging.
5.2 Code reviews

The different code reviews performed by the senior developers, see Appendix A -
Developer backgrounds and Code reviews, indicates that the application was built using
best practices for an .NET Web API, with no changes needed for the application to
function more optimized in regards to the experiment. One of the developers mentioned
the risk of validity due to isolation, however, this has been addressed/discussed in the
implementation chapter (4. Implementation). The application will be modified in the
final delivery to the partnered accounting firm to better suit their development style and
make integration to their own environments easier.
5.3 Results of experiments

By running BenchmarkDotNet through the implementation, the below results were
received. For a summarized table of all results see Appendix C - Summarized results of
experiments where all the test results are presented side by side in a single table, instead
of in a test by test case.
Experiment 1 was performed to test hypothesis 1 using an unmodified GET request
configuration.
26
Table 5.2: Execution time of GET REST request and POST GraphQL requests without modification.
Run Rest (μs) HotChocolate (μs) GraphQLNet (μs)
1 291.43 239.24 428.76
2 293.90 237.52 428.49
3 291.00 239.48 429.80
4 287.30 237.78 429.96
5 294.20 237.30 428.99
6 291.69 237.68 428.81
7 289.30 238.52 429.86
8 286.92 238.50 428.63
9 286.90 237.76 430.45
10 290.10 238.53 431.30
11 293.46 237.73 430.21
12 285.51 237.77 430.74
13 287.27 238.19 430.29
14 285.58 238.63
15 286.39 239.52
MEAN 289.40 238.28 429.71
N 15 15 13
The results presented in table 5.2 show that HotChocolate was faster than the two others
on average, while REST was faster than GraphQL for .NET.
Experiment 2 was performed to test hypothesis 2 using an over-fetching of GET
requests configuration.
Table 5.3: Execution time of GET REST request and POST GraphQL requests when requesting less data than what is provided from
a one REST endpoint.
Run Rest (μs) HotChocolate GraphQLNet
1 273.32 208.74 306.16
2 276.88 209.17 308.89
3 282.66 207.98 308.52
4 275.33 209.78 308.34
5 270.50 208.11 306.12
6 268.18 211.30 305.65
7 280.16 209.70 305.87
8 282.72 211.38 306.77
9 283.34 208.92 311.10
10 274.60 210.25 312.39
11 279.40 209.68 312.78
12 280.96 210.37 312.58
13 278.35 211.54 311.43
14 280.90 211.89 314.27
15 278.38 211.50 313.77
MEAN 277.71 210.02 309.64
N 15 15 15
27
The results presented in table 5.3 show that HotChocolate was faster than the two
others on average, while REST was faster than GraphQL for .NET. However the
average response time of GraphQL for .NET was decreased when compared to
experiment 1.
Experiment 3 was performed to test hypothesis 3 using an under-fetching of GET
Table 5.4: Execution time of GET REST request and POST GraphQL requests when data must be retrieved from two REST
endpoints.
1 544.72 319.48 522.98
2 561.17 319.07 524.18
3 542.83 318.19 526.78
4 546.10 317.68 527.24
5 557.09 316.09 524.27
6 545.70 316.57 523.87
7 542.62 317.18 526.35
8 536.53 318.49 525.73
9 537.64 316.36 526.21
10 556.12 315.41 531.56
11 561.96 316.43 532.93
12 547.25 315.94 529.66
13 537.30 316.89 529.85
14 549.97 320.48 528.54
15 315.04 528.07
MEAN 547.64 317.29 527.21
N 14 15 15
The results presented in table 5.4 show that HotChocolate was faster than the two
others on average, while GraphQL for .NET was faster than REST.
Lastly, experiment 4 was performed to test hypothesis 4 using an unmodified POST
28
Table 5.5: Execution time of POST REST request and POST GraphQL requests without modification.
1 276.84 210.13 325.41
2 287.00 210.01 325.55
3 283.46 213.35 326.75
4 279.43 211.00 327.49
5 287.01 209.90 327.65
6 280.75 211.24 327.77
7 284.82 212.03 326.55
8 278.44 208.84 328.53
9 283.24 210.03 326.37
10 280.35 211.35 325.86
11 277.14 213.07 326.17
12 287.23 210.82 325.36
13 280.28 210.74
14 294.02 212.02
15 291.19 210.60
MEAN 283.41 211.01 326.62
N 15 15 12
The results presented in table 5.5 show that HotChocolate was faster than the two others
on average, while, similar to experiment 1, REST was faster than GraphQL for .NET.
29
6. Analysis
In the below chapter, the result will be analyzed using BenchmarkDotNet to present
box plots and tables of data. Furthermore, the statistical methods Levene’s test and the
Kruskal–Wallis one-way analysis of variance tests are applied to the data retrieved
from the benchmark analysis to consider if the performance between REST and
GraphQL is statistically significant and to what significance level.
6.1 Performance benchmarks

In the following tables “Ratio” corresponds to the mean of the ratio distribution
([Current]/[Baseline]), “Rank” relates to the relative position of current benchmark
mean among all benchmarks (Arabic style). “Gen 0” or “Gen 1” instead means GC
Generation 0 or 1 collects per 1000 operations and lastly “Allocated” means allocated
memory per single operation (managed only, inclusive, 1KB = 1024B).
6.1.1 Testing of the endpoint BasicInformation (no modification)
Figure 6.1: Execution time of GET REST request and POST GraphQL requests without modification. Generated by
BenchmarkDotNet.
30
Table 6.1: Execution time and memory benchmarks of GET REST request and POST GraphQL requests without modification.
Method Mean (μs) Min (μs) Max (μs) Ratio Rank Gen 0 Gen 1 Allocated (KB)
Rest 289.398 285.508 294.202 1 2 3.4180 1.4648 28
HotChocolate 238.276 237.301 239.515 0.82 1 4.3945 0.9766 34
GraphQLNet 429.714 428.488 431.304 1.48 3 6.8359 1.4648 52
6.1.2 Testing of the endpoint BasicInformation (over-fetching)
Figure 6.2: Execution time of GET REST request and POST GraphQL requests when requesting less data than what is provided
from a one REST endpoint. Generated by BenchmarkDotNet.
Table 6.2: Execution time and memory benchmarks of GET REST and POST GraphQL requests when requesting less data than
what is provided from a one REST endpoint.
Rest 277.712 268.184 283.337 1 2 3.4180 1.4648 28
HotChocolate 210.020 207.979 211.889 0.76 1 3.1738 0.7324 25
GraphQLNet 309.642 305.648 314.271 1.12 3 4.3945 0.9766 37
31
6.1.3 Testing of the endpoint BasicInformation (under-fetching)
Figure 6.3: Execution time of GET REST request and POST GraphQL requests when data must be retrieved from two REST
endpoints. Generated by BenchmarkDotNet.
Table 6.3: Execution time and memory benchmarks of GET REST request and POST GraphQL requests must be retrieved from two
REST endpoints.
Rest 547.645 536.534 561.960 1 3 6.8359 2.9297 54
HotChocolate 317.285 315.038 320.481 0.58 1 4.8828 1.4648 38
GraphQLNet 527.215 522.979 532.931 0.96 2 7.8125 1.9531 61
32
6.1.4 Testing of the endpoint CreateToken (no modification)
Figure 6.4: Execution time of POST REST request and POST GraphQL requests without modification. Generated by
BenchmarkDotNet.
Table 6.4: Execution time and memory benchmarks of POST REST request and POST GraphQL requests without modification.
Rest 283.413 276.836 294.025 1 2 3.9063 1.9531 30
HotChocolate 211.008 208.837 213.347 0.74 1 4.1504 0.9766 31
GraphQLNet 326.622 325.364 328.533 1.16 3 4.8828 0.9766 39
6.2 Statistical test considerations

To be able to use many of the more common tests reliably (e.g. t-test) a set of
assumptions needs to be met [27]. One of the more important assumptions is the
homogeneity of variances which usually is tested through Levene’s test [27]. In the
testing using the raw data above in 5.3 Results of experiments (can also be found in
Appendix C - Summarized results of experiments) the result of the testing indicates that
the variances were not homogenous, see Table 6.5.
33
Table 6.5: Results of Levene’s test for the different groups of population
Test Statistical Conclusion

significance
(Two-tailed)
GetBasicInformation p < 0.001 There is a statistically significant

difference between the variances in the
populations REST, HotChocolate and
GraphQLNet
GetBasicInformationOverFetching p < 0.001 There is a statistically significant

GraphQLNet
GetBasicInformationUnderFetching p < 0.001 There is a statistically significant

GraphQLNet
PostCreateToken p < 0.001 There is a statistically significant

GraphQLNet
Since Levene’s test displayed a significant difference in variances a non-parametric

independent group test, Kruskal–Wallis one-way analysis of variance, was used to
conclude on the significance between the groups (REST, HotChocolate and GraphQL
for .NET) in the difference tests conducted above [28].
Additionally, due to the use of the same implementation in several tests, Bonferroni
Correction was used to adjust the significance level needed to be met in order to
counteract the multiple comparisons problem [53]. Since the data is used from the same
implementation in four different cases the significance level of 0.05 is divided by four,
giving a value of 0.0125 which needs to be met in order for significance to be
concluded.
6.3 Statistical tests

The Kruskal–Wallis one-way analysis of variance tests were conducted using the raw
data in the Result (5.3 Results of experiments) through the web application [54]. The
below results present the statistical significance between the different groups in the
tests.
34
Table 6.6: Statistical results of the One-way Kruskal–Wallis test for the test GetBasicInformation
Test GetBasicInformation
Total number of samples N = 43
Groups Rest: Count =15, Mean rank=23.000

HotChocolate: Count =15, Mean rank=8.000
GraphQLNet: Count =13, Mean rank=37.000
Test statistic H(2) = 37.294
Statistical significance (One-tailed) p < 0.001
Conclusion Based on a significance level of 0.0125, there is a

statistically significant difference between the groups
Rest, HotChocolate and GraphQLNet.
Table 6.7: Statistical results of the One-way Kruskal–Wallis test for the test GetBasicInformationOverFetching
Test GetBasicInformationOverFetching


Table 6.8: Statistical results of the One-way Kruskal–Wallis test for the test GetBasicInformationUnderFetching
Test GetBasicInformationUnderFetching


35
Table 6.9: Statistical results of the One-way Kruskal–Wallis test for the test PostCreateToken
Test PostCreateToken


6.4 Null hypotheses

Taking the above statistical results of the raw data in consideration and comparing with
the testing done in chapter 5. Results, conclusions can be drawn on the different defined
null hypotheses.
Following the test results, testing in 6.1.1 GET - BasicInformation - No modification
concluded that REST is faster (289.4 μs) compared to GraphQL (429.7 μs).
Additionally, tests in tests 6.1.2 GET - BasicInformation - Over-fetching concluded that
REST is faster (277.7 μs) compared to GraphQL (309.6 μs). Furthermore, in the test
6.1.3 GET - BasicInformation - Under fetching the response time of REST (547.6 μs) is
compared to GraphQL (527.2 μs) showing the result of GraphQL being slightly faster
than REST. Lastly, when looking at test 6.1.4 POST - CreateToken - No modification
no changes to the result appeared compared to the no modification GET request above
(6.1.1 GET - BasicInformation - No modification).REST was still faster (283.4 μs) than
GraphQL (326.6 μs). The different results are all significant on the 0.0125 level (as
concluded in 6.4 Statistical tests) and are summarized in Table 6.10.. The only null
hypothesis that was not rejected was the testing conducted on over-fetching of data.
Table 6.10: Null hypothesis results when comparing REST to GraphQL for .NET
Null hypothesis (REST vs GraphQL for .NET) Result
𝐻0: REST APIs (RAPI) GET requests are slower or identical in response time (mean false
time measured in microseconds) compared to GraphQL (GQL) POST requests
when requesting the exact data that is available from one endpoint.
𝐻0: GraphQL (GQL) POST requests are slower or identical in response time (mean true
time measured in microseconds) compared to REST APIs (RAPI) GET requests
when requesting less data than available from one endpoint.
𝐻0: GraphQL (GQL) POST requests are slower or identical in response time (mean false
when requesting more data than available from one endpoint.
𝐻0: REST APIs (RAPI) POST requests are slower or identical in response time false
(mean time measured in microseconds) compared to GraphQL (GQL) POST
requests when requesting the exact data that is available from one endpoint.
36
Testing in 6.1.1 GET - BasicInformation - No modification concluded that, as opposed
to the testing of GraphQL for .NET, HotChocolates is faster (238.3 μs) compared to
REST (289.4 μs). The testing in 6.1.2 GET - BasicInformation - Over fetching and 6.1.3
GET - BasicInformation - Under fetching follows the defined hypotheses and seems to
suggest that GraphQL is faster (210.0 respectively 317.3 μs) compared to REST (277.7
respectively 547.6 μs). Lastly, tests in 6.1.4 POST - CreateToken - No modification
noted that GraphQL (211.0 μs) was faster than REST (283.4 μs) unlike testing
performed on GraphQL for .NET which showed opposite results. The different results
are all significant on the 0.0125 level (as concluded in 6.3 Statistical tests) and can be
summarized in the below table. The null hypotheses that were not rejected were the
testing conducted on the GET and POST requests with no modification to the response.
Table 6.11: Null hypothesis results when comparing REST to HotChocolate
Null hypothesis (REST vs HotChocolate) Result
𝐻0: REST APIs (RAPI) GET requests are slower or identical in response time (mean true
time measured in microseconds) compared to GraphQL (GQL) POST requests
when requesting the exact data that is available from one endpoint.
when requesting less data than available from one endpoint.
when requesting more data than available from one endpoint.
𝐻0: REST APIs (RAPI) POST requests are slower or identical in response time true
(mean time measured in microseconds) compared to GraphQL (GQL) POST
requests when requesting the exact data that is available from one endpoint.
37
7. Discussion
This thesis aimed at comparing the response time of GraphQL versus REST when
implementing an .NET Web API. To this end four different tests were conducted and
analyzed within an experiment to gain insights.
Results display that variance in response time is much larger when contacting an
REST endpoint compared to GraphQL, see Figure 6.1 to 6.4. This means that even
though REST in some cases is faster than GraphQL, there is a more consistent response
from GraphQL which can be an important factor depending on the use case.
Furthermore, the general memory allocation is much larger in GraphQL compared to
REST which could be explained by the extra layers and work needed for GraphQL to
evaluate and execute queries.
The performance benchmark results are mostly not comparable with the result found
in other contributions and as such it does not correlate to previous research. This is due
to testing being performed in different ways and using a variety of internally/externally
used GraphQL servers as well as different applications and development environments.
With that said, inconsistencies have also been presented in research before. REST has
been presented both as faster than GraphQl [18], as well as inconsistent in terms of
response time [19], [22].
7.1 REST vs GraphQL for .NET

To implement GraphQL for .NET in the application, 373 lines of code were required
over 21 classes, in comparison to REST which only required 79 lines of code over two
classes. Reason being the code first approach being one of few options when
implementing this framework [45]. The amount of classes also contributes to the
increased lines of code, since there is mandatory syntax in each class causing extra lines
of code. The code first approach with defined types in its own classes has advantages as
well, e.g. a more granular and controllable way of creating mappings. However, if the
aim is to implement GraphQL using as few lines of code as possible it might not be the
best solution.
In regards to the testing of performance, beginning with the GraphQL for .NET
framework, the results are mostly consistent with the different defined hypotheses
where expectantly, REST is faster than GraphQL for no modification GET and POST
request. Furthermore, as expected and defined in the hypothesis statements, GraphQL is
found to be faster than REST in an under-fetch scenario. There is one unexpected and
interesting finding that is not consistent with the defined hypotheses (where the data
favors the alternative hypothesis); the testing of response time when performing REST
over-fetching. By performing comparative analysis between the different test cases it
was noted that while the response time of the REST requests is roughly the same
between the first (289.4 μs) and second (277.7 μs) query, the response time decreases
for GraphQL when not returning the whole object as in the first query (429.7 μs)
compared to the second query (309.6 μs), see 6.1.1 GET - BasicInformation - No
modification and 6.1.2 GET - BasicInformation - Over fetching. Even though this
particular test result suggests that GraphQL is slower when performing REST
over-fetching, this might not always be entirely true. What can be seen in the above
testing is an indication of an identified overhead in how GraphQL is executing and
evaluating queries as suggested by the hypotheses and the information gathered in the
theoretical background, see 3.2 GraphQL. What this test is showing is that for this
particular response the overhead is not offset by the smaller response requested by
38
GraphQL, which means that if the REST response is sufficiently large and the GraphQL
response is slimmed down enough the GraphQL could become faster than REST.
7.2 REST vs HotChocolate

When implementing the framework HotChocolate GraphQL required 81 lines of code
compared to the 79 needed for REST. The requirements as defined in 3.2 GraphQL, to
map objects to types is still existing in HotChocolate, however the framework allows
for a much easier way to do this mapping through an annotation based approach, with
much more happening behind the scenes when compared to GraphQL for .NET.
Furthermore, two separate classes were created, one for queries and one for mutations,
as per best practices in GraphQL. Had the implementation been done in a single class
the total lines of code for GraphQL would be less than that of REST.
Coming back to the performance testing on HotCocolate, the results from the testing
was perhaps the most interesting finding throughout the experiment. In this case,
HotChocolate not only outperforms the framework GraphQL for .NET in all the
different tests, see 6. Analysis, but also REST in the test where no modifications were
made, see 6.1.1 GET - BasicInformation - No modification and 6.1.4 POST -
CreateToken - No modification. This was not expected and also somewhat contradicting
the result in the related work, stating that GraphQL should be slower than REST when
no change is made to the returned response.
Due to the different and unexpected result in performance when performing testing
of HotChocolate, a senior developer of the HotChocolate framework was contacted in
order to build an understanding of the possible reasons for the application being not
only more performant than GraphQL for .NET but also compared to REST. The senior
developer indicated that there is a mistake in the initial assumption that REST is faster
than GraphQL (HotChocolate) and adds that REST potentially can be faster, but that
this requires optimization work from the developers side and is not provided out of the
box. Additionally, the developer confirms that there exists an overhead in the GraphQL
execution engine but mentions that for HotChocolate this overhead is quite small.
Automatic optimization of requests is performed which makes the GraphQL requests
perform better than most requests done through REST. Examples of this include sending
through the GraphQL query in byte format through the transport layer instead of
allocating string objects (which would add overhead in terms of
serialization/deserialization). Moreover, the query that gets sent through the transport
layer is not the query that gets executed and this modification/optimization enables
HotChocolate to fetch the information in parallel and assemble the result faster. It was
also stated that the performance of HotChocolate is, and continues to be, a focus area
(with performance improvements of around 300 % coming to the next major version,
13). Reasons for it being faster than both GraphQL for .NET and REST being
optimization of the GraphQL execution engine. This is further corroborated by the
benchmark results generally showing better memory allocation and handling of the
garbage collector compared to both REST and GraphQL for .NET. E.g. in the test 6.1.1
GET - BasicInformation - No modification the total Gen 1 collections per 1000
operations was 0.9766 compared to 1.4648 for both GraphQL for .NET and REST,
which is consistent throughout the whole experiment. Finally, HotChocolate are
continually measuring its performance against other libraries e.g. GraphQL for .NET
and while that is not directly comparable with results from this thesis, those results are
also indicating that HotChocolate in many ways is more optimized than GraphQL for
.NET [55].
39
8. Conclusions and Future Work
This thesis has presented how the implementation of two GraphQL frameworks
(GraphQL for .NET and HotChocolate) behaves compared to REST in an .NET 6
environment. By conducting a controlled experiment, containing five different
hypotheses, the experiment intended to answer the following research questions: What
are the performance implications when using GraphQL over REST in Web APIs? The
author partnered with one of the big four accounting firms in the world to implement an
application to act as a test case and at the same time solve the problem of digitally
integrating to the Swedish Companies Registration Office and to register auditor reports
through an API.
When looking at performance, HotChocolate outperforms both GraphQL for .NET
and REST, in a .NET 6 Web API which was a surprising finding since other related
work suggest that REST is faster than GraphQL when requesting the same data. With
the testing performed using BenchmarkDotNet the results indicate that there are several
factors that play a role in performance and code maintainability, with perhaps the most
important finding being the importance of which framework is used in the
implementation. The inner workings of a GraphQL engine is without doubt complex,
adding extra layers that need to receive, validate and resolve queries and what is to be
returned/dropped, causing an overhead which will need to be handled for GraphQL to
be feasible to use over REST. The different frameworks also implement GraphQL in
different ways, allowing e.g. different methods to make mapping of types easier, as well
as different optimizations which in turn affects both performance and code
maintainability (compare e.g. the code first approach in GraphQL for .NET with the
annotation based approach in HotChocolate). The previous related work does not focus
on testing different GraphQL frameworks within the same development environment,
but instead focus on only comparing REST to GraphQL, which makes this an
interesting finding in not only a .NET environment, but potentially to other
development environments as well.
8.1 Future Work

The framework requiring the least amount of effort to implement was REST if looking
at the lines of code and classes required, but interestingly enough, closely rivaled by
HotChocolates implementation which automated many of the mandatory steps to get
GraphQL to work. Furthermore it was only the fact that the author opted to use best
practices when implementing HotChocolate that caused extra overhead in the amount of
lines of code required. If not opting to use best practices, or changing the REST
implementation to use two separate controller classes, HotChocolate would instead be
the one requiring the least amount of code to implement. Looking at the related work,
research regarding the amount of lines of code in the different developed applications
are sparsely reported [19], with REST requiring more lines of code in two of three
rewritten applications. This is an interesting finding, and more research could be made
to conclude how code maintainability is affected in the different frameworks using
different metrics, like lines of code, Halstead volume, Cyclomatic Complexity or others.
As noted in the testing of GraphQL for .NET the response time of a REST over-fetch
was faster than that of GraphQL, but the analysis indicated that there is an overhead
threshold before GraphQL becomes more performant in these cases. Since our
responses were limited by the relatively small test data (e.g. total allocation of between
25-61 kb), as defined by the Swedish Companies Registration Office, one future
research direction could be to investigate the performance behaviors on larger requests
40
and responses, as suggested by the performance benchmarks performed by
HotChocolate in its own comparison between other frameworks. A separate view to this
would be to find the exact threshold limit in request/response size, to see when
GraphQL outperforms REST. Moreover, this thesis has only focused on testing
performed on GraphQL POST requests since this is considered standard. Another
interesting research area would be to also get a sense of how GraphQL GET requests
are behaving from a performance perspective. Lastly, this thesis concluded that the
framework used is an important factor in deciding what architectural type is more
performant. Testing has only been performed on the two most popular GraphQL
frameworks that exists for C# while other frameworks could have called for a different
implementation to GraphQL, making it better or worse in different situations.
41
References
[1] E. Spjut and M. Ulfhielm, “Revisionsberättelser : Vilka faktorer har ett samband
med orena revisionsberättelser?,” Dissertation, Handelshögskolan vid Umeå
universitet, Umeå, 2008. Accessed: Feb. 11, 2022. [Online]. Available:
http://urn.kb.se/resolve?urn=urn:nbn:se:umu:diva-1710
[2] Riksdagsförvaltningen, “Årsredovisningslag (1995:1554) Svensk
författningssamling 1995:1995:1554 t.o.m. SFS 2021:1216 - Riksdagen.”
https://www.riksdagen.se/sv/dokument-lagar/dokument/svensk-forfattningssamli
ng/arsredovisningslag-19951554_sfs-1995-1554 (accessed Feb. 11, 2022).
[3] “Sverige i toppen i ny rankning över EU:s mest digitaliserade länder,”
Regeringskansliet, Nov. 12, 2021.
https://www.regeringen.se/pressmeddelanden/2021/11/sverige-i-toppen-i-ny-rank
ning-over-eus-mest-digitaliserade-lander/ (accessed Feb. 11, 2022).
[4] Uppdrag att införa en tjänst för ingivning av finansiell information avseende
årsredovisningar m.m. Regeringen och Regeringskansliet, 2016. Accessed: Feb.
11, 2022. [Online]. Available:
https://www.regeringen.se/regeringsuppdrag/2016/08/uppdrag-att-infora-en-tjanst
-for-ingivning-av-finansiell-information-avseende/
[5] Uppdrag att utveckla tjänsten för digital ingivning av årsredovisningar m.m. för
alla företagsformer. Regeringen och Regeringskansliet, 2018. Accessed: Feb. 11,
2022. [Online]. Available:
https://www.regeringen.se/regeringsuppdrag/2018/03/uppdrag-att-utveckla-tjanst
en-for-digital-ingivning-av-arsredovisningar-m.m.-for-alla-foretagsformer/
[6] “Programvara och leverantörer – Lämna in årsredovisningen digitalt,”
Bolagsverket.
https://bolagsverket.se/pb/etjanster/lamna-in-arsredovisning/programvara-och-lev
erantorer-lamna-in-arsredovisningen-digitalt-1.17543 (accessed Feb. 11, 2022).
[7] S. Hadjipetri Glantz, “Obligatorisk digital årsredovisning rör upp känslor,”
Balans, Jun. 07, 2021.
https://tidningenbalans.se/artikel/obligatorisk-digital-arsredovisning-ror-upp-kans
lor (accessed Feb. 11, 2022).
[8] “Företagsregistret,” Statistiska Centralbyrån.
http://www.scb.se/vara-tjanster/bestall-data-och-statistik/foretagsregistret/
(accessed Feb. 11, 2022).
[9] “Vad gör en revisor?,” FAR.
https://www.far.se/kunskap/branschens-yrkesroller/vad-gor-en-revisor/ (accessed
May 14, 2022).
[10] M. Lo, “RESTful API 101,” Geek Culture, Aug. 25, 2021.
https://medium.com/geekculture/restful-api-101-b61671e5a3ea (accessed Apr.
28, 2022).
[11] S. B. Avraham, “What is REST — A Simple Explanation for Beginners, Part 1:
Introduction,” Extend, Sep. 05, 2017.
https://medium.com/extend/what-is-rest-a-simple-explanation-for-beginners-part-
1-introduction-b4a072f8740f (accessed Apr. 28, 2022).
[12] “GraphQL | A query language for your API.” https://graphql.org/ (accessed Apr.
26, 2022).
[13] W. Trocki, “GraphQL Performance Explained,” Medium, Nov. 15, 2019.
https://medium.com/@wtr/graphql-performance-explained-cb4b43412fb4
(accessed May 17, 2022).
42
[14] ChilliCream GraphQL Platform. ChilliCream, 2022. Accessed: Apr. 25, 2022.
[Online]. Available: https://github.com/ChilliCream/hotchocolate
[15] GraphQL for .NET. graphql-dotnet, 2022. Accessed: Apr. 25, 2022. [Online].
Available: https://github.com/graphql-dotnet/graphql-dotnet
[16] R. Anderson, D. Roth, and S. Luttin, “Overview of ASP.NET Core.”
https://docs.microsoft.com/en-us/aspnet/core/introduction-to-aspnet-core
(accessed Jun. 02, 2022).
[17] G. Brito and M. T. Valente, “REST vs GraphQL: A Controlled Experiment,” in
2020 IEEE International Conference on Software Architecture (ICSA), Mar.
2020, pp. 81–91. doi: 10.1109/ICSA47634.2020.00016.
[18] D. A. Hartina, A. Lawi, and B. L. E. Panggabean, “Performance Analysis of
GraphQL and RESTful in SIM LP2M of the Hasanuddin University,” in 2018
2nd East Indonesia Conference on Computer and Information Technology
(EIConCIT), Nov. 2018, pp. 237–240. doi: 10.1109/EIConCIT.2018.8878524.
[19] M. Seabra, M. F. Nazário, and G. Pinto, “REST or GraphQL? A Performance
Comparative Study,” in Proceedings of the XIII Brazilian Symposium on Software
Components, Architectures, and Reuse, New York, NY, USA, Sep. 2019, pp.
123–132. doi: 10.1145/3357141.3357149.
[20] O. Hartig and J. Pérez, “Semantics and Complexity of GraphQL,” in Proceedings
of the 2018 World Wide Web Conference, Republic and Canton of Geneva, CHE,
Apr. 2018, pp. 1155–1164. doi: 10.1145/3178876.3186014.
[21]. “NET and .NET Core official support policy,” Microsoft.
https://dotnet.microsoft.com/en-us/platform/support/policy/dotnet-core (accessed
Feb. 11, 2022).
[22] S. L. Vadlamani, B. Emdon, J. Arts, and O. Baysal, “Can GraphQL Replace
REST? A Study of Their Efficiency and Viability,” in 2021 IEEE/ACM 8th
International Workshop on Software Engineering Research and Industrial
Practice (SER IP), Jun. 2021, pp. 10–17. doi: 10.1109/SER-IP52554.2021.00009.
[23] “GraphQL Code Libraries, Tools and Services.” https://graphql.org/code/#c-net
(accessed Apr. 25, 2022).
[24] A. Gupta and P. Jalote, “An Experimental Evaluation of the Effectiveness and
Efficiency of the Test Driven Development,” in First International Symposium on
Empirical Software Engineering and Measurement (ESEM 2007), Sep. 2007, pp.
285–294. doi: 10.1109/ESEM.2007.41.
[25] C. Wohlin, P. Runeson, M. Höst, M. C. Ohlsson, B. Regnell, and A. Wesslén,
Experimentation in Software Engineering. Springer, 2012. doi:
10.1007/978-3-642-29044-2.
[26] dotnet/BenchmarkDotNet. .NET Platform, 2022. Accessed: Apr. 23, 2022.
[Online]. Available: https://github.com/dotnet/BenchmarkDotNet
[27] “Independent t-test in SPSS Statistics - Procedure, output and interpretation of
the output using a relevant example | Laerd Statistics.”
https://statistics.laerd.com/spss-tutorials/independent-t-test-using-spss-statistics.p
hp (accessed May 15, 2022).
[28] Y. Chan and R. P. Walmsley, “Learning and understanding the Kruskal-Wallis
one-way analysis-of-variance-by-ranks test for differences among three or more
independent groups,” Phys. Ther., vol. 77, no. 12, pp. 1755–1762, Dec. 1997, doi:
10.1093/ptj/77.12.1755.
[29] R. C. Martin, Clean Architecture: A Craftsman’s Guide to Software Structure and
Design, 1st ed. USA: Prentice Hall Press, 2017.
[30] R. C. Martin, J. M. Rabaey, A. P. Chandrakasan, and B. Nikolić, Agile Software
43
Development: Principles, Patterns, and Practices. Pearson Education, 2003.
[31] A. Akinshin, Pro .NET Benchmarking: The Art of Performance Measurement, 1st
ed. edition. Apress, 2019.
[32] G. Mavroudeas et al., “Learning GraphQL Query Cost,” in 2021 36th IEEE/ACM
International Conference on Automated Software Engineering (ASE), Nov. 2021,
pp. 1146–1150. doi: 10.1109/ASE51524.2021.9678513.
[33] “Schemas and Types | GraphQL.” https://graphql.org/learn/schema/ (accessed
May 09, 2022).
[34] “Execution | GraphQL.” https://graphql.org/learn/execution/ (accessed May 09,
2022).
[35] G. Cleeren, “Architecting ASP.NET Core 3 Applications: Best Practices,” Jan.
08, 2021. Accessed: May 13, 2022. [Online]. Available:
https://app.pluralsight.com/library/courses/architecting-asp-dot-net-core-applicati
ons-best-practices
[36] A. Baco, “CQRS Pattern with C#,” Medium, Nov. 16, 2020.
https://abdelmajid-baco.medium.com/cqrs-pattern-with-c-a9aff05aae3f (accessed
Apr. 23, 2022).
[37] S. Kudchikar, “CQRS Design Pattern C#,” Code with Shadman, Mar. 14, 2020.
https://codewithshadman.com/cqrs-design-pattern-csharp/ (accessed Apr. 23,
2022).
[38] S. Kudchikar, “Mediator Pattern C#,” Code with Shadman, Nov. 06, 2019.
https://codewithshadman.com/mediator-pattern-csharp/ (accessed Apr. 23, 2022).
[39] A. Tarek, “Mediator Design Pattern In .NET C#,” Medium, Jan. 08, 2022.
https://levelup.gitconnected.com/mediator-design-pattern-in-net-c-e1bfcc96789d
[40] “GraphQL 5.1.1.” https://www.nuget.org/packages/GraphQL/ (accessed May 05,
2022).
[41] “HotChocolate 12.7.0.” https://www.nuget.org/packages/HotChocolate/ (accessed
May 05, 2022).
[42] “BenchmarkDotNet 0.13.1.” https://www.nuget.org/packages/BenchmarkDotNet/
[43] “Moq 4.17.2.” https://www.nuget.org/packages/Moq/ (accessed May 05, 2022).
[44] Rick-Anderson, “Tutorial: Create a web API with ASP.NET Core.”
https://docs.microsoft.com/en-us/aspnet/core/tutorials/first-web-api (accessed
Apr. 24, 2022).
[45] “GraphQL .NET.”
https://graphql-dotnet.github.io/docs/getting-started/introduction (accessed Apr.
24, 2022).
[46] “Queries.” https://chillicream.com/docs/hotchocolate/defining-a-schema/queries
[47] “Mutations.”
https://chillicream.com/docs/hotchocolate/defining-a-schema/mutations (accessed
Apr. 24, 2022).
[48] AutoMapper/AutoMapper. AutoMapper, 2022. Accessed: Apr. 23, 2022.
[Online]. Available: https://github.com/AutoMapper/AutoMapper
[49] K. Larkin, S. Gordon, G. Condron, and R. Nowak, “Make HTTP requests using
IHttpClientFactory in ASP.NET Core.”
https://docs.microsoft.com/en-us/aspnet/core/fundamentals/http-requests
[50] J. Calvarro Nelson, S. Smith, and J. van der Til, “Integration tests in ASP.NET
44
Core.” https://docs.microsoft.com/en-us/aspnet/core/test/integration-tests
[51] moq. Moq, 2022. Accessed: Apr. 23, 2022. [Online]. Available:
https://github.com/moq/moq4
[52] K. Larkin, R. Anderson, T. Dykstra, and S. Smith, “Filters in ASP.NET Core.”
https://docs.microsoft.com/en-us/aspnet/core/mvc/controllers/filters (accessed
May 06, 2022).
[53] R. A. Armstrong, “When to use the Bonferroni correction,” Ophthalmic Physiol.
Opt. J. Br. Coll. Ophthalmic Opt. Optom., vol. 34, no. 5, pp. 502–508, Sep. 2014,
doi: 10.1111/opo.12131.
[54] “AI-Therapy | Statistics for Psychologists | ANOVA.”
https://www.ai-therapy.com/psychology-statistics/hypothesis-testing/anova
[55] M. Staib, “Say hello to Hot Chocolate 12!,” Say hello to Hot Chocolate 12!, Sep.
27, 2021. https://chillicream.com/blog/2021/09/27/hot-chocolate-12 (accessed
May 04, 2022).
45
Appendix A - Developer backgrounds and Code reviews
Max Fiske
Background
A senior architect and backend developer with nine years of combined practical
development experience at different companies (currently at Forefront Consulting
Group AB). Max has worked primarily with C# and has implemented several
applications using Clean Architecture. Some of Max's previous engagements include
Riksrevisionen, Telia, Kronfågeln and many more.
Code review
I think it looks good. I have some general comments regarding the solution.
Things that are good:

- You are following the principles of clean architecture which I personally like.
Everything is pointing inwards against the domain.
- Good abstractions.
- Nice integration test.
- Clean and easy to understand code base.
- BenchmarkDotNet is a good library and well established in the community.
Things that can be improved:

- Using a http factory.
- Some Swedish words in the code base that could be in English.
Things to think about

- BenchmarkDotNet works for this comparison between rest and GraphQL but the
result may vary if it's not run in an isolated environment. For example if you run
the benchmark on your computer it will vary depending on resources available
during the test run.
- If the code would query an actual database you may actually get some different
results.
Daniel Gustafsson
Background
A senior DevOps and backend developer with twelve years of combined practical
development experience at different companies (currently at Netlight Consulting AB
(publ)). Daniel has worked primarily with C# and has implemented countless API based
applications/wrappers in a Web API environment. Some of Daniels previous
engagements include Elgiganten AB, Betsson AB (publ), Universitets- och
högskolerådet (UHR) and BAE Systems AB.
Code review
The code looks good. Please take a look at the below parts:
- The field “Client” in the ScroW.Infrastructure project has been made public, but
is never used outside the ScroClient class. Consider making it private.
- There is no interface for the Query or Mutation class in the ScroW.API project.
How are these classes instantiated?
46
Joakim Ahlén
Background
A senior fullstack developer with 21 years of experience at Digistrada Sweden AB.
Joakim has worked in several different projects as a consultant, Region Stockholm, Alot
Technology Ltd and Norconsult AB to name a few, and has worked in several different
development stacks, ranging from Javascript/Typescript, Java to C#.
Code review
In general I think it looks good. Some comments:
- Clean and easy to understand code base.
- The Swedish Companies Registration Office is exposing their API in Swedish.
To tackle this I would keep the classes' property names in Swedish, but rename
the Class names to English.
- A large part of the code handles logic for instance, moving data from one class
to another. Another way of handling this would be to use attributes in C# to
instead decorate each individual property and keep one set of classes.
- As I see it you are implementing and exposing the Swedish Companies
Registrations Office API through your own set of endpoints without necessarily
changing the response format. Your solution is good for any party choosing to
implement and apply its own logic/authentication to it.
47
Appendix B - Repository and benchmark outputs
Repository:
https://github.com/LNU-Developer/ScroW-Public
Benchmark Outputs:
https://github.com/LNU-Developer/ScroW-Public/tree/master/tests/ScroW.Api.Perform
anceTests/BenchmarkDotNet.Artifacts
48
Appendix C - Summarized results of experiments
GetBasicInformation (μs) GetBasicInformationOverFetching (μs) GetBasicInformationUnderFetching (μs) PostCreateToken (μs)
Run Rest HotChocolate GraphQLNet Rest HotChocolate GraphQLNet Rest HotChocolate GraphQLNet Rest HotChocolate GraphQLNet
1 291.43 239.24 428.76 273.32 208.74 306.16 544.72 319.48 522.98 276.84 210.13 325.41
2 293.90 237.52 428.49 276.88 209.17 308.89 561.17 319.07 524.18 287.00 210.01 325.55
3 291.00 239.48 429.80 282.66 207.98 308.52 542.83 318.19 526.78 283.46 213.35 326.75
4 287.30 237.78 429.96 275.33 209.78 308.34 546.10 317.68 527.24 279.43 211.00 327.49
5 294.20 237.30 428.99 270.50 208.11 306.12 557.09 316.09 524.27 287.01 209.90 327.65
6 291.69 237.68 428.81 268.18 211.30 305.65 545.70 316.57 523.87 280.75 211.24 327.77
7 289.30 238.52 429.86 280.16 209.70 305.87 542.62 317.18 526.35 284.82 212.03 326.55
8 286.92 238.50 428.63 282.72 211.38 306.77 536.53 318.49 525.73 278.44 208.84 328.53
9 286.90 237.76 430.45 283.34 208.92 311.10 537.64 316.36 526.21 283.24 210.03 326.37
10 290.10 238.53 431.30 274.60 210.25 312.39 556.12 315.41 531.56 280.35 211.35 325.86
11 293.46 237.73 430.21 279.40 209.68 312.78 561.96 316.43 532.93 277.14 213.07 326.17
12 285.51 237.77 430.74 280.96 210.37 312.58 547.25 315.94 529.66 287.23 210.82 325.36
13 287.27 238.19 430.29 278.35 211.54 311.43 537.30 316.89 529.85 280.28 210.74
14 285.58 238.63 280.90 211.89 314.27 549.97 320.48 528.54 294.02 212.02
15 286.39 239.52 278.38 211.50 313.77 315.04 528.07 291.19 210.60
MEAN 289.40 238.28 429.71 277.71 210.02 309.64 547.64 317.29 527.21 283.41 211.01 326.62
N 15 15 13 15 15 15 14 15 15 15 15 12
49

Graphql Vs Rest Api

Uploaded by

Copyright:

Available Formats

Graphql Vs Rest Api

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Graphql Vs Rest Api

Uploaded by

Copyright:

Available Formats

Bachelor Degree Project

Evaluating GraphQL over

Author: Rickard Marjanovic

Keywords: Swedish Companies Registration Office, Controlled Experiment, Clean

8. Conclusions and Future Work 40

Appendix A - Developer backgrounds and Code reviews 46

Appendix B - Repository and benchmark outputs 48

Appendix C - Summarized results of experiments 49

1.2 Related work

1.7 Target group

2.1 Controlled Experiment

2.1.1 Dependent and independent variables

2.2.1 Hypothesis statement

2.2 Other Method Considerations

2.3 Reliability and Validity

2.4 Ethical Considerations

3.3 Clean Architecture

4.1 Software and Hardware

OS: Windows 10.0.19044.1645 (21H2)

4.2 General structure

public ScroController(IMediator mediator) => this.mediator = mediator;

public GraphQLController(IDocumentExecuter executer, ISchema schema)

return new ExecutionResultActionResult(result);

public class CustomWebApplicationFactory<TStartup> :

var expectedBasicInformationResult = MockResponse.GetExpectedBasicInformationResponse();

public class GetBasicInformationEndpointBenchmarks : BenchmarkBase

5.1 General result of implementation

5.2 Code reviews

5.3 Results of experiments

6.1 Performance benchmarks

6.1.1 Testing of the endpoint BasicInformation (no modification)

6.1.2 Testing of the endpoint BasicInformation (over-fetching)

6.2 Statistical test considerations

Test Statistical Conclusion

GetBasicInformation p < 0.001 There is a statistically significant

GetBasicInformationOverFetching p < 0.001 There is a statistically significant

GetBasicInformationUnderFetching p < 0.001 There is a statistically significant

PostCreateToken p < 0.001 There is a statistically significant

Since Levene’s test displayed a significant difference in variances a non-parametric

6.3 Statistical tests

Total number of samples N = 43

Groups Rest: Count =15, Mean rank=23.000

Test statistic H(2) = 37.294

Statistical significance (One-tailed) p < 0.001

Conclusion Based on a significance level of 0.0125, there is a

Total number of samples N = 45

Groups Rest: Count =15, Mean rank=23.000

Test statistic H(2) = 39.130

Statistical significance (One-tailed) p < 0.001

Conclusion Based on a significance level of 0.0125, there is a

Total number of samples N = 44

Groups Rest: Count =14, Mean rank=37.500

Test statistic H(2) = 38.227

Statistical significance (One-tailed) p < 0.001

Conclusion Based on a significance level of 0.0125, there is a

Total number of samples N = 42

Groups Rest: Count =15, Mean rank=23.000

Test statistic H(2) = 36.329

Statistical significance (One-tailed) p < 0.001