Source Code in Database (SCID)
Source Code in Database (SCID)
Development
Environment for
Java
Final Report
May 2003
Abstract
Software development requires the aid of many tools to accomplish a variety of tasks.
Java development tools are a dime a dozen, but finding a good Integrated
Development Environment (IDE) is rare.
The purpose of this project is to design and develop a free, handy IDE to the Java
community that can be used by both amateur and professional software engineers to
write Java programs quickly and efficiently.
The system uses an intuitive Graphical User Interface with dockable toolbars and
workspace windows. It provides an editor with syntax highlighting and allows easy
navigation through source code with the help of a single tree. It also has a code
palette, which enables code templates, ranging from classes and applets to simple
statements, to be inserted at the click of a button.
The project also explores the area of Source Code In Database (SCID) and moves
on to the design and implementation of a tool that parses source code and stores
useful information (such as classes, interfaces, fields, methods and their locations) in
a conventional SQL database which is accessible to user queries. The inability to
locate a piece of information that you know is out there somewhere, but you cannot
find it, can be extremely frustrating. SCID can ameliorate this problem.
Finding errors has been made very easy, since the application has both a parser and
compiler. The parser alerts users to syntax and lexical errors before compiling. This
means that the compiler only returns semantic errors and the number of times the
compiler has to be run is significantly reduced. All errors are linked to the source
code, so users can jump right to the problem and correct it.
i
Acknowledgments
Acknowledgements
I wish to thank Mr Graham Knight, my project supervisor for his help, encouragement
and advice throughout the year. Thanks must also go to my friends and colleagues for
their useful comments and suggestions in the early stages of the project and also to all
those who were kind enough to feign enthusiasm for my demonstrations.
I would also like to thank my family and friends for their support, especially my
brother for his professional proofreading.
Finally, I would also like to thank the developers of the various tools and libraries that
I have used and the Java community in general.
ii
Contents
Contents
Abstract i
Acknowledgements ii
Contents iii
Chapter 1 - Introduction 1
1.1 What is an Integrated Development Environment? 1
1.2 Motivation 1
1.3 Features 2
1.4 The Database 2
1.5 Intended Audience 3
1.6 Structure of the Report 3
1.7 Summary 4
Chapter 2 - Background 5
2.1 General Area 5
2.2 Existing Tools 5
2.3 Tools and Libraries 6
2.4 Source Code in Database (SCID) 7
2.5 Implementation Language 7
2.6 Summary 8
iii
Contents
Chapter 6 – Implementation 36
6.1 Choosing an Implementation Language 36
6.2 Package Hierarchy 36
6.3 The Parser 37
6.4 The Database 39
6.5 Executing System Commands 45
6.6 API Viewer 46
6.7 Code Palette 48
6.8 Class Browser 49
6.9 File Explorer 50
iv
Contents
Chapter 7 - Testing 58
7.1 Testing Strategy 58
7.2 Testing the Parser 58
7.3 Testing the Database 59
7.4 Unit Testing 60
7.5 Testing the GUI 61
7.6 Usability Testing 61
7.7 Testing Platform Independence 61
7.8 Summary 62
Chapter 8 - Conclusions 63
8.1 Evaluation 63
8.2 Performance 63
8.3 Further Work 64
8.3.1 Extending the Database 64
8.3.2 Extending the Code Palette 64
8.3.3 More Options 64
8.3.4 Code Completion 65
8.3.5 UML Visualisation 65
8.3.6 Refactoring 65
8.3.7 Wizards 65
8.4 Summary 66
v
Contents
Chapter 9 - Endmatter 67
A System Manual 68
B User Manual 70
C Test Results 77
D Code Listing 81
E Project Plan 95
F Interim Report 100
G Bibliography 103
H Table of Figures 106
I Glossary 107
vi
Chapter 1 Introduction
Chapter 1
Introduction
An apprentice carpenter may want only a hammer and saw, but a master
craftsman employs many precision tools. Computer programming likewise
requires sophisticated tools to cope with the complexity of real applications,
and only practice with these tools will build skill in their use.
Robert L. Kruse, Data Structures and Program Design
1.2 Motivation
Until recently, computer programs were written using simple text editors (e.g.
Notepad and vi). They were compiled in external console windows and if there were
any compilation errors, programmers would have to switch back to their editors, find
the specified line and correct the error. After a successful compilation, programs
would then be executed in the console window.
The problem with this process is that amateur programmers face the daunting task of
learning how to use their editors and learning obscure compile and execute
commands.
1
Chapter 1 Introduction
for the missing construct. Storing the main constructs of a program into a database
can solve this problem. Constructs can be found by querying the database.
The primary aim of this project is to tie all these tools together in one nice package
with a common interface. This will aid both amateur and professional software
engineers and programmers in developing software.
1.3 Features
The system uses an intuitive Graphical User Interface with dockable toolbars and
workspace windows. As you might expect, it provides an editor with syntax
highlighting. Syntax highlighting is the display of programming language tokens
using different fonts and colors. This makes code easier to follow and errors such as
misplaced quotes easier to find.
Errors detected during compilation are linked to the source code, so users can jump
right to the problem and correct it. It allows easy navigation through source code with
the help of a single tree. It also allows code writing using templates and a
documentation generator and viewer.
The functionality of the system is extensible so other tools can be added at a later
date. For example, a future final year project could involve extending the system to
allow drag-and-drop GUI creation.
The current breed of IDEs simply store source code in files. This system is also
connected to a database as explained in the next section.
In addition to storing source code in files, the system also parses java source files and
stores important information in a database. Any errors during parsing are highlighted
in the source code, so that users can fix them immediately. This significantly reduces
the number of times users have to compile, since the program is already syntactically
valid. After a successful parse, the following information is recorded in the tables of
the database:
• Classes
• Interfaces
• Methods
• Fields
• Since the locations of these constructs are also stored, you can easily find the
declaration of any language construct.
2
Chapter 1 Introduction
• Programmers can concentrate on what they are editing instead of figuring out
which file to edit and where the construct in question exists in that file.
• Programs can be displayed in a variety of ways.
Chapter 4 Describes the analysis and design phase of the project. It starts by
analysing the requirements and splitting the system up into modules
and classes. It also describes why particular decisions in the design
were made. This chapter also illustrates the overall system structure.
Chapter 5 Describes the design of the database. In particular it discusses why this
kind of database was chosen and the structure of the tables in the
database schema.
Chapter 8 Analyses the system that has been presented and evaluates the work
that has been done in this project. Also describes how the work could
be taken further.
Chapter 9 Contains other bits and pieces of information such as the appendices
and bibliography.
3
Chapter 1 Introduction
• Bibliography
• Glossary
Since each chapter builds on the preceding one, it is recommended that these chapters
be read sequentially from start to finish.
1.7 Summary
This chapter discussed some of the problems with existing IDEs and demonstrated the
need of having a database to store source code. The structure of the report was also
presented.
4
Chapter 2 Background
Chapter 2
Background
This chapter presents an overview of the research done in the areas of programming,
databases and Integrated Development Environments. The research was carried out by
means of books, the Internet, newsgroups and conversations with both amateur and
professional programmers and members of the java community in general.
This project aims to solve these common problems by developing a handy, open-
source and free IDE, which is extensible so other tools can be added at a later date.
The user interface will be designed to be similar to existing applications, in terms of
buttons, menus and shortcuts, in order to reduce the learning curve associated with a
new application. It will be designed to run on multiple platforms and will be multi-
threaded for quicker user response.
• Sun’s Forte for Java [1]: This IDE is implemented completely in Java and so
runs on all platforms. It is open source and free. Has a nice auto-update feature
and one of the best help facilities. However, the coding paradigm and overall
5
Chapter 2 Background
• Microsoft Visual J++ [2]: Has some nice features if you are using it just for
editing; however, if you use its visual tools, COM tools, database (like RDO),
or J/Direct, you will be stuck with a Windows only Java program. According
to the java community, Microsoft feels threatened by Java's platform-
independent features, and is designing tools such as Visual J++ and Internet
Explorer in such a way as to sabotage its platform-independence.
• JBuilder [3]: This IDE provides a broad variety of components and is highly
customizable. It includes the Open Tools API that allows a development team
to customize the IDE and add additional features not packed with it.
According to research and personal use it was found that after extended use
(around four or five hours), memory seems to increase out of control. At this
point, you simply have to shut down JBuilder and restart; otherwise, typing
becomes a tedious process, with constant "delays" every couple of lines.
This evaluation reiterates the need for a lightweight, inexpensive IDE that runs on a
variety of platforms.
One specific area in which use of an external library is important is that of parsing the
Java source code itself. For this a parser generator is required.
• SableCC [5]: This is another parser generator library for Java. It is quite
widely used in the Java community, and also has grammars for several
different languages.
• JavaCC [6]: This is the most popular parser generator library for Java. In
addition to the parser generator itself, JavaCC provides other standard
capabilities related to parser generation such as tree building (via a tool called
JJTree included with JavaCC), actions, debugging, etc. JavaCC was used in
6
Chapter 2 Background
this project because it is easily available and comes with a javaCC grammar
repository containing example grammars.
Another external resource that is important to this project is that of editing text.
Clearly, the system must provide an effective text editor with basic word processor
functions such as opening files, saving, cut, copy and paste, find and replace etc. To
implement these features code samples were taken from the famous WordProcessor
example in "Swing, 2nd Edition" by Robinson and Vorobiev [8].
The concept of SCID was first developed by Roedy Green in the early 1970s [9].
Since then a lot of research has been done in this field and SCID-think is gradually
catching on. The concept has not yet been implemented in full in any existing IDE.
The basic idea is to pre-parse source code and put it in a database. This eventually
leads to the elimination of java files, since all source code is stored exclusively in the
database. This is an important step away from thinking of programs strictly as linear
streams of ASCII characters. It would allow a GUI-based data entry system with all
sorts of point and click features, extreme data validation, and ability to reuse that data,
view it in many ways, and search it by any key.
There are many possible kinds of data structures that could be used for this purpose.
These include, but are not limited to, parse trees, xml files and traditional SQL
databases.
This is a very vast field. This project explores the idea of storing only important
information extracted from source code, such as classes, fields and methods, in the
form of rows and columns in an SQL database. Research was carried out in defining
what tables would be required and normalizing them.
It was decided to use MySQL [10]; the most popular open source database also
available on departmental machines. Driver classes (package org.gjt.mm.mysql)
had to be downloaded from the Internet [11].
7
Chapter 2 Background
The object-oriented language chosen was Java, mainly because it enjoys "write once
run anywhere" flexibility and is highly secure, open and robust [12].
Java can tend to be quite slow. In order to match the speed of applications written in
other languages, code must be optimised. A lot of research was done, in the different
ways used to speed up Java [13]. The following ways were identified:
• Compile using optimizations turned on (e.g., javac -O)
• Optimize algorithms, especially those involved in string manipulation (use a
string buffer instead)
• Improve performance by multi-threading
2.6 Summary
After reviewing the work that has been done already, one can conclude that it is
within the scope of this project to create an IDE that not only provides basic functions
but also stores source code in a database.
There are several tools and libraries that might be employed to avoid re-implementing
work which has already been done and allow the project to progress more quickly.
Research has highlighted several points where decisions must be made about how a
particular part of the framework should be implemented. The merits of the various
options have been examined and decisions will be made in the design section later in
the report.
8
Chapter 3 Requirements Gathering
Chapter 3
Requirements Gathering
To develop software projects on time and budget there is nothing more important than
gathering and clearly defining the requirements. This chapter describes the
requirements workflow of the system. To gather requirements, the functionality of
existing systems and how they could be improved was studied, in order to determine
the following:
After exploring and analysing the requirements, these were then articulated in the
form of a Requirements Specification Document, which provides a common
vocabulary, and understanding of the problem space and the desired system
behaviour.
The system will store source code in a MySQL database, which will allow searching.
This will be implemented as a plug-in, so that users, who do not have MySQL
installed on their computers, can still benefit from other features that the system
provides.
In addition, the system will speed up code writing by using templates and a point-and-
click interface. It will also allow users to navigate through source code, by displaying
it in the form of a single tree in which fields and methods are defined as nodes of the
tree.
9
Chapter 3 Requirements Gathering
• M- must have (mandatory). These include the most import requirements such
as editing code, the database for storing and searching code, parser and java
features (compiling and running programs)
• S – should have. These include a file explorer and class navigator.
• C – could have (optional) such as a help system
• W – would have (these can wait), such as UML diagrams, an API Viewer etc
Priorities are used to distinguish user needs (features that are critical to the system's
success) and user wants (features that would be nice to have but are not essential).
ID Requirements Priority
General
1 Platform Independent. M
Engaging user interface with switchable look-and-feel and theme
2 S
support.
3 Support for application, applet and web application development M
4 Users should not have to leave the IDE at any stage. S
5 Built in file browser. S
6 Dockable toolbars and sidebars. S
7 Help system. C
Editor
Output Window
10
Chapter 3 Requirements Gathering
Database
Class Browser
Coding Tools
11
Chapter 3 Requirements Gathering
A use case diagram separates the system into actors and use cases.
The actors of this system (i.e. the IDE) are programmers who could be either
professionals or amateurs. They make use of the system for writing Java programs. In
addition they can browse through class files, insert code templates, search the
database and execute commands such as compiling and running.
1. User selects a file from the file chooser and clicks Open.
2. The system parses the file and updates the Database.
3. At the same time, it also creates a tree view of the class structure of the file and
displays it in the Class Browser.
4. The file is then opened in a new editor window.
If after step 1, any of the above occur, the system informs User and returns.
12
Chapter 3 Requirements Gathering
1. User selects the Add File option from the Database menu.
2. User then selects a file or directory to add to the database.
3. The system parses the file or directory recursively and adds items (fields, methods
etc) to the database.
13
Chapter 3 Requirements Gathering
Error Case 1: If, after step 5, the class does not exist, the system informs the user and
returns.
System
Open File
Save File
«extends» Compile
«extends»
Execute Commands Run
«extends»
Generate Javadoc
Insert Code
User
Modify Options
Add/Remove File(s)
Database Operations
Search Database
Change Look and
Feel
14
Chapter 3 Requirements Gathering
3.5 Scenarios
A scenario is an instance of a use case and corresponds to a possible interaction
between the system and its actors. Some scenarios of key use cases are listed below:
Scenario 1: John presses Ctrl-O and selects Program1.java from the file chooser.
A new editor window pops up and displays the file. The file is added to the Class
Browser tree, and John clicks on the items in tree to view the file’s structure.
Scenario 2: John presses Ctrl-O and selects Program1.java from the file chooser.
He gets a message that the file is already open.
Scenario 3: Chris clicks on the Open button on the toolbar and enters Hello as the
filename. He gets a message that it is not a valid java file. He tries again, but this time
he types Hello.java. The file is displayed in a new window.
Scenario 4: Sam clicks on the Open button on the toolbar and selects a directory. She
gets a warning that the file he selected is a directory. She tries again.
Scenario 1: Peter has just installed this system. He wishes to continue work on an
existing project and wants to upload all the classes in this project to the Database. He
selects Add/Remove Files from the Database menu. A window is displayed which
lists all the files currently in the Database. He clicks on the Add button and selects his
project directory and clicks Ok. He watches, as all the files in his project are
recursively added to the Database. Consequently the list grows. He receives a
message that all the files have been successfully added to the Database.
Scenario 2: James Gosling is working on a massive project. Fortunately, all his files
are stored in the database. He is writing a subclass and wants a list of all the methods
of the super class so that he can override them. He selects Search Database from the
Database menu and selects method search. He enters the name of the super class and
finally hits Search. A results table appears which shows him all the methods in the
super class; their names, return types and parameters. He copies them into the
subclass and overrides them.
15
Chapter 3 Requirements Gathering
Scenario 1: Jim presses the Compile button on the toolbar. An output window pops
up with the compile command. Unfortunately, Jim finds he has forgotten a semicolon
at the end of a statement on line 70. He clicks on the hyperlinked error in the output
window and line 70 is highlighted. He adds the semicolon and recompiles.
Scenario 2: Bob is writing his first program. He presses the Compile button and holds
his breath. The output window does not display any errors. He sighs with relief.
Scenario 1: Joe wishes to write a web application that makes use of servlets. He
clicks New to open a blank editor. He then types the name of the servlet class, which
he has chosen to be MyServlet. He then highlights this name and chooses Servlet
from the Code Palette. A class containing the standard Servlet template, with the
doGet and doPost methods, is displayed in the editor. He just fills it up.
Scenario 2: Fatima thinks that a method in her program is redundant. To test her
theory, she highlights the method and clicks on Comment in the Code Palette. Her
method is commented out. She then saves the file and compiles. No errors are
detected. She concludes that the method serves no purpose and so deletes it from the
program.
Scenario 1: Harry is working on a program, which has a few 100 lines of code so far.
There are lots of fields and methods. He wants to view the code of a particular method
called drawDiagram. He is pleased that he does not need to scroll up a 100 lines to
find the method declaration. He simply scans the class tree and clicks on the node that
says drawDiagram. Automatically, the method declaration is highlighted in the
editor.
Scenario 1: Ahmed wishes to use jikes instead of javac for compiling his
programs. He selects Options from the Edit menu and changes the existing compile
command. He then clicks Apply. After editing his program, he presses Compile on the
toolbar and the jikes compiler is used. The next day he starts up the system again
and is pleased to see that the change he made has been saved.
3.6 Summary
Requirements are bound to change during the course of the project. Unmanaged
changes, however, can be harmful to the project. They can cause delays, increase cost,
decrease quality, or outright kill the project. Only with rigorous requirements
definition and management is it possible to finish your project on time and on budget.
16
Chapter 3 Requirements Gathering
This chapter outlined the requirements workflow of the system. Each requirement had
an associated priority in order to distinguish the important features from the less
important ones. Key use cases and example scenarios for the system were outlined.
These will be very useful in the analysis and design phase for partitioning the
functionality of the system.
17
Chapter 4 Analysis & Design
Chapter 4
Analysis and Design
This chapter describes the analysis and design of the system. It is about refining and
structuring the requirements to get a better understanding of them. It also seeks to
explain the reasoning behind the decisions made and assess their effectiveness in
solving the problems involved. An object-oriented (OO) approach has been used since
it offers a more natural way of conceptualizing and developing software systems.
Unified Modelling Language (UML) has been used as the means of expressing OO
concepts.
Each stage of the design process began before the previous stage was entirely
complete, and as such resulted in alterations being made to the previous stage, such
that some design decisions resulted in a slight change to the specification, and so on.
• Editor: Plays the role of a word processor. It is responsible for displaying files
on the screen for handling edit commands such as cut, copy, paste, undo, redo,
find and replace etc. It highlights source code according to syntax, matches
parentheses and displays line numbers.
18
Chapter 4 Analysis & Design
• Java Parser: Parses java source files and highlights any errors that occur
during parsing. It sends tokens (information about the file being parsed) to the
Database and Class Browser, which are consequently updated.
Collaborators: Database, Class Browser
• Class Browser: Displays class information in the form of a tree. How the tree
is built depends on the information is sent by the parser. The nodes of tree are
coloured according to the access level of the item at that node.
Collaborator: Java Parser
• Pretty Printer: Parses java files and converts them into HTML. These files
can then be viewed as web pages.
Collaborator: Editor
With these responsibilities and collaborations in mind, a basic system model can be
drawn:
Options
Printer
19
Chapter 4 Analysis & Design
• LineNumber: A component used to display line numbers on the left hand side
of the editor window.
• SyntaxDocument: The document on which text is written. It is responsible for
inspecting words and highlighting them if they are keywords, strings or
comments. Also responsible for indentation and parenthesis matching.
• CustomTextPane: A special text pane, which contains the Syntax Document
and Line Number component. It is responsible for editing operations such as
cut, copy, paste, undo, redo etc.
• JavaEditor: Provides a window for containing all of the above. It listens for
window events. For example, it checks if a file has been saved when the
window is closed. Is also responsible for file operations such as opening,
closing, saving etc.
• JavaParser: The tool that does the actual parsing. It also informs the user of
any errors during parsing and updates the Database and Class Browser.
20
Chapter 4 Analysis & Design
• ASCII_UCodeESC_CharStream
• ParseException
• JavaParser
• ParserManager Generated by JavaCC
• JavaParserConstants
• Token
• JavaParserTokenManager
• TokenMgrError
• TreeBuilder: The parser sends information about any class and the
TreeBuilder is responsible for building a tree depicting the structure of the
class. Items in the tree are coloured according to the access level. For example,
private fields are coloured red whereas public ones are green.
• ClassBrowser: Provides a panel for displaying the tree built by the
TreeBuilder. It is also responsible for handling tree events, so for instance, if a
method node is clicked, the corresponding method is highlighted in the editor.
21
Chapter 4 Analysis & Design
In the end it was decided to display code templates in the form of a tree. This way
different code items can be classified appropriately. For example, the tree would have
a branch containing loop constructs such as while, for and do-while and
another branch containing statements such as if, if-else and switch etc. Users
can therefore expand only those branches that they are interested in.
The code constructs and templates that were to be inserted had to be stored
somewhere. It was decided to store them in XML format. An XML file can then be
parsed using an XML DOM parser (part of the java swing library) [16] and the tree
built during parsing. This is a flexible approach since more templates and constructs
can be added at a later date. The user can also edit the existing file to change the
layout (such as brace placing) of the code. An extract of the file is shown below:
<codetype name="Comments">
<code name="//">//|</code>
<code name="/* */">/*|*/</code>
</codetype>
<codetype name="Loops">
<code name="do" >do\n{\n|\n}\nwhile();</code>
<code name="while" >while()\n{\n|\n}</code>
<code name="for" >for( ; ; )\n{\n|\n}</code>
</codetype>
Only one class called CodePalette seemed necessary for this feature. This class
creates a new window, parses the XML file, builds the code tree and listens for user
selections. It then inserts the selected template into the editor and highlighted by the
syntax highlighter.
4.3.7 Options
It was decided, for the sake of simplicity, to store user preferences in a simple text
file. When the system is started up, this file is read and all of the system variables are
initialised using the file data. If a user decides to change their options, this module
updates the file. An extract of the options file is below:
NAME:Fahd Shariff
COMPILECMD:javac -classpath .:<dir> <filepath>
22
Chapter 4 Analysis & Design
4.3.8 Printer
This module is responsible printing. It also allows print previewing. The following
classes were identified to carry out the responsibilities of this module.
This module has been taken from the author’s previous year group project, in which it
was implemented as a web application using JSP and Java Servlets. It parses java
code and marks it up in HTML. It has the following classes:
23
Chapter 4 Analysis & Design
Fig 3 shows the associations between different packages in the system. It also lists all
the classes that each package contains.
It can be seen that the database communicates with the parser and is also connected
with the options package in order to read the user’s database configuration. All
packages are connected to the GUI in order to provide feedback to the user.
24
Chapter 4 Analysis & Design
The DBManager class manages all operations of the database and liaises with the
parser in order to receive information. The AddRemoveDialog class needs to know
what files are in the database and is therefore connected to the extracter class.
25
Chapter 4 Analysis & Design
Some classes are generated from the grammar using JavaCC so it is not very
important to include their attributes and methods. The ParserManager class manages
all parsing operations that are mainly carried out by the JavaParser. The
JavaParser communicates with the external classes, DBInserter and TreeBuilder
in order to keep the database and class browser up-to-date.
Other class diagrams such as those of the Graphical User Interface, are not very
important from the point of view of the reader since they just show how components
are associated with each other.
26
Chapter 4 Analysis & Design
The figure above shows the sequence of activities taking place when a user wishes to
open a file. If the file is a valid java file, then it is parsed and if there are no errors, the
database and class browser are updated concurrently.
27
Chapter 4 Analysis & Design
This is another activity that updates the database. When the user saves a file, the
system first checks to see whether there have been any changes to the file since the
last time it was saved. If there are no changes then there is no need to reparse the file
and update the database since the results of these operations would not make any
difference. However is there have been changes, these are saved by writing the code
to the file. The database and browser are updated to reflect these changes. (More
about this in the implementation section).
28
Chapter 4 Analysis & Design
The diagram above shows the sequence of activities that take place when the user
wants to search for an item in the database. The user’s search information is converted
into an SQL query and executed. If there are no results, the user is informed about this
via a dialog. If results are found, they are displayed to the user.
A good well-designed interface plays a major role in the success of any application.
The application's interface should operate smoothly and as predicted, anticipating
what the user will do next.
29
Chapter 4 Analysis & Design
The interface of this system is designed to follow common sense. This means that the
interface will interact with the user in a natural way that will meet the user’s
expectations. It is also designed to be consistent with other applications by using the
standard menu and windowing tools, button names and locations, dialog boxes, and so
forth.
As seen in the figure, the GUI has been made as intuitive as possible i.e. symbols and
styles the user knows [18].
The interface will be designed to allow multiple look and feels. In this way, users can
choose the look and feel that they feel most comfortable with.
In order to avoid a cluttered interface, it was decided to use Tabbed Panes to contain
the file explorer, class browser and code palette. These three features are represented
as trees and so require sufficient height to expand. They were therefore placed on the
extreme left of the main window.
It is important that the system responds quickly when a user clicks a mouse button or
types on the keyboard. If there is only one thread, when the system is busy (e.g.
compiling etc), it ignores user actions, giving the user the impression that the system
has crashed. In order to avoid this problem, it was decided to use multi-threading i.e.
have one or more threads managing the user interface while others perform different
background tasks.
30
Chapter 4 Analysis & Design
It seemed a good idea to have two separate toolbars: one for formatting and the other
for standard operations. In this way buttons can be categorised for easy identification
and selection. Users have the option of removing the toolbar that they do not use
frequently. Familiar icons were used on all buttons.
Windows covered up by others can lead to “out of sight, out of mind” [19]. To allow
multi-file editing and efficient window management a virtual desktop was designed.
This is essentially a large panel holding multiple editors (as internal frames), which
allows easy window management.
4.7 Summary
This chapter described the Analysis and Design workflow of the system. It analysed
the requirements gathered earlier and broke the system down into modules and then
into classes according to their responsibilities and collaborations. In particular, the
generation of a parser using JavaCC, the initial design of the database using JDBC
and SQL and the design of the Code Palette using XML were discussed. A class
diagram was presented that showed the association between classes. The next chapter
focuses on the design of the database in totality.
31
Chapter 5 Database Design
Chapter 5
Database Design
The design of the database is one of the key ingredients to a successful
implementation. This chapter describes how the database was designed from the point
of choosing a suitable database to the actual logical design.
• Flat: Store data in one big table. An example would be a phone book.
• Relational: Instead of a single table, data is stored in many tables that relate to
each other.
• Object-oriented: Support objects and classes. They allow structured sub
objects; each object has its own identity, or object-id (as opposed to a purely
value-oriented approach) and provide support for methods and inheritance.
It was decided to use a relation database for the purpose of this project. Relational
databases are the dominant paradigm for new applications, and have been used for
most applications developed within the past decade. (Object-oriented databases are
fading in the marketplace, so are only used for compelling situations in which they
clearly surpass relational databases. For example, some engineering and multimedia
applications can benefit from an object-oriented database [21].) The information we
wish to store would fit well in a relational database too.
There are a number of DBMSs suitable for managing relational databases. There are
proprietary DBMSs, which are commercial products e.g. Microsoft’s SQL Server and
Oracle’s Oracle 8i. There are also Open Source DBMSs, which are free of charge,
normally under the GNU General Public License e.g. Interbase and MySQL.
There was no other alternative but MySQL since it is readily available and installed
on departmental machines. However, MySQL does not support stored procedures or
nested statements.
32
Chapter 5 Database Design
The purpose of the database is to store information about java source code. Since Java
is a vast language and programs are becoming increasingly complex, it must be
decided what information the tables in the database will hold. The initial database can
then be extended at a later date to incorporate other information.
Highlight in
Editor
Open File
Store in
Add File to
Database
Get Location of
Database
Item
Search
Search in
Database
Return results
33
Chapter 5 Database Design
The diagram above illustrates the tables used to store information in the database.
Each table consists of columns. PK is the primary key of the table.
34
Chapter 5 Database Design
5.6 Relations
The relations used in the database can be read off from the ER-diagram above. All
relations have to be normalized. Normalization is used to eliminate repeating groups
by placing the repeating data in a separate relation. For example, a field can have
zero, one or more modifiers. Instead of duplicating field information (such as name,
type etc) in every row, a new relation is created called FieldModifiers, which contains
the field identifier (FieldID) and the approporiate modifier (ModifierID). This
minimizes duplication of data i.e. reduces storage space required and speeds up
queries.
The relations for the database in third normal form (3NF) are:
• Modifier (ModifierID, ModifierName): contains modifiers e.g. public, static…
• Type (TypeID, TypeName): contains primitive types, classes & interfaces
• Filepath (TypeID, Filepath): contains a list of files that are in the database
• TypeModifiers (*TypeID, ModifierID)
• Class (ClassID, ExtendsClassID, Location)
• Interface (InterfaceID, Location)
• ClassInterfaces (ClassID, InterfaceID)
• Field (*FieldID, FieldName, TypeID, ClassID, Location)
• FieldModifiers (*FieldID, ModifierID)
• Function (FunctionID, ClassOrInterfaceID): this is a method or constructor
• Method (MethodID, MethodName, ReturnTypeID, Location)
• Constructor (ConstructorID, Location)
• FunctionModifiers (*FunctionID, ModifierID)
• FunctionExceptions (FunctionID, ExceptionName)
• FunctionParameters (FunctionID, ParamName, ParamTypeID)
(The primary and foreign keys have been underlined. Where there are no foreign
keys, the primary key is the one with the star (*) prefix.)
Some relations also have a column called “Location”. This is used by the
ClassBrowser to navigate quickly to where the item is located in the file. For example,
if the user clicks on a particular method in the browser, the browser looks up its
location in the Database and then jumps to that location in the editor window.
5.7 Summary
This chapter described the design of the database. A suitable data structure i.e. a
relational database was chosen followed by a suitable DBMS (MySQL). The items
that needed to be stored in the database were identified. A data flow diagram was
designed to inspect how different parts of the system would interact with the database.
Finally, an ER diagram was created to show the entities and relations between them.
35
Chapter 6 Implementation
Chapter 6
Implementation
This chapter describes the implementation of the design models developed in the
Design phase. The goal is to produce source code and ultimately the executable
system.
Apart from the above reasons, Java is free, flexible, robust and very popular in the
programming community.
36
Chapter 6 Implementation
The following sections describe how the design of each part of the system was
transformed into an implementation in Java. Only the most important and interesting
aspects will be discussed.
A complete Java grammar was obtained from an online grammar repository [15]. This
was edited so that whenever any important tokens were encountered, they were sent to
the database. At the same time, they were also sent to the class browser to be added to
the tree representation of the class. The user would be informed of any errors during
parsing. Since the parser also informs the user of syntax and lexical errors, the user
only has to run the compiler to check for semantic errors.
The three main aspects of parsing that we are interested in are: parsing class
declarations, field declarations and method declarations.
Parsing Class Declarations: The class declaration is usually the first line of the
program and contains information such as the class name, modifiers, super class and
implementing interfaces. An example is:
The Java grammar contains the following method for parsing class declarations:
void UnmodifiedClassDeclaration() :
{
“class” <IDENTIFIER> [ “extends” Name() ][ “implements” NameList()]
ClassBody()
}
37
Chapter 6 Implementation
This is edited so that tokens are passed to the database. The code added is shown in
bold:
void UnmodifiedClassDeclaration() :
{
Token type ; //the class token
Token extendToken=null; //superclass
ArrayList interfaces = new ArrayList() ; //a list for interfaces
}
{
“class” type=<IDENTIFIER> [ “extends” extendToken=Name() ]
[“implements” interfaces =NameList() ]
{
try{
String className = type.toString() ;
String extendsName ;
if(extendToken == null) extendsName = “Object” ;
else extendsName = extendToken.toString() ;
dbInserter.clearOldData(className) ;
dbInserter.addClass(className,
extendsName,
filepath,
getLocation(type)) ;
dbInserter.addClassInterfaces(interfaces,className) ;
}catch(Exception e){System.out.println(e) ;}
}
ClassBody()
}
In the code above, the values of tokens are stored in variables and then sent to the
database using the appropriate methods of the dbInserter object.
The Java grammar contains the following construct for parsing fields:
void FieldDeclaration() :
{}
{
(“public” | “protected” | “private” | “static” | “final” |
“transient” | “volatile” )*
Type() VariableDeclarator() ( “,” VariableDeclarator() )* “;”
}
The grammar is edited so that information such as modifiers, name and type can be
extracted and put in the database. This is similar to class declarations and the full code
can be found in the code listing.
38
Chapter 6 Implementation
The Java grammar contains the following construct for parsing methods:
void MethodDeclaration() :
{}
{
(“public” | “protected” | “private” | “static” | “abstract” |
“final” | “native” | “synchronized” )*
ResultType() MethodDeclarator() [ “throws” NameList() ]
( Block() | “;” )
}
Once the grammar has been edited in this way, JavaCC is used to generate the source
code for the parser.
Running the parser: The ParserManager class is responsible for sending files to the
parser. This process is implemented in the following way:
try{
FileInputStream in = new FileInputStream(filepath) ;
if(parser == null ) parser = new JavaParser(in)
else parser.ReInit(in) ;
39
Chapter 6 Implementation
• Adding/removing files
• Searching
Establishing a connection: The first thing that needs to be done (before SQL
statements can be executed) is to establish a connection with the DBMS, in this case,
MySQL. This involves two steps:
• Loading the driver
• Making the connection
The DBConnection class is responsible for this. It reads the database settings, which
include driver name, database URL, username and password from the configuration
file and establishes a connection in the following way:
Class.forName(driverName);
Connection con = DriverManager.getConnection(url, username, passwd);
Creating Tables: The relations designed previously in the ER diagram were turned
into tables using the SQL CREATE TABLE statement.
In the Modifiers table, java modifiers had to be inserted and in the Types table, java
primitive types were inserted. This was done using the INSERT INTO statement. The
type of data that the columns can contain was declared, such as INT or VARCHAR etc
and primary keys were defined.
The statements used to create the tables are in an SQL script file. This file is executed
when the system is first set up. The following statements set up all the required tables
in the database, in accordance with the design plan.
CREATE TABLE Modifier (
ModifierID INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
ModifierName VARCHAR(255)
);
40
Chapter 6 Implementation
Filepath VARCHAR(255)
);
41
Chapter 6 Implementation
Function(FunctionID),
MethodName VARCHAR(255) NOT NULL,
ReturnTypeID INT REFERENCES Type(TypeID),
Location VARCHAR(255) NOT NULL
);
Inserting Data: The class that is responsible for inserting information is called
DBInserter. It contains the following methods to add data into the database:
42
Chapter 6 Implementation
Searching the Database: A database search dialog was implemented which enables
users to search the database by either filling in their search criteria into a form or by
executing SQL statements directly. In the former case, user’s search criteria is
converted by the system into complex SQL queries and executed. A result set is
returned in the form of a table. The dialog is shown in the figure below:
Users enter information into the slots in the table. In this case, the user is looking for
all fields that have been declared private and contain the string name. Hence the user
enters “%name%” as the Field Name and chooses “private” from the drop down menu
under Access Modifiers. The user then presses Field Search.
43
Chapter 6 Implementation
In the query above, various the Field, Type, FieldModifiers and Modifier tables are
joined using their primary keys. ‘%’ is used as a wildcard. After the statement is
executed the following results are displayed:
Adding/Removing File(s) from the Database: A dialog was needed to show the user
what files the database contains, so that the user can remove and add more files. The
following dialog was created:
The file list is displayed in the dialog. Users can click on any item in the list and then
click Remove. A sequence of DELETE statements is then executed to remove it and all
of its references from every table in the database. This means that all its fields and
methods are removed too.
Remove All clears all the tables in the database, except the Modifiers table and the
Primitive Types. The statements used are shown below:
44
Chapter 6 Implementation
Finally, the dialog has an Add button. This button displays a JFileChooser, which
allows users to select a file or directory to add to the database. If a directory is
selected, all the files in it are recursively added. For example, in the figure above, the
directory called “dbsearcher” was selected, which resulted in all the java files within it
being parsed and added to the database.
This task was designated to the OutputFrame class. It uses the java Runtime classes
to execute commands that belong to the operating system, in the following way:
In the code above, the input stream of the process (or the error stream if compiling) is
obtained (which contains the results of the process) and a buffered reader put on it. A
while loop is used to append each line onto the output window. If the command being
executed was a compile command such as javac, then the error is parsed and is
displayed in the form of hyperlinks.
45
Chapter 6 Implementation
For example when compiling a file called Hello.java the following errors were
displayed:
Since all errors have a fixed format, they can be parsed easily and can be broken
down into the file path, line number and description. The file path and line number are
displayed in the form of hyperlinks:
Users can click on these links to jump to the correct line in the source code. The error
is highlighted in red. In this way the user can avoid delving through lots of lines of
code looking for the error.
http://www.cs.ucl.ac.uk/teaching/java/javadoc/api/index.html
Base URL
The URLs for the packages and classes can be derived from the base URL by
appending the following:
Packages: http://www.cs.ucl.ac.uk/teaching/java/javadoc/api/overview-frame.html
Classes: http://www.cs.ucl.ac.uk/teaching/java/javadoc/api/allclasses-frame.html
46
Chapter 6 Implementation
The API Viewer connects to all the URLs given above and parses the HTML files. It
builds a hash map, which maps all the classes, packages and interfaces to their
corresponding URLs. It then displays a list of all the items on the screen. When an
item is clicked, its URL is looked up in the hash map and the page corresponding to
the URL is displayed.
Users can change the main URL of their documentation in the options window. In this
way they can view the documentation of different projects with ease.
The API Viewer has a search box in which users can search for specific items.
Wildcards are matched too. This means that users do not need to scroll down a long
list of classes to find the one that they are looking for (which is the case when viewing
documentation via a web browser). So, for example, if the user enters “String*” into
the search box, a list of possible matches is displayed including String, StringBuffer,
StringContent and so on. The user can then choose the one s/he wishes to view.
47
Chapter 6 Implementation
<codetype name="Comments">
<code name="//">//|</code>
<code name="/* */">/*|*/</code>
</codetype>
<codetype name="Loops">
<code name="do" >do\n{\n|\n}\nwhile();</code>
<code name="while" >while()\n{\n|\n}</code>
<code name="for" >for( ; ; )\n{\n|\n}</code>
</codetype>
The XML code above contains a special character ‘|’ in some places. For example, a
print statement is represented as:
This character is used to indicate where user’s selected text is to be inserted. Hence if
the user selected the string “Hello World” in the editor and then clicked on the print
item in the tree, the character would be replaced by the selection and the following
would be displayed:
System.out.println(“Hello World”) ;
If no text is selected:
System.out.println(); is displayed.
48
Chapter 6 Implementation
Information is displayed in the form of nodes, which display the file path, class name,
super class, interfaces, methods and fields. Nodes are colored according to the traffic
light scheme; items that are public are green, protected are yellow and private are red.
The tree listens for mouse clicks. When the user clicks on a particular node, an SQL
query is executed to find out the location of the item that was clicked. For example, if
a field was clicked:
This returns the location of the field. If the file is open, the location is highlighted in
the editor. This feature will obviously not work if the database is not connected.
49
Chapter 6 Implementation
System.getProperty("user.home");
This statement returns the root directory, which is then recursively traversed using
methods from the java.io package such as directory.list()(returns a list of all
files in the directory). During this process a tree is built showing the file hierarchy. A
filter is applied so that only directories and files with a “.java” extension are
displayed in the tree. These files are displayed with a special icon so that can be easily
identified.
When a node representing a file is clicked, its path is obtained and the file is opened
in a new editor, provided it is not already open. The file is parsed and the database
and class browser are updated.
50
Chapter 6 Implementation
using a VetoableChangeListener, which listens for the window closing event and
then checks to see if the file has been saved.
Syntax Highlighting: The editor holds a SyntaxDocument, which is responsible for
highlighting code as the user types. This is implemented using a document listener.
Whenever a character is inserted into the document, the resulting word is looked up in
a table of keywords and types. If it is found then it is highlighted in accordance with
the keyword color scheme. It is then checked whether it is a String or part of comment
and then colored accordingly.
Line numbers: This is an external component that fits onto the side of the editor and
displays line numbers. It uses a canvas to paint on which the numbers are painted. The
spacing between numbers is determined by the font size of the editor.
Bracket Matching: This was implemented using a CaretListener. Whenever the caret
encountered a bracket it searched for the matching one. The code below shows how
an opening bracket found is found given a closing bracket, the current position and
the string to look in.
/**
* Finds an opening bracket that matches the closing bracket.
*
* @param closeBracket the closing bracket.
* @param pos the position of the closing bracket.
* @param data the string to look in.
* @return the position of the opening bracket.
*/
public static int findOpeningBracket
(char closeBracket, int pos, String data)
{
char openBracket;
//find the opening bracket that matches this one
switch(closeBracket)
{
case ')': openBracket = '(' ; break ;
case '}': openBracket = '{' ; break ;
default: openBracket = '[' ; break ;
}
//now iterate backwards along the string until you find the
//desired bracket
int count = 0 ;
for(int i = pos-2 ; i>=0 ; i--)
{
char c = data.charAt(i) ;
51
Chapter 6 Implementation
In this way the location of the opening bracket is found. There is a similar method for
finding the closing bracket. Once a matching bracket is found, the region between the
brackets is highlighted.
Popup menu: Like any text editor, this editor has a popup menu, which is triggered
when the user clicks on the text area using the right mouse button.
52
Chapter 6 Implementation
The file contains a variety of information including the name of the user, java
commands and database settings. This gives the user freedom to execute different
commands, for example, they can change their classpath in the compile command or
their database URL in order to use a different database.
The options file contains tags such as <dir>, <filepath> and <classname>.
These are determined by the system during execution and refer to the directory, file
name and class name of the java program currently being edited.
Options Dialog: This is a dialog that allows users to change their preferences. It is
built using a JTable.
6.12 Printing
The system would not be complete without a printing facility. Files can be printed
uses the Java Printing API [24] and syntax highlighting and user fonts are all retained
on the paper version as well. Code for this operation and for print previewing was
adapted from the famous WordProcessor example in "Swing, 2nd Edition" by
Matthew Robinson and Pavel Vorobiev [8]. It had to be modified to work with this
system.
When the Print button is pressed a standard Java Print Dialog is displayed and the file
is then printed.
53
Chapter 6 Implementation
catch (PrinterException e)
{System.out.println(e) ;}}
When the Save As Webpage button is clicked, the user is asked to enter the name of a
web file to save the java file as. The java file is parsed, and HTML tags are emitted
depending on the type of token encountered by the parser.
6.14 Toolbars
The system has two toolbars located at the top of the screen. They can be removed if
desired. The toolbars are:
For example, compilation takes place in a separate thread in the following way:
outputFrame.setFile(file) ;
Thread runner = new Thread()
{
public void run()
{
try
{
outputFrame.compileCode() ;
}
catch(Exception e) {System.out.println(e) ;}
54
Chapter 6 Implementation
}
};
runner.start() ; //starts the thread
A new thread is created which compiles the file. If a new thread were not created, the
user would have to wait until during the compilation process. This way, the user can
continue working on the file while compilation takes place. Opening a file, saving,
parsing and database operations run in separate threads too.
Formatting Toolbar: This holds buttons for formatting text. Users can change the
font type, size and style of selected text. Text alignment can also be changed.
Once a new item (font name, size, style etc) is selected from a drop down menu, it is
applied to the selected text through the use of an attached ActionListener. The item
is assigned to a SimpleAttributeSet instance using the appropriate method from
the StyleConstants class:
The setAttributeSet() method first determines the start and end positions of the
selected text. The setCharacterAttributes() method is called to assign the given
set of attributes to the selection.
6.15 Menus
The system has menus that categorize the buttons on the toolbars and also provide
additional features. The menus are similar to those found in other applications. Each
menu item has a mnemonic and accelerator.
For example, the code below creates a new menu item called Open. This is activated
when the user presses Ctrl-O (accelerator) and also if the user presses Alt-O
(mnemonic).
55
Chapter 6 Implementation
Instead of implementing these actions twice in the menu and toolbar classes, it was
decided use action objects. Each action has its own class and all the actions are
present in a single java file called AppAction.java. An extract of this file is shown
below:
/**
* New File Action
* Opens a New Blank Java Editor
*/
class NewAction extends AppAction
{
public NewAction()
{
super("New", new ImageIcon("app/images/New.gif")) ;
}
/**
* Cut Action
*/
class CutAction extends AppAction
{
public CutAction()
{
super("Cut", new ImageIcon("app/images/Cut.gif")) ;
}
56
Chapter 6 Implementation
In this way, all actions shared by different components are implemented as individual
actions. Buttons and menu items that use these actions can then be created by:
This creates a button called New. When the button is pressed the actionPerformed
method of class NewAction is invoked.
6.18 Summary
This chapter described the implementation of the system based on the design given in
previous chapters. In particular it discussed how the parser was implemented from a
grammar using JavaCC and how the database was built. Various other major
components of the system, such as the Code Palette, Class Browser and API Viewer
were implemented. Minor components such as toolbars and menus used by the system
were discussed briefly. The factors taken into account while implementing each
feature were also presented.
57
Chapter 7 Testing
Chapter 7
Testing
This chapter describes testing phase of the software development cycle. Due to large
number of components, only the important ones have been discussed below. The
different strategies used and any bugs found have also been presented here. Test
results can be found in the appendix.
Individual java classes were tested using the JUnit framework [25]. The concept of
using unit testing to drive design has been employed to a limited extent. Every
method in each class of the system was tested for a variety of inputs and expected
outputs. The testing of each part of the system was carried out as each method was
written, so as to save time that would otherwise be used on a specific testing phase.
This process identified several errors in the code mainly NullPointerExceptions.
GUI components and tools were mostly tested in an interactive fashion. User testing
was carried out as well.
Since, one of the major features of the system is to parse source code and add it to a
database, the parser and database classes had to be tested thoroughly. These have been
explained in some detail below.
A set of dummy files was created. For thorough testing, each file contained many
different constructs such as a super class, interfaces, exceptions, modifiers, parameters
etc. One of these files is shown below:
58
Chapter 7 Testing
It is hard and time consuming to automate the testing of a parser, mainly because a
grammar is used which is converted to java code by JavaCC. An alternative was to
insert print statements at vital points in the code. The words printed out to the console
were then compared with what was expected.
Testing whether the correct information is extracted from the Class Declaration is
shown below:
void UnmodifiedClassDeclaration() :
{
Token type ; //the class token
Token extendToken=null; //superclass
ArrayList interfaces = new ArrayList() ; //a list for interfaces
}
{
"class" type=<IDENTIFIER> [ "extends" extendToken=Name() ]
["implements" interfaces =NameList() ]
{
try{
String className = type.toString() ;
String extendsName = extendToken.toString();
System.out.println(“Name:” +className) ;
System.out.println(“SuperClass:” +extendsName) ;
for(int i = 0 ; i < interfaces.size() ; i++)
System.out.println(“Interface:” +interfaces.get(i)) ;
}catch(Exception e){System.out.println(e) ;}
}
ClassBody()
}
In the code above, print statements were inserted to print out the class name, super
class and interfaces. In the same way, they ere inserted into the FieldDeclaration
and methodDeclaration constructs.
59
Chapter 7 Testing
Once again a set of dummy java files were created. They were sent to the parser,
where they were parsed and added to the database. The database was then checked
using SQL select statements to see whether items were indeed added in the correct
tables and columns. The files were changed, by adding a new method or changing an
existing file name, and then re-parsed. The database was checked again to ensure that
it had been updated correctly.
Using the system interface, files were removed from the database and the database
was once again checked.
Different combinations of database searches were carried out via the GUI and the
results compared to expected results. Due to the complexity of some of the SQL
queries a few minor bugs were encountered but were ironed out immediately.
For example, the StringOperator class has a method for replacing all occurrences of
a substring with another string. This method is shown below:
/**
* Returns a new string resulting from replacing all occurrences of
* "repl" in this string with "with".
*/
public static String replaceString(String text, String repl,
String with)
{
StringBuffer buffer = new StringBuffer(text.length());
int start = 0;
int end = 0;
while( (end = text.indexOf(repl, start)) != -1 ) {
buffer.append(text.substring(start, end)).append(with);
start = end + repl.length();
}
buffer.append(text.substring(start));
return buffer.toString();
}
The method was tested using JUnit. The extract from the JUnit test class is shown
below:
60
Chapter 7 Testing
Unit testing was useful in identifying errors due to boundary conditions such as
ArrayIndexOutOfBoundsException and NullPointerException.
As a result of this process, some errors were found and corrected. Errors were mainly
due to action events e.g. tree not responding, errors not highlighted etc.
Since the system is intended for programmers, it was decided that a computer science
student at university would be an ideal candidate for user testing. The testing process
involved observing users interacting with the system. Users were requested to open
java files and try out the database, class browser, code palette and compiler. Other
features were also tested in this way.
Due to lack of users, user testing was also carried out by taking on the role of the user
and walking through the system. The system was also used to work on the rest of the
system i.e. to edit its own source code.
Due to its multithreaded nature, the system may crash when carrying out large tasks
e.g. opening a file.
61
Chapter 7 Testing
7.8 Summary
The system was tested using a variety of methods such as JUnit testing, print
statements and inspection. Usability testing was carried out to check whether the
system performed well in the real world context. Any bugs found during testing were
rectified immediately. The system works as expected. It may crash unexpectedly, very
rarely, due to multithreading. In order to avoid this, a single thread could be used for
the entire system, but this will slow things down tremendously.
62
Chapter 8 Conclusions
Chapter 8
Conclusions
The aim of this chapter is to evaluate the key achievements and limitations of the
work undertaken, and to arrive at some conclusion about the overall worth of the
project.
8.1 Evaluation
The following main criteria were specified at the outset to define successful
completion of the project:
It can be said that these goals have been broadly achieved and the project does seem
to have been largely successful. A fully working IDE has been produced that satisfies
its goals to a large extent. It was used for further development of the project i.e. to edit
its own source code and worked very well.
MySQL was the only DBMS available in the department. It does not support a
number of features such as stored procedures, transactions and most importantly
nested SELECT statements. As a result, queries involving lots of tables became quite
complex.
8.2 Performance
The final system allows users to write programs quickly and easily. The speed is on
par with (if not faster than) other development environments. This is due to the fact
that tasks requiring lots of resources are started up in new threads. This keeps the
system responsive to user actions. All implementation was performed on UNIX
63
Chapter 8 Conclusions
Solaris Version 8 using Java 1.4.1. The program performed as expected in that
environment but display and speed may differ on other operating systems.
One way of doing this would be to create new relations in the database to hold all the
extra information. For example, another table would be created to hold nested classes
and so on. As a result code will not be stored in files, but will be stored exclusively in
the database. This is an important step away from thinking of programs strictly as
linear streams of ASCII characters. This is what SCID hopes to achieve in the future.
Once stored in the database, users can pick whatever information they wish to view
and are not forced to view the whole file. For compiling source code, information
would be extracted from the database, stored in a temporary file which would them be
compiled normally.
In addition, users could be given the option of entering information into dialog boxes
when a code item is clicked. For instance, if the user presses the for loop button, then
a dialog would appear prompting the user to enter the initial, terminator and increment
expressions.
64
Chapter 8 Conclusions
In the future, the system could enable users to specify their formatting preferences and
automatically format new source code or apply to existing source code. Preferences
include indenting, tab size, braces, spacing, syntax highlighting colours etc. Users
could also be allowed to customise their toolbars and menus.
In addition a more efficient method could be developed for storing options. The
current system uses a text file for this purpose; but in the future options could be
stored in an XML file, perhaps in the following way:
<options>
<name>Fahd Shariff</name>
<compilecommand>javac</compilecommand>
</options>
8.3.6 Refactoring
A refactoring is an operation, which modifies code to improve its design without
affecting the meaning of the program [26]. Possible refactorings include: extract a
method, introduce a variable, change a method, surround with try/catch, and
automatically update all references to a package, class, or method being renamed or
moved.
8.3.7 Wizards
Wizards can speed up code writing.
• The New wizard could allow users to fill in information such as class name,
super class, interfaces, fields and methods into a dialog box. Java source
would be generated automatically from this information.
65
Chapter 8 Conclusions
• The JUnit wizard could automatically generate test classes for java source
code.
• The Javadoc wizard could automatically insert documentation above methods
and fields by looking at their declarations.
• The Jar wizard could allow java projects to be zipped up into jar files.
8.4 Summary
This project has achieved its aims – namely to create an Integrated Development
Environment for Java that implements Source Code In Database, allows class and
error navigation, code insertion and increases the productivity of programmers with a
number of other tools.
The project has resulted in the creation of a very useful piece of software with a
database, code navigation, code insertion and error linking. There are many
advantages of the database, which provides more motivation to develop this further in
the future. Individual tools, such as the API Viewer, can be used separately if desired.
The scope for further work on this IDE is immense. This chapter outlined some of the
improvements to the existing system and new features that can be plugged into it. In
particular it was discussed how the extension of the database could lead to the
extinction of source code in files.
66
Chapter 9 Endmatter
Chapter 9
Endmatter
This chapter contains the various appendices, the bibliography and glossary.
Contents
• A System Manual
• B User Manual
• C Test Results
• D Code Listing
• E Project Plan
• F Interim Report
• G Bibliography
• H Table of Figures
• I Glossary
68
Appendix A System Manual
Appendix A
System Manual
This document contains all the technical details such as how the system should be
compiled and run. It also describes where the code is stored, so that it can be modified
in the future.
For the purposes of this document, the software will be referred to as JIDE (Java
Integrated Development Environment).
• You will need approximately 4.5 MB of free disk space to complete the
installation.
• If you wish to use the database feature, you will need MySQL, which can be
downloaded free from: http://www.mysql.com/downloads. You will not need
to download database drivers since they are provided.
2. Copy the jide.zip file from the disk into this directory.
3. Execute the command to extract all the contents of the zip archive. This will
vary according to which tool you use, but the command line format for using
unzip would be: unzip jide.zip
69
Appendix A System Manual
shell> safe_mysqld&
on UNIX machines.
This command uses a batch file to build a database called jide and set up all
the tables.
7. Installation Complete!
You will need to set up your preferences (particularly your database configuration) in
the Options menu.
Simply type the name of the batch file at the command prompt to execute it.
70
Appendix B User Manual
Appendix B
User Manual
This document attempts to provide help and assistance to programmers using this
Integrated Java Development Environment. There are many advantages to using this
IDE over other editors, although the biggest, may be that source code is parsed and
information about classes, methods and fields are stored in a database. This has
proven to be very useful when searching for something that you know is out there
somewhere, but you don't know where.
The application has a class browser for easy navigation through classes. It also has a
code palette, which enables code templates ranging from classes and applets to simple
statements to be inserted at the click of a button.
Finding errors is very easy, since the application has both a parser and compiler. The
parser alerts you to syntax and lexical errors (which make up approx 80% of errors)
before you press the compile button. This means that the compiler only returns
semantic errors and the number of times the compiler has to be run is significantly
reduced. All errors are linked to the source code via hyperlinks so that they can be
rectified easily.
The sections below cover basic usage of the different features provided by the IDE.
2 Opening Files
Menu: File>Open
Toolbar: Open button
Shortcut: Ctrl-O
File Explorer: Popup>Open File
This displays a file chooser dialog box and loads the specified file into a new
editor. Alternatively, a file can be selected from the File Explorer. By default,
opening a file also adds it to the class browser and to the database.
71
Appendix B User Manual
3 Saving Files
Menu: File>Save
File>Save As
Toolbar: Save button or Save As button
Shortcut: Ctrl-S
This saves the current file to disk. If Save As is used, then the current file is
saved to the new specified location. Saving a file, updates the class browser
and the database.
6 File Explorer
This is located on the extreme left of the application. The tab is named File
Explorer and contains a list of all the files in the system in the form of a tree.
The explorer can be closed by clicking on the X button in the top right hand
corner. It can be viewed as a floating frame by clicking on the Pop button. It
can be docked back into the main application by using the Dock button.
To browse a directory double-click it. Java files are displayed with a ‘cup’
symbol. Unopened Java files can be displayed by double-clicking them.
Clicking a file or directory with the right mouse button displays a popup menu
containing various commands. The Open File button simply opens the file and
is equivalent to clicking on the file. The Browse button adds the files to the
class browser. The Add To Database button adds the selected file to the
database. The Reload button refreshes the file explorer to show any new files
or whether any files have been deleted from the system.
72
Appendix B User Manual
To remove a file, select it from the list and press Remove. To remove
everything in the database press Remove All. This will clear the database.
The SQL Search tab allows you to execute SQL queries on the database.
73
Appendix B User Manual
directory is selected, all the java files under it are added to the browser
recursively.
Items are displayed in different colours depending on what they are. The
traffic light scheme has been used where green refers to all items that are
public, red are private and yellow are protected.
If the database is connected, clicking on an item will highlight where that item
occurs in the source, provided the source file is open. This feature does not
work if there is no database connection.
Code has been categorised according to what it is e.g. loops, statements, comments
etc. Code can be inserted by clicking on the desired node in the tree, which is inserted
at the current caret position in the active file. Alternatively, code typed in the editor
can be highlighted and then an item selected from the tree. This way the highlighted
text is embedded within the code template.
74
Appendix B User Manual
2 Running Programs
Toolbar: Run button
This compiles the current file using java. An output frame displays the results
of executing the program. The run command can be changed in the options
dialog.
4 Running Applets
Toolbar: Applet Viewer
This will ask you to enter the URL of the applet. The applet is displayed using
the applet viewer in a new window.
5 Generating Documentation
Toolbar: Javadoc button
This generates documentation for the current file using javadoc. An output
frame displays the results of executing the command.
75
Appendix B User Manual
The editor has line numbers along the left hand side of the window. It also highlights
syntax while you type.
The title bar of the editor contains the name of the file. If the file has unsaved
changes, the title will have the word “(modified)” appended to it. This disappears
when the file is saved.
Bracket Matching: Positioning the caret immediately before or after a bracket will
highlight the corresponding closing or opening bracket (assuming it is visible) and all
the text within it. In this way it is easier to the bounds of, say loops.
Formatting Text: The formatting toolbar contains buttons for changing the font face,
style and size and for aligning text.
The Popup Menu: The editor has an associated popup menu, which is invoked
whenever the right mouse button is clicked on the editor. The menu contains simple
text editing commands such as cut, copy and paste.
76
Appendix B User Manual
The Look and Feel menu allows you to change the look and feel of the application.
The default is the Metal Look and Feel. You can only select those that are supported
by the system. This means that UNIX machines will not allow you to set a Windows
look and feel. The Themes sub-menu allows you to change the colour coding of the
toolbars, buttons and menus. Themes will only take effect if the Metal Look and Feel
is selected.
77
Appendix C Test Results
Appendix C
Test Results
This document contains the actual results of the tests carried out in the Testing Phase
of the development process. In particular it contains the test results of the database
and parser. Other classes were tested using JUnit and all tests were successful. The
major part of the project is GUI based and hence there are no test results.
public Dummy()
{
}
The source code above is challenging since it contains all the constructs of the java
language that we are interested in. It has a super class, interfaces, two kinds of field
declarations, two constructors and methods with parameters and exceptions.
78
Appendix C Test Results
Class: Dummy
Super class: Parent
Implementing Interfaces: <Imp1, Imp2, Imp3>
Class Modifiers: <public>
Fields:
field1, int, <private, final>
s1, String, <private>
s2, String, <private>
s3, String, <private>
Methods:
method, void, <public, synchronized>
getName, String, <protected>
setVars, void, <public>, <i:int, s:String>,<SQLException>
Constructors:
<public><f:int><Exception>
<public>
Parsing successful.
+--------+----------+
| TypeID | TypeName |
+--------+----------+
| 1 | boolean |
| 2 | char |
| 3 | byte |
| 4 | short |
| 5 | int |
| 6 | long |
| 7 | double |
| 8 | float |
| 9 | Dummy |
| 10 | Parent |
| 11 | Imp1 |
| 12 | Imp2 |
| 13 | Imp3 |
| 14 | String |
| 15 | void |
+--------+----------+
+---------+----------------+-----------+
| ClassID | ExtendsClassID | Location |
+---------+----------------+-----------+
| 9 | 10 | 2,14,2,18 |
+---------+----------------+-----------+
79
Appendix C Test Results
+--------+---------------------------------+
| TypeID | Filepath |
+--------+---------------------------------+
| 9 | ~/jide/src/app/tests/Dummy.java |
+--------+---------------------------------+
+---------+-------------+
| ClassID | InterfaceID |
+---------+-------------+
| 9 | 11 |
| 9 | 12 |
| 9 | 13 |
+---------+-------------+
+--------+------------+----------+
| TypeID | ModifierID | Location |
+--------+------------+----------+
| 9 | 6 | 2,1,2,6 |
+--------+------------+----------+
+-------------+-----------+
| InterfaceID | Location |
+-------------+-----------+
| 11 | 2,46,2,49 |
| 12 | 2,52,2,55 |
| 13 | 2,58,2,61 |
+-------------+-----------+
+---------+-----------+--------+---------+-----------+
| FieldID | FieldName | TypeID | ClassID | Location |
+---------+-----------+--------+---------+-----------+
| 1 | field1 | 5 | 9 | 4,17,4,22 |
| 2 | s1 | 14 | 9 | 5,20,5,21 |
| 3 | s2 | 14 | 9 | 5,24,5,25 |
| 4 | s3 | 14 | 9 | 5,28,5,29 |
+---------+-----------+--------+---------+-----------+
+---------+------------+----------+
| FieldID | ModifierID | Location |
+---------+------------+----------+
| 1 | 4 | 4,5,4,11 |
| 2 | 4 | 5,5,5,11 |
| 3 | 4 | 5,5,5,11 |
| 4 | 4 | 5,5,5,11 |
+---------+------------+----------+
+------------+--------------------+
| FunctionID | ClassOrInterfaceID |
+------------+--------------------+
| 1 | 9 |
| 2 | 9 |
| 3 | 9 |
| 4 | 9 |
| 5 | 9 |
+------------+--------------------+
+----------+------------+--------------+-------------+
| MethodID | MethodName | ReturnTypeID | Location |
+----------+------------+--------------+-------------+
| 3 | method | 15 | 16,30,16,35 |
| 4 | getName | 14 | 20,22,20,28 |
| 5 | setVars | 15 | 25,17,25,23 |
+----------+------------+--------------+-------------+
80
Appendix C Test Results
+------------+------------+-------------+
| FunctionID | ModifierID | Location |
+------------+------------+-------------+
| 1 | 6 | 7,5,7,10 |
| 2 | 6 | 12,5,12,10 |
| 3 | 6 | 16,5,16,10 |
| 3 | 9 | 16,12,16,23 |
| 4 | 5 | 20,5,20,13 |
| 5 | 6 | 25,5,25,10 |
+------------+------------+-------------+
+------------+---------------+-------------+
| FunctionID | ExceptionName | Location |
+------------+---------------+-------------+
| 1 | Exception | 7,32,7,40 |
| 5 | SQLException | 25,56,25,67 |
+------------+---------------+-------------+
+------------+-----------+-------------+-------------+
| FunctionID | ParamName | ParamTypeID | Location |
+------------+-----------+-------------+-------------+
| 1 | f | 5 | 7,22,7,22 |
| 5 | i | 5 | 25,29,25,29 |
| 5 | j | 5 | 25,36,25,36 |
| 5 | s | 14 | 25,46,25,46 |
+------------+-----------+-------------+-------------+
+---------------+-------------+
| ConstructorID | Location |
+---------------+-------------+
| 1 | 7,12,7,16 |
| 2 | 12,12,12,16 |
+---------------+-------------+
From the results above the file was successfully added to the database. The file was
then altered and the results inspected again. Finally another file was added to the
database. The database passed all tests. It was also tested using the Search Dialog in
the interface.
81
Appendix D Code Listing
Appendix D
Code Listing
The application consists of a total of:
• 8 packages
Only the code for the named classes below is in this document. These classes were
chosen because they deal with interesting and important aspects of the system such as
the database and parser.
• DBStatement.java
• DBInserter.java
• DBExtracter.java
• DBManager.java
• ParserManager.java
• OutputFrame.java
82
Appendix E Project Plan
Appendix E
Project Plan
This document was drafted in early November in collaboration with the project
supervisor.
It includes the:
• Work plan – a pert chart showing how the workload was structured over the
project period (October – April)
95
Appendix F Interim Report
Appendix F
Interim Report
This document was outlined in January in collaboration with the project supervisor.
100
Appendix F Interim Report
101
Appendix F Interim Report
1. Continuation of Implementation:
Documentation: A Java API Viewer that helps users quickly find the
documentation of the classes they are looking for. The system will also
allow automatic insertion of documentation.
Wizards: The “New” wizard to generate code for classes given methods
and fields. The “Unit Test” wizard to generate code for JUnit test Classes.
3. Final Report: Compilation of the final report will be carried out during the
beginning of March.
Supervisor’s Signature:
102
Appendix G Bibliography
Appendix G
Bibliography
References cited in the text:
103
Appendix G Bibliography
Other Resources:
• Aubjex, an IDE that transforms Java code into an especially efficient and
complete database form.
http://www.alajava.com/aubjex/index.htm
104
Appendix G Bibliography
• Java XBrowser
http://www.xbrowser.sourceforge.net/
105
Appendix H Table of Figures
Appendix H
Table of Figures
Design Models
Figure 1 – Use Case Diagram .................................................................................. 14
Figure 2 – Basic System Model ............................................................................... 19
Figure 3 – Package Dependencies............................................................................ 24
Figure 4 – Database Class Diagram ......................................................................... 25
Figure 5 – Parser Class Diagram.............................................................................. 26
Figure 6 - Open File Activity Diagram .................................................................... 27
Figure 7 - Save File Activity Diagram ..................................................................... 28
Figure 8 - Search Database Activity Diagram .......................................................... 29
Figure 9 – Initial GUI Design .................................................................................. 30
Figure 10 - Data Flow Diagram ............................................................................... 33
Figure 11 - Entity Relation Diagram ........................................................................ 34
Figure 12 - System Package Hierarchy .................................................................... 36
Screenshots
Figure 13 - Database Search Dialog ......................................................................... 43
Figure 14 - Database Search Results ........................................................................ 44
Figure 15 - Add/Remove File(s) Dialog ................................................................... 44
Figure 16- Ouput Window....................................................................................... 46
Figure 17 - API Viewer ........................................................................................... 47
Figure 18 - Code Palette .......................................................................................... 48
Figure 19 – Class Browser....................................................................................... 49
Figure 20 – File Explorer......................................................................................... 50
Figure 21 – Java Editor............................................................................................ 52
Figure 22 – Options Dialog...................................................................................... 53
106
Appendix I Glossary
Appendix I
G l o s s a ry
This appendix provides short definitions of the jargon terms used throughout the
report.
107
Appendix I Glossary
Mnemonic Shows the user which key to press (in conjunction with the
Alt key) to activate a command.
Modifier Keywords used to modify code e.g. public, private, static etc.
Multithreading Carrying out more than one task at a time.
MySQL A true multi-user, multi-threaded SQL (Structured Query
Language) database server.
Object Oriented Allows structured sub objects; each object has its own
Database identity, or object-id (as opposed to a purely value-oriented
approach).
Options User preferences.
Output Window The text area which displays the results of executing java
commands such as javac and java. Compiler errors are linked
to the source code.
Parentheses Positioning the caret immediately before or after a bracket
Matching will highlight the corresponding closing or opening bracket
(assuming it is visible) and all the text within it.
Parse To analyse a sentence in order to extract tokens corresponding
to syntax .
Parser A tool used for parsing programs.
Parser Generator A program which, given a formal description of a language,
can generate a parser. E.g. JavaCC.
Platform Code that once written, can be run anywhere.
Independent
Popup menu A menu displayed when the right mouse button is clicked.
Pretty Printer A module responsible for saving files as colourful web pages
in HTML format.
Print Preview A feature that shows how the document will look when
printed.
Refactoring An operation which modifies code to improve its design
without affecting the meaning of the program.
Relational Stores data in many tables that relate to each other.
Database
SCID (Source Code in Database) The process of storing source code
in a database instead of files to aid searching and navigating.
Semantic error An error in the meaning of the statements in the program.
Shared action An action used by more than one class e.g. toolbars and
menus.
SQL (Structured Query Language) A language used to create,
maintain, and query relational databases.
Standard Toolbar A toolbar used for common operations such as file handling
and java operations.
Status Bar A panel on the bottom of the main window which displays
messages to inform the user of what the system is doing.
Syntax error An error in the use of the rules of language.
Syntax Colouring text according to syntax
Highlighting
108
Appendix I Glossary
Highlighting
Theme A feature that enables the user to specify alternative colours
and fonts across the entire application.
Toolbar A vertical or horizontal bar containing icons that represent the
commands that can be used in an application.
Type A class or interface.
UML Creating UML representations of projects.
Visualisation
Unit Test Test of one part of the system to see if remediation efforts
were successful.
URL (Uniform Resource Locator) An Internet address which tells a
browser where to find an Internet resource.
Virtual Desktop A feature allowing multiple windows to be opened within the
application to prevent cluttering up the operating system's
desktop.
Wildcard A character that can represent any group of characters e.g. '*'.
Window Arranging windows in the virtual desktop. They can be
Management arranged in a number of ways including vertically and
horizontally.
XML (Extensible Mark-up language) A way of storing data in files.
Used by the code palette.
XML DOM An XML parser.
Parser
109