Design and Implementation of the Advanced Technology for Remote Laboratory

Ning Wang

Download Free PDF

Design and Implementation of the Advanced Technology for Remote Laboratory

Ning Wang

Remote laboratory (RLab) is the use of telecommunications to remotely conduct real experiments, and also is an inevitable necessity for Internet enabled education in Science, Technology, Engineering, and Math (STEM) fields due to their effectiveness, flexibility and cost savings. Currently, plenty of remote experiments have been developed based on various methods. However, most of them need to install some external software plug-in components to address different platforms and network firewalls, which negatively impact the utilization of the remote laboratories. To achieve the remote laboratory without the need of any extra plug-in, we designed and implemented a unified remote laboratory framework in our previous works. A Comet solution via Socket.IO which is the package of Node.js is implemented in the server side, and a new web socket protocol which lets the experiment communicate with Socket.IO is created for the workstation. With the Comet solution, the real-time experiment live streaming video is transferred via network port 1026 and the real-time control command and experiment data are transferred via network port 1029. Consequently, the network firewall issue still can’t be resolved in the Comet solution. To resolve all of issues found in the previous works and integrate all of improvement for the unified framework, we propose a novel unified framework for the remote laboratory development. The novel unified framework includes three application packages, client web application, server application and experiment control application. The client web application uses the Web 2.0 Technology which includes HyperText Markup Language (HTML), Cascading Style Sheets (CSS), and JQuery/JQuery-Mobile JavaScript libraries. In addition, the Mashup technology for user interface integration is employed. The server application is directly built on the top of MySQL Database, Apache web server engine and Node.js web server engine. The experiment control application uses Web Service technology that is based on the LabVIEW (Laboratory Virtual Instrumentation Engineering Workbench) and runs in workstation. To implement the real-time communication between each application, we use JavaScript Object Notation (JSON) and Socket.IO which was developed based on web socket protocol to implement the communication module. To solve the traversing network firewalls issue, we integrate one core component, Node-HTTP-Proxy, in our current framework to share network port 80. With this solution, we can achieve real-time remote experiment live streaming video transmission and real-time experiment data transmission traversing the network firewalls. With the novel unified framework, we implement two important remote mechanical engineering experiments, remote Shape Memory Alloy (SMA) experiment and remote Smart Vibration Platform (SVP) experiment. With the remote laboratory which is enabled by the novel unified framework, the terminal users can use the real-time remote experiments on the most of popular web browsers without the firewall issues and the need for third party plug-in. This novel unified framework will significantly benefit the remote laboratory development in future.

DESIGN AND IMPLEMENTATION OF THE ADVANCED TECHNOLOGY FOR REMOTE LABORATORY THESIS Presented in Partial Fulfillment of the Requirements for the Master of Science Degree in the Graduate School of Texas Southern University By Ning Wang, B.S. Texas Southern University 2014 Approved By: ____________________________________ Chairperson, Thesis Committee ____________________________________ Dean, The Graduate School Approved By: ______________________________________ Wei Li, Ph.D., Chairperson, Thesis Committee _____________ Date ______________________________________ Xuemin Chen, Ph.D., Co-Advisor Graduate School Representative _____________ Date ______________________________________ Aladdin M. Sleem, Ph.D., Committee Member _____________ Date ______________________________________ Miao Pan, Ph.D., Committee Member _____________ Date ii TABLE OF CONTENTS Page LIST OF TABLES .............................................................................................................. v LIST OF FIGURES ........................................................................................................... vi LIST OF ABBREVIATIONS .......................................................................................... viii VITA .................................................................................................................................. ix ACKNOWLEDGMENTS .................................................................................................. x CHAPTER 1 INTRODUCTION ........................................................................................................... 1 1.1 Background ............................................................................................................... 1 1.2 Motivation ................................................................................................................. 4 1.3 Objective ................................................................................................................... 7 1.4 Research Outcomes ................................................................................................... 7 1.5 Outline ....................................................................................................................... 8 2 A NOVEL UNIFIED FRAMEWORK FOR REMOTE LABORATORY DEVELOPMENT ............................................................................................................... 9 2.1 Previous Works ......................................................................................................... 9 2.2 Introduction of a Novel Unified Framework........................................................... 11 2.3 Vital Improvement of the Novel Unified Framework ............................................. 14 2.3.1 The Node-HTTP-Proxy for The Network Port Sharing ................................... 15 2.3.2 Real-Time Experiment Video Streaming Approach ........................................ 16 2.4 New Isolated Experiment Network Design for Remote Laboratory ....................... 17 2.5 New Authentication URL Design for Remote Experiment Webpage .................... 17 2.6 Remote Experiment Scheduler and User Management System .............................. 18 3 A NOVEL REAL-TIME EXPERIMENT VIDEO TRANSMISSION APPROACH ... 19 3.1 Previous Works ....................................................................................................... 19 3.2 Proposed Approach ................................................................................................. 20 3.3 Architecture of the Novel Approach ...................................................................... 22 3.3.1 Streaming Protocol Introduction ...................................................................... 23 3.3.2 The FFmpeg for the Real-Time Video Encoder ............................................... 27 iii 3.3.3 The Real-Time Video Segment Module Introduction ...................................... 29 3.3.4 Web Server Configuration and Video Transmission ........................................ 31 3.4 The Novel Real-Time Video Transmission Approach Implementation ................. 33 4 A NOVEL REAL-TIME EXPERIMENT DATA TRANSMISSION APPROACH .... 34 4.1 Proposed Novel Approach ...................................................................................... 35 4.1.1 The Node-HTTP-Proxy Introduction ............................................................... 36 4.1.2 The Server-Side Software System Introduction ............................................... 39 4.1.3 The Real-Time Web Application Introduction................................................. 42 4.2 Approach Implementation ....................................................................................... 45 4.2.1 Node-HTTP-Proxy Configuration .................................................................... 45 4.2.2 LabVIEW to Node.js via Socket.IO ................................................................. 47 5 THE IMPLEMENTATION OF TWO REMOTE EXPERIMENTS ............................. 49 5.1 The Implementation of the Two Remote Experiments ........................................... 49 5.1.1 The Remote SMA Experiment ......................................................................... 49 5.1.2 The Remote SVP Experiment........................................................................... 52 5.1.3 The User Interface Development ...................................................................... 54 5.1.4 Server Side Integration ..................................................................................... 56 5.1.5 Integrate LtoN Module to Two New Remote Experiment ............................... 57 5.2 Experiment Data Analysis....................................................................................... 58 5.3 Advantages of New Remote Experiments based on the Novel Unified Framework ....................................................................................................... 61 6 CONCLUSION AND FUTURE WORKS .................................................................... 64 6.1 Conclusion.............................................................................................................. 64 6.2 Future Works .......................................................................................................... 65 REFERENCES ................................................................................................................. 66 iv LIST OF TABLES Table Page 1 Challenges in remote laboratory development. ............................................................... 6 2 Research outcomes........................................................................................................... 8 3 Technology/protocol/software list of the new framework. ............................................ 12 4 The comparison of main video streaming transmission protocol. ................................. 24 5 Technology/protocol/software list of the SMA client web application. ........................ 55 6 Technology/protocol/software list of the SMA server application. .............................. 57 7 Technology/protocol/software comparison between two solutions ............................... 61 8 The advantages of the new remote SMA experiment. ................................................... 62 v LIST OF FIGURES Figure Page 1 System architecture of Comet solution (Published at the World Congress on Engineering Education 2013)............................................................... 10 2 The architecture and data transmission flow of the new framework. .................................................................................................................. 13 3 The new remote laboratory solution network architecture. .......................................... 14 4 Scheduler, experiment conflict control and experiment management page screenshots. ................................................................................. 18 5 The working process of the new real-time video transmission approach. ..................................................................................................................... 21 6 The architecture of the novel real-time video transmission solution........................................................................................................................ 23 7 HLS basic configuration (Published at HLS protocol draft in IETF 2012 ). ................................................................................................................ 26 8 FFmpeg trans-coding process. ..................................................................................... 28 9 Segment software package working process. ............................................................... 31 10 The architecture of the novel real time experiment control commands and data transmission solution.................................................................. 34 11 The new data transmission working process. .............................................................. 36 12 Node-HTTP-proxy deployment principle. ................................................................... 38 13 Node.js architecture. ................................................................................................... 40 14 Socket.IO working process. ......................................................................................... 43 15 Macro and micro views of SMA spring deformation and transformation. ............................................................................................................ 50 16 SMA experiment device. ............................................................................................. 52 17 Sample paradigms of the new SMA experiment user interface. .................................. 56 vi 18 The diagram of bridge experiment. .............................................................................. 53 19 SVP experiment device ................................................................................................ 54 20 Sample paradigms of the new SVP experiment user interface. ................................... 56 21 Arbitrary displacement control (with amplitude of 1.205 in) in remote SMA experiment. ............................................................................................ 59 22 DC voltage of 15 V in square waveform applied on SMA wire with frequency of 0.01 Hz. ......................................................................................... 60 23 AC voltage of 15 V in sinusoidal waveform applied on SMA wire with frequency of 0.025 Hz (displacement change in voltage series). ............................................................................................................ 60 24 Screenshots on PC and portable devices...................................................................... 63 vii LIST OF ABBREVIATIONS AJAX Asynchronous JavaScript and XML CSS Cascading Style Sheets CDN Content Delivery Network GUI Graphical User Interface HLS HTTP Live Streaming HTML Hyper Text Markup Language HTTP Hypertext Transfer Protocol JSON JavaScript Object Notation LabVIEW Laboratory Virtual Instrument Engineering Workbench LtoN LabVIEW to Node.js MIME Multipurpose Internet Mail Extensions MJPEG Motion Joint Photographic Experts Group OS Operation System REST REpresentational State Transfer RTSP Real-time Streaming Protocol RTMP Real-time Message Protocol SMA Shape Memory Alloy SVP Smart Vibration Platform SOAP Simple Object Access Protocol XML Extensible Markup Language URL Uniform Resource Locator UDP User Datagram Protocol viii VITA October 14, 1978 ……………………. Born – Lanzhou, Gansu, China 1998-2002 ……………………. B.S., China Agriculture University Beijing, China 2005-2008 ……………………. M.S., Hong Kong Polytechnic University Hongkong, China 2012-2014 ……………………. M.S., Texas Southern University Houston, Texas, USA Major Field ……………………. Computer Science ix ACKNOWLEDGMENTS In fall 2012, I enrolled in the Computer Science graduate program of Texas Southern University (TSU). Pursuing the graduate education at TSU is such a meaningful journey, and I am so proud of my demonstrated courage, perseverance, ingenuity and inspiration at TSU. I express my sincere appreciation to the following people: my coadvisor, Dr. Xuemin Chen, who is a kind and strict tutor, and always taught me the earnest work attitude and realistic spirit in research through engineering perspective, and supports me so much in my research and life; my advisor, Dr. Wei W. Li, who is a perfect and professional scholar on Computer Science and always provided me guidance and support. I would like to appreciate Dr. Gangbing Song, a professor at University of Houston (UH), and Dr. Hamid Parsaei, a professor at Texas A&M University at Qatar (TAMUQ), who are the Lead PI and Co-lead PI of project “Hands-on Laboratory via Internet-To Develop a Unified Remote Laboratory Framework For Cross Nation Engineering Education ” which I have been working on. They also gave me very strong support in my research. I am also grateful to my colleagues, Ming Bao who is a graduate student at TSU, and Jun Weng who is a graduate student at UH, for the helps they provided in my research. This thesis is based on the research supported by Qatar National Research Fund (Grant No. NPRP 4-892-2-335). Thanks the Qatar National Research Fund support my research and NSF CREST Center for Research on Complex Networks (Grant No. 1137732) for providing me the financial support as a Graduate Research Assistant. x Finally, I especially appreciate my wife's strong support for my study and research, which greatly strengthens my heart and provide me impetus to move forward and become an expert in my research area. xi DESIGN OF THE ADVANCED TECHNOLOGY FOR REMOTE LABORATORY DEVELOPMENT By Ning Wang, B.S. Texas Southern University, 2014 Professor Wei Li, Advisor Associate Professor Xuemin Chen, Co-Advisor Remote laboratory (RLab) is the use of telecommunications to remotely conduct real experiments, and also is an inevitable necessity for Internet enabled education in Science, Technology, Engineering, and Math (STEM) fields due to their effectiveness, flexibility and cost savings. Currently, plenty of remote experiments have been developed based on various methods. However, most of them need to install some external software plug-in components to address different platforms and network firewalls, which negatively impact the utilization of the remote laboratories. To achieve the remote laboratory without the need of any extra plug-in, we designed and implemented a unified remote laboratory framework in our previous works. A Comet solution via Socket.IO which is the package of Node.js is implemented in the server side, and a new web socket protocol which lets the experiment communicate with Socket.IO is created for the workstation. With the Comet solution, the real-time experiment live streaming video is transferred via network port 1026 and the real-time control command and experiment 1 2 data are transferred via network port 1029. Consequently, the network firewall issue still can’t be resolved in the Comet solution. To resolve all of issues found in the previous works and integrate all of improvement for the unified framework, we propose a novel unified framework for the remote laboratory development. The novel unified framework includes three application packages, client web application, server application and experiment control application. The client web application uses the Web 2.0 Technology which includes HyperText Markup Language (HTML), Cascading Style Sheets (CSS), and JQuery/JQuery-Mobile JavaScript libraries. In addition, the Mashup technology for user interface integration is employed. The server application is directly built on the top of MySQL Database, Apache web server engine and Node.js web server engine. The experiment control application uses Web Service technology that is based on the LabVIEW (Laboratory Virtual Instrumentation Engineering Workbench) and runs in workstation. To implement the real-time communication between each application, we use JavaScript Object Notation (JSON) and Socket.IO which was developed based on web socket protocol to implement the communication module. To solve the traversing network firewalls issue, we integrate one core component, Node-HTTP-Proxy, in our current framework to share network port 80. With this solution, we can achieve real-time remote experiment live streaming video transmission and real-time experiment data transmission traversing the network firewalls. With the novel unified framework, we implement two important remote mechanical engineering experiments, remote Shape Memory Alloy (SMA) experiment and remote Smart Vibration Platform (SVP) experiment. 3 With the remote laboratory which is enabled by the novel unified framework, the terminal users can use the real-time remote experiments on the most of popular web browsers without the firewall issues and the need for third party plug-in. This novel unified framework will significantly benefit the remote laboratory development in future. CHAPTER 1 INTRODUCTION 1.1 Background Information technology has had a great impact on education, by providing additional teaching and researching strategies such as online learning, online researching, etc. The 2013 Sloan survey of online learning revealed that the proportion of all students taking at least one online course is at an all-time high of 32 percent. This survey of more than 2500 colleges and universities in United States found that over 6.7 million students were taking at least one online course, an increase of 570,000 students over the previous year (Allen and Seaman, 2013). To provide such online courses, remote access and control laboratory equipment are inevitably necessary. During the last decades, remote laboratories have been introduced into engineering education processes as well as integrated within e-learning frameworks offered to engineering and science students. It is not only personalizing the learning pathways, but also reduces the costs significantly and makes lab experiments available almost at anytime and anywhere (Gomes and Bosgoyan, 2009; Rodriguez-Andina, Gomes and Bogosyan, 2010). Remote laboratory, by definition, is a laboratory which lets the user to conduct experiment remotely through the Internet, and it accessible through the Internet can resolve cost and access constraints as they can be used at flexible time and from various locations (Melkonyan, Gampe, Pontual and Huang, 2014). Compared to traditional hands-on experiments, experiments operated remotely over the Internet offer many advantages. One of these benefits is the capability to handle a large number of students to conduct experiments through scheduling. The remote laboratory allows a 1 2 workaround for complex logistics, such as staff, space, scheduling, budget, and commuting. The remote laboratory equipment can always be made available to students, who may conduct the experiments from any Internet accessible devices. Remotely controlled laboratories could set the stage for shared infrastructure between departments and colleges, as well as among different institutions around the world. The capability to share experiments makes providing experiments significantly more affordable. Moreover, a substantially increased number of experiments could be available for use, including experiments involving expensive equipment, rare materials and in remote locations. Furthermore, in some societies where male and female student access different campuses, such remote laboratories can avoid laboratory facility duplication, offering the same remote experiments for both genders (Khalil et al., 2009). In the last decades, the engineering education community has carried out numerous initiatives to develop and implement remote laboratory activities in engineering education. This trend resulted concomitantly from the availability of innovative software packages for instrumentation and simulation, as well as the necessity to better support active and collaborative learning. In these solutions for the remote laboratory development, there are mainly two kinds of system architecture, Client-Server architecture and Browser-Server architecture. In the early stage of remote laboratory development, Client-Server architecture software system is used for achieving the highperformance real time experiment data transmission (Li, Huai and Bhargava, 1995; Gillet, Salzmann, Longchamp and Telepresence, 1997). One of the popularly deployed technologies at that time for remote panel over the Internet is National Instrument’s LabVIEW (Laboratory Virtual Instrument Engineering Workbench) (N. Duro, R. 3 Dormido, H. Vargas, S. Dormido-Canto 2008). Currently, with Client-Server architecture, some other new integrated learning environments for remote experimentation were developed based on Web services and .NET remote services. These new learning environments were deployed for students in disparate locations to simultaneously and collaboratively complete complex experimental exercises (Callaghan, Harkin, McColgan, McGinnity and Maguire, 2007). With the development of computer technology, the Browser-Server architecture technology is becoming more stable. Especially, the REST (REpresentational State Transfer) provides a lightweight protocol accessible to a wide variety of clients. To adopt the Browser-Server architecture, LabVIEW integrated a new feature to interact with the Virtual Instruments by using the RESTful web services technology. This architecture does not require complex message passing and provides a simple interface for the user to use Web services in LabVIEW. However, it requires the client interface to be developed using different technologies, such as LabVIEW plug-in, Flash plug-in, etc (Olmi, Cao, Chen and Song, 2011). In addition, as the number of remote experiments increases, the software is not capable of handling multiple users with multiple resources. Consequently, a unified framework for remote laboratory experiments development was proposed in order to allow the set up of a distributed network of online experiments that work in any Internet browser without the need of any additional plug-in (Olmi, Chen and Song 2011). Nevertheless, some new challenges still were faced during the unified framework development and deployment. The real-time communication between browser and server, and traversing network firewall are two critical issues we strived to resolve. 4 1.2 Motivation In the remote laboratory research, there already are many teams working in this field. They have made many excellent and significant research achievements. Some projects and achievements in different organizations and universities are listed in following: 1). The LiLa (Library of Labs) project is from the alliance of the European eight universities and three enterprises. The goal of this project is the composition and dissemination of a European infrastructure for mutual exchange of experimental setups and simulations, specifically targeted at undergraduate studies in engineering and science (Richter, Boehringer and Jeschke, 2009). In LiLa project, there are virtual laboratories (simulation environments) and remote laboratories (real laboratories which are remotely controlled via the Internet). For the remote laboratories implementation, the LiLa project mainly through a browser plug-in and gain control over all devices. It is the BrowserServer architecture remote laboratory system with plug-in. 2). The NetLAB project is funded by the School of Electrical and Information Engineering in the Division of Information Technology, Engineering and the Environment, University of South Australia. The main goal of the NetLab project was to create a laboratory experience that would allow students to perform these common experiments on any PC, and have it feel as close as possible to using a real laboratory. NetLAB is designed for learning computer networks and doing research. It is the Client-Server architecture software and uses Netkit/UML emulated networks software package with a GUI for creating and operating network configurations. NetLAB provides an oracle that monitors, diagnoses and explains network behavior. 5 3). The iCampus iLabs project is from the Massachusetts Institute of Technology. The Massachusetts Institute of Technology’s iLab project has developed a distributed software toolkit and middleware service infrastructure to support Internet accessible laboratories and promote their sharing among schools and universities on a worldwide scale. The iLab is the remote laboratory based on Client-Server Architecture. 4). The UTS Remote Labs project is developed by the University of Technology Sydney (UTS). UTS Remote Labs is part of Lab share, an Australian Government funded project that aims to create a national network of shared, remotely accessible laboratories. It is also the Client-Server architecture software, and a remote laboratory consists of experiment apparatus that can be remotely monitored and controlled over the Internet. The students can control equipment located in universities and conduct experiments from the school science classroom. 5). The remote laboratory is developed by the research team in Smart Materials and Structure Laboratory (SMSL) at University of Houston. The goal of this project is to provide users with an interface that will work in most Internet-enabled web browsers without the need to install most of software plug-in (Olmi, Cao, Chen and Song, 2011). They proposed a unified framework for remote laboratory development and it is the Browser-Server architecture software system, and includes three sections: web side, web server, and experiment server. Nevertheless, the real-time experiment video and real-time experiment data transmission need to use the special network port. 6). With the National Science Foundation (NSF) support, the Virtual and Remote Laboratory (VR-Lab) is developed at Texas Southern University (TSU). They collaborate with the SMSL at UH to successfully develop a plug-in free SVP remote experiment. 6 They also develop the Data Communication (DC) virtual and remote experiments and the remote Digital Signal Processing (DSP) experiment. Their remote laboratory is also the Browser-Server architecture software system. To our best knowledge, all remote laboratories above are based on two software architecture paradigms, Client-Server architecture and Browser-Server architecture. The common software architecture is composed as follows: the experiment device itself, a local controlled workstation connected to the device, which plays the role of a gateway between the experiment device and the remote computer of the user, and the associated. In fact, most of devices must be locally handled by a workstation in order to be remotely controlled over the Internet. From all of above illustration, there are still some challenges not to be resolved yet. The challenges in remote laboratory development are listed in Table 1. Table 1 Challenges in remote laboratory development. Problem Challenges 1 How to improve Remote laboratories lack reusability issue? 2 How to improve remote laboratories Localization transparency issue? 3 How to improve remote laboratory system Cross-Platform performance and resolve the Client side software without plug-in issue? 4 How to resolve the real-time experiment data and real-time experiment video traversing network firewall transmission issues? 5 How to improve Substitution of devices issue in remote 7 laboratory? 6 How to improve the remote laboratory system security issue? 1.3 Objective The general objective of our research is to design and implement an innovative advanced remote laboratory technology to resolve some challenges in development of remote laboratory. More specifically, the goals are to: 1) Develop a seamless remote laboratory technology that is platform (PC, MAC, PDA, etc.), web browser (IE, Firefox, Safari, etc.), and operating system (Windows, UNIX, Linux, etc.) independent. 2) Develop a rich, user friendly interface with a leading edge feature named web perspective. 3) Develop an easy-to-use unified remote laboratory publishing tool for remote laboratory developers. 4) Develop a generic scheduler web server for remote experiment farm. 1.4 Research Outcomes With the novel unified framework, some challenges are resolved and improved. Our solutions are listed in the Table 2. 8 Table 2 Research outcomes. Problem Solutions 3 A novel unified framework allows online experiments to be accessed by most of popular Internet browsers without the need of any extra plug-in To improve the real-time experiment data transmission performance, a 3 new web socket protocol, LabVIEW to Node.js (LtoN) which was implemented with the Socket.IO was created in the workstation. A novel unified framework also resolved the real-time experiment data and 4 real-time experiment video traversing network firewall transmission issue through the Node-HTTP-Proxy software module. The new isolated experiment network and the authentication URL were 6 designed to improve the remote laboratory system security issue. 1.5 Outline In chapter 1, the background of remote laboratory is introduced. The research Objective, motivations and outcomes of my research are presented as well. In chapter 2, a novel unified framework for remote laboratory development is presented. In chapter 3, a novel real-time video transmission approach is presented. In chapter 4, a novel real-time experiment data transmission approach is presented. In chapter 5, the implementation of two remote experiments is presented. In chapter 6, conclusion of my research and future works are presented. CHAPTER 2 A NOVEL UNIFIED FRAMEWORK FOR REMOTE LABORATORY DEVELOPMENT In this chapter, a novel improved unified framework for the remote laboratory development is presented. This novel improved unified framework is used to improve the Problem 3, problem 6 and resolve the problem 4 which were mentioned in Chapter 1. The novel unified framework includes three parts: the client web application, the server application and the experiment control application. Socket.IO, a JavaScript library of Node.js, is used to support the real-time web application development. Node-HTTPProxy is used to support the traversing network firewall (Wang and Chen 2013, Wang 2014). To address the real-time experiment video transmission, HTTP Live Streaming (HLS) is used for real-time video transmission across network firewall. 2.1 Previous Works In our previous works, we developed a Comet solution which allows distributed online experiments to work in the most of popular Internet browsers independent of any additional plug-ins. We also resolved some challenge issues of developing cross-browser web user interface (Olmi, Cao, Chen and Song, 2011; Olmi, Chen and Song, 2011). We have used this framework to implement some remote control engineering experiments. For example, the new SVP remote experiment is now used to teach students in mechanical engineering courses at the University of Houston. This remote control experimentation is aimed to offer student hands-on experience through the Internet on structural vibration control by using a Magneto-Rheological (MR) and SMA braces to control the vibration. To improve the experiment data transmission performance of the 9 10 unified framework, we developed a Comet solution via Socket.IO. Socket.IO is a software package of Node.js, and is implemented on the server side. On the experiment control workstation, a new web socket connection is created by Socket.IO to handle the experiment communication between workstation and server (Chen, Osakue, Wang, Parsaei and Song, 2013). Nevertheless, as Figure 1 depicts, the real-time experiment live streaming video is transferred via network port 1026 and the real-time experiment data is transferred via network port 1029 based on this Comet solution. With the Comet solution, users can remotely conduct the experiments by using the most of popular Internet browsers without installing any plug-in via the two special network ports 1026 and 1029. Experiment Control and Data Show Web Page Framework Video Server Node.js Server Port 80 Port 1029 Web Page Port 1026 Video show Web Server Server Experiments Equipment Figure 1 System architecture of Comet solution (Published at the World Congress on Engineering Education 2013). Normally, the special network ports are restricted and blocked by network firewalls in most of universities, institutes and companies based on the security regulations. Only public network port 80 is open for accessing the web contents on web 11 server. Thus, we were motivated to find a new approach to solve the traversing network firewall issue. 2.2 Introduction of a Novel Unified Framework The Novel Unified framework is based on the Web 2.0 technology. The client web application is based on HTML, CSS, and JQuery/JQuery-Mobile JavaScript libraries. The Mashup technology is used for user interface implementation. The client web application can be run in most of current popular browsers such as IE, Firefox, Chrome, Safari etc. The server application is based on Web Service technology, and is directly built on top of MySQL database, Apache web server engine and Node.js web server engine (Hughes-Croucher and Wilson, 2012; Wikipedia, 2013). The server application uses JSON and Socket.IO which is developed based on web socket protocol to implement the real-time communication between the server application and the client-web application (Wikipedia, 2013). The server application runs on Centos 6.0 Linux server. The experiment control application is based on the LabVIEW, and uses Socket.IO for real time communication with server application. The experiment control application runs on experiment control workstation which runs Window 7 operation system (OS). Our new unified framework for remote laboratory development is based on three vital technologies, which are the Socket.IO, Node-HTTP-Proxy, and HLS protocol. NodeHTTP-Proxy is used for experiment data and control commands transmission and traversing firewall and the novel video transmission approach is based on HLS protocol for real-time system monitoring. The Mashup technology is used for user interface implementation. Table 3 depicts the technical characteristics of the novel unified framework in detail. 12 Table 3 Technology/protocol/software list of the new framework. 1 2 3 4 5 6 7 Name HTTP Proxy Data Protocol Real-time experiment video Transmission Technology/Protocol/Software Node-HTTP-Proxy Socket.IO Http Live Streaming Protocol /FFmpeg /Segmenter software package Database MySQL Client – User Interface Mashup technology, JavaScript Server - Web Service Apache , Node.js , JSON Equipment Control LabVIEW Remark Part of Node.js Part of Node.js LtoN (LabVIEW to Node.js) The system architecture of the new unified framework and the data transmission flow of the novel unified framework are shown in Figure 2. 13 SVP Webpage SMA Webpage EEWSN Webpage Server-Based Marshup Technology to generate the user interface Experiment Control and Data Show Video show Client User Interface Network port 80 Network port 80 Node-HTTP-Proxy Network port 80 Video HLS Server Apache Web server engine Network port 5001 Node.js Web server engine Server LabVIEW in Workstation Web Camera Web Camera and Work Station SVP experiment EEWSN Experiment SMA Experiment Experiments Equipments Figure 2 The architecture and data transmission flow of the new framework. 14 The network architecture of the new remote laboratory solution is shown in Figure 3. Internet User Equipments Center Server Internal Experiment Network Network Switch Experiment Workstation Experiment IP Camera Experiment Workstation SMA Experiment Experiment IP Camera SVP Experiment Figure 3 The new remote laboratory solution network architecture. 2.3 Vital Improvement of Novel Unified Framework In this new framework, there are two key improved implementations, i.e., HTTP proxy implementation and real time video processing module implementation. To address 15 the real-time experiment control command and data transmission, we configure and run Node-HTTP-Proxy software application on the server side. The proxy application monitoring the network port 80 looks up the appropriate web server application request from the client and then proxies the data directly from the web server application to the client web browser. To address the real-time experiment live steaming video transmission, we implement a new real-time video transmission solution via HLS protocol combined with FFmpeg, which is a powerful cross platform command line video trans-code / encoding software package. 2.3.1 The Node-HTTP-Proxy for The Network Port Sharing We need to address a complicated problem this is that we cannot set Apache and Node.js to listen on the network port 80 at the same time. In addition, we do not have an option to deactivate Apache and only run Node.js. To solve this issue, we propose to set up Node-HTTP-Proxy to get both Apache and Node.js working together and sharing the same network port 80 on the server side. Node-HTTP-Proxy is a full-featured HTTP proxy which is involved in Node.js server software package, and is a free and open source software package. Currently there are one Apache web application and one Node.js web application which work together in our web server. We use the Apache web application to generate the user interface framework, and the Socket.IO which is a module of the Node.js web engine to transfer the experiment data and experiment equipment control commands. In order to share the network port 80 for these two applications, we used the Node-HTTPProxy to handle the HTTP requests and data transmission. The Node-HTTP-Proxy is used the stream for HTTP requests and data transmission. The concept of a stream in 16 Node.js lends itself beautifully to working with HTTP requests. One of the strengths of Node.js is its capability for streaming data; a great deal of performance can be achieved simply by tunneling the request and response streams to other destinations. 2.3.2 Real-Time Experiment Video Streaming Approach The purpose of the new approach is to transfer the real-time experiment video via port 80. To achieve the goal, we design and implement a new real-time video transmission solution via HLS protocol combined with FFmpeg, which is a powerful cross platform command line video trans-code/ encoding software package. To address the network firewall and flash plug-in issue, our new approach works by breaking the overall real-time experiment video stream into a sequence of small HTTP-based file downloads. Each download consists of one short chunk of an overall potentially unbounded transport stream. As the stream, the client may select from a number of different alternate streams containing the same video content encoded at a variety of data rates, allowing the streaming session to adapt to the available data rate. Briefly, the client (normally the Web browser) loads a playlist file of these video segments which is created following HLS protocol on the Web server (we use the Apache Web Server). The contents of this playlist are short clips (about 10 seconds) residing on a Web server. We need to create and maintain a playlist file and the short segments of the real-time video stream. Therefore, the sequence of small HTTP-based file downloading is transferred via HLS protocol, and we use the FFmpeg software package to break the overall real-time experiment video stream into short download segments (Foresman, 2009). Finally, the real-time experiment video segments are transferred via HLS protocol through network 17 port 80, and the real-time video will be reassembled in the Web browser and shown to end users. 2.4 New Isolated Experiment Network Design for Remote Laboratory To address the security issue of network, we designed and implemented an isolated experiment network at Texas A&M University at Qatar (TAMUQ). All of the experimental equipment and experimental control workstations are put in this isolated experiment network. Our central server connects with the experiment network, and it manages all the communications between experimental control workstations and the users over the Internet. In order to implement the isolated experiment network, the following tasks were done: 1) We designed and developed the DHCP configure and management application on central server, and assigned a group of the private IP addresses (the IP address is from 192.168.0.1 to 192.168.0.100) for every experimental control workstation and every web camera. 2) Every experiment control workstation and IP Camera can automatically obtain their private IP address from the DHCP management application. Every experimental control workstation and web camera can only communicate with other devices in the isolated experiment network. 2.5 New Authentication URL Design for Remote Experiment Webpage We use PHP language to implement an authentication URL function for the experiment operation page. To address the authentication URL issue, we used the MD5 encryption and decryption algorithm in experiment operation page URL generation 18 process. The experiment operation page URL is encrypted in server side using MD5 algorithm, and it is decrypted in browser side. Our design for the authentication URL issue is controlled by a system timer. The remote experiment URL will be invalidated if user fails to open the authentication URL in certain time range. If the experiment operation page cannot be shown correctly, an error message will be shown instead. Currently, the timer is set as 10 seconds. 2.6 Remote Experiment Scheduler and User Management System To manage the remote experiments, we implement a scalable experiment scheduler system and a user management system. We use PHP language to implement the systems, and use MySQL 5.5 database management system to manage the experimental data, user information and other system information. Figure 4 shows the screenshots of the scheduler, the experiment conflict control and the experiment management page. Scheduler Experiment conflict control Experiment management Figure 4 Scheduler, experiment conflict control and experiment management page screenshots. CHAPTER 3 A NOVEL REAL-TIME EXPERIMENT VIDEO TRANSMISSION APPROACH In this chapter, the design and implementation process of a novel real-time experiment video transmission approach are presented. This novel transmission approach is used to resolve the Problem 4 which was mentioned in Chapter 1. The novel solution includes three parts, HLS protocol, FFmpeg and Segmenter software package. FFmpeg is a powerful cross-platform command line video trans-code/encoding software package. Segmenter software package is the cost-free Video streaming segments software package. 3.1 Previous Works In our previous works, we designed and implemented a unified framework for remote laboratory development (Olmi, Cao, Chen and Song, 2011). In this unified framework, we resolved some challenges of developing a cross-browser web user interface for improvement of the unified framework (Olmi, Chen and Song, 2011). With this unified framework, the terminal users can remotely conduct the experiments by using the most of popular Internet browsers without installing any plug-ins. However, we still face the real-time experiment video transmission traversing the network firewall issue. To resolve this essential issue, we need to look for a new approach to transfer the real-time video via network port 80. Meanwhile, all Web contents of the experimentation are also transferred to the terminal users via the same network port. Therefore, the goal of this novel approach is to solve the real-time experiment video transmission across the network firewall issue. 19 20 Nowadays, with the fast development and improvement of network technology, a novel, reliable and free video steaming transmission protocol, HTTP Live Streaming protocol, is more and more popularly used for real-time video transmission across a network firewall. HLS is an HTTP-based media streaming communications draft protocol implemented by Apple Inc. Apple used this protocol on September 1, 2010 to stream their iPod Keynote event live over the Internet, and on October 20, 2010 to stream their ‘Back to the Mac’ Keynote event live over the Internet as part of their QuickTime and iOS software (Foresman, 2009). Currently, there are more and more giant software companies’ solutions to support HLS, such as Adobe Flash Media Server (Adobe FMS), Microsoft IIS Media Server, Google Android Honeycomb, etc. Since it requests to use only standard HTTP transaction, HLS is capable of traversing any firewall or proxy server that lets through standard HTTP traffic. Unlike User Datagram Protocol (UDP) based protocols (such as Real-time Transport Protocol (RTP)), HLS also allows content to be delivered over a widely available Content Delivery Network (CDNs). 3.2 Proposed Approach The basic purpose of the new approach is to improve the remote laboratory video transmission function. To achieve this goal, we proposed the new real-time video transmission solution via HLS protocol in combination with FFmpeg. To address the network firewall and flash plug-in issue, our new approach works by breaking the overall real-time experiment video stream into a sequence of small HTTP-based file downloads. Each download contains one short chunk of an overall potentially unbounded transport stream. As the stream is played, the client may select from a number of different alternate streams containing the same material encoded at a variety of data rates, allowing the 21 streaming session to adapt to the available data rate. Briefly, the client (Web browser) loads a playlist file of these video segments which was created following HLS protocol on the Web server (we proposed Apache Web Server). The content of this playlist are short clips (regularly proposed 10 seconds) residing on a Web server. We need to create and maintain a playlist file and the short segments of the real-time video stream in the server. Consequently, the sequence of small HTTP-based file downloads is transferred via HLS protocol, and we used a FFmpeg software package to break the overall real-time experiment video stream into short segments. Finally, the real-time experiment video segments are transferred via HLS protocol through network port 80 and the real-time video will be reassembled in the Web browser and shown to the end users. Real Time Video Streaming Segment, Enclosure, and transmission Video Source Network Camera FFMPEG Segmenter HTTP Live Streaming Protocol Media Server HTTP Server Real Time Video Streaming collection and Process Internet (HTTP protocol) Terminal Users Figure 5 The working process of the new real-time video transmission approach. 22 Figure 5 illustrates the working process of the novel real-time experiment video transmission approach. It is very clear that we not only need to setup the FFmpeg software package in media server side, but we also need to compile and install the Segmenter software package and configure the Apache Web server in HTTP server side. We build up the real-time experiment video processing and transmission environment, then the new solution will be implemented to resolve the real-time video transmission problem mentioned above. 3.3 Architecture of the Novel Approach The novel real-time experiment video transmission approach has been developed based on the HLS protocol which is a novel, good performance and free video steaming transmission protocol proposed by Apple. As depicted in Figure 6, the novel approach consists of the three basic entities: HLS, FFmpeg and Segment software package. 23 The real time experiment video receiver H.264 codec FFMPEG Network Camera .ts files Segmenter .m3u8 files HTTP Live Streaming HTTP Web browsers in Various Terminal equipment Figure 6 The architecture of the novel real-time video transmission solution. 3.3.1 Streaming Protocol Introduction For the real-time experiment video transmission, we need to select a stable transmission protocol that performs well. Currently, there are three popular streaming transmission protocols which are widely used, Real-time Message Protocol (RTMP), HLS and Real-time Streaming Protocol (RTSP). As depicted in Table 4, we know the detailed correlation between these three main streaming transmission protocols. Based on the requirements, we need to look for a real-time video transmission protocol which is stable, free, and has good capability across a network firewall. Consequently, after comparing, it is the best selection for us that the HLS will be used in our new solution. 24 Table 4 The comparison of main video streaming transmission protocol. Name Upper Protocol Mode Main Force Client request. Video request. Server Request. Real-time Live Request. Play format Request Delay Video Recoder (VCR) Network Address Transiation (NAT) traversal RTMP Real-time Message Protocol HLS HTTP Live Stream RTSP Real-time Streaming Protocol TCP / HTTP HTTP RTP / RTCP C/S Adobe Flash supported Browser HTML5 supported Browser B/S Apple C/S Microsoft HTML5 supported Browser Players FLV, F4V MP4 NA FMS Server/Flash Server/Red 5 Normal HTTP Server RTSP Streaming Server dedicate Encoder Flash Media Encoder a. media encoder(H.264 & AAC) into MPEG-2 streaming b. streaming segmenter from MPEG-2 streaming to segments for live c. file segmenter from file into segments for on-demand related with server .ts media data file, .m3u8 index file related with server & player 5~30 sec < 2 sec Support with high precision Support with high precision Support with normal precision Good Fair Good FLV/F4V file divided into F4f media data file f4x index file Dependent server performance 25 HLS protocol is a way to send video and audio over HTTP protocol from a Web server via network port 80 to client software (Web browsers and other client applications) on desktops, laptops or to other portable devices (including IOS-based portable devices, Android-based portable devices, Window mobile-based portable devices, etc.). Accordingly, the HLS Protocol mainly consists of three parts which are the server component, the distribution component and the client software. The server component is responsible for taking input streams of media and encoding them digitally, encapsulating them in a format suitable for delivery, and then preparing the encapsulated media for distribution. The distribution component consists of standard Web servers and they are responsible for accepting client requests and delivering prepared media and associated resources to the client. For large-scale distribution, edge networks or other content delivery networks can also be used. The client software is responsible for determining the appropriate media to request, downloading those resources, and then reassembling them so that the media can be presented to the terminal users in a continuous stream. The HLS protocol is also an Internet-Draft protocol submitted by Apple, Inc to the Internet Engineering Task Force (IETF), and the Internet-Drafts are working documents of the IETF, its areas, and its working groups (Roger and William May, 2012). 26 Figure 7 HLS basic configuration (Published at HLS protocol draft in IETF 2012 ). Figure 7 shows the HLS protocol standardized basic configuration, which was defined by Apple, Inc (Roger and William May, 2012). As illustrated in Figure 7, there are four main critical parts of the process, which are video and audio source collection, video and audio streaming encoder module, media segment module and video and audio files distribution. In our remote experiment development, the video source is the real-time experiment video, which is input from the network camera directly and is the H264 format video. Currently, we only need to collect the video streaming without audio streaming to show the whole procedure of remote experimentation for the terminal users. As it fulfills our project requirements and is easy to maintain and improve, we prefer to use the FFmpeg software package as the encoder to process the experiment video. Furthermore, FFmpeg is an open source free segment software package to break up the video into the little video clips. To setup the Web server, there are some good Web server 27 software packages, such as Nignx Web server, Apache HTTP server, etc. In our previous unified framework development, we selected the Apache 2.0 Web server software package to setup the web server. Consequently, we implement the new video streaming transmission approach based on the Apache 2.0 Web server as an improvement of the unified framework. 3.3.2 The FFmpeg for the Real-Time Video Encoder Normally, the encoder is used to produce a MPEG transmission stream. For large audio and video producers (such as television, radio, etc.), they usually have hardwarebased encoders that output exactly MPEG transmission stream. As the purpose of our research is to design a new approach which is easily extended and maintained in future, we need to find a good performance and stable software solution. Consequently, an open source software solution will be our objective. With this goal, we prefer to select the FFmpeg as the video encoder in our solution. FFmpeg is a complete, cross-platform command line software package which can record, convert and stream audio and video. It includes libav-codec which is a leading audio/video codec library. FFmpeg is also a leading multimedia framework used to decode, encode, trans-code, Mux, deMux, stream, filter and play video and audio source files which are created and collected by humans and machines. FFmpeg not only can encode the streams in all required Codecs (such as HE-AAC, MP3, MP4, etc.), but also output these elementary streams in MPEG-TS. The FFmpeg software package is free software licensed under the Lesser General Public License (LGPL) or General Public License (GPL) depending on your choice of configuration options and it supports the 28 most obscure, ancient formats up to those that are cutting edge. No matter if they were designed by some standards committee, the community or a corporation. FFmpeg is a very fast video and audio converter that can grab from a live video and audio source (such as a network camera), and also can convert between arbitrary sample rates and resize video on the fly with a high quality poly-phase filter. Normally, the FFmpeg software reads from an arbitrary number of input files which can be regular files, pipes, network streams, grabbing devices, etc., and writes to an arbitrary number of output files which are specified by a plain output filename. In principle, each input file or output file can contain any number of streams as different types (such as video, audio, subtitle, attachment, data, etc.). The streams which were allowed numbers and types can be limited by the container format. Therefore, as a general rule, FFmpeg options are applied to the next specified file. Consequently, when we used the FFmpeg operation commands to process video, it was very important that the FFmpeg operation commands input must be ordered. And you can also have the same option on the command line multiple times. Each occurrence is then applied to the next input or output file. The detailed working process of the FFmpeg software package is shown in Figure 8. Input Files demuxer Encoded data packets decoder Decoded Frames encoder muxer Encoded data packets Output Files Figure 8 FFmpeg trans-coding process. FFmpeg calls upon the “libavformat” library (containing “deMuxers”) to read input files (our input files are the real-time experiment video streaming files) and get 29 packets containing encoded data from these files. When multiple input files are coming, FFmpeg tries to keep them synchronized by tracking the lowest timestamp on any active input stream. Afterwards, Encoded packets are passed to the decoder. The decoder produces uncompressed frames (for example, raw video, PCM audio, etc.) which can be processed further by filtering. After filtering, the frames are passed to the encoder, which encodes them and outputs encoded packets again. Finally, those are passed to the Muxer, which writes the encoded packets to the output file. Example 1 shows the command line which is the FFmpeg commands used in our new video transmission solution to collect the real-time video resource file from the network camera device. FFmpeg -i "rtsp://10.3.52.36/axis-media/media.amp? videocodec =h264&streamprofile =svp" -vcodec libx264 -subq 1 -g 250 -qmin 10 -qmax 51 -i_qfactor 0.71 /var/www/html/Hls_Test /Record. mp4 Example 1 FFmpeg command line. Previously, we used to output real-time video files and constantly decompose, transfer and compose them to achieve the real-time monitor remote experiment process. Therefore, we also developed an automatically executing command file using shell in the Linux server. 3.3.3 The Real-Time Video Segment Module Introduction In the architecture of our new solution, we need to segment the video which is output from the FFmpeg software package to the short clips (regularly proposed 10 30 seconds). Therefore, we need to be looking for one software package for this purpose. Currently, there are some good segment software packages for selection, and the Segmenter is a good candidate for us. The Segmenter is a software package which splits a transmission stream into chunks and then updates a playlist file with these chunk files. Most Segmenters are not open source. Consequently, they are difficult to improve and maintain in the future. As an example, Apple's Segmenter is a good solution and a costfree Segmenter, but it is a binary program made for the Mac and does not run very well on the Linux server. It is not suitable for our new solution, since our approach needs to implement the segment function on the Linux server. By the above description, the Free and Open Source Segmenter solution which was written by Carson McDonald, is the best selection for us up to date. It can run well on a Linux server, and we can freely download the source codes of this solution from his blog (McDonald 2009). The segment software package, which is used in our novel solution, is based on the open source Segmenter solution and we modify some parts to match our requirements for the real-time experiment’s video transmission. Figure 9 shows the detailed working process of the segment software packages in our new solution. 31 Segment Commands input Segment Software Package Index Files and structure Video Streaming resource FFmpeg Software Package Segment software create the relation between Index file with clip files .ts video clip files Figure 9 Segment software package working process. Example 2 shows that the command line which is used to create a stream from a video file created with the above FFmpeg command split into 10 second intervals. Segmenter Record.ts 10 Record Record.m3u8 http://vr-lab. engineeringtech. tsu.edu/ Hls_Test/ Example 2 Segmenter command line. 3.3.4 Web Server Configuration and Video Transmission To transfer the real-time video stream via network port 80, we need to configure the Web server that used the Apache 2.0 HTTP server software package to match the HLS protocol requirement. Actually, Apache is probably not the best choice for delivering files via HLS protocol. However, our previous tasks were already finished on the Apache Web server and there is a low possibility that we can change the Web server for our current project status. Therefore, we decided to go with the very flexible and 32 customizable Apache instead of the Nginx or Lighty Web servers, which are better suited for HLS. At this point, we should have a set of files that represent a stream and this stream definition file. Those files can be uploaded to a Web server. The other important step is to ensure that those files will be downloaded correctly and setup to Multipurpose Internet Mail Extensions (MIME) types code. There are two pieces of MIME type code which are important for the streaming content. They need to be added in the Web server configuration files. Therefore, we need to add the following two examples, example 3 and example 4, MIME type codes in the Apache Server configuration file, httpd.conf, in order to let the Apache Web server support the HLS protocol and files. .m3u8 application/x-mpegURL .ts video/MP2T Example 3 Apache configuration MIME types code AddType application/x-mpegURL .m3u8 AddType video/MP2T .ts Example 4 Apache configuration MIME types code We can create the corresponding folder to manage the m2u8 files on the Web server side. In the client side, we only need to add the HTML code or use JWPlayer.js functions in the Web pages, and then the real-time video will be shown in the web page for end users. It is necessary to point out that the browser must support HTML5, otherwise we need to use the JWPlayer.js to support the HLS video. So far, most of the 33 popular newest browsers (such as, Safari, Chrome, IE10, Firefox, etc.) almost support the HTML5. 3.4 The Novel Real-Time Video Transmission Approach Implementation To achieve the purpose of the real-time experimentation live video streaming with HLS protocol across a network firewall, we create two video files in HTTP server side. Meanwhile, we develop a script program to automatically run the command file, which runs in Linux server side to record the video from the network camera. Finally, with HLS protocol, the real-time experiment video files are automatically decomposed, transferred and composed for the end users. The switch time slot was set to 1 minute, and it means that one video file time slot is 1 minute. Currently, there is an problem which is found in low hardware configuration smart phone. It is that the real-time experiment video can not be continuously playbacked through the low hardware configuration smart phone. When we use the low configuration smart phone to load the video stream, the video playback is not smooth. Anyway, this problem is not found in the IPhone and other high-performance smart phones. CHAPTER 4 A NOVEL REAL-TIME EXPERIMENT DATA TRANSMISSION APPROACH In this chapter, the design and implementation process of a novel real-time experiment data transmission approach are presented. This novel transmission approach also is used to resolve the Problem 4 which was mentioned in Chapter 1. This experiment data transmission approach via HTTP proxy gets both Apache web server application and Node.js with its Socket.IO working together (Wang and Chen 2014). Figure 10 illustrates the architecture of the novel real-time experiment data transmission solution. This novel solution includes parts, HTTP proxy, Node.js and Socket.IO. In additional, we use Node-HTTP-Proxy, which is the good and free software package developed by Node.js development team, to implement the HTTP proxy. Clients Network Port 80 Node-HTTP-Proxy Spec Network Port Spec Network Port Spec Network Port Apache Node.js Node.js Database LabVIEW Server Figure 10 The architecture of the novel real time experiment control commands and data transmission solution. 34 35 4.1 Proposed Novel Approach The purpose of this novel solution is to transfer the experiment control command and experiment data between client and server across network firewall. This novel solution is also a great improvement for our remote laboratory unified framework. To address the network firewall issue, we set up an HTTP proxy using Node-HTTP-Proxy in server-side. It is used to monitor the connections on the public network port 80 in the server-side. The HTTP proxy switches the requests from the client browsers to the proper software applications which run in the server-side, and each application is listening for its own special network port based on the hostname in each request. Now, in our web server, we only configure and run an HTTP proxy application which is built up by Node-HTTPProxy software package to fulfill the requests transferring tasks. This proxy application is monitoring the public network port 80, and looks up the appropriate web server application for a request from the client. The experiment data also is directly switched from the web server applications to the client web browsers. Currently, our web server includes one Apache web server application and one Node.js web server application. We also use the Apache web server application to generate the user interface framework, and use Node.js web server application to handle the experiment data and experiment equipment control commands. Figure 11 illustrates the new data transmission solution working process. As the Figure 11 illustration, it is very clear that the Node.js web server software need to be setup in the server-side. We also need to compile and install the Node-HTTP-Proxy software package and reconfigure the web server. Meanwhile, the web server must 36 include Apache web server and Node.js web server to build up the real time control and experiment data running environment to support our novel data transmission solution. End user Public Port 80 Node-HTTP-Proxy Proxy Port 80 Special Port 5001 Apache Web Server Node.js Web Server Experiment Video and other web content Experiment Control Command and Data Web Server Figure 11 The new data transmission working process. 4.1.1 The Node-HTTP-Proxy Introduction In our previous unified remote laboratory framework, we faced the critical problem. We cannot set Apache web server application and Node.js web server application to listen on the same public network port 80 at the same time. We also cannot find a suitable approach to have the option of deactivating Apache web server application and just run Node.js web server application. To resolve the complicated problem mentioned above, we design and implement a new solution to setup one HTTP proxy by Node-HTTP-Proxy software package. Apache web server application and Node.js web server application can work together well through this novel approach. It also can share the same public network port 80 between these two web server applications in the server-side. 37 Node-HTTP-Proxy is a full-featured HTTP proxy involved in node.js web server software system. It is also the free and open source software package. Node-HTTP-Proxy is designed as middleware concept. The middleware is a simple way to add new features to your application stack. Nevertheless, the problem is that the extremely popular middleware style of web application development is often incompatible. For example, if you use a body decoder middleware, every request will need to be buffered in full before the body can be properly decoded. Consequently, this approach will increase memory usage and reduce the system performance. It is compared to simply relaying unaltered requests and responses to other destinations. All the tasks of the middleware has to be waited for and read into memory, possibly altered, then sent again and adding to both the time and resource cost of each request. The concept of a stream in Node.js lends itself beautifully to working with HTTP requests. One of Node.js strengths is its facility for streaming data. Meanwhile, a great deal of performance can be achieved simply by piping the request and response streams to other destinations and then back again. More importantly, if any subsequent task in the Node-HTTP-Proxy tasks chain tries to listen for 'data' or 'end' events, or otherwise treat the current HTTP connection as a stream, it just won't work. As a buffered request is no longer a stream, it is now just a data object. Consequently, it is why the Node-HTTPProxy was designed, and it works well with excellent performance. 38 WebServer Application (Apache) WebServer Application (Node.js) WebServer Application (Nginx) Node-Http-Proxy HTTP/Socket.IO Client HTTP/Socket.IO HTTP/Socket.IO Client Client Node-HTTP-Proxy Deployment Principle Figure 12 Node-HTTP-proxy deployment principle. Figure 12 illustrates the deployment principle of the Node-HTTP-Proxy. In the deployment principle, it is very clear that the Node-HTTP-Proxy is working as an agent to transfer the request and data between the clients and web server applications through the public network port 80. Node-HTTP-Proxy is also an excellent HTTP proxy for the web sockets protocol. Therefore, you can let your applications run independently on different special network ports while serving everything to the end user over public network port 80, such as we use software like Socket.IO for our data transmission solution. 39 var HTTP = require('HTTP'), HTTPProxy = require('HTTP-proxy'); HTTPProxy.createServer({ hostnameOnly: true, router: { //web-development.cc 'www.my-domain.com': '127.0.0.1:80', 'www.my-other-domain.com': '127.0.0.1:5001' } }).listen(80); Example 5 Node-HTTP-Proxy code line. As Example 5 illustration, the code segment is using Node.js with Node-HTTPProxy on the public network port 80. With this example code, it creates the basic HTTP proxy server application and listens for the public network port 80. It will switch any requests from the clients to two server applications which separately listen for network port 80 and network port 5001. 4.1.2 The Server-Side Software System Introduction Nowadays, there are three most popular web server software engines, Apache, Microsoft IIS and Nginx. However, we must select a stable server-side software engine which has the excellent performance to support the real time communication web application development. Based on this purpose, the Node.js web server engine comes in our view with its excellent performance to support the real time communication web 40 application development. The biggest distinction with Apache web server software engine is that Node.js is an incredibly fast and efficient server software system that scales well. If you have configured Apache's 'HTTPd.conf' file, you have no doubt seen the Start Servers and Min-Spare Servers settings which keep a specified number of idle Apache servers running in order to bypass the time required to start servers for new connections. In contrast, Node.js tells the operating system (through e-poll, k-queue, /dev/poll, or select) that it should be notified when a new connection is made, and then it goes to sleep. If any clients new connect, then it executes the callback. Each connection is only a small heap allocation. Figure 13 illustrates the Node.js software system architecture. JavaScript Node Standard Library Node Binding Google V8 JavaScript Engine Thread Pool Event Loop Crypto DNS (Open SSL) C / C++ Figure 13 Node.js architecture. Node.js is a server-side software system designed to develop scalable Internet applications and notably setup web servers. Node.js is a packaged compilation of Google's V8 JavaScript engine, which include the “libuv” platform abstraction layer and a core library which is primarily written in JavaScript. Meanwhile, node.js is using event driven operation mode, asynchronous I/O port to minimize overhead and 41 maximize scalability. The original goal of Node.js is to create web sites with push capabilities as seen in web applications like Gmail. Nevertheless unlike most other JavaScript programs, node.js is not executed in a web browser and it instead as a serverside JavaScript application. In the server-side, Node.js not only implements some Common JavaScript specifications, but also provides a Read-eval-print loop (REPL) environment for interactive testing. Node.js is a JavaScript environment running in Google’s V8 JavaScript engine. As depicted in Figure 13, Node.js only exposes non blocking asynchronous interfaces to the developer. It also has very few abstractions. The power of Node.js lies in the fact that it moves the developer away from certain interfaces like synchronous I/O. Each node.js application is a single thread. If there are more tasks need to be done, the multiple node.js instances must be started. At the same time, the kernel will do the load balancing, and the memory isolation is enforced at the process boundary. Node.js uses the module architecture to simplify the creation of complex applications. Modules are akin to libraries in C language, or units in Pascal language. Each module contains a set of functions related to the 'subject' of the module. For example, the Node-HTTP-Proxy module contains functions specific to HTTP Proxy. Node.js provides some core modules out of the box to help you access files on the file system, create HTTP Proxy and Socket.IO, and perform other useful functions. As Example 6 illustration, it is very simple to use the 'require ( )' function to include the modules in your program. Anyway, Node.js is a promising technology and an excellent choice for a high load application. It already has been proven by corporations, like Microsoft, eBay, and Yahoo. 42 var comunicate_ io = require('socket.io'); var HTTPProxy = require('HTTP-proxy'); Example 6 Node.js example code in server. 4.1.3 The Real-Time Web Application Introduction To support the real time communication web application development, normally, the web socket protocol is used to implement this requirement. Web socket protocol is most often used to describe what it makes possible is the “Real-time Web”. Basically web socket protocol is breaking the limitations of the request/response protocol of HTTP. Instead, with event-handling implemented at each side, it creates a socket connection between your browsers and the back-end. As the events cause something to be happened in the end users’ browsers, it is possible to be triggered at the back-end without the browsers having to see whether anything have been happened at the back-end. When a message is sent via the web socket protocol connection to the browsers, the message triggers a pre-defined event. At the same time, something immediately happens in the browsers. Web Socket protocol is bi-directional, so events which occur in the browser can generate some messages. Those messages are sent via the web socket protocol connection to the back-end. Meanwhile, they are effectively providing an alternative to the use of AJAX over HTTP protocol. As above mentioned, we have to build our real time communication application based on a web socket protocol. In the interest of adapting our server-side architecture where we use Node.js web server software system, 43 we decide to use Socket.IO. Socket.IO primarily uses the web socket protocol to connect the client web pages with web server. Socket.IO not only primarily uses Web Socket protocol, but also is supported and relied on some other software packages. If there are the requirements from the users, it can fall back on multiple other methods, such as Adobe Flash sockets, JavaScript Object Notation with Padding (JSON-P) polling, and AJAX long polling, etc, while continuing to provide the same interface. Although it can be used as simply a wrapper for the web socket protocol, it provides many more features including broadcasting to multiple sockets, storing data associated with each client, and asynchronous I/O. Figure 14 illustrates the Socket.IO working process. Client Socket.IO handles communication with the browser Node.js Http Server Socket.IO Data Stream The data provider Data Source Figure 14 Socket.IO working process. Socket.IO enhances the web socket protocol by providing built-in multiplexing, horizontal scalability, automatic JSON encoding/decoding, and more. Socket.IO also uses long polling technology. Long polling technology addresses the weakness of traditional polling methods by asking the server for new information on a certain interval, 44 nevertheless keeping the connection open if the server has nothing new for us. This technique dramatically decreases latency and network traffic, which means that it efficiently disguises itself as a server-push technique. Socket.IO is a JavaScript library and aims to support real time web applications possible in every browser and mobile device via blurring the differences between the different transmission mechanisms. Socket.IO mainly includes two parts: a clientside library that runs in the browser and a server-side library for Node.js. Both components have a nearly identical API. Similar with the Node.js, it also uses eventdriven operational principle. The Example 7 and Example 8 illustrate the example codes of these two parts. var io = require('socket.io').listen(80); io.sockets.on('connection', function (socket) { socket.emit('news', { hello: 'world' }); socket.on('my other event', function (data) { console.log(data); }); }); Example 7 Socket.IO example code in server. 45 <script src="/socket.io/socket.io.js"></script> <script> var socket = io.connect('HTTP://localhost'); socket.on('news', function (data) { console.log(data); socket.emit('my other event', { my: 'data' }); }); </script> Example 8 Socket.IO example code in client. 4.2 Approach Implementation 4.2.1 Node-HTTP-Proxy Configuration In the interest of the real time experiment control command and experiment data transmission via public network port 80, we need to set up and configure the new architecture in our web server to get the Apache 2.0 web server application working well with Node.js V0.10.20 web server application together. And we also need to configure the new architecture to match the Node-HTTP-Proxy requirement. To achieve this goal, we must finish the following tasks. At first, The Express software package and Node-HTTP-Proxy software package have to be installed in the server-side. Accordingly, the virtual hosts which like normal on Apache web server software system need to be set up. Secondly, all virtual hosts will be placed in a common directory (for example: "/localhost"). Then the Apache web server software application is 46 listening for the public network port 80. It only has to be configured the port in file "httpd.conf" (such as changed "Listen 80" to "Listen other special port"). At last, all virtual hosts must be made sure, as defined in "extra/HTTPd-vhosts.conf" were set to an IP based name Virtual Host (127.0.0.1) instead of using a port (such as HTTP://vrlab.engineeringtech.tsu.edu:80). On the Node.js web server software system, the application/server (it is the node.js virtual host) that listened for the special network port (such as port 5001) which is different with Apache port (somewhat arbitrarily choice of port number) is created. Then in the "/localhost" directory, a file called "nodeHTTPProxy.js" is created. As shown in Example 9, the example code of the NodeHTTP-Proxy is implemented in our server. With Node-HTTP-Proxy, a proxy server case is also created in nodeHTTPProxy.js to be listening for the public network port 80. / Module dependancies var HTTPProxy = require('/usr/local/lib/node_modules/HTTPproxy/lib/node-HTTP-proxy') , express = require('/usr/local/lib/node_modules/express/lib/express'); // HTTP proxy-server HTTPProxy.createServer(function (req, res, proxy) { // Array of node host names var nodeVhosts = [ 'vhost1' , 'vhost2' ] , host = req.header('host') , port = nodeVhosts.indexOf(host) > -1 ? 80 : 5001; // Now proxy the request proxy.proxyRequest(req, res, { host: host , port: port }); }) .listen(80); Example 9 NodeHTTPProxy.js example code. 47 After all of the issues mentioned above are solved, nodeHTTPProxy.js needs to be run. Each of the node.js applications has a map file that contains the port which is listened on by the application, as well as a map that indicates the expected path which the application is being served on. 4.2.2 LabVIEW to Node.js via Socket.IO To implement the real time communication between client application and server application, we design and implement the new application transmission protocol based on Socket.IO software package. This new application communication protocol includes two parts, client part runs in browsers and server part runs in web server. They are developed by JavaScript language and also enhance the web socket protocol. In this new application transmission protocol, we define our own special communication instruction set to implement real time experiment control commands and experiment data transmission. In our new protocol, with using special short instruction to control experiment process, the experiment data transmission performance is improved. To implement the new protocol, we only need to create two JavaScript program files, one for client application runs in web browsers and the other for server application runs in web server. Meanwhile, we have to create the new node.js task to run the protocol in server-side. The server-side application must hold running status forever. Because the server-side protocol application can hold the active status, it can ensure the normal real time communication. In client side, the communication instruction functions set file is used to support the client application normal operation in web browsers. 48 After all of the issues mentioned above are solved, we need to run the server-side protocol file using ‘forever start’ command of node.js in server-side. The server-side protocol application will be the active status, and the real-time communication can be started. CHAPTER 5 THE IMPLEMENTATION OF TWO REMOTE EXPERIMENTS In this chapter, two new remote experiments based on the novel unified framework are presented. Two new remote experiments are successfully implemented under this novel unified framework, and their user interface can be run on most of popular web browsers without installing any additional software plug-in. Some advantages of two new remote experiments also are discussed in this chapter. The capability of running remote experiments on portable device let users gain insights by observing and interacting with the real instrument in a convenient way. 5.1 The Implementation of the Two Remote Experiments 5.1.1 The Remote SMA Experiment The SMA experiment is developed to demonstrate and analyze the characteristics of Shape Memory Alloys (Wang and Weng 2014). It is used to study the hysteresis behavior of the wire actuator and how the driving frequency changes the hysteresis loop. The experiment can apply sinusoidal driving signals (AC) of 16.4 V in magnitude and respectively from 1/30 Hz to 1/50 Hz in frequency to the SMA actuator. Students can observe the wire movement through the webcam in the remote laboratory. They can record a number of cycles such as five for each test. Saved data include three types of data: the applied voltage value, the displacement of the wire actuator, and time in seconds. These data will be used for future analysis. SMA materials are metallic alloys which have the Shape Memory Effect (SME) of being able to return to a pre-determined, or “trained,” shape from a deformed state by 49 50 applying heat. A number of types of shape memory alloys are known to exhibit the Shape Memory Effect when heated, including gold-cadmium and nickel-titanium, or national. The SME follows from the changes in atomic crystal structure that the alloy undertakes at different temperatures. At low temperatures, the material behaves like a normal, malleable metal alloy. However, the SME takes place when the alloy is heated above the temperature which causes the crystal structure of the atoms to change, known as the transformation temperature. As the atomic crystal structure changes, so does the overall shape of the material, reverting to a pre-determined shape created by training the material above the transformation temperature over time. Due to the training, the overall austenite form is maintained through cooling and deformation. The process of deformation and transformation for an SMA spring is illustrated in Figure 15. Figure 15 Macro and micro views of SMA spring deformation and transformation. 51 The experiment consists of an SMA wire connected to the electronic circuitry that controls the voltage going to SMA wire. A displacement sensor is also attached to the SMA wire to sense the motion caused by the SMA during heating and cooling through the applied electric voltage. Remote SMA Experiment is defined as real experiments conducted by Internet users. This experiment use real instrumentation and components at a different location than where they are being controlled by the user. As shown in Figure 16, the SMA device is assembled by using both fabricated and off-the-shelf components. The main frame uses L shape aluminum bar fixed by screws. There are plexi-glass sheets fixed on several sides of the SMA to protect and decorate the experiment. On the bottom, blue LED lights are used to light up the experiment. In the middle of SMA device, a red block is lifted by SMA wire and pulled down by two springs. The experiment consists of a workstation, NI DAQ 6008 USB, and a DC power supply, a Solid State Relay (SSR), a web camera, and SMA wires. A PC workstation runs LabVIEW to control the voltage applied on the SMA wire. NI DAQ 6008 USB is a data acquisition device. It generates voltage output based on the signal from LabVIEW and senses voltage changes in the experiment to control the amount of voltage to SMA wire. The DC power supply is used to generate a consistent voltage on SMA. Pulse-width Modulation (PWM) is used to control the voltage by using a SSR. The SSR can switch on and off swiftly to have a voltage value defined by the user even using a DC power supply. The camera is used to view the experiment change remotely. By applying voltage on SMA wire, the temperature of SMA will increase and the length of the wire will decrease. 52 Figure 16 SMA experiment device. In order to implement the new remote SMA experiment, there are three tasks to be done, the user interface development, system integration in server side and integrate LtoN module to the new SMA experiment. 5.1.2 The Remote SVP Experiment The design of the SVP experiment is shown in the schematic Figure 18. The mechanical frame depicted in Figure 18 can be viewed as a simplified cable stay bridge. The later is composed by one or more piers that tightly hold the cables that suspend the bridge upper deck. The piers are strong and subject to compression forces, which translates into high stiffness, while the cables holding the desk are less rigid and subject to tension forces, which translates into lower stiffness compared to the piers. Thus from Figure 18, the strong bridge piers are provided by the two thicker vertical bottom walls, while the more flexible upper walls simulate the low stiffness of the bridge upper deck. The external sinusoidal force provided by the imbalance mass motor simulate an external force capable of inducing vibrations on the bridge, such as: wind, rain, etc. 53 Bridge Deck SMA MR Damper Bridge Columns Figure 17 The diagram of bridge experiment. A steel tongue attached to the upper platform dips into a small MR fluid reservoir damper which is mounted to the rigid base. An electromagnet is placed around the MR fluid reservoir damper such that the terminating ends of the electromagnet “clamp” the MR reservoir damper. When the electro-magnet creates a magnetic field through the MR fluid damper the MR fluid increases in viscosity, making it difficult for the tongue to move through, damping the vibration of the structure. Alternately, stretching diagonally across the steel frame are two sets of SMA wire braces, forming an “X” shape inside of the steel frame. On the ends of each set of SMA braces, alligator clips are attached to allow current to flow through the wires and induce electrical heating. As the wires are heated, they contract, stiffening the structure, illustrating the effects of stiffness on the damping ratio of the overall structure. The electrical wires providing power to the electro-magnet, the SMA wire braces, and the rotary motor lead down to a power supply encased in a plastic sub-base which contains the circuitry for the piece, including a microcontroller allowing for computer 54 controlled demonstration, analogy input/output ports for demonstration and feed-back control with a PC link, manual control buttons, and an LCD user display. Two accelerometer sensors mounted on the top and middle bars allow the user to monitor the system vibration. The bridge experiment device is shown in Figure 19. Figure 18 SVP experiment device In order to implement the new remote SVP experiment, there are three tasks to be done, the user interface development, system integration in server side and integrate LtoN module to the new SVP experiment. 5.1.3 The User Interface Development Table 5 depicts the technology, protocol and software used in the client web application development. 55 Table 5 Technology/protocol/software list of the SMA client web application. Name Technology/Protocol/Software Remark 1 Development Language HTML , CSS, JavaScript Using HTML5 2 Socket.IO Part of Node.js 3 Real-Time communication Protocol Integration technology 4 Widgets JQuery/ JQuery-Mobile Mashup technology In addition, the Mashup technology for user interface integration is employed in our client application. We use server-based Mashup technology to generate our client application. Server-based Mashup analyzes and reformats the data on a remote server and transmits the data to the user's browser. Figure 19 and Figure 20 depict three sample paradigms of the client application which shows three different user interfaces for two remote experiment. The architecture of our Mashup scheme is divided into three layers: 1) Presentation / user interaction: this is the user interface of Mashup. Our novel design uses HTML, CSS, JavaScript and Asynchronous JavaScript. 2) Web Services: the system functionality can be accessed using the API services. Our novel design used JSON-RPC, REST and SOAP. 3) Data: the data is handled in three ways, i.e., sending, storing and receiving. Our novel design uses JSON and Socket.IO for data transmission. 56 User Control interface Experiment management interface Record Experiment data Figure 19 Sample paradigms of the new SMA experiment user interface. User Control interface Experiment Conduct interface Record Experiment data Figure 20 Sample paradigms of the new SVP experiment user interface. 5.1.4 Server Side Integration The server application is directly built on the top of a novel assembled server engine scheme, and it includes two server engines working together, Apache HTTP server engine and Node.js server engine. Based on the server-based Mashup technology used in the novel framework, the Apache HTTP server application integrates the user interface widgets with web content and the real-time experiment video, and Node.js server application handles the real time experiment data transmission. Table 6 depicts the technology, protocol and software used in the server application implementation. 57 Table 6 Technology/protocol/software list of the SMA server application. 1 2 3 4 5 Name Server Engine Technology/Protocol/Software Apache, Node.js Real-Time experiment data Protocol Database HTTP Proxy Real-Time video transmission Socket.IO MySQL Node-HTTP-Proxy Http Live Streaming Protocol /FFmpeg /Segmenter software package Remark Using the newest stable version Part of Node.js Part of Node.js We implemented the experiment scheduler system and user management system on the server-side. For the data management, we use the MySQL Database management system. 5.1.5 Integrate LtoN Module to Two New Remote Experiment We design and dimplement a new real time experiment data transmission protocol based on Socket.IO and JSON. It is named LtoN. Using the new real time transmission protocol, an end user can conduct the experiment, save the experiment data, and load the experiment data file. More details of this new protocol are introduced in the following: 1) The new application communication protocol includes two parts, client part running in browsers and server part running in web server. They are developed by JavaScript language and enhanced by the web socket protocol. 2) In this new application transmission protocol, we defined our own special communication instruction set to implement real time experiment control commands and experiment data transmission. With the new communication 58 instruction set, we can secure the data communication when user is conducting the remote experiment. 3) In our new protocol, we designed some brief instruction to control experiment progress to improve the experiment data transmission performance. For the protocol implementation, we only need to create two JavaScript files, one for client application running in web browsers and the other one for the server application running in web server. Then we need to create the new Node.js task to run the protocol in server-side, and this server-side application must hold running status forever. Because only the server-side protocol application holds the status active, it can ensure the normal real time communication. On the client side, there is a set file of communication instruction functions to support the normal operation of the client application in web browsers. 5.2 Experiment Data Analysis The function of SMA experiment has been tested remotely by using the novel plug-in free remote laboratory at TAMUQ. The test performed a randomly chosen arbitrary values, 1.205 in, as the reference in displacement control. The reference switched between the arbitrary value and zero which was manually control in the GUI. The data containing the real displacement were recorded and were directly downloaded from the GUI. In each test, the GUI was recording the data for 30 seconds. Downloading of the data was available in the same GUI. The downloaded data for the real displacement and the displacement reference in each test was plotted in the same figure by using MATLAB. Figure 21 depicts the results of the displacement change of SMA in the test from plug-in free remote laboratory. 59 In the Figure 22, DC voltage of 15 volts in square waveform with frequency, 0.01 Hz, is applied on SMA wire and the displacement change of SMA are plotted in the figure. At lower frequencies the SMA wire has more time to heat up, the displacement change is larger at lower frequency. The hysteresis of SMA is affected by the frequencies of driving voltage. More demonstrations are performed using AC voltage applied on SMA wire. In the Figure 23, the displacement change of SMA wire is plotted in voltage series. The 15 volts is in sinusoidal waveform at frequencies and 0.025 Hz is applied on SMA wire. Similar to the results in DC voltage trials, AC voltage at lower frequencies produces more displacement change on SMA wire when the amplitude of voltage keeps the same. Figure 21 Arbitrary displacement control (with amplitude of 1.205 in) in remote SMA experiment. 60 Figure 22 DC voltage of 15 V in square waveform applied on SMA wire with frequency of 0.01 Hz. Figure 23 AC voltage of 15 V in sinusoidal waveform applied on SMA wire with frequency of 0.025 Hz (displacement change in voltage series). 61 5.3 Advantages of New Remote Experiments based on the Novel Unified Framework As Table 7 depicts, it lists the development technology, experiment data transaction protocol and software package which are used in two different remote experiments implementation process. The new remote experiments based on our novel unified framework makes some improvements in traversing network firewall function and software plug-in free. In order to provide a more realistic experience to the user for conducting the remote experiments, the new solution also implements the real-time experiment video function. Table 7 Technology/protocol/software comparison between two solutions Function Name 1 2 Traversing network firewall Data protocol 3 Real-time experiment video 4 5 Database Client – User Interface Software plug-in 6 7 8 Server - web service Equipment control New Remote Experiments with Remote Experiments with the Novel Unified Framework LabVIEW Node-HTTP-Proxy None LtoN (LabVIEW to Node.js) data transaction protocol HTTP Live Streaming Protocol /FFmpeg /Segmenter software package MySQL Mashup technology, JavaScript NI Publish-Subscribe Protocol (NI-PSP) None None Apache , Node.js , JSON LabVIEW run-time engine and browser plug-in LabVIEW web service LabVIEW LabVIEW MySQL LabVIEW remote panel Comparing to the traditional remote experiments with the LabVIEW remote panel, as the Table 8 depicts, the new SMA remote experiment solution has three advantages. The user interfaces of the new remote experiments are plug-in free and running in different web browsers, and the new solution also resolves the traversing network 62 firewall issue. End users only need to access the Internet and use a web browser to operate the remote experiment. The experiment control webpage is developed by JavaScript which is a universal and regular computer language. Meanwhile, any regular web browsers are good to use all the features of the remote panel without requiring any more extra software plug-ins. In addition, the new remote experiments provide the realtime experiment video function. It also provides a more realistic experience to the user for conducting the remote experiments. As Figure 24 depicts, the new SMA remote experiment user interface can also be used through smart phones and tablet computer. Table 8 The advantages of the new remote SMA experiment. Advantage Function Name 1 2 3 Traversing network firewall Real-time experiment video Software plug-in New Remote SMA Experiment Remote SMA Experiment with the Novel Unified with LabVIEW Framework Node-HTTP-Proxy None Http Live Streaming Protocol /FFmpeg /Segmenter software package None None LabVIEW run-time engine and browser plug-in 63 Sample Paradigm on Desktop Sample Paradigm on iPhone Sample Paradigm on iPad Figure 24 Screenshots on PC and portable devices. CHAPTER 6 CONCLUSION AND FUTURE WORKS 6.1 Conclusion In this thesis, a novel unified framework for remote laboratory development was presented. We solved and improved three problems which were mentioned in chapter 1. To improve problem 3, a novel unified framework was proposed for implementing the remote laboratory without the need of any extra plug-in. The user interface can be run on most of current popular Internet browsers in any platform without installing any additional plug-in. The capability of running remote experiment on portable device let users gain insights by observing and interacting with the real instrument in an efficient way. To resolve problem 4, two innovative solutions were proposed for remote laboratory development to solve the real-time experiment video and real-time experiment data transmission across network firewall issues. To improve problem 6, the new isolated experiment network and the authentication URL were designed to improve the remote laboratory system security issue. These solutions were implemented in remote laboratory at TAMUQ. Finally, two new remote experiments, remote SVP experiment and remote SMA experiment, were successfully implemented under this novel unified framework. This novel unified framework gives the future remote laboratory development a prodigious improvement. 64 65 6.2 Future Works This thesis presented the design and implementation approach of a novel unified framework for developing remote laboratory experiments. Meanwhile, it also illustrated the differences between the traditional remote experiment solution based on LabVIEW remote panel and the new remote experiment solution based on the novel unified framework. While the new remote experiment solution compared with the previous traditional solutions has been significantly improved, there are still more works which must be done to improve the remote laboratory system stability and security. The future works are listed below: 1) Based on the current solution of problem 3, the Microsoft IE browser cannot support MJPEG real-time video format very well until now. To solve the video display problem in Microsoft IE browser, we still have to install the Java applet plug-in for the video display. With the software technology development, I believe this problem can be resolved in near future. Anyway, our solution is working very well in the most of popular browsers, such as, Chrome, Firefox, Safari, etc. 2) Based on the current solution of problem 6, we used encrypted URL with the MD5 encryption and decryption algorithm to resolve authentication URL issue, and used 10 seconds timer to control authentication URL effectiveness. But its stability is still not very good, and need to be improved in future work. REFERENCES Allen, E., and Seaman, J. (2013). “Changing Course Ten Years of Tracking Online Education in the United States”, 2013. The Sloan Consortium. Gomes, L. and Bosgoyan, S. (2009) "Current trends in remote laboratories", IEEE. Trans. on Industrial Electronics, Vol 56, NO 12, pp. 4744-4756, December, ISSN: 0278-0046. Rodriguez-Andina, J., Gomes, L. and Bogosyan, S. (2010) "Current trends in industrial electronics education", IEEE. Trans. on Industrial Electronics, Vol 57, Nº 10, pp. 3242-3244, October, ISSN: 0278-0046. Melkonyan, A., Gampe, A., Pontual, M. and Huang, G. (2014). "Facilitating Remote Laboratory Deployments Using a Relay Gateway Server Architecture", IEEE. Trans. on Industrial Electronics, Vol 61(1), pp. 477-485, Jan, ISSN: 0278-0046. Li, S., Huai, J., and Bhargava, B. (1995). "Building High Performance Communication Services for Digital Libraries", Computer Science Technical Reports Paper 1212. Gillet, D., Salzmann, C., Longchamp, R. and Telepresence, D. B. (1997). “An Opportunity to Develop Real-World Experimentation in Education”, European Control Conference, Brussels, Belgium, Session WE-M-L 1, July 1-4. Callaghan, M.J., Harkin, J., McColgan, E., McGinnity, T.M. and Maguire, L.P. (2007). “Client–server architecture for collaborative remote experimentation” Journal of Networks and Computer applications, ISSN: 1084-8045, Volume 30,Issue 4, pages 1295-1308, November. Richter, T., Boehringer, D. and Jeschke, S. (2009). “LiLa: A European Project on Networked Experiments” Automation, Communication and Cybernetics In Science and Engineering , Part 2, 307-317, DOI: 10.1007/978-3-642-16208-4_27 Olmi, C., Cao, B., Chen, X. and Song, G. (2011). “A Unified Framework for Remote Laboratory Experiments”, In Proceedings of ASEE Annual Conference & Exposition (June 26 - 29), Vancouver, BC, Canada. Omli, C., Chen, X. and Song, G. (2011). “A Framework for Developing Scalable Remote Experiment Laboratory”, In Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2011, pp. 2045-2050, Honolulu, Hawaii. Chen, X., Osakue, D., Wang, N., Parsaei, H. and Song, G. (2013). “Development of a Remote Experiment under a Unified Remote Laboratory Framework,” in 66 67 Proceedings of the World Congress on Engineering Education 2013. H.R. Parsaei and K.S. Warraich, eds. Olmi, C., Song, G. and Mo, Y. L. (2007). “An innovative and multi-functional smart vibration platform.” Smart Materials and Structures, 16, 1302–1309. Hughes-Croucher, T. and Wilson, M.,(2012). “Up and Running with Node.js (First ed.)”, O'Reilly Media(April), p. 204, ISBN 978-1-4493-9858-3. Wikipedia, (2013). “Node.js – Wikipedia, the free encyclopedia”, Online, http:// en.wikipedia.org / wiki/ Node.js, accessed May 1. Wikipedia, (2013). “Socket.IO – Wikipedia, the free encyclopedia”, Online,http:// en.wikipedia.org / wiki/Socket.IO, accessed May 1. Wang, N., Chen, X., Song, G. and Parsaei, H., (2013). “A New Framework for Remote Laboratory Development,” IERI Journals: Lecture Notes in Information Technology 39, 205-211. Wang, N., Chen, X., Song, G. and Parsaei, H., (2014). “Remote experiment development using an improved unified framework,” In E-Learn: World Conference on ELearning in Corporate, Government, Healthcare, and Higher Education (pp. 20032010). Association for the Advancement of Computing in Education (AACE). 2014, October. Wang, N., Weng, J., Ho, M., Chen, X., Song, G. and Parsaei, H., (2014). “Development Of A Remote SMA Experiment-A Case Study,” In Qatar Foundation Annual Research Conference (No. 1, p. ITPP0944). Wang, N., Chen, X., Ho, M., Parsaei, H. and Song, G., (2014). “A Novel Solution For Addressing Network Firewall Issues In Remote Laboratory Development,” In Qatar Foundation Annual Research Conference (No. 1, p. ITPP0036). Foresman, C., (2009). “Apple proposes HTTP streaming feature as protocol standard”, Ars Technica, Retrieved 2009-07-10, in IETF. Roger, P. (Apple Inc.), William May, Jr. (Apple Inc.) (2012). "the IETF Internet-Draft of the HTTP Live Streaming specification." Internet Engineering Task Force, Online, Retrieved October. McDonald, C. (2009). “An open source segmenter– written by Carson McDonald” Subversion (SVN) online, Submitted: June 28. View publication stats

RELATED PAPERS

RELATED TOPICS

Log In

Design and Implementation of the Advanced Technology for Remote Laboratory

Design and Implementation of the Advanced Technology for Remote Laboratory

Related Papers

RELATED PAPERS

RELATED TOPICS