9th International Conference "Distributed Computing and Grid Technologies in Science and Education", 2021
There is no single diagnostic marker for neurodegenerative diseases. The biomedical data obtained... more There is no single diagnostic marker for neurodegenerative diseases. The biomedical data obtained during these studies have heterogeneous nature, which greatly complicates their collection, storage and complex analysis. Special methods of statistical analysis due to the described specifics of the data must be applied. The results obtained indicate that for a correct diagnosis, it is necessary to use a comprehensive assessment of all tests.
Computational Science and Its Applications – ICCSA 2020, 2020
Diffraction and radiation forces result from the interaction between the ship hull and the moving... more Diffraction and radiation forces result from the interaction between the ship hull and the moving fluid. These forces are typically simulated using added masses, a method that uses mass to compensate for not computing these forces directly. In this paper we propose simple mathematical model to compute diffraction force. The model is based on Lagrangian description of the flow and uses law of reflection to include diffraction term in the solution. The solution satisfies continuity equation and equation of motion, but is restricted to the boundary of the ship hull. The solution was implemented in velocity potential solver in Virtual testbed—a programme for workstations that simulates ship motions in extreme conditions. Performance benchmarks of the solver showed that it is particularly efficient on graphical accelerators.
Computational Science and Its Applications – ICCSA 2019, 2019
Virtual testbed is a computer programme that simulates ocean waves, ship motions and compartment ... more Virtual testbed is a computer programme that simulates ocean waves, ship motions and compartment flooding. One feature of this programme is that it visualises physical phenomena frame by frame as the simulation progresses. The aim of the studies reported here was to assess how much performance can be gained using graphical accelerators compared to ordinary processors when repeating the same computations in a loop. We rewrote programme’s hot spots in OpenCL to able to execute them on a graphical accelerator and benchmarked their performance with a number of real-world ship models. The analysis of the results showed that data copying in and out of accelerator’s main memory has major impact on performance when done in a loop, and the best performance is achieved when copying in and out is done outside the loop (when data copying inside the loop involves accelerator’s main memory only). This result comes in line with how distributed computations are performed on a set of cluster nodes, and suggests using similar approaches for single heterogeneous node with a graphical accelerator.
Computational Science and Its Applications – ICCSA 2017, 2017
In this article, we propose an approach that allows acceleration of the Time-of-Flight (ToF) even... more In this article, we propose an approach that allows acceleration of the Time-of-Flight (ToF) event reconstruction algorithm implementation, which is a part of the Multi Purpose Detector (MPD) Root application.
Virtualized computing infrastructures are often used to create clusters of resources tailored to ... more Virtualized computing infrastructures are often used to create clusters of resources tailored to solve tasks taking into account particular requirements of these tasks. An important objective is to evaluate such requirements and request optimal amount of resources which becomes challenging for parallel tasks with intercommunication. In previous works we investigated how light-weight container-based virtualization can be used for creating virtual clusters running MPI applications. Such cluster is configured according to the requirements of particular application and allocates only necessary amount of resources from the physical infrastructure leaving space for co-allocated clusters running without conflicts or resource races. In this paper we investigate similar concepts for MapReduce applications based on Hadoop framework that use Cloudply virtualization tool to create and manage light-weight virtual Hadoop clusters on Amazon cloud resources. We investigate performance of several Ha...
In particle accelerator physics the problem is that we can not see what is going on inside the wo... more In particle accelerator physics the problem is that we can not see what is going on inside the working machine. There are a lot of packages for modelling the behaviour of the particles in numerical or analytical way. But for most physicists it is better to see the picture in motion to say exactly what is happening and how to influence on this. The goal of this work is to provide scientists with such a problem-solving environment, which can not only do some numerical calculations, but show the dynamics of changes as a motion 3D picture. To do this we use the power of graphical processors from both sides: for general purpose calculations and for there direct appointment – drawing 3D motion. Besides, this environment should analyse the behaviour of the system to provide the user with all necessary information about the problem and how to deal with it.
To represent the space charge forces of beam a software based on analytical models for space char... more To represent the space charge forces of beam a software based on analytical models for space charge distributions was developed. Special algorithm for predictor-corrector method for beam map evaluation scheme including the space charge forces were used. This method allows us to evaluate the map along the reference trajectory and to analyze beam envelope dynamics. In three dimensional models the number of computing resources we use is significant. For this purpose graphical processors are used. This software is a part of Virtual Accelerator concept which is considered as a set of services and tools of modeling beam dynamics in accelerators on distributed computing resources.
Architecture of a digital computing system determines the technical foundation of a unified mathe... more Architecture of a digital computing system determines the technical foundation of a unified mathematical language for exact arithmetic-logical description of phenomena and laws of continuum mechanics for applications in fluid mechanics and theoretical physics. Deep parallelization of the computing processes serves to the revival of application of functional programming at a new technological level. The efficiency of computations is provided by true reproduction of the fundamental laws of physics and continuum mechanics. Tensor formalization of numerical objects and computing operations serves to spatial interpolation of rheological state parameters and laws of the fluid mechanics as mathematical models in the local coordinates of the elementary numeric cells — large liquid particles. The proposed approach allows the use of explicit numerical scheme, which is an important condition for increasing the efficiency of the algorithms developed by numerical procedures with natural parallel...
Computational Science and Its Applications – ICCSA 2020, 2020
Strong wind causes heavy load on the ship in a seaway bending and pushing it in the direction of ... more Strong wind causes heavy load on the ship in a seaway bending and pushing it in the direction of the wind. In this paper we investigate how wind can be simulated in the framework of Virtual testbed—a near real-time ship motion simulator. We propose simple model that describes air flow around ship hull with constant initial speed and direction which is based on the law of reflection. On the boundary the model reduces to the known model for potential flow around a cylinder, and near the boundary they are not equivalent, but close enough to visualise the effect of the hull on the flow. Then we apply this model to simulate air flow around real-world ship hull and conclude that for any real-world situation ship roll angle and ship speed caused by the wind is small to not cause capsizing, but large enough to be considered in onboard intelligent systems that determine real roll, pitch and yaw angles during ship operation and similar applications.
International Journal of Business Intelligence and Data Mining, 2017
Distributed computing clusters are often built with commodity hardware which leads to periodic fa... more Distributed computing clusters are often built with commodity hardware which leads to periodic failures of processing nodes due to relatively low reliability of such hardware. While worker node fault-tolerance is straightforward, fault tolerance of master node poses a bigger challenge. In this paper master node failure handling is based on the concept of master and worker roles that can be dynamically re-assigned to cluster nodes along with maintaining a backup of the master node state on one of worker nodes. In such case no special component is needed to monitor the health of the cluster while master node failures can be resolved except for the cases of simultaneous failure of master and backup. We present experimental evaluation of the technique implementation, show benchmarks demonstrating that a failure of a master does not affect running job, and a failure of backup results in re-computation of only the last job step.
In the problem of simulation of marine object behaviour in a seaway determination of pressures ex... more In the problem of simulation of marine object behaviour in a seaway determination of pressures exerted on the object is often done on assumption of ocean wave amplitudes being small compared to wave height, however, this is not the best approach for real ocean waves. This was done due to underlying wind wave models (such as Longuet—Higgins model) lacking ability to produce large amplitude waves. The other option is to use alternative autoregressive model which is capable of producing real ocean waves, but in this approach pressure calculation scheme should be extended to cover large-amplitude wave case. It is possible to obtain analytical solutions for both two- and three-dimensional problem and it was found that corresponding numerical algorithms are simple and have efficient implementations compared to small amplitude case where the calculation is done by transforming partial differential equations into numerical schemes. In the numerical experiment it was proved that obtained for...
ABSTRACT Efficient management of a distributed system is a common problem for university’s and co... more ABSTRACT Efficient management of a distributed system is a common problem for university’s and commercial computer centres, and handling node failures is a major aspect of it. Failures which are rare in a small commodity cluster, at large scale become common, and there should be a way to overcome them without restarting all parallel processes of an application. The efficiency of existing methods can be improved by forming a hierarchy of distributed processes. That way only lower levels of the hierarchy need to be restarted in case of a leaf node failure, and only root node needs special treatment. Process hierarchy changes in real time and the workload is dynamically rebalanced across online nodes. This approach makes it possible to implement efficient partial restart of a parallel application, and transactional behaviour for computer centre service tasks.
Computational Science and Its Applications – ICCSA 2014, 2014
ABSTRACT One of efficient ways to conduct experiments on HPC platforms is to create custom virtua... more ABSTRACT One of efficient ways to conduct experiments on HPC platforms is to create custom virtual computing environments tailored to the requirements of users and their applications. In this paper we investigate virtual private supercomputer, an approach based on virtualization, data consolidation, and cloud technologies. Virtualization is used to abstract applications from underlying hardware and operating system while data consolidation is applied to store data in a distributed storage system. Both virtualization and data consolidation layers offer APIs for distributed computations and data processing. Combined, these APIs shift the focus from supercomputing technologies to problems being solved. Based on these concepts, we propose an approach to construct virtual clusters with help of cloud computing technologies to be used as on-demand private supercomputers and evaluate performance of this solution.
Master node fault-tolerance is the topic that is often dimmed in the discussion of big data proce... more Master node fault-tolerance is the topic that is often dimmed in the discussion of big data processing technologies. Although failure of a master node can take down the whole data processing pipeline, this is considered either improbable or too difficult to encounter. The aim of the studies reported here is to propose rather simple technique to deal with master-node failures. This technique is based on temporary delegation of master role to one of the slave nodes and transferring updated state back to the master when one step of computation is complete. That way the state is duplicated and computation can proceed to the next step regardless of a failure of a delegate or the master (but not both). We run benchmarks to show that a failure of a master is almost “invisible” to other nodes, and failure of a delegate results in recomputation of only one step of data processing pipeline. We believe that the technique can be used not only in Big Data processing but in other types of applications.
Nowadays supercomputer centers strive to provide their computational resources as services, howev... more Nowadays supercomputer centers strive to provide their computational resources as services, however, present infrastructure is not particularly suited for such a use. First of all, there are standard application programming interfaces to launch computational jobs via command line or a web service, which work well for a program but turn out to be too complex for scientists: they want applications to be delivered to them from a remote server and prefer to interact with them via graphical interface. Second, there are certain applications which are dependent on older versions of operating systems and libraries and it is either non-practical to install those old systems on a cluster or there exists some conflict between these dependencies. Virtualization technologies can solve this problem, but they are not too popular in scientific computing due to overheads introduced by them. Finally, it is difficult to automatically estimate optimal resource pool size for a particular task, thus it o...
9th International Conference "Distributed Computing and Grid Technologies in Science and Education", 2021
There is no single diagnostic marker for neurodegenerative diseases. The biomedical data obtained... more There is no single diagnostic marker for neurodegenerative diseases. The biomedical data obtained during these studies have heterogeneous nature, which greatly complicates their collection, storage and complex analysis. Special methods of statistical analysis due to the described specifics of the data must be applied. The results obtained indicate that for a correct diagnosis, it is necessary to use a comprehensive assessment of all tests.
Computational Science and Its Applications – ICCSA 2020, 2020
Diffraction and radiation forces result from the interaction between the ship hull and the moving... more Diffraction and radiation forces result from the interaction between the ship hull and the moving fluid. These forces are typically simulated using added masses, a method that uses mass to compensate for not computing these forces directly. In this paper we propose simple mathematical model to compute diffraction force. The model is based on Lagrangian description of the flow and uses law of reflection to include diffraction term in the solution. The solution satisfies continuity equation and equation of motion, but is restricted to the boundary of the ship hull. The solution was implemented in velocity potential solver in Virtual testbed—a programme for workstations that simulates ship motions in extreme conditions. Performance benchmarks of the solver showed that it is particularly efficient on graphical accelerators.
Computational Science and Its Applications – ICCSA 2019, 2019
Virtual testbed is a computer programme that simulates ocean waves, ship motions and compartment ... more Virtual testbed is a computer programme that simulates ocean waves, ship motions and compartment flooding. One feature of this programme is that it visualises physical phenomena frame by frame as the simulation progresses. The aim of the studies reported here was to assess how much performance can be gained using graphical accelerators compared to ordinary processors when repeating the same computations in a loop. We rewrote programme’s hot spots in OpenCL to able to execute them on a graphical accelerator and benchmarked their performance with a number of real-world ship models. The analysis of the results showed that data copying in and out of accelerator’s main memory has major impact on performance when done in a loop, and the best performance is achieved when copying in and out is done outside the loop (when data copying inside the loop involves accelerator’s main memory only). This result comes in line with how distributed computations are performed on a set of cluster nodes, and suggests using similar approaches for single heterogeneous node with a graphical accelerator.
Computational Science and Its Applications – ICCSA 2017, 2017
In this article, we propose an approach that allows acceleration of the Time-of-Flight (ToF) even... more In this article, we propose an approach that allows acceleration of the Time-of-Flight (ToF) event reconstruction algorithm implementation, which is a part of the Multi Purpose Detector (MPD) Root application.
Virtualized computing infrastructures are often used to create clusters of resources tailored to ... more Virtualized computing infrastructures are often used to create clusters of resources tailored to solve tasks taking into account particular requirements of these tasks. An important objective is to evaluate such requirements and request optimal amount of resources which becomes challenging for parallel tasks with intercommunication. In previous works we investigated how light-weight container-based virtualization can be used for creating virtual clusters running MPI applications. Such cluster is configured according to the requirements of particular application and allocates only necessary amount of resources from the physical infrastructure leaving space for co-allocated clusters running without conflicts or resource races. In this paper we investigate similar concepts for MapReduce applications based on Hadoop framework that use Cloudply virtualization tool to create and manage light-weight virtual Hadoop clusters on Amazon cloud resources. We investigate performance of several Ha...
In particle accelerator physics the problem is that we can not see what is going on inside the wo... more In particle accelerator physics the problem is that we can not see what is going on inside the working machine. There are a lot of packages for modelling the behaviour of the particles in numerical or analytical way. But for most physicists it is better to see the picture in motion to say exactly what is happening and how to influence on this. The goal of this work is to provide scientists with such a problem-solving environment, which can not only do some numerical calculations, but show the dynamics of changes as a motion 3D picture. To do this we use the power of graphical processors from both sides: for general purpose calculations and for there direct appointment – drawing 3D motion. Besides, this environment should analyse the behaviour of the system to provide the user with all necessary information about the problem and how to deal with it.
To represent the space charge forces of beam a software based on analytical models for space char... more To represent the space charge forces of beam a software based on analytical models for space charge distributions was developed. Special algorithm for predictor-corrector method for beam map evaluation scheme including the space charge forces were used. This method allows us to evaluate the map along the reference trajectory and to analyze beam envelope dynamics. In three dimensional models the number of computing resources we use is significant. For this purpose graphical processors are used. This software is a part of Virtual Accelerator concept which is considered as a set of services and tools of modeling beam dynamics in accelerators on distributed computing resources.
Architecture of a digital computing system determines the technical foundation of a unified mathe... more Architecture of a digital computing system determines the technical foundation of a unified mathematical language for exact arithmetic-logical description of phenomena and laws of continuum mechanics for applications in fluid mechanics and theoretical physics. Deep parallelization of the computing processes serves to the revival of application of functional programming at a new technological level. The efficiency of computations is provided by true reproduction of the fundamental laws of physics and continuum mechanics. Tensor formalization of numerical objects and computing operations serves to spatial interpolation of rheological state parameters and laws of the fluid mechanics as mathematical models in the local coordinates of the elementary numeric cells — large liquid particles. The proposed approach allows the use of explicit numerical scheme, which is an important condition for increasing the efficiency of the algorithms developed by numerical procedures with natural parallel...
Computational Science and Its Applications – ICCSA 2020, 2020
Strong wind causes heavy load on the ship in a seaway bending and pushing it in the direction of ... more Strong wind causes heavy load on the ship in a seaway bending and pushing it in the direction of the wind. In this paper we investigate how wind can be simulated in the framework of Virtual testbed—a near real-time ship motion simulator. We propose simple model that describes air flow around ship hull with constant initial speed and direction which is based on the law of reflection. On the boundary the model reduces to the known model for potential flow around a cylinder, and near the boundary they are not equivalent, but close enough to visualise the effect of the hull on the flow. Then we apply this model to simulate air flow around real-world ship hull and conclude that for any real-world situation ship roll angle and ship speed caused by the wind is small to not cause capsizing, but large enough to be considered in onboard intelligent systems that determine real roll, pitch and yaw angles during ship operation and similar applications.
International Journal of Business Intelligence and Data Mining, 2017
Distributed computing clusters are often built with commodity hardware which leads to periodic fa... more Distributed computing clusters are often built with commodity hardware which leads to periodic failures of processing nodes due to relatively low reliability of such hardware. While worker node fault-tolerance is straightforward, fault tolerance of master node poses a bigger challenge. In this paper master node failure handling is based on the concept of master and worker roles that can be dynamically re-assigned to cluster nodes along with maintaining a backup of the master node state on one of worker nodes. In such case no special component is needed to monitor the health of the cluster while master node failures can be resolved except for the cases of simultaneous failure of master and backup. We present experimental evaluation of the technique implementation, show benchmarks demonstrating that a failure of a master does not affect running job, and a failure of backup results in re-computation of only the last job step.
In the problem of simulation of marine object behaviour in a seaway determination of pressures ex... more In the problem of simulation of marine object behaviour in a seaway determination of pressures exerted on the object is often done on assumption of ocean wave amplitudes being small compared to wave height, however, this is not the best approach for real ocean waves. This was done due to underlying wind wave models (such as Longuet—Higgins model) lacking ability to produce large amplitude waves. The other option is to use alternative autoregressive model which is capable of producing real ocean waves, but in this approach pressure calculation scheme should be extended to cover large-amplitude wave case. It is possible to obtain analytical solutions for both two- and three-dimensional problem and it was found that corresponding numerical algorithms are simple and have efficient implementations compared to small amplitude case where the calculation is done by transforming partial differential equations into numerical schemes. In the numerical experiment it was proved that obtained for...
ABSTRACT Efficient management of a distributed system is a common problem for university’s and co... more ABSTRACT Efficient management of a distributed system is a common problem for university’s and commercial computer centres, and handling node failures is a major aspect of it. Failures which are rare in a small commodity cluster, at large scale become common, and there should be a way to overcome them without restarting all parallel processes of an application. The efficiency of existing methods can be improved by forming a hierarchy of distributed processes. That way only lower levels of the hierarchy need to be restarted in case of a leaf node failure, and only root node needs special treatment. Process hierarchy changes in real time and the workload is dynamically rebalanced across online nodes. This approach makes it possible to implement efficient partial restart of a parallel application, and transactional behaviour for computer centre service tasks.
Computational Science and Its Applications – ICCSA 2014, 2014
ABSTRACT One of efficient ways to conduct experiments on HPC platforms is to create custom virtua... more ABSTRACT One of efficient ways to conduct experiments on HPC platforms is to create custom virtual computing environments tailored to the requirements of users and their applications. In this paper we investigate virtual private supercomputer, an approach based on virtualization, data consolidation, and cloud technologies. Virtualization is used to abstract applications from underlying hardware and operating system while data consolidation is applied to store data in a distributed storage system. Both virtualization and data consolidation layers offer APIs for distributed computations and data processing. Combined, these APIs shift the focus from supercomputing technologies to problems being solved. Based on these concepts, we propose an approach to construct virtual clusters with help of cloud computing technologies to be used as on-demand private supercomputers and evaluate performance of this solution.
Master node fault-tolerance is the topic that is often dimmed in the discussion of big data proce... more Master node fault-tolerance is the topic that is often dimmed in the discussion of big data processing technologies. Although failure of a master node can take down the whole data processing pipeline, this is considered either improbable or too difficult to encounter. The aim of the studies reported here is to propose rather simple technique to deal with master-node failures. This technique is based on temporary delegation of master role to one of the slave nodes and transferring updated state back to the master when one step of computation is complete. That way the state is duplicated and computation can proceed to the next step regardless of a failure of a delegate or the master (but not both). We run benchmarks to show that a failure of a master is almost “invisible” to other nodes, and failure of a delegate results in recomputation of only one step of data processing pipeline. We believe that the technique can be used not only in Big Data processing but in other types of applications.
Nowadays supercomputer centers strive to provide their computational resources as services, howev... more Nowadays supercomputer centers strive to provide their computational resources as services, however, present infrastructure is not particularly suited for such a use. First of all, there are standard application programming interfaces to launch computational jobs via command line or a web service, which work well for a program but turn out to be too complex for scientists: they want applications to be delivered to them from a remote server and prefer to interact with them via graphical interface. Second, there are certain applications which are dependent on older versions of operating systems and libraries and it is either non-practical to install those old systems on a cluster or there exists some conflict between these dependencies. Virtualization technologies can solve this problem, but they are not too popular in scientific computing due to overheads introduced by them. Finally, it is difficult to automatically estimate optimal resource pool size for a particular task, thus it o...
Uploads
Papers by Alexander Degtyarev