MapReduce and Hadoop

We introduced a new computation model that reflects the assumptions of the map-reduce framework, but allows for networks of processes other than Map feeding Reduce. We illustrated the benefit of our model by showing how to improve the computation of the 3-way join and by developing algorithms for merging and sorting. The cost measures by which we evaluate the algorithms are the communication among processes and the processing time, both total over all processes and elapsed (i.e., exploiting parallelism). The input file is hold on as a group of files on persistent storage. The goal of a sorting system is to rework this input file into associate ordered set of output files, conjointly hold on persistent storage, such the concatenation of those output files so as constitutes the sorted version of the input file. Our goal is to style and implement a sorting system that may sort datasets of the targeted size whereas achieving a good exchange between speed, resource utilization, and cost.

The widespread use of digital images has led to a new challenge in digital image forensics. These images can be used in court as evidence of criminal cases. However, digital images are easily manipulated which brings up the need of a method to verify the authenticity of the image. One of the methods is by identifying the source camera. In spite of that, it takes a large amount of time to be completed by using traditional desktop computers. To tackle the problem, we aim to increase the performance of the process by implementing it in a distributed computing environment. We evaluate the camera identification process using conditional probability features and Apache Hadoop. The evaluation process used 6000 images from six different mobile phones of the different models and classified them using Apache Mahout, a scalable machine learning tool which runs on Hadoop. We ran the source camera identification process in a cluster of up to 19 computing nodes. The experimental results demonstrate exponential decrease in processing times and slight decrease in accuracies as the processes are distributed across the cluster. Our prediction accuracies are recorded between 85 to 95% across varying number of mappers.

While advanced analysis of large dataset is in high demand, data sizes have surpassed capabilities of conventional software and hardware. Hadoop framework distributes large datasets over multiple commodity servers and performs parallel computations. We discuss the I/O bottlenecks of Hadoop framework and propose methods for enhancing I/O performance. A proven approach is to cache data to maximize memory-locality of all map tasks. We introduce an approach to optimize I/O, the in-node combining design which extends the traditional combiner to a node level. The in-node combiner reduces the total number of intermediate results and curtail network traffic between mappers and reducers.

To buy this project in ONLINE, Contact:
Email: jpinfotechprojects@gmail.com,
Website: https://www.jpinfotech.org


On Scalable and Robust Truth Discovery in Big Data Social Media Sensing Applications

Identifying trustworthy information in the presence of noisy data contributed by numerous unvetted sources from online social media (e.g., Twitter, Facebook, and Instagram) has been a crucial task in the era of big data. This task, referred to as truth discovery, targets at identifying the reliability of the sources and the truthfulness of claims they make without knowing either a priori. In this work, we identified three important challenges that have not been well addressed in the current truth discovery literature. The first one is "misinformation spread" where a significant number of sources are contributing to false claims, making the identification of truthful claims difficult. For example, on Twitter, rumors, scams, and influence bots are common examples of sources colluding, either intentionally or unintentionally, to spread misinformation and obscure the truth. The second challenge is "data sparsity" or the "long-tail phenomenon" where a majority of sources only contribute a small number of claims, providing insufficient evidence to determine those sources' trustworthiness. For example, in the Twitter datasets that we collected during real-world events, more than 90% of sources only contributed to a single claim. Third, many current solutions are not scalable to large-scale social sensing events because of the centralized nature of their truth discovery algorithms. In this paper, we develop a Scalable and Robust Truth Discovery (SRTD) scheme to address the above three challenges. In particular, the SRTD scheme jointly quantifies both the reliability of sources and the credibility of claims using a principled approach. We further develop a distributed framework to implement the proposed truth discovery scheme using Work Queue in an HTCondor system. The evaluation results on three real-world datasets show that

— Cloud computing is one of the emerging techniques to process the big data. Cloud computing is also, known as service on demand. Large set or large volume of data is known as big data. Processing big data (MRI images and DICOM images) normally takes more time. Hard tasks such as handling big data can be solved by using the concepts of hadoop. Enhancing the hadoop concept will help the user to process the large set of images. The Hadoop Distributed File System (HDFS) and MapReduce are the two default main functions which is used to enhance hadoop. HDFS is a hadoop file storing system, which is used for storing and retrieving the data. MapReduce is the combination of two functions namely map and reduce. Map is the process of splitting the inputs and reduce is the process of integrating the output of map's input. Recently, medical experts experienced problems like machine failure and fault tolerance while processing the result for the scanned data. A unique optimized time scheduling algorithm, called Dynamic Handover Reduce Function (DHRF) algorithm is introduced in the reduce function. Enhancement of hadoop and cloud and introduction of DHRF helps to overcome the processing risks, to get optimized result with less waiting time and reduction in error percentage of the output image.

In last two decades continues increase of comput-ational power and recent advance in the
web technology cause to provide large amounts of data. That needs large scale data
processing mechanism to handle this volume of data. MapReduce is a programming model
for large scale distributed data processing in an efficient and transparent way. Due to its
excellent fault tolerance features, scalability and the ease of use. Currently, there are several
options for using MapReduce in cloud environments, such as using MapReduce as a service,
setting up one’s own MapReduce cluster on cloud instances, or using specialized cloud
MapReduce runtimes that take advantage of cloud infrastructure services. Cloud computing
has recently emerged as a new paradigm that provide computing infrastructure and large
scale data processing mechanism in the network. The cloud is on demand, scalable and high
availability so implement of MapReduce on the top of cloud services cause faster, scalable
and high available MapReduce framework for large scale data processing. In this paper we
explain how to implement MapReduce in the cloud and also have a comparison between
implementations of MapReduce on AzureCloud, Amazon Cloud and Hadoop at the end.

Sentiment analysis on Twitter data has attracted much attention recently. One of the system’s key features, is the immediacy in communication with other users in an easy, user-friendly and fast way. Consequently, people tend to express their feelings freely, which makes Twitter an ideal source for accumulating a vast amount of opinions towards a wide diversity of topics. This amount of information oers huge potential and can be harnessed to receive the sentiment tendency towards these topics. However, since none can invest an innite amount of time to read through these tweets, an automated decision making approach is necessary. Nevertheless, most existing solutions are limited in centralized environments only. Thus, they can only process at most a few thousand tweets. Such a sample, is not representative to dene the sentiment polarity towards a topic due to the massive number of tweets published daily. In this paper, we go one step further and develop a novel method for sentiment le...

With cloud computing, cheap storage and technology advancements, an enterprise uses multiple applications to operate business functions. Applications are not limited to just transactions, customer service, sales, finance but they also include security, application logs, marketing, engineering, operations, HR and many more. Each business vertical uses multiple applications which generate a huge amount of data. On top of that, social media, IoT sensors, SaaS solutions, and mobile applications record exponential growth in data volume. In almost all enterprises, data silos exist through these applications. These applications can produce structured, semi-structured, or unstructured data at different velocity and in different volume. Having all data sources integrated and generating timely insights helps in overall decision making. With recent development in Big Data Integration, data silos can be managed better and it can generate tremendous value for enterprises. Big data integration offers flexibility, speed and scalability for integrating large data sources. It also offers tools to generate analytical insights which can help stakeholders to make effective decisions. This paper presents the overview on data silos, challenges with data silos and how big data integration can help to stun them.

The explosive growth of data produced by different devices and applications has contributed to the abundance of big data. To process such amounts of data efficiently, strategies such as De-duplication has been employed. Among the three different levels of de-duplication named as file level, block level and chunk level, De-duplication at chunk level also known as byte level is the most popular and widely deployed. Many chunking techniques are also available which are categorised as Whole File Chunking, Fixed Size Chunking (FSC) and Content Defined Chunking (CDC). The objective of this paper is to analyse the performance of different existing chunking techniques based on their characteristics. In this study the significance of each technique provides insight to enable researchers understand and select a technique for their research.

The emergence of big data analytics as a way of deriving insights from data brought excitement to mathematicians, statisticians, computer scientists and other professionals. However, the absence of a mathematical foundation for analytics has become a real challenge amidst the flock of big data marketing activities, especially in healthcare insurance. This paper developed a mathematical model for the analytics of healthcare insurance data using set theory. We implemented a prototype of the model using Java Programming Language, MapReduce Framework, Association Rule Mining and MongoDB. Also, we tested the model accuracy using data from the National Health Insurance Scheme in Nigeria to reduce delay. The result showed that the accuracy level was 97.14% on average, which depicts a higher performance for the model. This result implies that delays affecting the processing of data submitted by the providers and enrollees to the HMOs reduced drastically leading to the improvement in the eas...

We have implemented a Hadoop compatible framework that helps in detection of suspected hardware failures in DataNodes within a Hadoop cluster and in signalling the nodes accordingly. Based on the status of the various hardware components of the DataNodes, the master node signals the DataNode so that appropriate actions can be taken. In a Hadoop cluster, the knowledge of the network is important for the master node to configure itself to meet varying needs of the DataNodes. It is important to keep track of the current topology of the Hadoop network and track the status of the network services in Hadoop. We have also added the functionality to discover the network topology in a Hadoop cluster. This discovered topology information will be useful in performing load balancing in the network, or in making intelligent decisions in data replication in case of node failures.

Data outsourcing allows data owners to keep their data at untrusted clouds that do not ensure the privacy of data and/or computations. One useful framework for fault-tolerant data processing in a distributed fashion is MapReduce, which was developed for trusted private clouds. This paper presents algorithms for data outsourcing based on Shamir's secret-sharing scheme and for executing privacy-preserving SQL queries such as count, selection including range selection, projection, and join while using MapReduce as an underlying programming model. The proposed algorithms prevent the untrusted cloud to know the database or the query while also preventing output size and access-pattern attacks. Interestingly, our algorithms do not need the database owner, which only creates and distributes secret-shares once, to be involved to answer any query, and hence, the database owner also cannot learn the query. We evaluate the efficiency of the algorithms on parameters: (i) the number of communication rounds (between a user and a cloud), (ii) the total amount of bit flow (between a user and a cloud), and (iii) the computational load at the user-side and the cloud-side.

DNA Sequencing is a process where we determine and identify every single DNA base and element that is in the genome of an individual. There are six billion of those in every normal cell in every person. When we apply DNA sequencing in the Cancer project. We first figure out what those six billion DNA bases are in the normal cells in that person and then we take some of the tumor in that person and figure out what the DNA bases are in the tumor. In this paper, Here we discuss implements and future works in bioinformatics with Hadoop. We study the MapReduce algorithm from algorithm lay by point and demonstrate the appropriates of our approach by tracing and analyzing efficient MapReduce algorithms for sorting and simulation problem of parallel algorithms specified in the help of pigeonhole principle. The big data is a great computational challenge to statistical analysis of DNA big data. We can get general statistical analysis through R language. After studying the survey paper various approaches of using GPU and MapReduce. We adopted the best solution to using R with MapReduce. An R package is created to shift a set of critical R functions on GPU card. It allows users to run R code with GPU spread that enable much faster large data set of computation.

سال گذشته بیش از 14 سیستم های توصیه گر برای مقالات علمی  به طور فزآینده ای محبوب شده اند، به طوری که در  مقالات علمی، اختراع ثبت شده، صفحات وب و سایر موارد در این زمینه منتشر شده اند. سیستم های توصیه گر مقالات 170 با استفاده از اطلاعات زمینه ای سعی دارند بر اساس عملکرد، سلیقههای شخصی، رفتارهای کاربر و بسته به زمینهای که در آن مورد استفاده قرار گرفتهاند، به هر کاربر پیشنهادهایی را ارائه دهند که با تمایلات شخصی کاربر تطابق داشته باشند. از آنجایی که حجم مقالات روز به روز در حال افزایش است و سیستم های توصیه گر توانایی پوشش این حجم عظیم را برای پردازش مقالات با توجه به سلایق کاربر را ندارند، به همین دلیل برای پوشش و پردازش سریع این حجم از مقالات نیاز به استفاده از  کاهش) است. سیستم پیشنهادی این پژوهش ابتدا برای هر مقاله پروفایلی را تشکیل –پردازش موازی(برنامه نویسی نگاشت می دهد که حاوی اطلاعات زمینه و دامنه ی مقاله می باشد سپس با توجه به دامنه کاری کاربر و دامنه مقالات، سیستم به کاربر مقالاتی را توصیه خواهد کرد. از آنجایی که حجم داده های مورد استفاده جزء داده بزرگ می باشد و زمان نیز یک فاکتور مهم در این پژوهش است به همین علت از بستر هدوپ و برنامه نویسی موازی برای پیاده سازی سیستم استفاده شده است . عملکرد سیستم پیشنهادی با معیارهایی همچون رضایت کاربر و دقت سنجیده شد که نتایج رضایت بخش را به همراه داشته است

—The operations of the Internet have led to a significant growth and accumulation of data known as Big Data. Individuals and organizations that utilize this data, had no idea, nor were they prepared for this data explosion. Hence, the available solutions cannot meet the needs of the growing heterogeneous data in terms of processing. This results in inefficient information retrieval or search query results. The design of indexing strategies that can support this need is required. A survey on various indexing strategies and how they are utilized for solving Big Data management issues can serve as a guide for choosing the strategy best suited for a problem, and can also serve as a base for the design of more efficient indexing strategies. The aim of the study is to explore the characteristics of the indexing strategies used in Big Data manageability by covering some of the weaknesses and strengths of B-tree, R-tree, to name but a few. This paper covers some popular indexing strategies used for Big Data management. It exposes the potentials of each by carefully exploring their properties in ways that are related to problem solving.

A flexible, efficient and secure networking architecture is required in order to process big data. However, existing network architectures are mostly unable to handle big data. As big data pushes network resources to the limits it results in network congestion, poor performance, and detrimental user experiences. This paper presents the current state-of-the-art research challenges and possible solutions on big data networking theory. More specifically, we present the state of networking issues of big data related to capacity, management and data processing. We also present the architectures of MapReduce and Hadoop paradigm with research challenges, fabric networks and software defined networks (SDN) that are used to handle today's idly growing digital world and compare and contrast them to identify relevant problems and solutions.

IEEE Hadoop Big Data Project Titles 2017 | 2018 IEEE Big Data Projects
A Scalable Data Chunk Similarity based Compression Approach for Efficient Big Sensing Data Processing on Cloud
A Systematic Approach Toward Description and Classification of Cybercrime Incidents
Achieving Efficient and Privacy-Preserving Cross-Domain Big Data Deduplication in Cloud
aHDFS: An Erasure-Coded Data Archival System for Hadoop Clusters
Big Data Privacy in Biomedical Research
Cost-Aware Big Data Processing across Geo-distributed Datacenters
Disease Prediction by Machine Learning over Big Data from Healthcare Communities
DyScale: a MapReduce Job Scheduler for Heterogeneous Multicore Processors
Efficient Processing of Skyline Queries Using MapReduce
FiDoop-DP: Data Partitioning in Frequent Itemset Mining on Hadoop Clusters
Hadoop MapReduce for Mobile Clouds
Mining Human Activity Patterns from Smart Home Big Data for Healthcare Applications
PPHOPCM: Privacy-preserving High-order Possibilistic  c-Means Algorithm for Big Data Clustering with Cloud Computing
Practical Privacy-Preserving MapReduce Based K-means Clustering over Large-scale Dataset
Public Interest Analysis Based on Implicit Feedback of IPTV Users
Ring: Real-Time Emerging Anomaly Monitoring System over Text Streams
Robust Big Data Analytics for Electricity Price Forecasting in the Smart Grid
Scalable Uncertainty-Aware Truth Discovery in Big Data Social Sensing Applications for Cyber-Physical Systems
Self-Adjusting Slot Configurations for Homogeneous and Heterogeneous Hadoop Clusters
Service Rating Prediction by Exploring Social Mobile Users’ Geographical Locations

According to Soule (n.d), the surge towards a semantic web has created a need for efficient querying methods over large datasets that include statements about web resources. Husain et al (2011) state that semantic web technologies are being developed in order to present data that can be retrieved and understood by both humans and machines. The authors posit that these technologies have been standardized to address certain limitations such published web pages that aren’t suitable for reasoning. The most prominent standard is the Resource Description Framework (RDF) because of its ability to represent data in a machine readable format. According to the World Wide Web Consortium (W3C) (n.d). RDF has features that facilitate data merging regardless of the underlying schemas and also supports the evolution of the schemas over time without the need to change the consuming clients.

ETL process (Extracting-Transforming-Loading)is  responsible  for  (E)xtracting  data  from  heterogeneous sources,  (T)ransforming  and  finally  (L)oading  them  into  a data warehouse (DW). Nowadays, Internet and Web 2.0 are generating data at an increasing rate, and therefore put the information  systems  (IS)  face  to  the  challenge  of  big  data.Data integration systems and ETL, in particular, should be revisited and adapted and the well-known solution is based on  the  data  distribution  and  the  parallel/distributed  processing.  Among  all  the  dimensions  defining  the  complexity of  the  big  data,  we  focus  in  this  paper  on  its  excessive"volume"  in  order  to  ensure  good  performance  for  ETL processes. In this context, we propose an original approach called  Big-ETL  (ETL  Approach  for  Big  Data)  in  which  we define ETL functionalities that can be run easily on a cluster of  computers  with  MapReduce  (MR)  paradigm.  Big-ETL allows, thereby, parallelizing/distributing ETL at two levels:(i)  the  ETL  process  level  (coarse  granularity  level),  and(ii) the functionality level (fine level); this allows improving further the ETL performance.

Hadoop an apache product which is an open-source, Java based programming framework is used to support large data sets in a distributed environment. Hadoop has maximum advantage over scalable and fault-tolerant distributed processing technologies. Also,(HDFS)Hadoop Distributed File System is highly fault tolerant and used for applications that have large data sets. Hence HDFS file system using name node, data node and task tracker will perform distribution of job in Hadoop environment .Since Hadoop has overwhelming advantage in optimizing big data , we prefer to use Hadoop to analyze large datasets of weather processing.

Guest talk in the following graduate courses of University of Arkansas at Little Rock:
1. Information System Analysis and Design
2. Social Computing

Also presented a shorter version of it at the Arkansas Chapter of ISACA (Information Systems Audit and Control Association)

Mining big data requires a deep investment in people and time. How can you be sure you’re building the right models? With this hands-on book, you’ll learn a flexible toolset and methodology for building effective analytics applications with Hadoop.

Using lightweight tools such as Python, Apache Pig, and the D3.js library, your team will create an agile environment for exploring data, starting with an example application to mine your own email inboxes. You’ll learn an iterative approach that enables you to quickly change the kind of analysis you’re doing, depending on what the data is telling you. All example code in this book is available as working Heroku apps.

Create analytics applications by using the agile big data development methodology

Build value from your data in a series of agile sprints, using the data-value stack

Gain insight by using several data structures to extract multiple features from a single dataset

Visualize data with charts, and expose different aspects through interactive reports

Use historical data to predict the future, and translate predictions into action

Get feedback from users after each sprint to keep your project on track

Accurate recognition and differentiation of the human facial expressions require substantial computational power, where the efficiency of algorithm plays a vital role. Recent advancement in the human computer interaction and object recognition in terms of facial expressions and gesture demanded realistic facial animation models, smart algorithms for massive data handling, as well as sophisticated graphical user interfaces. A rapid escalation in the photo upload to the online social networks web sites such as Facebook and Twitter is evident, where huge dataset handling became the key issue. Competent search and manipulations within a large dataset for image reproductions posed a new challenge, where standard tools cannot achieve the desired target. Often, images possessing intricate multidimensional attributes involve robust computational techniques for pragmatic recognition. Thus, developing a novel robust facial recognition platform emerged as an urgent necessity. We introduce a new algorithm for facial image tagging and classification in cloud environments using the Hadoop and MapReduce based k-nearest neighbor algorithms. Experiments are performed on 3120 images from 120 individuals (65 male, 55 female) from the AR Face Database. The efficiency of the proposed algorithm is evaluated in terms of recognition rates and processing time. Significant improvements in the performance are demonstrated.

( Alternatif Link download Buku Analisis Big Data: http://bit.ly/2x8ta9S )

Alhamdulillahhi robbil alamin, puji syukur kehadirat Allah SWT atas segala rahmat dan karunia-Nya dengan terselesaikannya penulisan buku ini dengan judul “Analisis Big Data”. Buku ini merupakan uraian untuk memudahkan pemahaman konsep, tingkat dasar sampai lanjut dalam sistem cerdas dan penerapannya melalui pemanfaatan teknologi Big Data, dengan mengedepankan keterampilan dalam pembuatan dan hasil implementasi dengan berbagai kombinasi algoritma berbasis sistem cerdas maupun dengan perpaduan berbagai macam tools untuk membangun ekosistem analisis Big Data yang powerfull. Konsep sederhana Analisis Big Data mencakup Volume, Velocity, dan Variety (3V), dan pengembangannya ada yang menyebut (7V) termasuk Volume, Velocity, Variety, Variability, Veracity, Value dan Visualization, atau 10V bahkan lebih dari itu, tetapi sebaiknya tidak membatasi pemahaman Big Data ini hanya dengan sedikit banyaknya istilah dari huruf V yang digunakan :). Kemudian dengan adanya ekosistem tersebut, jika dibutuhkan analisis sederhana maupun yang lebih kompleks, maka harapannya tidak akan ada kendala dari besarnya data yang diolah. Adanya kemajuan teknologi dalam hal penyimpanan, pengolahan, dan analisis Big Data meliputi (a) penurunan secara cepat terhadap biaya penyimpanan data dalam beberapa tahun terakhir; (b) fleksibilitas dan efektivitas biaya pada pusat data dan komputasi awan untuk perhitungan dengan konsep elastisitas dan penyimpanannya; serta (c) pengembangan kerangka kerja baru seperti Hadoop ecosystem (salah satu peluang bisnis yang besar untuk developer untuk saat ini dan ke depannya dalam rangka membangun ekosistem analisis Big Data yang sangat powerfull sekelas Cloudera, Hortonworks, etc), yang memungkinkan pengguna untuk mengambil manfaat dari sistem komputasi terdistribusi, misal untuk menyimpan sejumlah data yang besar melalui pemrosesan parallel, dukungan database NoSQL, dan komputasi berbasis streaming. Sehingga kemajuan teknologi ini telah menciptakan beberapa perbedaan yang sangat signifikan, misal dalam hal kecepatan maupun ketepatan dari hasil yang didapatkan antara analisis tradisional dengan tools yang bukan dengan konsep Big Data versus analisis modern untuk saat ini dan masa depan dengan membangun ekosistem Big Data yang sangat powerfull.
Penulis mengucapkan terimakasih yang sebesar-besarnya kepada kedua orang tua penulis, spesial kepada Istriku tercinta “Alfun Lailatul Musabbikha”, bapak ibu mertua, dik Wiwik, dik Ganda, kak Mat, kak Deri, Bude Gi, Pakde Dul, Bude Gani, Pak Abu, mbak Nik, dik Andre, Mbak As, Mbak Lia, Mas Mualif, Mbak Lis, Pakde Naji, Bulik Yul, Pakde Tasrun, Pakde Zen, Pakde Zud, Paklik Fiq, mas Bowo dan semua sanak saudaraku yang tak bisa kusebutkan seluruhnya, serta beberapa pihak yang telah membantu penyelesaian buku ini:
1. Para penulis artikel Analisis Big Data di forum, web, blog dan buku yang menjadi referensi buku ini untuk memberikan masukan yang sangat berharga sekali untuk perbaikan dan penyelesaian buku ini.
2. Mbak Efi Riyandani, yang telah banyak membantu penulisan buku, dan mahasiswa-mahasiswa terbaik saya semester Ganjil 2016/2017, yaitu: Maryamah, Moh. Fadel Asikin, Daisy Kurniawaty, Selly Kurnia Sari, Nanda Agung Putra, Ardisa Tamara Putri, Dhimas Anjar Prabowo, Listiya Surtiningsih, Raissa Arniantya, Brillian Aristyo Rahadian, Diva Kurnianingtyas, Dyan Putri Mahardika, Tusty Nadia Maghfira.
3. Mahasiswa-mahasiswa terbaik saya semester Ganjil 2017/2018, yaitu: Yessica Inggir F., Kholifaul K., Ni Made Gita D. P., Ema Agasta, Retiana Fadma P. Sinaga, Fachrul Rozy Saputra Rangkuti, Yunita Dwi Alfiyanti, Dyah Ayu Wahyuning Dewi, Annisaa Amalia Safitri, Sarah Aditya Darmawan, Danastri Ramya Mehaninda, Eka Novita Shandra, Fakharuddin Farid Irfani, Rio Cahyo Anggono, Robih Dini, Yulia Kurniawati, Novirra Dwi Asri, Muhammad Vidi Mycharoka, Vania Nuraini Latifah, Olivia Bonita, Eka Miyahil Uyun, Cusen Mosabeth, Evilia Nur Harsanti, Ivarianti Sihaloho.

Semoga kontribusi kalian menjadi ilmu yang barokah dan bermanfaat. Aamiin. :). Tidak ada gading yang tak retak. Maka penulis memohon kritik dan saran untuk perbaikan dan penyempurnaan buku ini. In Syaa Allah pada edisi berikutnya, kami akan memberikan manualisasi Map Reduce, Spark, etc. dari setiap algoritma pada setiap contoh kasusnya. Selamat membaca buku ini dan semoga bermanfaat.

Malang, 19 Juli 2016-24 Mei 2018

Penulis

Data-intensive systems encompass terabytes to petabytes of data. Such systems require massive storage and intensive computational power in order to execute complex queries and generate timely results. Further, the rate at which this data is being generated induces extensive challenges of data storage, linking, and processing. A data-intensive cloud provides an abstraction of high availability, usability, and efficiency to users. However, underlying this abstraction, there are stringent requirements and challenges to facilitate scalable and resourceful services through effective physical infrastructure, smart networking solutions, intelligent software tools, and useful software approaches. This paper analyzes the extensive requirements which exist in data-intensive clouds, describes various challenges related to the paradigm, and assess numerous solutions in meeting these requirements and challenges. It provides a detailed study of the solutions and analyzes their capabilities in meeting emerging needs of widespread applications.

The Predictive Retailer is a retail company that utilizes the latest technological developments to connect with its customers to deliver an exceptional personalized experience to each and every one of them. Today, technology such as AI, Machine Learning, Augmented Reality, IoT, Real-time stream processing, social media, and wearables are altering the Customer Experience (CX) landscape and retailers need to jump aboard this fast moving technology or run the risk of being left out in the cold.

The Predictive Retailer reveals how these and other technologies can help shape the customer journey. The book details how the five types of analytics—descriptive, diagnostic, predictive, prescriptive, and edge analytics—affect not only the customer journey, but also just about every operating function of the retailer.

An IoT connected retailer can make its operations smart. Connected devices can help with inventory optimization, supply chain management, labor management, waste management, as well as keep its data centers green and its energy use smart.
Social media is no longer a vanity platform, but rather it is a place to both connect with current customers as well as court new ones. It is also a powerful branding channel that can be utilized to both understand a retailer's position in the market, as well as a place to benchmark its position against its competitors.
Today, technology moves at break-neck speed and it can offer the potential of anticipatory capabilities, but it also comes with a confusing variety of technology and technological terms--Big Data, Cognitive Computing, CX, Data Lakes, Hadoop, Kafka, Personalization, Spark, etc., etc. The Predictive Retailer will help make sense of it all, so that a retail executive can cut through the confusing technological jargon and understand why a Spark-based real-time stream processing data stream might be preferable over a TIBCO Streambase one, or an IBM Streaming Analytics one. This book will help retail executives break through the technological clutter so that they can deliver an unrivaled customer experience to each and every patron that comes through their doors.

The Predictive Casino is a casino that utilizes the latest technological developments to connect with its customers to deliver an exceptional personalized experience to each and every one of its patrons. Today, technology such as AI, Machine Learning, Augmented Reality, IoT, Real-time stream processing, social media, and wearables are altering the Customer Experience (CX) landscape and casino operators need to jump aboard this fast moving technology or run the risk of being left out in the cold.
The Predictive Casino reveals how these and other technologies can help shape the customer journey. The book details how the five types of analytics—descriptive, diagnostic, predictive, prescriptive, and edge analytics—affect not only the customer journey, but also just about every operating function of the casino.
Facial recognition technology can spot a customer stepping onto a casino bus at the Macau/China border and that set off alerts that can notify a host on the floor, a restaurant manager, a dealer, or even the hotel’s GM should the player be a high-end VIP. A whole other sequence of events gets triggered as the player’s favorite table is prepared, his favorite, and his or her Theo gets added to a real-time table games revenue management model that takes into account his personal play. Labor needs are also considered and alerts can be sent to the required or unneeded staff.
An IoT connected casino can make its operations smart. Connected devices can help with inventory optimization, supply chain management, labor management, waste management, as well as keep its data centers green and its energy use smart. 
Social media is no longer a vanity platform, but rather a place to both connect with current customers as well as court new ones. It is also a powerful branding channel that can be utilized to both understand a casino's position in the market, as well as a place to benchmark its position against its competitors.
Today, technology moves at break-neck speed and it can offer the potential of anticipatory capabilities, but it also comes with a confusing variety of technology and technological terms--Big Data, Cognitive Computing, CX, Data Lakes, Hadoop, Kafka, Personalization, Spark, etc., etc. The Predictive Casino will help make sense of it all, so that a casino executive can cut through the confusing technological jargon and understand why a Spark-based real-time stream processing data stream might be preferable over a TIBCO one, or an IBM one. This book will help casino executives break through the technological clutter so that they can deliver an unrivaled customer experience to each and every patron coming through their doors.

Abstract — Big data analytics technologies are to extract value from very large data volume, variety of data, and highly rate of data stream. With the fast deployment of cloud services with mobile devices, big data analytics is shifting from personal computer to mobile devices. But, significant limitations of mobile devices are less storage amount and processing power. This paper proposes a big data analytic platform on mobile cloud computing with efficient query execution time by developing MapReduce Transformation Process and query operation based on input query’s complexity level. Furthermore, this paper presents the process of RESTful web service for providing seamless connectivity between mobile devices and cloud storage, where store the data and all of necessary processing steps are done by cloud backend. This proposed platform is evaluated by using Census dataset and compared the result with other traditional high level query languages, such as Pig, Hive, and Jaql.

Keywords- big data analytic; mobile cloud computing; Hadoop MapReduce; RESTful web serivce

International Journal of Computer Science and Information Security (IJCSIS),
Vol. 13, No. 9, September 2015

https://sites.google.com/site/ijcsis/
ISSN 1947-5500

These last years, the new technologies produce each day large quantities of data. Companies are faced with certain problems of collecting, storing, analyzing and exploiting these large volumes of data in order to create the added value. The whole issue, for companies and administrations, is not to pass by valuable information drowned in the mass. It is here where the technology of the "Big Data" intervenes. This technology is based on an analysis of very fine masses of data. It is interesting to note that there are several publishers who offer distributions ready to use for managing a system Big Data namely HortonWorks [1], Cloudera [2], MapR [3], IBM Infosphere BigInsights [4], pivotal HD [5], Microsoft HD Insight [6], etc. The different distributions have an approach and a different positioning in relation to the vision of a platform Hadoop. These solutions are the Apache Projects and therefore available. Yet, the interest of a complete package resides in the compatibility between the components, the simplicity of installation, support, etc. In this article, we shall discuss the world of big data by defining these characteristics and its architecture. Then we shall talk about some distributions Hadoop, and finally, we shall conclude by a comparative study on the top five suppliers of Hadoop distributions of Big Data.

Apriori is designed to operate on databases containing transactions (for example, collections of items bought by customers, or details of a website frequentation). Other algorithms are designed for finding association rules in data having no transactions , or having no timestamps (DNA sequencing). Each transaction is seen as a set of items (an item set). Given a threshold , the Apriori algorithm identifies the item sets which are subsets of at least transactions in the database. Apriori uses a "bottom up" approach, where frequent subsets are extended one item at a time (a step known as candidate generation), and groups of candidates are tested against the data. The algorithm terminates when no further successful extensions are found.